When people first see data in minware, they are often surprised with the level of detail and have questions about how to use this information effectively and responsibly. Rolling out engineering analytics across an organization to achieve their full potential while avoiding pitfalls involves a lot of stakeholders and can be tricky to do well.
One concern that comes up with adopting engineering metrics is whether managers and executives will use the data in a fair and appropriate way rather than leaping to conclusions without having adequate context, especially if they will be a factor in performance reviews.
However, it’s important for everyone to understand that without data, managers are already prone to bias and have to make decisions with less information, primarily relying on what they hear in discussions. This risks favoring the more politically savvy engineers at the expense of quieter hard workers.
Another question that often arises during the engineering analytics adoption process is whether they will guide people toward the organization’s true goals, or whether people will game the metrics. This is definitely a legitimate worry, as evidenced by the long list of corporate scandals involving manipulation of financial metrics.
However, this is not an insurmountable hurdle. Though there is some manipulation, financial metrics still form the backbone of our economy. For engineering metrics, the right combination of incentives, culture, and alignment to goals can significantly reduce the risk of unintended outcomes.
Finally, there is the question of how to integrate engineering metrics into regular workflows and conversations so that they lead to the appropriate actions.
In this article, we walk through steps you can take to successfully adopt engineering analytics in your organization, based on our experience helping customers roll out minware.
Also, we are here to help. If you’d prefer, we can walk through some questions about your organization and goals on a brief call and create a custom timeline and plan for you, just let us know!
The first step in a smooth adoption process is to identify all of the stakeholders and what their roles will be. Neglecting to do this significantly increases the risk of the rollout failing or only partially achieving its goals, either due to outright resistance from overlooked stakeholders, or by missing important requirements.
For this step, be recommend using a decision-making framework like DACI, which provides clarity for everyone on their roles.
Here is the typical set of roles that we have seen and would recommend as a starting point for adopting engineering analytics.
The person leading the project and driving things forward. For engineering analytics, this may be an individual engineering or project manager, but is often the CTO or VP of Engineering themselves. Either way, it is important that the ultimate leader of the engineering team strongly supports the initiative and the person in this role is empowered to act on behalf of the leader.
These are people with veto power over the rollout who need to provide approval. The exact list may vary between organizations, but we often see the following:
These are people who are involved in early demos and discussions about the process, giving them the opportunity to provide feedback prior to important decisions. Their buy-in is important, but they do not have veto power. We typically see people in these roles as contributors:
People in this category are informed about the decision so that they have the opportunity to ask questions prior to the final implementation. It is important to include everyone who will be affected in this category to build their support and avoid them feeling blindsided or excluded. We typically see in this category:
Sometimes the question comes up of whether to inform everyone about the adoption of engineering analytics. Not informing individuals may make sense in certain limited situations, like if the organization only plans to use metrics for cost capitalization and effort allocation reporting to the CEO, but not look at metrics at an individual level.
However, we do not recommend looking at individual metrics without giving those people access to the same metrics so that they understand what the data says and can have a transparent conversation with their manager. For example, a manager bringing up things they saw in engineering metrics without saying where they came from creates a culture of distrust that is ultimately counter-productive.
The next thing that is important for a successful rollout is building a timeline of all the events that have to occur to complete the adoption. Here is a list of the important events that you can use as a starting point. We recommend building this list to include all relevant meetings and attach specific dates to each item.
This timeline may vary depending on your needs, but has all the key elements for organizations of any size to successfully adopt engineering analytics. You may also want to pare this down a bit, but keep in mind that the most important thing is to create a written adoption plan and make sure all key stakeholders have the opportunity to provide feedback before moving forward.
The key artifact in the adoption process is a written plan that describes how the organization will integrate engineering analytics into their processes and workflows. We have put together an Engineering Analytics Adoption Plan Template that you can use as a starting point.
Importantly, the adoption plan is the place where you address potential gaming of metrics, affirming that people should do what is best for the organization. If they believe this to be in conflict with the metrics, they should feel supported and encouraged to reconcile this with their manager rather than incentivized to subvert the intent of the metrics.
This template covers most of the questions that typically come up during the adoption process, and is geared toward organizations that are fully rolling out minware to their whole engineering organization.
If your situation is different, let us know, and we’d be happy to tailor the template to your needs.
Successfully managing complex organizational changes can be difficult and time-consuming. We respect our customers’ time and appreciate the level of effort it takes to roll out engineering metrics to an organization.
Part of the process has to happen internally, but our goal is to provide as much planning support as we can to ensure success. If there is any part of the adoption process where you think we can help save you time, please reach out!