minware Engineering Analytics Adoption and Change Management Guide

When people first see data in minware, they are often surprised with the level of detail and have questions about how to use this information effectively and responsibly. Rolling out engineering analytics across an organization to achieve their full potential while avoiding pitfalls involves a lot of stakeholders and can be tricky to do well.

One concern that comes up with adopting engineering metrics is whether managers and executives will use the data in a fair and appropriate way rather than leaping to conclusions without having adequate context, especially if they will be a factor in performance reviews.

However, it’s important for everyone to understand that without data, managers are already prone to bias and have to make decisions with less information, primarily relying on what they hear in discussions. This risks favoring the more politically savvy engineers at the expense of quieter hard workers.

Another question that often arises during the engineering analytics adoption process is whether they will guide people toward the organization’s true goals, or whether people will game the metrics. This is definitely a legitimate worry, as evidenced by the long list of corporate scandals involving manipulation of financial metrics.

However, this is not an insurmountable hurdle. Though there is some manipulation, financial metrics still form the backbone of our economy. For engineering metrics, the right combination of incentives, culture, and alignment to goals can significantly reduce the risk of unintended outcomes.

Finally, there is the question of how to integrate engineering metrics into regular workflows and conversations so that they lead to the appropriate actions.

In this article, we walk through steps you can take to successfully adopt engineering analytics in your organization, based on our experience helping customers roll out minware.

Also, we are here to help. If you’d prefer, we can walk through some questions about your organization and goals on a brief call and create a custom timeline and plan for you, just let us know!

Identify the Stakeholders

The first step in a smooth adoption process is to identify all of the stakeholders and what their roles will be. Neglecting to do this significantly increases the risk of the rollout failing or only partially achieving its goals, either due to outright resistance from overlooked stakeholders, or by missing important requirements.

For this step, be recommend using a decision-making framework like DACI, which provides clarity for everyone on their roles.

Here is the typical set of roles that we have seen and would recommend as a starting point for adopting engineering analytics.

Driver

The person leading the project and driving things forward. For engineering analytics, this may be an individual engineering or project manager, but is often the CTO or VP of Engineering themselves. Either way, it is important that the ultimate leader of the engineering team strongly supports the initiative and the person in this role is empowered to act on behalf of the leader.

Approver

These are people with veto power over the rollout who need to provide approval. The exact list may vary between organizations, but we often see the following:

CTO/VP Engineering - If not also the driver, the engineering team’s leader is the most important person who needs to approve the overall project.
HR - It is important for HR to understand how managers will or will not use analytics data in their performance reviews to ensure that it is done in a fair and appropriate way. They should also be comfortable with the permissions that people have to look at data for other teams and individuals to assess whether it aligns with the company’s desired culture.
CEO - The CEO may be an approver, especially if the plan is to share engineering metrics with the board or CEO. We recommend doing this because it helps the engineering leader communicate where the organization is putting its effort and show improvements to productivity. This creates stronger alignment between the engineering team and the rest of the organization, making it easier to justify the resources that the engineering leader needs to be successful.
Finance - Someone in finance may be an approver if they plan to utilize minware’s cost capitalization functionality.
Standard Purchasing (IT, Security, Legal, Budget) - Last, there may be a standard set of approvers involved with the process of purchasing and administering the engineering analytics software.

Contributor

These are people who are involved in early demos and discussions about the process, giving them the opportunity to provide feedback prior to important decisions. Their buy-in is important, but they do not have veto power. We typically see people in these roles as contributors:

Engineering Managers and Directors - These are the people who will be responsible for integrating engineering analytics into their regular conversations with direct reports, and possibly performance reviews. It’s important for them to be on board and have their concerns addressed for successful adoption.
A Principal Engineer - It can be helpful to involve a senior and respected individual engineer to ensure that they believe the metrics are accurate and align with their understanding of what success looks like for individual contributors on the team.
A Senior Scrum Master / Project Manager / Product Owner - Since a lot of the metrics relate to agile and project planning processes, it is similarly important that the people responsible for these processes agree that the metrics are accurate and align with their goals.

Informed

People in this category are informed about the decision so that they have the opportunity to ask questions prior to the final implementation. It is important to include everyone who will be affected in this category to build their support and avoid them feeling blindsided or excluded. We typically see in this category:

All Engineers and Scrum Team Members - These are the people responsible for day-to-day work that the engineering analytics will be measuring, so it is important to know what will be expected of them.
Everyone in the Product Organization - This can depend on how involved product managers are in day-to-day project management, which varies between organizations. As a general guide, anyone who participates in scrum ceremonies should probably be informed.

Is It Okay Not to Inform Everyone?

Sometimes the question comes up of whether to inform everyone about the adoption of engineering analytics. Not informing individuals may make sense in certain limited situations, like if the organization only plans to use metrics for cost capitalization and effort allocation reporting to the CEO, but not look at metrics at an individual level.

However, we do not recommend looking at individual metrics without giving those people access to the same metrics so that they understand what the data says and can have a transparent conversation with their manager. For example, a manager bringing up things they saw in engineering metrics without saying where they came from creates a culture of distrust that is ultimately counter-productive.

Create a Timeline

The next thing that is important for a successful rollout is building a timeline of all the events that have to occur to complete the adoption. Here is a list of the important events that you can use as a starting point. We recommend building this list to include all relevant meetings and attach specific dates to each item.

Initial Demo - The Driver receives a demo and decides to move forward with a trial.
(If necessary) Security Review - The Driver facilitates any security review needed to kick off the trial.
Data Connection - The Driver facilitates connecting data sources to start the trial.
Driver Demo - The Driver receives an initial demo with their own data to review insights and learn how to navigate the system.
Driver Review - The Driver reviews the reports following the insights demo to ensure that the metrics meet requirements. The Driver may ask questions or have ad hoc calls with minware during this time to ensure they have a full understanding of the metrics and minware is configured properly for the organization. The Driver may also wish to review minware with key Contributors or Approvers at this time.
Contributor Demo - The Driver, Contributors, and Approvers have a call with minware to walk through the tool. The goal of the walkthrough is to answer any questions, as well as identify any special configuration or functionality that the organization will need.
Contributor Review - The Driver, Contributors, and Approvers meet internally to share any questions, concerns, or requirements that will need to be addressed to successfully adopt engineering analytics. This may happen in multiple meetings if there are different groups of stakeholders with different roles, like a separate meeting for engineering managers and for HR/CEO/finance.
Draft Adoption Plan - The Driver drafts an adoption plan (described in the next section) that resolves any outstanding questions or concerns. The Driver may also wish to draft part or all of this plan prior to the contributor review to facilitate that conversation.
Decision - The Driver obtains approval to move forward with the adoption plan from all of the Approvers and purchase the software.
Initial Announcement - The Driver announces the adoption plan to everyone who should be Informed.
General Feedback - The Driver solicits questions and feedback from everyone who is informed, possibly in a call, in writing, or anonymously depending on the communication norms of the organization.
(Optional) Q&A Sessions - Depending on the feedback, the Driver, engineering leader, and/or individual engineering managers may want to have further internal sessions for discussing the adoption plan and answering questions.
(Optional) General Demo - Everyone who is informed participates in a demo to hear more from minware about their product vision and how to successfully adopt engineering analytics. The goal of this call is to build support for the adoption plan, and also help minware understand how to best serve the organization with customer success and future product updates.
Final Adoption Plan - The Driver should finalize the adoption plan, including answers to any outstanding questions, and share it with everyone.
Invite Everyone to minware - Finally, the Driver should invite everyone to minware who will have access so that they can begin using the software.
Ongoing Quarterly Business Reviews - After finalizing the adoption plan, we recommend having business reviews at least quarterly to go over progress towards organizational goals and provide feedback to minware about how to improve the product.

This timeline may vary depending on your needs, but has all the key elements for organizations of any size to successfully adopt engineering analytics. You may also want to pare this down a bit, but keep in mind that the most important thing is to create a written adoption plan and make sure all key stakeholders have the opportunity to provide feedback before moving forward.

Draft an Adoption Plan

The key artifact in the adoption process is a written plan that describes how the organization will integrate engineering analytics into their processes and workflows. We have put together an Engineering Analytics Adoption Plan Template that you can use as a starting point.

Importantly, the adoption plan is the place where you address potential gaming of metrics, affirming that people should do what is best for the organization. If they believe this to be in conflict with the metrics, they should feel supported and encouraged to reconcile this with their manager rather than incentivized to subvert the intent of the metrics.

This template covers most of the questions that typically come up during the adoption process, and is geared toward organizations that are fully rolling out minware to their whole engineering organization.

If your situation is different, let us know, and we’d be happy to tailor the template to your needs.

The Bottom Line

Successfully managing complex organizational changes can be difficult and time-consuming. We respect our customers’ time and appreciate the level of effort it takes to roll out engineering metrics to an organization.

Part of the process has to happen internally, but our goal is to provide as much planning support as we can to ensure success. If there is any part of the adoption process where you think we can help save you time, please reach out!