How to Measure AI ROI: A Step-by-Step Framework

TL;DR: Measuring AI return on investment (ROI) means reading the slope of a delivery outcome regressed against token spend. Pick a value delivery metric such as roadmap value delivery or on-time delivery rate, then capture token spend by team and time period. Run a linear regression and read the slope, with the R-squared value showing how much of the outcome's variation spend explains. Same-team trends over time are the supporting check, while bug rate, PR lead time, and ticket cycle time act as guardrails. minware runs this methodology on connected data, with the slope and R-squared value shown in its pre-built AI impact reports.

A before/after baseline is a common way to measure AI ROI, and it assumes the world stood still while your team adopted AI. In most organizations it did not: tooling changed, priorities shifted, and teams reorganized. A static snapshot struggles to separate AI's contribution from everything else that moved.

AI tool vendors report usage climbing while the board asks whether higher spend buys more delivery or just a bigger bill. This article walks through five measurement steps built to hold up in an executive review, then reads a real regression example.

Replace static baselines with a regression

A regression reads AI's contribution from variation across teams and time periods, so it does not depend on the world standing still. Two shifts make that possible: measuring value instead of adoption, then naming the selection bias that skews any comparison of AI-assisted vs. non-AI work.

Move from adoption to value metrics

Adoption metrics show whether a tool is running, while throughput metrics show how much code it produces. Neither answers whether delivery improved. The most common version of this mistake is pull request merge counts presented as AI ROI, which prove code output rose without showing whether that code delivered value.

Value delivery metrics answer the ROI question directly because they measure what the organization completed: roadmap value delivery and on-time delivery rate. Both frame results in terms a CFO can weigh against the spend.

Accounting for selection bias

Comparing AI-assisted work to non-AI work is losing meaning as adoption spreads, and the non-AI groups that remain are prone to selection bias. An engineer who skips AI may simply be working where it is less likely to help, such as a legacy repository with poor test coverage. Selection bias means the groups being compared differ systematically because of who chose to adopt the tool, so the comparison struggles to attribute outcomes to AI adoption alone.

The standing confounders include task-level selection (engineers reach for AI where it helps), person-level selection (early adopters may already be strong performers), and covariates such as code base age or code quality.

Understand the regression behind the framework

AI ROI is a marginal figure: the change in a delivery outcome per unit change in token spend. That makes the calculation a linear regression, with the slope as the answer.

Defining the slope and R-squared

Each regression fits one metric pair: an outcome on the y-axis, token spend on the x-axis. Each data point represents one team in one time period. The slope is the change in the outcome per dollar of spend, read at the margin.

The R-squared value shows how much of the outcome's variation the spend explains, a goodness-of-fit measure that cannot establish cause. A low R-squared means token spend is only weakly correlated with the outcome, without indicating what else drives it. Correlating spend against three outcomes takes three separate regressions, one for each metric pair.

minware has no built-in statistical calculator for group comparison, so export comma-separated values (CSV) data or use an external calculator for confidence intervals.

Gathering the required data

The framework draws on three data sources: your version control system, your project management system, and your AI coding tools.

Data source	What it provides	Why it matters
Version control (GitHub, GitLab, Bitbucket, Azure DevOps)	Commits, pull requests, PR lead time, bug rate denominator	Code-side guardrail signals and AI authorship
Project management (Jira, Linear, Azure Boards, GitHub Issues)	Story points completed, ticket cycle time, bugs created	The ticket data behind the delivery outcome
AI coding tool	Token spend, agent sessions (individual units of AI agent work)	The spend data behind the investment

Connecting the required inputs

Before running your first regression, confirm each input is in place:

• Version control connected: Commits, pull requests, and branch-to-ticket links are flowing, with initial backfill complete.

• Project management connected: Tickets include story point estimates or another value field, with status changes captured for ticket cycle time.

• AI tool telemetry configured: Token spend data arrives at the most granular level your tools support, ideally through OpenTelemetry for prompt-level detail.

• Data connected across tools: Relationship recovery connects agent sessions to commits, pull requests, and tickets. Identity resolution matches each developer across systems, while data normalization maps every vendor's format into one model.

• Team and time period filters defined: You can slice data by team, by week, or by sprint, with active contributor work days available for per-person-day normalization.

• Hygiene metrics reviewed: Linking branches to tickets and estimating tickets show where process gaps will undercount AI-assisted work.

Many analyses work as soon as data is connected and initial backfill completes, including the regression of story points completed against token spend in minware's pre-built AI impact reports. Configuration comes in when a question needs a new or adjusted metric or report.

Step 1: Select your primary outcome metrics

The first decision is which outcome you will regress against spend. Choose wrong and every later step produces a defensible number for the wrong question.

Lead with value delivery metrics

Lead with value delivery: roadmap value delivery and on-time delivery rate. When no explicit value field is configured, roadmap value delivery defaults to story points completed on tickets that are not bugs and have a parent epic. On-time delivery rate divides each roadmap item's estimated duration by its actual duration, capped at 100%. Both answer the board's question in units it already cares about.

Sprint completion is a predictability metric and an earlier leading indicator of on-time project delivery. That makes it useful context, though it never serves as the ROI evidence itself.

Not every team estimates in story points, so the unit of value depends on how your team plans work: tickets completed, story points completed, story point velocity, roadmap value delivery, or project completion. Pick one per report. Story points completed is already roadmap value delivery's default calculation, so listing both would count the same outcome twice.

Assign each metric a role

The table below shows each metric's category and its role in the analysis.

Metric	Category	Role
Roadmap value delivery	Value delivery	Primary outcome
On-time delivery rate	Value delivery	Primary outcome
Overhead cost	Cost	Primary outcome
Bug rate	Quality	Guardrail
PR lead time	Efficiency	Guardrail
Ticket cycle time	Efficiency	Guardrail

Avoid flawed indicators

These metrics show activity or code output, so none of them can carry an ROI claim on its own:

• Pull requests merged: The metric teams still reach for out of habit. A study of Microsoft engineers found command-line agent adopters merged roughly 24% more pull requests. Its authors note that a merged PR serves as a proxy for output, which differs from the value it delivers.

• Commits: One study of GitHub projects found Cursor adoption produced a large but transient increase in commits and lines added. Commit counts can jump with a rollout, then fade, and neither movement says whether delivery changed.

• Lines of code: Google said in April 2026 that 75% of its new code is AI-generated and approved by engineers, so lines of code increasingly tracks AI output volume.

• Completions accepted: Accepting a suggestion is not the same as delivering value.

• Seat counts and logins: Show whether the tool is in use, which says nothing about outcomes.

Step 2: Pair each outcome with guardrail metrics

Guardrails keep a delivery gain from hiding trouble elsewhere. They also help explain why an outcome moves when it does.

Each value delivery metric pairs with two guardrails, bug rate for quality and cycle time for efficiency. By default, bug rate is bugs created divided by pull requests merged. Bug rate is a superset of change failure rate and captures quality problems change failure rate misses. A high change failure rate still signals a more severe problem needing immediate attention.

As a guardrail, cycle time should hold steady or fall as AI spend rises, which confirms the spend is not creating a bottleneck further down the pipeline. As a diagnostic, when delivery outcomes do not move, consider confounders first, such as teams taking on more complex work. Ticket cycle time and bug rate are where you look next.

Step 3: Capture token spend by team and time period

Token spend is the dollar cost of AI tool usage. Token consumption and token cost are separate data points, so confusing them can produce a regression against the wrong variable.

Sourcing and normalizing token data

Token usage data is broadly available across AI coding tools through vendor APIs and OpenTelemetry integrations. GitHub Copilot reports token and credit usage through its usage metrics API. Cursor's Admin API returns per-request token consumption and cost. Claude Code offers both an API and OpenTelemetry, while Codex uses OpenTelemetry.

The Claude Code Analytics API reports sessions, token usage, and estimated cost per user, per model, per day. Reported spend is an approximation of what you pay. On flat-rate plans such as Claude Pro, it can be more or less than the amount actually billed.

minware's regressions default to grouping by team and time period, normalizing each data point to per-person-day. Token spend and the outcome metric are both divided by active contributor work days. Team-size differences then cannot distort the slope, because a 10-person team and a 2-person team compare on the same unit. The same normalization absorbs holidays and vacations, since a short week has fewer active contributor work days in the denominator.

Connecting telemetry early

Connect telemetry as soon as you can, because OpenTelemetry reports at per-prompt, per-session granularity with no historical backfill. Earlier usage is still recoverable for Console organizations: the Claude Code Analytics API can backfill at a coarser per-user, per-model, per-day grain. The Claude Enterprise Analytics API covers data from January 1, 2026 onward.

OpenTelemetry for Claude Code, Codex, and GitHub Copilot is set in local configuration files. Administrators can also push the same settings centrally through managed settings or mobile device management (MDM) tools.

Step 4: Join token spend to delivery outcomes

This step joins spend data to outcome data, which is where the ROI calculation takes shape.

A regression against token spend compares metrics per team per time window, so it needs teams and periods that differ. The variation comes from teams and time periods, while a same-team trend over time serves as the supporting check.

minware's Lean AI Framework regresses three outcomes separately against token spend: roadmap value delivery, on-time delivery rate, and overhead cost. Read the incremental ROI off the slope of each regression. The framework matches metrics to reporting level, from business outcomes for the board down to best practice diagnostics for engineering teams.

Do not bucket token spend into low, moderate, and high spend groups. Bucketing a continuous variable is still cohort analysis, with the same selection bias problems as an AI vs. non-AI comparison. Keep spend continuous, then name the confounders whenever you draw a conclusion from the regression. The continuous methodology also does not need a clean pre-AI baseline.

Step 5: Read the slope against same-team trends

Once the regression runs, read the slope correctly, then check it against the same team's trend over time.

The slope is the ROI figure, read at the margin: story points completed per additional dollar of token spend, for example. Swapping the axes expresses the same relationship as dollars per story point, the cost-per-outcome figure stakeholders often ask for. Dividing total delivery by total spend gives the average version instead, which attributes all delivery to AI spend, including everything humans would have shipped anyway. That average is also undefined for non-AI work, because spend of zero cannot be a denominator.

Same-team trends over time control for differences between teams, because each team is compared against its own history as AI maturity increases. If the regression slope is positive but a given team's trend is flat, the slope may be driven by differences between teams. That is the confound the supporting check exists to catch.

The whole method is observational analysis, controlling for bias in data your teams already generate.

Work through an example readout

An example regression on minware's AI ROI report plots story points completed on the y-axis against token spend in dollars on the x-axis.

Reading a real slope

The regression returns a slope of 0.02 story points completed per dollar of token spend, with an R-squared of 0.022. The slope reads as 2 additional story points for every additional $100 of token spend. The R-squared says token spend explains about 2% of the variation in story points completed, so on that data, spend is only weakly correlated with delivery.

A weak correlation does not identify what else drives delivery. Name the standing confounders before concluding anything, starting with task-level and person-level selection.

Next comes the supporting check. Plot each team's story points completed over the same period. If a team's trend rises as its spend rises, the relationship holds within that team. If the trend is flat, the slope may reflect differences between teams instead.

Presenting the result to leadership

The board slide leads with the slope and the R-squared value. It then adds the same-team trend as the supporting check, with bug rate and cycle time trends as the guardrail metrics. Name roadmap value delivery as the outcome, then let PR lead time follow as a supporting signal. The sentence that survives an executive review states the slope, the R-squared value, and whether the trend holds within the same team over time.

Decide whether to build or buy

The framework only works once version control, project management, and AI tool data are linked, whether by an internal pipeline or by minware. The patented hypercube data model recovers relationships between commits, tickets, pull requests, and agent sessions that carry no structured link. It also resolves identities across systems and normalizes vendor formats, so the regression runs on connected data from every source.

Consideration	Internal build	minware
Initial setup effort	High: typically multiple weeks of engineering time to build a custom extract, transform, load (ETL) pipeline across version control, project management, and AI tool APIs	Connect data sources through standard integrations, with backfill running automatically
Ongoing maintenance	Someone owns API changes, schema drift, and answering "how is this calculated" indefinitely	minware maintains compatibility with vendor APIs and answers calculation questions through support
Org-specific questions	Answered by whoever built the pipeline	A new or adjusted metric or report needs configuration, which customer success typically handles in a single call and about 24 hours
Metric transparency	Visible if you document the formulas and keep the documentation current	Visible formulas via minQL, editable without a code deploy
Time to first regression	Weeks to months depending on data normalization complexity	Pre-built AI impact reports work as soon as data is connected

Run the framework this week

The five steps fit into one week: pick a value delivery outcome, pair it with guardrails, capture token spend per team per period, join spend to outcomes, then read the slope against same-team trends. The output is a number you can defend, with the methodology visible behind it.

Start a 14-day free trial at minware.com, no credit card required, then connect version control, project management, and an AI coding tool. The pre-built AI impact reports include the token-spend-to-outcome regression, so you can read the slope with your own data before talking to anyone on our team.

FAQs

What if our data is too messy to run this analysis?

minware works with data as it exists today, and its hygiene metrics such as linking branches to tickets surface where the process is breaking down as itemized findings. Because the hypercube data model recovers missing relationships itself, analysis can start before source data is cleaned up.

How long does it take to collect enough data for a regression?

The regression needs teams and time periods that differ, so a steadier slope comes from more teams over more weeks. minware backfills one year of history on Professional, with unlimited backfill on minware's Enterprise plan. First-connection backfill can take hours depending on repository size.

Can we use this framework with multiple AI tools?

Yes. minware normalizes token data from Claude Code, Cursor, GitHub Copilot, and Codex into the same report. AI commit authorship can also be read from version control for any tool with no AI integration at all.

What do we do if the slope is negative?

A negative slope means higher spend is associated with less delivery, so start by considering confounders. The teams spending more on tokens may be doing more complex work or handling more disruptions, either of which can lower delivery even when AI is neutral or helpful. minware's reports can be configured to add potentially confounding variables. Then look at the guardrails: a rising bug rate or ticket cycle time points to quality or flow problems behind the drop. Once you know the cause, test specific practice changes against delivery data from your own teams.

Do we need a data scientist to run this analysis?

No. minware's regression charts include the slope and R-squared value. Customer success handles most customization, typically within a single call and about 24 hours. For confidence intervals, export CSV data or use an external calculator.

Key terms glossary

Baseline: A reference point for comparison, typically a pre-rollout snapshot. The continuous regression methodology does not require a clean pre-AI baseline.

Bug rate: A quality metric measuring the number of bugs created divided by the number of code changes, by default the number of pull requests merged into a main branch. Some customers configure the denominator to story points completed instead, to reduce the risk of PR volume changes distorting the metric.

Linear regression: A statistical model charting the best-fit line between an outcome metric and an input variable. It is the primary AI ROI methodology, fitting a line through token spend and a delivery outcome metric across teams and time periods. Each regression fits one metric pair, correlating spend against multiple outcomes requires a separate regression for each.

On-time delivery rate: The estimated roadmap item duration (time from when a roadmap item is marked in progress until the original due date set at that time) divided by the actual roadmap item duration (time from in progress until it is marked done), capped at 100% per roadmap item. By default, minware treats each epic as a roadmap item, and this can be configured to use higher-level tickets instead.

Overhead cost: Personnel cost and AI token spend that goes toward work not counted in roadmap value delivery, i.e. cost not traceable to any ticket, or traceable to a ticket excluded from roadmap value delivery (by default, a bug ticket or a ticket with no parent epic/project ticket).

PR lead time: The time from a pull request's first commit to when it was deployed. Deployment defaults to when the code merges into a main branch, and can be configured to use Git tags or CI/CD deployment pipeline runs. An efficiency metric, alongside ticket cycle time.

R-squared (R²): In a regression chart, the share of the outcome metric's variation explained by the input variable, on a 0-to-1 scale.

Regression slope: The change in an outcome metric per unit change in the input variable, such as story points completed per dollar of token spend.

Roadmap value delivery: The expected value of completed work items. Configurable to any project management field where a team sets explicit value estimates during roadmap planning, or to a custom spreadsheet upload. If no explicit value field is configured, defaults to story points completed on tickets that are not bugs and have a parent epic/project ticket, with 1 point assigned per ticket if no estimate is set.

Same-team trend: The change in an outcome metric over time for the same team, used as a supporting check to control for team differences.

Ticket cycle time: The time from when a ticket was first marked in progress to when it was completed.

Token spend: The dollar cost of AI tool usage, as reported by AI tool APIs or OpenTelemetry. On flat-rate, tiered, or discounted plans, reported spend may differ from the amount actually invoiced.