Cursor Analytics Mistakes to Avoid: 9 Pitfalls That Distort Your AI ROI Story
TL;DR: Most attempts to show Cursor's return on investment (ROI) fail because they present pull requests merged or adoption counts as evidence the spend paid off. The method that holds up in executive reviews, used in minware's Lean AI Framework, is a set of linear regressions, one per outcome, of roadmap value delivery, on-time delivery rate, and overhead cost against token spend. The ROI is read off each slope. Each team's trend over time is the supporting check, since AI-vs-non-AI cohort comparisons are now prone to selection bias. Every calculation needs to be visible enough to defend when the board asks how it works.
Cursor's dashboards show adoption climbing, and boards ask what that spend bought. Most reporting setups built around Cursor track activity and code output but have no methodology for connecting spend to delivery. The nine mistakes in this article are the specific errors that keep engineering leaders from building an ROI case that holds up when a board member asks how the number was calculated.
Finding the gap in Cursor's own analytics
Cursor's Admin API returns per-user daily metrics including tabs shown, tabs accepted, lines added, and agent requests. These are activity and throughput metrics that confirm the tool is running.
Cursor's analytics dashboard reports usage, including active users, accepted AI lines, and the share of committed code Cursor wrote, but it doesn't connect that usage to bugs in your project management system, review time, or delivery outcomes. Presenting usage figures as ROI evidence asks the board to take a leap of faith, and boards ask follow-up questions usage data can't answer.
Mistake 1: Using pull requests merged as an AI ROI proxy
Pull requests merged is the metric engineering leaders reach for out of habit, and it is the one that most reliably undermines an AI ROI story.
Mistaking code output for value
Pull requests merged is a throughput metric. It measures code output volume, which leaves open whether that code delivered value. It also can't separate AI-assisted work from human work without attribution.
Counting merged pull requests also rewards splitting work into smaller pull requests, so the number can rise without any change in what ships.
A Microsoft study of its own rollout of Claude Code and GitHub Copilot CLI found adopters merged roughly 24% more pull requests. The authors caution that a merged pull request and the value it delivers are different things. DORA's 2025 research found AI adoption has a positive relationship with software delivery throughput and a negative relationship with delivery stability. Faster output with less stable delivery is the pattern a count of merged pull requests hides.
Tracking value delivery instead
Roadmap value delivery and on-time delivery rate are the value outcomes an ROI claim requires. Roadmap value delivery defaults to story points completed on tickets that are not bugs and have a parent epic. On-time delivery rate divides each roadmap item's estimated duration by its actual duration, capped at 100%. Pull requests merged can still be tracked as a measure of code output volume, but it can't carry the ROI claim. When a board member asks what the AI spend bought, the answer has to be in units of delivered value.
Mistake 2: Stopping at Cursor's own usage data
Cursor's usage data describes what the tool produced. The CFO's question is whether higher token spend is buying more delivery or just a bigger bill. Adoption metrics answer whether the tool is being used, while delivery metrics answer what the usage produced.
Connecting usage data to delivery systems
The fix is to connect Cursor usage data to your version control system, such as GitHub, GitLab, Bitbucket, or Azure DevOps, and your project management system, such as Jira, Linear, Azure Boards, or GitHub Issues. Roadmap value delivery and on-time delivery rate are then the outcomes that map to the commitments leadership tracks.
Cursor's Admin API returns per-request token and cost data. On Cursor Enterprise, the AI Code Tracking API also attributes AI-written lines to commits. Neither API links that usage to tickets or delivery outcomes, so that connection has to come from a normalized data layer or a custom pipeline.
Mistake 3: Dividing total outcomes by total spend
The tempting financial readout is total story points completed divided by total token spend, reported as story points per dollar. That figure credits AI with all delivery, including work people would have shipped anyway. It also breaks down for non-AI work, because a spend of zero can't be a denominator.
Token spend is the dollar cost of AI tool usage. Token consumption, the count of tokens, is a separate data point. For teams on flat-rate plans, reported spend may be more or less than the amount actually paid.
Reading ROI off the slope
The method that holds up is a linear regression of a delivery metric against token spend across teams and time periods. Each regression fits one metric pair, so the Lean AI Framework runs three separately: roadmap value delivery, on-time delivery rate, and overhead cost against token spend. Overhead cost is personnel cost and token spend that goes toward work outside roadmap value delivery, such as bug tickets or tickets with no parent epic. Treating spend as a continuous variable puts non-AI work at the zero-spend endpoint of the same regression, so no binary split is needed.
The regression slope is the ROI figure. If the slope of roadmap value delivery against token spend reads 0.01, using its story points completed default, each additional dollar is associated with 0.01 additional story points completed, roughly one story point for every $100 spent. The R-squared value shows how much of the outcome's variation spend actually explains.
Naming the confounders
Selection bias affects the regression too. Engineers tend to reach for AI on work where it is likely to help, so AI-assisted tasks and non-AI tasks aren't a like-for-like comparison. Early adopters are also often already the strongest performers on a team. Covariates such as code base age and code quality can shift the slope too. Name all three when you present the regression, because a slope that looks favorable may partly reflect who opted in and what they chose to work on.
Mistake 4: Relying on AI-vs-non-AI cohort comparisons
AI-vs-non-AI cohort comparison feels rigorous but collapses under scrutiny. It made sense when adoption was partial, but it no longer holds up because non-AI groups are disappearing. The ones that remain are prone to selection bias. An engineer who skips AI is often working where it is unlikely to help, such as a legacy repository with poor test coverage. Comparing that engineer's output to an AI user's output mostly measures the kind of work each one is doing.
The same selection effect shows up in product analytics. As William Gieng writes about opt-in AI features in Towards Data Science: "Nobody randomized the AI assistant. It shipped to eligible accounts, some of them turned it on, and the analytics team compared the ones that did against the ones that did not. That comparison is not an effect. It is a description of who opts in."
Treating spend as a continuous variable
Treating token spend as a continuous variable produces a spend-to-outcome relationship that remains valid as adoption becomes universal. A binary split treats a team spending $10 the same as one spending $10,000, so it can't show whether higher spend is buying more delivery.
Mistake 5: Ignoring quality and efficiency guardrails when measuring AI ROI
AI-assisted code tends to arrive with more issues attached. Ignoring the potential side effects on quality and efficiency turns a positive ROI story into a credibility problem once they surface.
Accounting for AI-introduced issues
A 2025 difference-in-differences study, which compares Cursor-adopting GitHub projects against a matched group of similar projects that don't use Cursor, found a 41.6% increase in code complexity and a 30.3% rise in static analysis warnings after adoption. The gains in code output were limited to the first two months, while the added complexity and warnings persisted.
A study of 302,600 AI-authored commits across 6,299 GitHub repositories found that 22.7% of tracked AI-introduced issues were still present in the latest version of each repository. Code smells made up 89.3% of all AI-introduced issues in the study, so the cost shows up as maintenance work an ROI story has to account for.
Using bug rate and cycle time as guardrails
Bug rate is the quality metric that shows when those issues turn into defects. It divides bugs created by pull requests merged, which captures quality problems throughout the development process.
PR lead time runs from a pull request's first commit to when it is deployed, which defaults to when the code merges into a main branch. Ticket cycle time runs from when a ticket moves to an in-progress status to when it is marked done. As a guardrail, cycle time should hold steady or fall as token spend rises. When delivery outcomes don't move, ticket cycle time and bug rate are where to look for the reason. The ROI evidence itself remains the regressions of roadmap value delivery, on-time delivery rate, and overhead cost against token spend.
"Minware gives us clear visibility into code quality, defect rates, and development health with quality SDLC metrics." - George V. on G2
Mistake 6: Accepting black-box ROI definitions
When the board asks "how is this calculated," someone has to answer. If the answer is "the vendor's proprietary algorithm," the credibility cost is immediate.
Counting the cost of opaque calculations
Many engineering analytics platforms keep their metric logic in behind-the-scenes structured query language (SQL). When a question comes up about how a metric is computed, finding the answer typically requires a support request and an engineering escalation.
That dependency puts the engineering leader back in the vendor's queue for every follow-up question. What the board cares about is whether you can explain the number you are presenting.
Inspecting the formula behind each metric
minware implements every metric in minQL, its formula language purpose-built for engineering data, and shows each metric's full data lineage in the UI. You can inspect how PR lead time handles merges, squashes, and rebases, or how a ticket's estimate change affects sprint metrics, without filing a support ticket.
Every org defines "done" differently: some teams count a ticket as done when it merges, while others wait for deployment. Some also exclude user acceptance testing from efficiency metrics. minware handles these differences by adjusting existing metrics and breakdowns in a report configuration, so customers rarely write minQL themselves. Customer success usually turns these changes around within a single call and about 24 hours.
"The platform comes with a great set of engineering reports out of the box, so you can start getting value immediately. At the same time, those reports are highly customizable, and minQL makes it possible to build your own metrics and dashboards tailored to your team's workflow rather than being limited to predefined reports." - Verified user on G2
Mistake 7: Waiting for a clean baseline before measuring
The conventional advice is to capture a clean pre-rollout baseline before turning on Cursor. That advice assumes a window most teams have already passed. In DORA's 2025 survey, 90% of respondents use AI at work.
Starting from the data you already have
The continuous token-spend regression doesn't need a pre-rollout baseline, because non-AI work already sits at its zero-spend endpoint. What matters is connecting Cursor usage data, version control, and your project management system as early as you can.
minware includes one year of historical backfill on Professional and unlimited backfill on Enterprise. Version control and ticket history are available as soon as the first backfill completes. First-connection backfill can take hours, depending on repository size and data volume. The mistake is telling yourself you missed the window, when the regression works with the data you have.
Mistake 8: Building internal tooling without counting the governance cost
A small team can build a custom pipeline connecting Cursor's API to version control and ticketing data. The mistake is underestimating both the upfront build and the upkeep that follows.
Budgeting for the build and its upkeep
Building the pipeline is a large upfront project, typically several weeks of engineering time. Connecting data across version control, ticketing, and AI tools requires three pieces of work:
-
Relationship recovery: linking agent sessions, commits, pull requests, and tickets that carry no structured connection
-
Identity resolution: matching the same developer across systems with different usernames or emails
-
Data normalization: canonicalizing each vendor's format into one consistent model
minware's patented hypercube data model handles this work, associating agent sessions with commits and pull requests, then linking those to tickets.
Vendor APIs change regularly. Cursor labels its AI Code Tracking API as alpha, with response shapes and fields that may change. Smaller undocumented changes can break an analysis in ways that stay hidden until a metric drifts.
Owning metric definitions over time
As AI coding tools spread across multiple teams, consistent metric definitions become a governance requirement. A custom pipeline that works for one team's definition of cycle time breaks when a second team defines it differently.
The recurring cost people underestimate most is the governance layer. Someone has to answer "how exactly is this calculated" every time a stakeholder asks, for as long as the pipeline runs. With an internal build, that's whoever built the pipeline, indefinitely. With minware, that question goes to customer success, though org-specific metrics still need configuring.
Mistake 9: Failing to audit data quality before presenting ROI
Story point velocity, roadmap value delivery, and sprint completion depend on ticket linkage. Weak linking makes these metrics quietly undercount AI-assisted work, which distorts the ROI story in the direction that hurts you. Code-side metrics like PR lead time and pull requests merged are calculated from version control alone.
Closing the linkage gaps
minware's hypercube data model recovers hidden associations by modeling what commit and ticket each contributor was working on at any given time, then associating usage with the most probable ticket context. Explicit branch-to-ticket linking still improves accuracy, because developers often write code before marking a ticket in progress.
Messy data also shows where cleanup should start. Gaps in the analytics output point to the process gaps behind them, and hygiene metrics turn each one into an item a manager can work through. The Best Practices report tracks PRs Traceable to Ticket and Tickets Completed with Estimate. Like minware's other target-based metrics, each one comes with a list of the specific failing items behind it.
Setting up Cursor analytics that defends ROI
The replacement methodology has three components.
Standardize your core metrics. Start with the metrics that answer the board's question. Roadmap value delivery and on-time delivery rate are the two value outcomes. Overhead cost is the cost outcome regressed alongside them. Bug rate, PR lead time, and ticket cycle time are the supporting signals that explain why a trend is moving. Activity and throughput metrics sit outside the business outcomes, which are the only metrics that can carry an ROI claim.
| Metric | Category | Role in the ROI case |
|---|---|---|
| Roadmap value delivery | Value | Carries the ROI claim |
| On-time delivery rate | Value | Carries the ROI claim |
| Overhead cost | Cost | Carries the ROI claim |
| Bug rate | Quality | Explains the trend |
| PR lead time | Efficiency | Explains the trend |
| Ticket cycle time | Efficiency | Explains the trend |
| Seats activated | Activity | Cannot carry the ROI claim |
| Agent session count | Activity | Cannot carry the ROI claim |
| Accept rate | Activity | Cannot carry the ROI claim |
| Lines of code | Throughput | Cannot carry the ROI claim |
| Commits | Throughput | Cannot carry the ROI claim |
| Pull requests merged | Throughput | Cannot carry the ROI claim |
Regress outcomes against token spend. minware's AI impact reports run the token spend regressions automatically, one each for roadmap value delivery, on-time delivery rate, and overhead cost. Each regression groups data by team and time period, normalizing each data point by active contributor work days so team size doesn't distort the slope.
Track same-team trends over time. The supporting check is each outcome's trend over time for the same team as AI maturity increases. This controls for differences between teams, because you are comparing a team to itself at different points in time. Differences within a team, such as new projects or personnel changes, still remain.
Building a Cursor ROI case that holds up
Most of the nine mistakes come from two habits: treating usage or code output as outcomes, and accepting calculations nobody can inspect. A Cursor ROI case holds up when it regresses roadmap value delivery, on-time delivery rate, and overhead cost against token spend. Read the answer off each slope, then check each team's trend over time before the next board review. That gives you a number you can defend line by line when the board asks how it was calculated.
Start a 14-day free trial at minware.com, no credit card required, and connect your first data source. Explore the pre-built AI impact reports with your own data before talking to anyone on our team.
FAQs
What's the most common Cursor analytics mistake engineering leaders make?
Presenting pull requests merged as ROI evidence. That count shows code output rose and says nothing about whether that code delivered value. Adoption metrics like seats and accept rates have the same limit, since they only confirm the tool is running. minware's AI impact reports answer the question with regressions of delivery outcomes against token spend.
Why doesn't an AI-vs-non-AI cohort comparison work anymore?
Non-AI groups are disappearing as adoption spreads, and the ones that remain are prone to selection bias because engineers who skip AI are often working where it is unlikely to help. Token spend level as a continuous variable is the durable replacement, which is why minware's Lean AI Framework regresses outcomes against spend.
How do I prove Cursor ROI without a clean pre-rollout baseline?
Run a linear regression of each value outcome, such as roadmap value delivery, against token spend across teams and time periods. Non-AI work sits at the zero-spend endpoint of the same regression, so no separate baseline is required.
What metrics should I use instead of pull requests merged?
Roadmap value delivery and on-time delivery rate. Roadmap value delivery defaults to story points completed on non-bug tickets with a parent epic. Pull requests merged can still be tracked as a measure of code output, but it can't carry the ROI claim. minware's Lean AI Framework regresses both against token spend, along with overhead cost.
How long does it take to get defensible Cursor analytics data?
Backfill on first connection can take hours, depending on repository size and data volume. minware shows a source's reports once its backfill completes. Professional includes one year of history, and Enterprise includes unlimited history.
Key terms glossary
Pull requests merged: The total number of pull requests merged, to or from any branch. A throughput metric that measures code output volume.
Token spend: The dollar cost of AI tool usage, as reported by AI tool APIs or OpenTelemetry. On flat-rate, tiered, or discounted plans, reported spend may differ from the amount actually invoiced.
Linear regression: A statistical model charting the best-fit line between an outcome metric and an input variable. It is the primary AI ROI methodology, fitting a line through token spend and a delivery outcome metric, with one data point per team per time period. Each regression fits one metric pair, so correlating spend against multiple outcomes requires a separate regression for each.
Roadmap value delivery: The expected value of completed work items. Configurable to any project management field where a team sets explicit value estimates during roadmap planning, or to a custom spreadsheet upload. If no explicit value field is configured, defaults to story points completed on tickets that are not bugs and have a parent epic/project ticket, with 1 point assigned per ticket if no estimate is set. It is one of the three outcomes minware regresses against token spend.
On-time delivery rate: The estimated roadmap item duration (time from when a roadmap item is marked in progress until the original due date set at that time) divided by the actual roadmap item duration (time from in progress until it is marked done), capped at 100% per roadmap item. By default, minware treats each epic as a roadmap item, and this can be configured to use higher-level tickets instead.
Overhead cost: Personnel cost and AI token spend that goes toward work not counted in roadmap value delivery, i.e. cost not traceable to any ticket, or traceable to a ticket excluded from roadmap value delivery (by default, a bug ticket or a ticket with no parent epic/project ticket).
Same-team trend: The practice of tracking a business outcome metric's movement over time for the same team as AI maturity increases. This controls for differences between teams and serves as the supporting check to the regression.
Supporting signals: Quality and efficiency metrics (bug rate, PR lead time, ticket cycle time) that sit alongside the ROI regression. They confirm the spend is not creating a bottleneck or quality problem further down the pipeline.
Hygiene metrics: Process metrics, such as PRs Traceable to Ticket and Tickets Completed with Estimate, that surface where data quality gaps may affect reporting accuracy.
Data lineage: The tracking of how data flows from raw sources through transformations to final metrics. In engineering analytics, data lineage shows the dependency chain from raw data to final calculation, making metrics defensible in an executive review.
DORA: The DevOps Research and Assessment program, which publishes annual research on software delivery performance. Its 2025 report is the source for this article's AI adoption figures.
Lean AI Framework: minware's methodology for measuring AI impact. For ROI, it runs three separate linear regressions of roadmap value delivery, on-time delivery rate, and overhead cost against token spend.
Selection bias: The tendency for AI adoption comparisons to reflect who opts in, because engineers who choose AI early differ systematically from those who do not.