Cursor Analytics Buyer's Checklist: 26 Questions to Ask Before You Sign

TL;DR: Ask methodology, data lineage, and outcome correlation questions before pricing. The most decisive question is whether the vendor runs a separate linear regression against Cursor token spend for each outcome in minware's Lean AI Framework: roadmap value delivery, on-time delivery rate, and overhead cost. Each chart should show the slope and the R-squared value. Ask which Cursor API the vendor reads for your plan, since Cursor exposes different APIs on different plans. minware shows the formula behind every metric, lists its pricing publicly, and offers a 14-day self-serve trial, so you can run these questions against your own data before a sales call.

Cursor analytics dashboards usually lead with adoption rates and pull requests merged. Neither answers the question a CFO eventually asks: is higher spend buying more delivery, or just a bigger bill? Whether a vendor can show how its own numbers are built is one of the clearest signals available during a demo.

The checklist below gives you 26 questions across six categories: methodology, data lineage, outcome correlation, pricing, integrations, and security. Three of them are specific to Cursor: which Cursor API the vendor reads, how it links Cursor sessions to commits, and whether it reports Cursor spend in dollars. Each question tests whether a vendor can prove AI return on investment (ROI) with a defensible method, or is selling a dashboard that will fail its first executive review.

Reviewing the six evaluation categories

This table shows what each category evaluates and why it matters before you commit to a vendor.

Category	What to evaluate	Why it matters for ROI
Methodology	Whether calculation formulas are visible in the UI	Defending a metric in an executive review requires knowing how it's calculated
Data lineage	Whether you can trace each metric back to source commits, tickets, or agent sessions	Numbers without clear lineage require vendor support to explain
Outcome correlation	Whether delivery outcomes are regressed against token spend	Adoption counts and pull requests merged alone don't prove delivery impact
Pricing	Whether pricing is publicly listed and a self-serve trial is available	Gated pricing limits cost modeling before a procurement commitment
Integrations	Whether the vendor natively supports your version control, project management, CI/CD, and AI coding tools	Missing integrations mean manual data stitching
Security	Whether the SOC 2 Type 2 report, data residency options, and business associate agreement (BAA) availability meet your requirements	Enterprise procurement requires verifiable compliance

Applying the checklist in vendor demos

Ask the questions in the order below: methodology, data lineage, outcome correlation, pricing, integrations, then security. A vendor that fails on methodology will not recover on pricing, so there is little value in negotiating terms before you have seen how its metrics are calculated.

The demo script vendors want to run shows you dashboards. Yours should show you calculations. Ask the vendor to open a specific metric, such as PR lead time or ticket cycle time, and walk through the formula behind it live. If the answer involves a support request or an engineering escalation, note it as a failure and move on to the next category.

Methodology: Auditing calculation formulas

The five questions below test whether the vendor's metric definitions are transparent and built to stay aligned with your org over time.

1. How does the vendor aggregate data across sources?

Ask whether the logic that combines version control, ticketing, and AI tool data is visible in the UI or kept in proprietary code. A formula you can inspect is one you can explain to a stakeholder without filing a ticket. minware implements every metric in minQL, its formula language for engineering data. Every value in the chart builder is written in minQL, so the logic behind any chart is visible.

2. How does the formula treat merges, squashes, and rebases?

Merges, squashes, and rebases are where metric definitions diverge most, so ask the vendor to show exactly how its formula treats each one. The answer changes PR lead time for any team with a complex Git workflow. If the question has to go to engineering, expect the same delay every time a stakeholder asks it.

3. What happens when a pull request has no linked ticket?

Ask whether the vendor surfaces unlinked work as a finding or drops it from the numbers. Code-side metrics such as PR lead time and bug rate come from version control, so they don't depend on ticket links. Story point velocity and roadmap value delivery do depend on them. Weak linking makes both undercount AI-assisted work. minware's hypercube data model figures out the most likely relationship between branches and tickets even without an explicit link. Hygiene metrics such as PRs Traceable to Ticket then list the specific pull requests that need a link, so each gap becomes an item someone can fix.

4. How do you customize a metric for your org?

Ask whether you can answer an org-specific question without waiting on the vendor's roadmap, and what turnaround to expect when you can't do it yourself. With minware, most customization means selecting pre-built metrics and breakdowns in a report configuration. You can also build reports through a point-and-click interface that combines data from version control, ticketing, and other sources. When a question needs a new or adjusted metric, customer success typically handles it in a single call and about 24 hours.

5. How are formula changes versioned over time?

Ask how the vendor versions and communicates formula changes. Also ask whether historical data is recalculated when a definition changes. A vendor that can't answer clearly risks metrics that drift out of alignment with your org's definitions, which undermines every trend line you report.

Data lineage: Auditing data sources and lineage

The five questions below test whether the vendor can trace every metric to its source and connect records that different systems never linked.

6. Can every metric be traced back to its source record?

Ask whether the vendor can drill from a reported number to the commits, tickets, or agent sessions behind it, with each item linked back to its source system. minware's hypercube data model links every entity from every data source. A flagged outcome metric leads to its supporting metrics, then to a slice by any dimension, then to the list of individual failing tickets, pull requests, or agent sessions, each linked to the original system for full context.

7. How does the vendor match developer identities across systems?

Ask how the vendor handles two edge cases: one developer using completely different names or emails across systems, and two people who share the same name. Identity resolution should handle both without manual cleanup on your end. Ask whether you can override a match the vendor gets wrong.

8. How does the vendor link Cursor sessions to commits and pull requests?

Cursor's API data can contain structured links to commits, but developers sometimes commit by hand after an agent did the work. Ask how the vendor connects those sessions to the right commits, pull requests, and tickets when no structured link exists. minware handles this relationship recovery, the harder half of entity resolution, with time-based linking in its patented hypercube data model. Without that recovery, AI-assisted work that was committed manually can end up with no link to the session that produced it.

9. How long does historical backfill take?

Ask for a realistic backfill timeline for a year of history, and whether partial reports appear mid-backfill. Treat backfill as a one-time setup step and plan it into your evaluation timeline.

10. How does the vendor normalize data from different tools?

Ask how the vendor's data normalization reconciles tools that describe the same concept in different formats, such as ticket statuses in Jira and Linear. Without a shared model, the same metric means something different in each report. A shared model is also what lets AI usage from Cursor and Claude Code appear in the same report. Ask too whether the vendor flags records it couldn't normalize.

Outcome correlation: Correlating Cursor spend with delivery outcomes

The six questions below test whether the vendor can connect Cursor token spend to the delivery outcomes your board will ask about.

11. Does the vendor regress delivery outcomes against token spend?

Ask for the specific method behind the vendor's AI ROI claims. The answer to listen for is a linear regression of each delivery outcome against token spend, with the same team's trend over time as the supporting check. Adoption and activity metrics measure whether Cursor is running, while delivery outcomes measure whether it is working. A vendor that answers with pull requests merged is measuring code output, which says nothing about whether that output delivered value.

12. Which outcomes does the vendor regress against spend?

The right answer is a separate regression for each of the three Lean AI Framework outcomes: roadmap value delivery, on-time delivery rate, and overhead cost. Bug rate and cycle time follow as supporting signals. minware's AI impact reporting includes these regressions, with the slope and the R-squared value shown on each chart. The slope is the change in the outcome per unit change in spend, and R-squared is the share of the outcome's variation that spend explains. Ask the vendor to name the confounders too: engineers reach for AI on work where it is likely to help, and early adopters are often already the strongest performers.

13. Does the vendor report Cursor spend in dollars?

Token spend means dollar cost, since a token count alone can't show whether higher spend is buying more delivery. Ask whether the vendor reports Cursor spend in dollars and how it handles plan allotments or negotiated discounts. On flat-rate, tiered, or discounted plans, reported spend can differ from the amount actually invoiced. Ask whether the vendor can scale reported costs to your total bill, so the regression reflects your average discount.

14. Does the vendor need a pre-rollout baseline?

Ask whether the vendor requires a clean pre-AI baseline before it can measure ROI. The continuous token spend regression doesn't need one. What matters is connecting Cursor and your other AI tools as early as possible. For tools that report through OpenTelemetry, such as Claude Code and Codex, detailed per-prompt usage is captured from the point it's configured.

15. Does the vendor track same-team trends over time?

Ask whether the vendor tracks each outcome metric's trend over time for the same team as AI maturity increases, which controls for differences between teams. AI-vs-non-AI cohort comparison doesn't work as a check, because non-AI groups are disappearing and the ones that remain are prone to selection bias. An engineer who skips AI is often working where it is unlikely to help, such as a legacy repository with poor test coverage.

16. Does the vendor pair delivery metrics with guardrails?

Ask whether the vendor pairs delivery metrics with guardrails such as bug rate and PR lead time. Ask too whether it can explain why a delivery trend is moving. Quality and efficiency metrics have two jobs. As a guardrail, cycle time should hold steady or fall as token spend rises, which shows the spend isn't creating a bottleneck further down the pipeline. As a diagnostic, when roadmap value delivery or on-time delivery rate stays flat despite rising spend, ticket cycle time and bug rate show where the constraint is.

Pricing: Checking pricing and contract terms

The four questions below surface pricing access, seat definitions, billing terms, and renewal costs before you enter a procurement cycle.

17. Is pricing public, and is there a self-serve trial?

Gated pricing limits cost modeling before the first vendor call, while listed pricing lets you check budget fit early. minware lists Professional at $25/contributor/month with no seat minimum, up to 100 contributors, billed monthly or annually. Enterprise is $45/contributor/month with a 50-seat minimum, no contributor cap, and annual billing. The trial runs 14 days, with no credit card required. LinearB lists Essentials at $29 per user per month and Enterprise at $59 per user per month, both billed annually, as of this writing. Jellyfish's only fixed price is a 12-month AWS Marketplace bundle covering platform access and 15 engineering seats, with other configurations quoted by the vendor.

18. What is the minimum seat commitment?

Ask about minimum seat counts, how the vendor defines a contributor, and whether inactive contributors are billed. A definition based on recent activity, with control over which active contributors count, keeps the bill tied to the people actually doing the work.

19. Is monthly billing available?

Ask whether monthly billing is available or whether an annual commitment is required. Monthly billing lets you validate the platform against real reporting cycles before committing budget for a year. Ask what notice period applies if you decide not to renew.

20. What costs can appear at renewal?

Ask about price changes at renewal, data retention limits, and fees for additional integrations or custom metrics. A per-contributor price that looks low can grow if custom metrics or longer history are sold as add-ons. Ask for the renewal terms in writing before you sign.

Integrations: Confirming tool compatibility

The three questions below confirm whether the vendor supports your existing stack and how much setup your team owns.

21. Which tools does the vendor support natively?

Ask which version control systems (such as GitHub, GitLab, Bitbucket, or Azure DevOps), project management systems (such as Jira, Linear, Azure Boards, or GitHub Issues), CI/CD pipelines, and AI coding tools the vendor supports natively. Ask what setup each one requires. Cursor returns per-request token usage with a dollar cost through its admin API, while GitHub Copilot reports adoption and usage through its usage metrics API. Claude Code sends cost data through OpenTelemetry, with separate analytics APIs for historical usage. Codex reports per-turn token usage through OpenTelemetry. AI commit and pull request authorship can also be read from version control for any tool, with no AI integration at all.

22. Which Cursor API does the vendor read for your plan?

The API a vendor reads determines which Cursor data it can pull, so its answer should depend on your plan. Ask which API the vendor reads for your plan and which admin role creates the key. minware's Cursor integration reads the Admin API, which is available on Cursor Teams and Cursor Enterprise plans. The Analytics API plus the AI Code Tracking API are Cursor Enterprise only, with minware's integration for them on minware's Enterprise plan.

23. How long from contract to first useful report?

Ask for a realistic timeline from signature to first useful report, and who does the setup work. The answer tells you how much of the rollout your own engineers will own. Ask whether customer success handles configuration and custom reporting. Also ask how long a new integration takes to build.

Security: Assessing security and data residency

The three questions below cover data residency, SOC 2 scope, and regulated health data.

24. Where is data stored, and can access stay in your environment?

Ask where data is stored, whether custom data residency is available, and whether single-tenant hosting is an option. For teams that need extra access control, ask whether an on-premise ingest agent is available. If one is offered, ask whether it changes what data is collected or only who holds the credentials. An agent that runs with your own credentials keeps API keys and system access inside your environment, even when the data collected stays the same.

25. Which Trust Services Criteria does the SOC 2 report cover?

Ask for the SOC 2 Type 2 report and which criteria it covers, then ask whether the vendor is ISO 27001 certified. SOC 2 reports are built on the Trust Services Criteria (TSC) set by the American Institute of Certified Public Accountants (AICPA). The five categories are Security, Availability, Processing Integrity, Confidentiality, and Privacy. Security is the one category included in every SOC 2 report, so check which of the other four a report adds. minware's SOC 2 Type 2 report covers the security criterion only, and minware isn't ISO 27001 certified.

26. Can the vendor sign a BAA?

If your org handles protected health information (PHI), ask whether the vendor can sign a BAA and on which plans. Under the Health Insurance Portability and Accountability Act (HIPAA), a BAA is the written assurance a covered entity needs before it discloses PHI to a vendor that handles it on the entity's behalf.

Choosing a vendor that can defend its numbers

A vendor that can show its formulas, trace each metric to source records, and regress delivery outcomes against Cursor token spend gives you numbers that hold up in an executive review. Running these 26 questions in the demo shows you which vendors meet that bar before you commit budget.

Start a 14-day free trial at minware.com, no credit card required, and connect your first data source to test these questions against your own data.

FAQs

Do you need OpenTelemetry to track Cursor?

No. minware's Cursor integration reads Cursor's APIs with an admin key to track adoption, usage, and cost. The Admin API covers Cursor Teams and Cursor Enterprise plans. Cursor Enterprise also unlocks the Analytics API plus the AI Code Tracking API, an integration that requires minware's Enterprise plan. OpenTelemetry matters for tools like Claude Code and Codex, which report detailed per-prompt usage through it.

How do I know if a vendor can actually prove AI ROI?

Ask whether the vendor regresses roadmap value delivery, on-time delivery rate, and overhead cost against token spend. minware shows the slope and R-squared value on each regression chart.

Should I build Cursor analytics internally instead?

Building your own pipeline starts with a large upfront project, typically multiple weeks of engineering time, before the first report. Maintenance continues after launch. A late-2025 Fivetran survey of 500 data leaders found data engineers spend 53% of their time on pipeline maintenance, with legacy and DIY pipelines breaking 30% to 47% more often than fully managed ones. A vendor takes on that maintenance. The tradeoff is configuration: org-specific questions need a new or adjusted metric or report, which minware's customer success team typically handles within about a day.

What if the vendor won't let me trial before a sales call?

Push for access to the product on your own data before a longer evaluation, since a scripted demo only shows the data the vendor chose. minware's 14-day trial is self-serve, with no credit card required.

Key terms glossary

Data lineage: The traceable path from a reported metric back to the original source data.

Data normalization: Standardizing messy vendor data formats into one consistent model so concepts can be connected across tools.

Entity resolution: The combined capability of recovering relationships between commits, tickets, pull requests, and AI agent sessions that carry no structured connection, plus matching a developer's identity across systems with different usernames or emails.

Identity resolution: Matching a developer's identity across systems with different usernames or emails. The easier half of entity resolution. The harder half, linking sessions to commits and tickets with no structured connection, is relationship recovery.

Relationship recovery: Linking commits, tickets, pull requests, and AI agent sessions that carry no structured connection between them. The harder half of entity resolution, and the subject of minware's patent.

Hypercube data model: minware's patented data model that links every entity from every data source, so any metric can be broken down by any dimension.

Linear regression: A statistical model charting the best-fit line between an outcome metric and an input variable. It is the primary AI ROI methodology, fitting a line through token spend and a delivery outcome metric across teams and time periods. Each regression fits one metric pair, so correlating spend against multiple outcomes requires a separate regression for each.

Regression slope: The change in an outcome metric per unit change in the input variable, such as story points completed per dollar of token spend.

R-squared (R²): In a regression chart, the share of the outcome metric's variation explained by the input variable, on a 0-to-1 scale.

Token spend: The dollar cost of AI tool usage, as reported by AI tool APIs or OpenTelemetry. On flat-rate, tiered, or discounted plans, reported spend may differ from the amount actually invoiced.

Roadmap value delivery: The expected value of completed work items. Configurable to any project management field where a team sets explicit value estimates during roadmap planning, or to a custom spreadsheet upload. If no explicit value field is configured, defaults to story points completed on tickets that are not bugs and have a parent epic/project ticket, with 1 point assigned per ticket if no estimate is set.

On-time delivery rate: The estimated roadmap item duration (time from when a roadmap item is marked in progress until the original due date set at that time) divided by the actual roadmap item duration (time from in progress until it is marked done), capped at 100% per roadmap item. By default, minware treats each epic as a roadmap item, and this can be configured to use higher-level tickets instead.

Overhead cost: Personnel cost and AI token spend that goes toward work not counted in roadmap value delivery, i.e. cost not traceable to any ticket, or traceable to a ticket excluded from roadmap value delivery (by default, a bug ticket or a ticket with no parent epic/project ticket).

Cursor: An AI coding tool that suggests code for a developer to accept, or works as a coding agent that executes multi-step tasks autonomously in agent mode.

Business associate agreement (BAA): A contract required under HIPAA when a covered entity shares protected health information with a vendor that creates, receives, maintains, or transmits that data on its behalf.

Protected health information (PHI): Individual health information that is created, received, maintained, or transmitted by covered entities or business associates, protected under federal HIPAA privacy and security standards.

Trust Services Criteria (TSC): The AICPA framework for evaluating service organization controls, covering Security, Availability, Processing Integrity, Confidentiality, and Privacy. Security is the only category included in every SOC 2 report.