Code Quality Metrics That Matter and 5 That Don't
TL;DR: Four code quality metrics predict delivery problems: bug rate, defect escape rate, post-merge churn rate, and change failure rate. Each one measures a failure mode in the development process. Five others mislead when used as targets or as proof of value: lines of code, commit frequency, pull requests merged, individual contributor rankings, and code coverage percentage. Pull requests merged still earns a place as a guardrail, because it shows when a team improves quality by shipping less. Every primary metric needs a guardrail metric, or teams game the number. minware tracks bug rate, post-merge churn rate, and change failure rate beside the delivery outcomes they should move. Those outcomes are roadmap value delivery and on-time delivery rate.
Engineering leaders face a specific problem: the metrics easiest to track are the most likely to mislead. Many dashboards track how much code a team writes, while the metrics that predict delivery problems track how much of that code fails or gets removed. This article names the four code quality metrics that predict delivery problems and the five that invite gaming. It explains the calculation behind each, then shows how to pair every primary metric with a guardrail so the numbers hold up in an executive review.
The danger of tracking vanity metrics over outcomes
Vanity metrics are activity and output measures presented as value. They show that work happened without connecting it to delivery outcomes. When a board asks whether engineering is performing, a dashboard of commit frequency and lines of code answers a question nobody asked.
Output metrics such as pull requests and commits can be measured relatively easily from engineering systems, but they incentivize the wrong behaviors. Lines of code rewards verbose code, while commit frequency rewards small, meaningless commits. Numbers go up while delivery outcomes stay flat.
The shift from activity counts to outcomes
Roadmap value delivery and on-time delivery rate are the value outcomes boards care about. Efficiency and quality signals support them by explaining why delivery is moving. DORA's research program uses five software delivery metrics: deployment frequency, lead time for changes, change failure rate, failed deployment recovery time, and deployment rework rate. DORA covers throughput and instability with no value delivery metric of its own, so its metrics work best as supporting signals.
Separately from DORA's use of the term, minware groups pull requests merged, commits, and lines of code under its own Throughput category, which measures code output volume. On their own, these metrics prove nothing about delivery impact. When a VP of Engineering reports that pull requests merged went up, the number confirms code output rose while leaving open whether that code delivered value or introduced defects.
The test for an actionable code quality metric
A code quality metric is actionable when it meets three criteria. It connects to a delivery outcome, it has a visible calculation a skeptical stakeholder can inspect, and it pairs with a guardrail metric that catches gaming. A metric without a guardrail invites gaming, because when a measure becomes a target, it ceases to be a good measure.
The four code quality metrics below predict delivery problems:
| Metric | What it measures | Why it matters | Suggested guardrail pair |
|---|---|---|---|
| Bug rate | Bugs created divided by pull requests merged | Covers every defect, including those that never cause a failed deployment | PR lead time and pull requests merged |
| Defect escape rate | Share of all bugs found that reach production | Reflects quality failures that reach users | PR lead time |
| Post-merge churn rate | Merged code removed within 14 days of merge | Catches code-quality problems that lead to heavy refactoring or deletion, which bug rate and change failure rate miss | Ticket cycle time |
| Change failure rate | By default, highest-priority bugs divided by pull requests merged | Flags severe failures that need immediate attention | Pull requests merged, or deployment frequency in the deployments-based configuration |
The five metrics below mislead when used as targets:
| Metric | What it measures | Why it misleads | Use instead or pair with |
|---|---|---|---|
| Lines of code | Raw volume of code written | Rewards verbose code and punishes refactoring | None for quality, planning use only |
| Commit frequency | Number of commits made | Easy to inflate by splitting changes, measures code output only | Roadmap value delivery |
| Pull requests merged | Code output volume per period | Easy to inflate by splitting work into smaller PRs, so it misleads as a target. It stays useful as a throughput guardrail | Bug rate and PR lead time |
| Individual contributor rankings | Raw output compared across engineers | Reflects task assignment more than skill, invites gaming | Team-level metrics |
| Code coverage percentage | Lines executed by tests | Shows which code tests run, says little about whether tests catch defects | Bug rate and defect escape rate |
The 4 code quality metrics that predict delivery problems
Each of the four flags a different way delivery breaks down, from defects in shipped code to failed changes in production. Each has an inspectable calculation and pairs with a guardrail that catches gaming.
Tracking bug rate against code shipped
Bug rate measures quality problems relative to code shipped. minware's default calculation, set out in its Lean AI Framework, divides bugs created by pull requests merged into a main branch. Some teams switch the denominator to story points completed, so changes in pull request volume don't distort the ratio. Because it counts every bug created, bug rate also catches quality problems that never cause a failed deployment.
The efficiency guardrail is PR lead time, which minware measures from a pull request's first commit to deployment. If bug rate falls but PR lead time rises, the team may be over-reviewing to avoid defects. The throughput guardrail is pull requests merged. If bug rate falls alongside pull requests merged, the team may be shipping less code to avoid defects.
Catching defects that escape to production
Defect escape rate measures the share of all bugs found that reach production. Definitions vary across the industry, but a common calculation divides bugs found in production by total bugs found across testing and production. Organizations can customize minware's bug rate to separate pre-release bugs from production bugs, which makes it effectively a defect escape rate. The efficiency guardrail for defect escape rate is PR lead time. If the escape rate falls but PR lead time rises, the team may be holding changes back for extra testing.
Spotting unstable code through post-merge churn
Post-merge churn rate measures how much merged code is removed within 14 days of merge: lines removed in that window divided by total lines merged. Bug rate and change failure rate catch visible defects after release, but they miss code-quality problems that lead to heavy refactoring or deletion. Post-merge churn rate catches those escaped code-quality problems, including code that gets reverted or rewritten soon after merge.
Post-merge churn rate works best as a diagnostic, since some churn is healthy and judging it takes the team's context. Repeated rewrites of a ticket's code often mean the first implementation missed something, such as an unclear requirement, a gap in review, or a design flaw. The efficiency guardrail is ticket cycle time, from when a ticket is first marked in progress to when it is completed. If post-merge churn rate falls but ticket cycle time rises, the team may be over-planning to avoid rework, which delays delivery.
Measuring severe change failures
Change failure rate measures how often a change causes a severe failure. minware's default divides highest-priority bugs by pull requests merged to a main branch. Teams that want DORA's own definition can reconfigure it to count change failures over production deployments, taken from Git tags or CI/CD pipeline runs. Teams can also set which priorities, labels, or fields indicate a change failure in their project management system. Bug rate is a superset of change failure rate, but a high change failure rate still signals a more severe problem that needs immediate attention, so track both.
Failed deployment recovery time measures how long it takes to recover from a deployment that requires immediate intervention. DORA's research finds that top performers do well across all five metrics, so track failed deployment recovery time alongside change failure rate. Under the default configuration, the throughput guardrail is pull requests merged. Under the deployments-based configuration, it is deployment frequency. If change failure rate falls along with deployment frequency, the team may be deploying less often to avoid failures, which increases batch size.
5 deceptive metrics that hide real delivery issues
The five vanity metrics share a common failure: they measure activity, code output, or test execution, none of which predicts quality problems on its own. Each is easy to game, and none connects to delivery outcomes. This pattern costs credibility in a board review, when the engineering leader reports output is up while the roadmap is behind.
Why lines of code rewards the wrong behavior
Lines of code assumes that more code equals more progress. It measures size, which says little about complexity, quality, or value. It rewards verbose code and punishes refactoring, which often reduces line count while improving the codebase. In LeadDev's 2024 survey of nearly 1,000 engineering leaders, 70% of respondents actively avoid measuring lines of code. Nicole Forsgren, lead author of a 2021 research paper on developer productivity, calls the belief that productivity comes down to activity like lines of code one of the most common myths.
Effort estimation models such as the Constructive Cost Model (COCOMO) size projects in source lines of code, so the metric keeps a planning use. A quality dashboard has no place for it.
Why commit frequency and pull requests merged invite gaming
Commit frequency and pull requests merged both count code output, which says little about progress or quality. When either becomes a target, it gets gamed: developers can split changes into many small commits, while pull requests merged incentivizes trivial PRs and discourages complex work. That adds review overhead without adding delivery. Both metrics also miss the benefits of collaboration such as peer programming or brainstorming.
Pull requests merged misleads as a target or as evidence of value, but it still has a job as a guardrail. Beside a quality metric, it shows whether a team improved quality by shipping less code, which is why this article pairs it with bug rate and change failure rate. When pull requests merged is the number being tracked, it needs its own guardrails: bug rate as the quality counterpart and PR lead time as the efficiency counterpart. Bug rate is the natural fit, because its default denominator is pull requests merged. For commit frequency, the outcome metric to watch instead is roadmap value delivery, which by default counts story points completed on tickets that are not bugs and have a parent epic.
Why developer rankings mislead
Individual contributor rankings are biased because engineers work on different types of tasks. An engineer maintaining a legacy codebase with poor test coverage will post different numbers than an engineer building a greenfield feature with modern tooling. Raw output comparisons end up reflecting assignment difficulty more than skill. Researchers from GitHub, Microsoft, and the University of Victoria warn that activity metrics should never be used in isolation to reward or penalize developers. Team-level metrics with drill-down to specific items avoid this problem.
Why code coverage fails as a quality metric
Code coverage percentage shows which lines tests execute, which says little about whether those tests catch defects. Martin Fowler's writing on test coverage makes the point directly: coverage is useful for finding untested parts of a codebase, but of little use as a numeric statement of test quality. If coverage becomes a target, high numbers are too easy to reach with low-quality tests, at the extreme with tests that contain no assertions. The signals to watch for quality are bug rate and defect escape rate.
The link between code quality metrics and delivery outcomes
Code quality metrics often sit in isolation. A team tracks bug rate in one report, cycle time in another, and deployment frequency in a third, with no view connecting them. Connect quality metrics to outcomes by tracking them beside roadmap value delivery and on-time delivery rate for the same team over time, then drilling into the items behind any change.
minware's patented hypercube data model makes this possible. A hypercube is a multidimensional data model that extends a data cube across many dimensions, so any metric can be broken down by any dimension. A VP of Engineering can view bug rate beside roadmap value delivery by team and time period without custom queries. The model normalizes data from version control, project management, CI/CD, AI coding tools, and calendars into one consistent layer. That covers version control systems such as GitHub, GitLab, Bitbucket, or Azure DevOps, plus project management systems such as Jira, Linear, Azure Boards, or GitHub Issues.
Tracing a delivery slip to quality metrics
When on-time delivery rate falls, ticket cycle time shows where work is slowing, and bug rate shows whether defect work is competing with planned work for the same engineers. The diagnostic question is whether the delay comes from slow review, unclear requirements, or quality problems that force rework.
minware's drill-down chain makes this explicit. A flagged outcome metric leads to its supporting metrics, then a slice by any dimension, then a list of the individual failing tickets, pull requests, or agent sessions linked back to the source system. A manager sees on-time delivery rate falling, checks ticket cycle time, bug rate, and post-merge churn rate, then finds the specific tickets driving the delay.
A metrics structure that holds up in review
Structuring metrics takes three steps: focus on early warning metrics, add quality checks to output metrics, then define your calculation logic and audit it every quarter. The last step repeats, so the structure keeps pace with the team.
Focus on early warning metrics
Early warning metrics are leading indicators that flag delivery problems before they become missed commitments. Three useful ones are ticket work in progress (WIP), PR lead time, and pre-merge churn rate. High ticket WIP points to overloaded engineers, waits on slow automated processes, or interruptions from bugs or unplanned work. Rising PR lead time can point to review bottlenecks. A high pre-merge churn rate points to planning problems, such as unclear requirements.
Bug rate and change failure rate register problems once defects exist, early enough to predict delivery delays before a commitment slips. Ticket WIP and pre-merge churn rate are diagnostics that sit beneath the outcome metrics, giving front-line managers specific items to act on.
Add quality checks to output metrics
Every throughput or value metric needs quality and efficiency counterparts. A practical way to pick the guardrail is to look for the ratio metric that already uses the primary metric as its denominator.
Pull requests merged pairs with bug rate, because the default bug rate calculation divides bugs created by pull requests merged. Deployment frequency pairs with change failure rate under the deployments-based configuration, which divides change failures by production deployments. The table below adds story points completed and shows what each pair catches.
| Primary metric | Suggested guardrail metrics | What the pair catches |
|---|---|---|
| Pull requests merged | Bug rate, PR lead time | Teams splitting work into smaller PRs without improving delivery |
| Deployment frequency | Change failure rate in its deployments-based configuration | Teams deploying more often by skipping testing |
| Story points completed | Bug rate, ticket cycle time | Teams cutting corners on quality to inflate story points completed |
The manager's judgment is the final safeguard, with metrics there to inform that judgment. A metric that looks good but feels wrong deserves a closer look, and the drill-down to failing items provides the evidence.
Define your calculation logic, then audit it quarterly
Org-specific definitions matter because every organization has its own definition of done, estimate units, and capitalization criteria. Forcing a generic definition onto an org-specific process produces misleading numbers.
minware exposes every metric's calculation as a visible, editable formula written in minQL, its patented formula language. Most customization happens by selecting existing metrics and breakdowns in a report configuration, so customers rarely write minQL themselves. A customer success agent can typically adjust a metric definition in a single call, with about a 24-hour turnaround and no engineering escalation.
Definitions drift as processes, tools, and teams change, so a metric that was accurate six months ago may no longer reflect how the team works. Each quarter, review metric definitions against the current process, then check for gaming by comparing primary metrics against their guardrails. Update the guardrails when new gaming patterns emerge or the team adopts new tools.
Broken data as an improvement list
A common objection is "our data is too messy for metrics." It assumes clean data has to come first, but the relationship works both ways. minware works with data as it exists today, and its metrics help clean that data up by surfacing exactly where the process is breaking down. Each data gap becomes an improvement item.
Use hygiene metrics to reveal process gaps
Hygiene metrics measure process adherence, such as PRs Traceable to Ticket and Tickets Completed with Estimate. Weak ticket linking makes story point velocity, roadmap value delivery, and sprint completion quietly undercount work, including AI-assisted work. PR lead time, PR review time, bug rate, and pull requests merged come from version control data, so they stay accurate even when linking is weak.
minware's Best Practices report tracks PRs Traceable to Ticket, Tickets Completed with Estimate, and related hygiene metrics. A manager can see exactly which items are missing links, then fix them.
Audit your existing data sources
Auditing existing data sources means checking the quality of version control, project management, and CI/CD data. Confirm that all relevant repositories and projects are connected. Check that historical backfill has completed, then confirm team configuration reflects the current org structure. Finally, verify three things the data model handles: identities resolved across systems, relationships recovered between records, and vendor data normalized into one model.
Entity resolution covers two problems. Identity resolution matches a developer's identity across systems with different usernames or emails. Relationship recovery links commits, tickets, pull requests, and AI agent sessions that carry no structured connection between them, such as connecting an agent session to the ticket it belongs to when that session produced no commits. Relationship recovery is the harder half and the subject of minware's patent. The hypercube data model handles it by modeling what commit and ticket each person was working on at any given moment.
This does not make explicit linking redundant. Time-based linking works without ticket IDs. Explicit linking makes those associations more accurate, because developers often have several tickets in progress or write code before marking a ticket in progress.
Use data gaps to drive improvement
Data gaps become a prioritized to-do list when every target-based metric shows the specific failing items behind it. A manager sees that pull requests are not linked to tickets, drills down to those pull requests, then assigns the fix.
Competing platforms typically present metrics as averages broken down by team. That works for board slides but gives front-line managers little to act on. minware's failing items list gives managers a direct lever: fix these specific pull requests, and the metric improves.
minware's approach to code quality metrics
Engineering data is scattered across version control, project management, CI/CD, and AI coding tools, with no normalized view connecting them. Producing a single delivery report means pulling from each system, reconciling different definitions in a spreadsheet, and hoping the data is still current. minware normalizes data from all of these sources into a single layer, so pre-built DORA, AI impact, and cycle time reports reflect live pipeline activity.
Every metric's calculation is visible and editable in the UI. When a stakeholder asks how a number is derived, the answer is in the formula. The hypercube data model lets a VP of Engineering compare any metric against any dimension, so org-specific questions get answered without waiting on a vendor roadmap. Many analyses run as soon as the initial backfill completes. Org-specific questions usually need a new or adjusted metric or report, which customer success sets up.
Build vs. buy for code quality analytics
Engineering teams often weigh an internal build against a vendor platform. The decision comes down to the initial pipeline project, ongoing maintenance cost, and metric governance:
| Cost factor | Internal build | minware |
|---|---|---|
| Initial setup | High: a multi-week project to build the data pipeline before the first report | Connect data sources. Pre-built reports work once the initial backfill completes, while org-specific metrics need configuration from customer success |
| API maintenance | Engineer owns updates when vendor APIs change | Covered by minware's maintenance |
| Data normalization | Custom extract, transform, load (ETL) mapping for each vendor's data format, rewritten when formats change | One canonical data model per source, maintained by minware |
| Entity resolution | Custom logic to link commits, tickets, pull requests, and agent sessions, plus identity matching across systems | Patented hypercube data model recovers relationships and resolves identities across sources |
| Metric definition questions | Engineer answers "how is this calculated" indefinitely | Customer success handles methodology questions |
| Reporting overhead | Manual report assembly each cycle | Pre-built reports run on demand |
| Customization flexibility | Unlimited, but each change takes engineering time | Most changes by selecting pre-built metrics and breakdowns, with customer success handling new definitions |
The recurring cost teams underestimate most is that someone has to keep answering "how exactly is this calculated" every time a stakeholder asks, long after launch. With a vendor, that's a support conversation. With an internal build, it's whoever built the pipeline, indefinitely.
The case for pairing every metric with a guardrail
Code quality metrics earn a place in an executive review when they predict delivery problems and can be defended line by line. Bug rate, defect escape rate, post-merge churn rate, and change failure rate meet that bar, provided each sits beside a guardrail that catches gaming. Lines of code, commit frequency, pull requests merged, rankings, and coverage can't carry a quality claim on their own, though pull requests merged still works as a guardrail. Tracking each quality metric beside roadmap value delivery and on-time delivery rate gives the next board conversation a defensible answer.
Start a 14-day free trial at minware.com, no credit card required, then connect your first data source. You'll see how minware calculates bug rate, post-merge churn rate, and change failure rate from your own engineering data.
FAQs
Which code quality metrics actually matter?
Four code quality metrics predict delivery problems: bug rate, defect escape rate, post-merge churn rate, and change failure rate. Each measures a failure mode in the development process, and each pairs with a guardrail metric to catch gaming.
What is the difference between a vanity metric and an actionable metric?
A vanity metric measures activity or code output, such as lines of code or commit frequency, and invites gaming. An actionable metric connects to a delivery outcome, has a visible calculation, and pairs with a guardrail metric that catches gaming.
How do you prevent teams from gaming code quality metrics?
Pair every primary metric with a guardrail metric that catches gaming. Pull requests merged pairs with bug rate and PR lead time. Deployment frequency pairs with change failure rate in its deployments-based configuration. Story points completed pairs with bug rate and ticket cycle time. The manager's judgment is the final safeguard.
How long does it take to see trends in code quality KPIs?
Historical backfill on first connection can take hours, depending on repository size. minware backfills one year of data on Professional and unlimited history on minware's Enterprise plan, so trends start from historical data. Each metric's trend over time for the same team then shows whether quality moves with delivery.
Should you track individual developer metrics or only team-level metrics?
Team-level metrics with drill-down to specific items avoid the assignment-difficulty and gaming problems of individual rankings. Engineers work on different types of tasks, so comparing raw output reflects assignment difficulty more than skill. A manager can find where an engineer needs support by drilling into failing items, with no ranking of individuals against each other.
Key terms glossary
Bug rate: A quality metric measuring the number of bugs created divided by the number of code changes, by default the number of pull requests merged into a main branch. Some customers configure the denominator to story points completed instead, to reduce the risk of PR volume changes distorting the metric.
Defect escape rate: The share of all bugs found that reach production, calculated as bugs found in production divided by total bugs found across testing and production. In minware, a bug rate customized to separate pre-release bugs from production bugs works as a defect escape rate.
Post-merge churn rate: For each pull request merged into a main branch, the number of merged code lines removed within 14 days divided by the total lines merged, recorded at the end of the 14-day window. Weighted by line count, so larger PRs count for more.
Pre-merge churn rate: The total lines added that don't make it to the final merge across all commits in a pull request, divided by the total lines including both churned and merged lines.
Change failure rate: By default, minware defines change failure rate as the number of highest-priority bugs divided by the number of pull requests merged to a main branch. Customers may reconfigure the denominator to production deployments (from Git tags or CI/CD pipeline runs) to align with DORA's own definition: the number of change failures (deployments requiring immediate intervention, such as a rollback) divided by the number of deployments.
Lines of code: The total number of lines of code originally added in commits.
Commit frequency: The number of original, non-rebase commits made over a period, recorded at the time of authorship.
Pull requests merged: The total number of pull requests merged, to or from any branch.
Code coverage percentage: The share of code lines executed when a test suite runs.
Roadmap value delivery: The expected value of completed work items. Configurable to any project management field where a team sets explicit value estimates during roadmap planning, or to a custom spreadsheet upload. If no explicit value field is configured, defaults to story points completed on tickets that are not bugs and have a parent epic/project ticket, with 1 point assigned per ticket if no estimate is set.
On-time delivery rate: The estimated roadmap item duration (time from when a roadmap item is marked in progress until the original due date set at that time) divided by the actual roadmap item duration (time from in progress until it is marked done), capped at 100% per roadmap item.
PR lead time: The time from a pull request's first commit to when it was deployed. Deployment defaults to when the code merges into a main branch, and can be configured to use Git tags or CI/CD deployment pipeline runs.
Ticket cycle time: The time from when a ticket was first marked in progress to when it was completed.