Why measuring developer activity isn’t the same as measuring engineering performance.
Imagine you’re reviewing the monthly performance of two developers.
One merged 24 pull requests, closed 18 tickets, and consistently delivered ahead of schedule. The other merged 6 pull requests, closed 5 tickets, and spent much of their time reviewing code, helping teammates, and discussing architectural decisions.
Looking at a dashboard, the first developer would probably stand out. But what if the second helped prevent a production incident, resolved an issue that had been slowing down the entire team, and helped two junior developers become more independent?
It’s a hypothetical example, but it illustrates a common problem with engineering metrics: what’s easiest to measure isn’t always what matters most.
I’ve been thinking about this while working on ways to give clients better visibility into engineering performance. Pull requests, sprint metrics, and delivery trends all provide useful information, but deciding what they actually tell us is more complicated.
The risk isn’t necessarily collecting the wrong data. It’s drawing the wrong conclusions from perfectly valid measurements.
We measure what’s easy to see
Modern development tools generate enormous amounts of data. GitHub tracks commits, pull requests, and review activity. Jira records tickets and sprint completion. CI/CD pipelines provide deployment information, while monitoring tools help us understand incidents and reliability.
Turning this information into a dashboard is relatively straightforward. Interpreting it requires more care.
Consider pull request counts. They can reveal activity patterns and workflow bottlenecks, but a developer submitting 15 small PRs hasn’t necessarily contributed more than someone submitting 3 substantial ones.
Story points have similar limitations. They’re useful for planning within a team, but comparing developers or teams that estimate work differently can produce misleading results.
In 2021, researchers from Microsoft, GitHub, and the University of Victoria introduced the SPACE framework, which argues that developer productivity cannot be captured through a single metric.
SPACE considers five dimensions: satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow.
Activity is only one part of the picture, although it tends to receive the most attention because it’s easy to collect and compare.
What happens when metrics become targets?
Goodhart’s Law is commonly summarized as:
“When a measure becomes a target, it ceases to be a good measure.”
The idea is that once people are evaluated according to a number, their behavior may change in ways that make the number less useful.
Suppose a company starts evaluating developers based on tickets completed per sprint. Developers might begin favoring smaller, predictable tasks over complex work that takes longer to finish.
The same can happen with pull requests. If PR volume becomes a performance target, splitting changes into smaller PRs could improve someone’s numbers without necessarily improving delivery.
Of course, smaller PRs can be genuinely beneficial. They make code easier to review and can reduce the risks associated with large changes. The question is whether the practice improves the workflow or simply improves a metric.
Nobody needs to deliberately manipulate the system for this to happen. People naturally adjust their behavior based on how success is measured.
A dashboard could show improving numbers even while the team’s engineering practices remain unchanged.
Valuable engineering work doesn’t always show up in dashboards
Think about an experienced developer spending an afternoon helping a teammate debug a difficult problem.
The teammate eventually resolves the issue and merges the PR. Their activity appears in the dashboard, while the person who helped may have little to show for those hours.
Similar situations arise when engineers review designs, improve documentation, investigate recurring incidents, or help prevent poor technical decisions.
The SPACE research recognizes that many valuable engineering activities, including mentoring and architectural guidance, are difficult to quantify.
This becomes particularly relevant as developers move into senior and leadership roles. We expect them to do more than complete their own tickets. They should help others make better decisions, reduce unnecessary complexity, and improve how the team works.
Evaluating them primarily through individual output ignores a significant part of their responsibilities.
There’s also the question of attribution. A successful feature might involve a developer implementing the solution, another engineer providing architectural guidance, QA identifying an edge case, and someone from the business clarifying a requirement.
Giving all the credit to whoever merged the final PR wouldn’t reflect how the work actually happened.
This doesn’t make individual performance impossible to evaluate. It simply means activity data needs to be interpreted alongside responsibilities, context, and impact.
Measuring delivery instead of individual activity
The engineering industry has developed frameworks that offer a broader view of software delivery.
One of the best-known examples is DORA (DevOps Research and Assessment).
DORA’s current model includes five software delivery performance metrics covering throughput and instability. These examine how quickly changes reach production, deployment frequency, recovery from failed deployments, and how often changes require intervention or rework.
These measurements help us understand how effectively a team delivers software, rather than simply how much activity it generates.
Imagine a team that consistently merges dozens of PRs but takes several weeks to deploy changes to production.
The PR count might suggest healthy development activity, but the delivery metrics would reveal that work is accumulating somewhere between development and release.
Perhaps reviews are taking too long, testing is unreliable, or deployment requires excessive manual coordination. The dashboard should help identify where to investigate.
DORA also emphasizes measuring performance at the application or service level and tracking improvement over time. Comparing unrelated teams or systems can be misleading because their technical environments and constraints may differ considerably.
It’s also worth recognizing what DORA doesn’t measure. Delivery metrics alone cannot determine whether a feature solved the right customer problem or whether an individual developer performed well.
I see DORA and SPACE as complementary approaches. DORA helps us understand software delivery, while SPACE provides a broader perspective on productivity, collaboration, and the working environment.
What I’d want from an engineering dashboard
There’s a legitimate business need for visibility. Managers need to understand whether projects are progressing, clients want confidence that their investment is producing results, and teams need to identify problems before they become expensive.
If I were designing an engineering performance dashboard, I’d focus on three questions.
1. Is work moving through the system effectively?
Metrics such as lead time, PR review time, blocked work, and delivery trends can help identify friction.
For example, if changes regularly wait several days for review, that’s a reason to investigate the review process. It doesn’t automatically mean individual reviewers are underperforming.
The objective should be understanding where work slows down and why.
2. Are we delivering reliable, valuable software?
Delivery speed means little if changes regularly cause incidents or require extensive rework.
DORA’s stability measurements, defect trends, and production reliability can help provide context. Ideally, we’d also understand whether delivered features are being used and whether they’re solving the intended business problems.
Those business outcomes aren’t always attributable to engineering alone, but they’re relevant when evaluating delivery effectiveness.
3. Is the team becoming more effective over time?
This is probably the hardest question to answer through metrics.
I’d want to know whether engineers are becoming less dependent on a few senior people, whether recurring issues are being resolved, and whether collaboration is helping the team work more effectively.
Some of this information might come from surveys and operational data. Much of it would require conversations with the people doing the work.
I’d also avoid combining everything into a single productivity score. Faster delivery accompanied by more production incidents shouldn’t automatically result in a positive overall rating.
Different measurements provide different perspectives, and combining them can sometimes hide the problems we’re trying to identify.
Start with a problem, not a KPI
A useful engineering dashboard doesn’t need to track everything. It should help answer specific questions about how the team works.
Suppose releases have become slower. Rather than immediately examining individual developer activity, I’d start by looking at how changes move from development to production.
Are pull requests waiting too long for review? Is testing creating delays? Do deployments depend on manual approvals or coordination between teams?
Once the bottleneck is identified, the team can discuss potential improvements and track whether those changes produce the expected results.
This is similar to the improvement cycle recommended by DORA: establish a baseline, identify constraints, make an improvement, and measure the outcome.
It also gives engineering metrics a practical purpose beyond producing monthly reports filled with green, yellow, and red indicators.
Not every available metric needs a permanent place on a dashboard. If a measurement doesn’t help explain a problem, guide a decision, or track whether an improvement actually worked, perhaps it doesn’t belong there.
