Guide 187 measurement guide

How to track mean time to remediate without distorting incentives

Track time to remediate with explicit start and closure evidence, distinguish active work from operational waiting and avoid incentives that hide unresolved defects.

Define a consistent start event, a closure event that requires positive fix evidence, and the exact population of findings. Then publish mean time to verified remediation alongside the number still open, their age, assessment coverage, and the mix of work. A falling mean can be bad news if only easy findings are being closed. Do not stop the clock because a ticket was reassigned, a patch merged, an alert disappeared, or a risk was accepted.

For an AppSec lead, the metric is useful when it reveals a bottleneck: slow validation, missing code ownership, difficult fixes, deployment delay, or slow reassessment. The answer changes if your purpose is different. “Time from first alert to review” and “time from approved finding to verified fix” are separate clocks; mixing them makes neither actionable.

Define the clock before looking at the number

Use one record per stable finding, with an event history rather than a mutable created_at and closed_at pair. A workable definition is:

Time to verified remediation = timestamp when a fix is positively reassessed on the relevant new revision or deployment, minus timestamp when the finding met the team’s stated evidence criteria and was accepted for action. Identify the assessor and whether the check was independent; a second reviewer is not part of every team’s clock definition.

This definition deliberately excludes unverified scanner candidates. Measure time to initial triage from alert arrival to a named reviewer taking ownership. Measure time to final disposition from arrival to a documented approval, rejection, or other terminal decision under the team's policy. An alert left for investigation has not reached final disposition: show its age and owner in the unresolved-candidate count. Otherwise, delaying approval could make remediation appear artificially fast. Keep the first-seen date visible even when it is outside the remediation clock.

Decide whether closure means source revision, production deployment, or both. A team can prove a code change fixes the reviewed path before the new release is live, but customer exposure may persist until deployment. Record verified in code and verified deployed as separate events if both matter. A missing repeat alert is not verification; the original actor-to-effect path and relevant controls must be reassessed. This aligns with NIST SSDF's emphasis on analyzing vulnerabilities and addressing their causes, rather than merely clearing a tool notification.

Write the metric definition next to the dashboard. Include time zone, whether elapsed days are calendar days, how you treat duplicate reports and reopened findings, and whether a source-supported issue awaiting deployment is still open for the chosen clock. Do not retroactively change that definition to improve a quarter.

Use a cohort that cannot hide open work

Select all findings approved in a defined intake period, including those still open at the report's as-of time. The denominator for the cohort is that full set. Calculate the arithmetic mean only for findings whose chosen closure event occurred; label its denominator as closed findings. Then display open count and ages from approval to the same as-of time. Open ages are not completed remediation times; mathematically they are right-censored observations. Do not insert zeroes or pretend they disappeared.

Also show:

Companion view Why it matters
Approved in cohort; verified closed; still open Makes the missing denominator visible
Age bands for open findings Shows whether slow work is accumulating
High-impact and lower-impact cohorts separately Prevents a mix shift from masquerading as faster remediation
Verified in code versus verified deployed Exposes release lag where production exposure matters
Reopened after supposed closure Tests whether the closure rule is reliable
Assessment activity and scope Guards against celebrating a quiet month when less was examined

OWASP SAMM's defect-management metrics explicitly include time to detect and resolve, exposure windows, reopened vulnerabilities, accepted risk, and verification coverage. Its guidance is a useful check on an isolated MTTR number. NIST's enterprise patch-management guide likewise treats identifying, prioritizing, installing and verifying corrective actions as an operational process; a tool event is only one step.

Worked measurement: a faster average with an aging risk

Illustrative case; all findings, dates and numbers are invented. An AppSec team freezes a cohort of four approved findings from 1–31 January. It computes status on 5 February 2026 at 12:00 UTC. All event dates below are in 2026 at 12:00 UTC; elapsed time is calendar time in 24-hour days, with no rounding or paused clock. The closure rule is positive reassessment of the original path on a pinned new revision.

Finding Approved Fix verified Elapsed time or open age on 5 February Current status
A: unsafe export permission 2 January 6 January 4 days Verified closed
B: missing object check 4 January 16 January 12 days Verified closed
C: unsafe token reuse 8 January 3 February 26 days Verified closed
D: cross-account record read 5 January — 31 days open Open; fix spans two services

The closed-finding mean is 14 calendar days: (4 + 12 + 26) / 3. The denominator is three verified closures, while the intake cohort denominator is four approved findings. One remains open at age 31 days. Reporting “MTTR = 14 days” without those denominators hides D. Do not compute a percentile from three closures and give it a false air of precision. Do not call the team safer merely because A and B were quick.

The AppSec lead's decision is to keep D visible as an open risk and agree a two-service fix and interim containment with the owners. The issue was approved after review of the relevant source and production policy; the age number does not reveal whether work is blocked by design, ownership, deployment or reassessment. The metric card for leadership could read:

January approved-finding cohort, as of 5 February: 4 approved; 3 verified closed; 1 open at 31 days. Mean for closed = 14 calendar days (n=3). One closure was verified after January. Code-to-deployment lag is not included because deployment dates were not consistently recorded. Next measurement change: capture deployment evidence; next risk action: agree finding D's containment and fix owner.

This card is useful because the number points to an action and states its blind spots. It is not a team performance grade.

Replay the events a simple closed-date export loses

Add the following events to the same four-finding cohort. These are synthetic records, not measured performance. The stable finding identity survives reassignment, duplicate alerts and reopening.

Finding First seen → approved Later event Reporting treatment
A 1 Jan → 2 Jan: 1 day Fix verified 6 Jan First remediation episode: 4 days
B 2 Jan → 4 Jan: 2 days Fix verified 16 Jan First episode: 12 days
C 1 Jan → 8 Jan: 7 days Fix verified 3 Feb; same unsafe behavior reintroduced and confirmed 9 Feb First episode: 26 days; new open exposure episode begins 9 Feb
D 2 Jan → 5 Jan: 3 days Duplicate alert D2 linked on 12 Jan; owner reassigned 20 Jan One finding; original approval clock continues

C's seven-day approval delay is invisible in its 26-day remediation interval. Its first-seen-to-first-verified-fix time is 33 days. Showing the one-, two-, seven- and three-day disposition delays beside remediation reveals the review bottleneck without pretending all candidates were valid on arrival. If additional unapproved candidates existed, their open ages would need a separate register too; this small example contains only these four.

For reopening, choose and publish a convention. This example preserves completed remediation episodes and reports a reopened exposure episode separately. C genuinely met the selected source-closure rule on 3 February, then a later change reintroduced the behavior on 9 February. If the first closure had instead been erroneous, correct that record; do not manufacture a successful episode from an invalid verification.

Under this convention, the two snapshots are:

Quantity 5 February, 12:00 UTC 12 February, 12:00 UTC
Original cohort 4 approved findings Same 4
Completed first remediation episodes A 4d, B 12d, C 26d Same three episodes
Mean of completed first episodes 14d, n=3 14d, n=3
Findings currently closed A, B, C: 3 A, B: 2
Finding still awaiting first closure D: 31d since approval D: 38d since approval
Reopened exposure None C: 3d since 9 February confirmation
First-seen age of unresolved D 34d 41d

C's reopened age is not three days of proven historical exploitation. It measures time since the recorded reintroduction confirmation; exact earlier deployment exposure would require deployment history. Likewise, this example's source-closure clock omits production rollout, so it cannot be presented as customer exposure duration.

A common dashboard filters to items currently closed and recalculates the mean. It would show (4 + 12) / 2 = 8 days on 12 February, down from 14. The display looks better precisely because the slow closure C left its population. At the same time, D aged another week and C became unresolved again. Preserve the episode mean and current-state counts together so this selection effect is visible.

Reassigning D on 20 January does not give it a new approval date. Merging D2 does not add another finding or a fresh clock. Accepting D's risk could alter its response and work queue, but would not produce a positive fix-verification event. These are concrete event rules; a general instruction to “avoid gaming” is not enough.

The leadership action at the second cutoff is specific: allocate work to D's two-service dependency and investigate how C's regression escaped the change controls. Reducing the displayed mean is not itself the action. For the finding and deployment state accounting behind those decisions, use the remediation progress guide.

Diagnose the waiting time before setting a target

For D, split the 38-day open interval into operational stages where the event history supports them: awaiting ownership, designing the two-service correction, implementation, release scheduling and reassessment. Do not invent stage durations when timestamps are absent. The purpose is to identify the next constraint, not to assign every elapsed day to a person.

If the correction is ready but no reviewer is available, adding a stricter engineering deadline will not shorten the remaining wait. If the two services lack a shared release plan, closing one team's ticket may still leave the unsafe path available. The metric should direct the lead to the real dependency and keep the original clock intact.

A target can then be tied to an action, such as staffing a review window or naming the second service's release owner. Keep the target separate from the observed time. Revising a target after new facts does not change how long D has already been open, and a justified exception does not become a completed remediation episode.

Make the next report reproducible

Ask the workflow owner to export stable finding IDs, first-seen time, initial triage time, terminal disposition time where applicable, approval time, closure evidence time, deployment time where applicable, reopen events, current status, and the as-of timestamp. Freeze the cohort before calculating. Exclude rejected candidates from the approved-finding denominator but report their completed disposition latency separately; keep investigations in an open-age view. Review a small sample of “closed” records to confirm each has the promised fix evidence; if the sample fails, correct the underlying records before publishing a polished metric.

Start by publishing one cohort card with both denominators and an open-age list. If the team cannot yet verify closure, publish that limitation and fix the event model first. Use the fix-verification guide to establish the closure evidence, then the coverage report template so fewer alerts are not mistaken for faster security work.