A merged patch is a proposed remedy, not proof that the original harmful behavior is gone. To verify a fix, bind the original finding and the proposed change to immutable revisions, restate the actor-to-effect path, then show positive evidence that the new behavior prevents that path. Check relevant alternate routes and changed shared controls. Record one of three outcomes: fixed, still exists, or inconclusive. A clean scanner run alone is inconclusive if the rule, scope, deployment or code location changed. Source review can establish that a control enforces the required property under stated assumptions. A claim about the running application also needs deployment evidence and, where the property depends on runtime behavior, authorized validation.
This method works for a single-line correction and for a refactor, but the evidence differs. A direct change to a local authorization check may be easy to map. A deleted handler, moved function or modified shared permission service requires a wider trace before closure. The goal is to let a second reviewer reproduce the decision without trusting the patch author's description.
First write down what the old finding actually proved
Start from the recorded claim and its evidence, not just an alert title. For a consequential or disputed flaw, ask a reviewer other than the patch author to challenge the conclusion. The minimum record names the affected repository, immutable old revision, selected scope, actor, controlled input, entry point, relevant checks and harmful effect. Preserve the exact source ranges and any deployment assumption. If the old conclusion was provisional, the fix review cannot silently upgrade it into a confirmed vulnerability. Establish the old claim or keep the outcome qualified.
An authorization finding might say: “An ordinary user of account A can request an invoice belonging to account B through the PDF download route because the document lookup uses only the supplied invoice ID.” That is testable. “Missing authorization” is merely a category. OWASP's IDOR testing guidance focuses on object references controlled by the requester; its Authorization Cheat Sheet calls for checking permissions on every request. Neither source proves that a particular application has the flaw.
For a meaningful comparison, also retain the policy that should have held. In this example: a user may download an invoice only if the invoice belongs to the account under which the request was authorized. A patch can remove the flagged line while leaving that policy unenforced elsewhere.
Work an illustrative fix from old path to new evidence
Illustrative case, not a customer result or an executed exploit: Northstar Billing has GET /invoices/{id}/pdf. At revision OLD_SHA, an independently checked source path shows that the route authenticates the caller, reads a PDF by the supplied invoice ID, and returns it. The reviewed route and service have no invoice-to-account comparison. A separate deployment artifact indicates this route is in the intended service. The finding states a conditional cross-account disclosure risk; no production request was made to prove exploitation.
A developer proposes NEW_SHA. The change replaces the unscoped lookup with a repository method that takes both invoice_id and the server-derived current_account_id, and returns “not found” unless both match. The PDF route calls it. A background export route still calls the older lookup, however. This patch improves the original route, but the reviewer cannot close the broader finding until the export path is evaluated. The precise finding boundary matters: if the original finding covered only the PDF route, the PDF path may be fixed while a new or linked issue is opened for export. If it covered all invoice delivery, closure waits for both paths.
The review record can be as small as this:
| Item | Northstar example | Evidence to attach |
|---|---|---|
| Original claim | User in A could obtain B's invoice PDF through direct download if no later control intervened. | Reviewed finding version at OLD_SHA; route, service and deployment references. |
| Intended correction | Bind invoice lookup to the caller's authorized account. | Change description and policy owner approval. |
| New path | Direct download calls get_invoice_for_account(invoice_id, current_account_id). |
Exact source range at NEW_SHA, including account identity provenance and query behavior. |
| Countercheck | Export still calls get_invoice_by_id. |
Caller inventory and source range at NEW_SHA. |
| Authorized behavior | A user can still download an invoice owned by A. | Controlled validation with synthetic A data, if run. |
| Denied behavior | A request from A for B's ID returns no B data. | Controlled validation with synthetic A/B data, if run. |
| Decision | Direct route fixed in source; broader invoice-delivery claim remains open or inconclusive pending export review. | Named reviewer, scope and unresolved item. |
The two synthetic accounts are essential in a live check: a request for a nonexistent ID does not prove cross-account authorization works. The reviewer needs a real foreign object in an authorized test environment, a caller without permission to it, and confirmation that no protected content appears in the response. OWASP's authorization guidance also favors checking the rule at the server on each request, because a hidden button or client filter is not an enforcement point.
A six-step verification method
1. Bind both revisions and the finding version
Record the complete old and new commit IDs, repository identity, branch or release artifact, and finding version. Confirm that each name resolves to a commit. Git documents rev-parse --verify with the ^{commit} type suffix for this purpose; a branch name alone may move between review and deployment (Git revision verification). If the new commit is not the deployed artifact, label the result source fix reviewed, not production fixed.
Input: original finding and proposed commit. Owner: reviewer. Output: a revision binding that another reviewer can retrieve. Check: the original source citation belongs to the old revision and the proposed code belongs to the new one.
2. Reconstruct the old actor-to-effect path
Read the old entry, caller, control and effect. Write the policy as one sentence. Do not use an alert screenshot as a substitute for the reviewed path. The old path tells you what must stop working and what legitimate action must keep working. If the earlier investigation omitted a downstream check, investigate it before claiming a fix outcome; the original finding may itself need correction.
Input: old evidence and policy. Output: a short trace with explicit assumptions. Check: a colleague can point to the exact point where the alleged bypass occurred.
3. Map the old path onto the new revision
Inspect the change and follow functions across renames, copies, deletions and moved routes. git diff can compare two commits and detect renames, but a rename hint is not proof that security behavior stayed the same (Git diff documentation). Search for current callers of the changed service and for alternate delivery paths. If the handler was removed, trace how requests now reach the same asset, or verify that the capability itself is gone. A deleted file by itself is not positive security evidence.
Input: old path and changed-file list. Output: old-to-new path mapping, including unresolved edges. Decision: if you cannot locate the new implementation or determine whether it is deployed, mark the review inconclusive.
4. Inspect the control, not only the changed line
Ask where the decisive check happens, where its trusted inputs come from, and whether all routes to the protected action pass through it. For Northstar, the account ID must be authorized against current server-side membership. A URL selector is acceptable only after that check; blindly trusting it would merely move the problem. The repository query must bind invoice and account together before returning data. Check failure behavior: a denied request should not disclose the PDF, metadata or a usable download URL.
Input: mapped new code plus relevant callers and policies. Output: an explanation of why the original path is blocked or still open. Check: identify a specific source-supported counterexample that would falsify your conclusion.
5. Gather a positive observation when the claim needs runtime proof
Use an authorized test environment and synthetic data to compare allowed and denied cases. For Northstar: own invoice works; foreign invoice is denied; the response and any generated link contain no B data; the export route receives separate assessment. Save test setup, artifact revision and actual outcome. A passing check on a staging build does not establish that production runs the same commit or configuration. If a runtime check is unsafe, unavailable or outside authorization, state that limit and make a source-only decision with its assumptions.
Input: test accounts, known objects and deployed artifact identity. Output: observed allowed/denied results with timestamps. Check: the negative case uses an object that really exists and belongs to the other account. The general principle aligns with NIST's Secure Software Development Framework: identify vulnerabilities, remediate them and verify the result as part of a managed development process.
6. Decide and leave an auditable record
| Outcome | Required basis | Example |
|---|---|---|
| Fixed | Positive evidence shows the original unsafe behavior is prevented or the capability is removed, with reachable paths accounted for in the stated scope. | All relevant invoice delivery paths enforce the account-to-invoice rule at NEW_SHA; authorized validation supports the conclusion. |
| Still exists | A path to the harmful effect remains under the same prerequisites. | Export still returns an invoice from an unscoped lookup. |
| Inconclusive | The evidence cannot decide: missing deployment mapping, unknown control, incomplete source, failed validation, or an unexplained rename. | The PDF handler changed, but the source of its current_account_id is unavailable. |
Record who decided, when, against which revision and scope, plus the evidence and remaining uncertainty. “Not found by scanner” is an observation about a tool run, not a fourth outcome that closes the issue. The sibling guide on fixed versus no longer detected gives the full silence-check decision tree.
Compare the patch with a convincing near miss
Two patches can both add an account_id parameter while enforcing different properties. This pseudocode is constructed for inspection, not executed:
OLD: invoice = find_invoice(request.invoice_id)
PATCH A:
invoice = find_invoice_for_account(request.invoice_id, request.account_id)
PATCH B:
account = require_current_authorized_account(session)
invoice = find_invoice_for_account(request.invoice_id, account.id)
repository operation:
return invoice where id = invoice_id AND account_id = account_id_argument
return no result when the pair does not match
Patch A lets the caller choose both the invoice and the account predicate. For a caller authorized only for A who knows B's invoice and account IDs, the pair can still select B's record. The query is scoped, but its authority is not. Patch B binds the predicate to current server-verified authority; the repository must actually enforce both conditions before returning metadata, a PDF or a usable delivery link.
Use a concrete evidence sheet when an authorized owner performs validation. These are supplied example outcomes, not results of checks performed for this article:
| Synthetic fixture | Required result | Example recorded outcome at B's controlled artifact |
|---|---|---|
| A member requests existing A invoice | Legitimate PDF available | Own invoice returned. |
| A member requests known existing B invoice | No B content or usable B delivery link | No matching authorized invoice; denial before delivery. |
| A member supplies B as request account | Request selector cannot replace trusted authority | Lookup still uses A; B invoice denied. |
| A member requests nonexistent invoice | Safe absence behavior | No result, but this row alone proves no cross-account rule. |
The first three rows challenge the correction and its identity provenance. They do not establish all possible routes or production rollout. The export caller remains a separate source question. If F-17 originally covered only direct PDF download, close that scope when its required evidence is complete and open a linked export investigation. If F-17 explicitly covered both delivery paths, report partial repair. Do not broaden or shrink the original claim silently to make its status convenient.
A useful reviewer comment is: “Patch A adds a predicate whose value the requester controls. Please bind it to current authorized account context as in B, then show the actual query and denial-before-delivery path.” That identifies the property the patch must satisfy without prescribing a framework.
A closure note a reviewer can challenge
Finding: Cross-account invoice PDF access, finding version F-17 at
OLD_SHA.
Reassessed revision:NEW_SHA; staging artifactBUILD_SHA, deployment mapping supplied separately.
Policy: only the account that owns an invoice may retrieve its PDF.
Change: direct download now uses the server-derived account ID in the invoice lookup.
Positive source evidence: route and repository query atNEW_SHAbind invoice and account before data return.
Supplied exercise check: own-account download succeeds and foreign-account download is denied in the hypothetical controlled artifact; no actual validation was executed for this article.
Unresolved: export route still uses the older lookup; production artifact not yet confirmed.
Decision: direct-route fix supported; the broader finding remains open pending export and production review.
Owner and next check: billing service owner maps export callers; release owner supplies production artifact ID.
The note does not claim a production exploit or an all-path fix. It leaves enough detail for engineering and security to disagree productively. To use the method today, take one recently merged security patch and fill in the old path, new path and decision fields before moving its ticket to closed. If the handoff itself is thin, start with the security fix ticket template.