What to do when two security reviewers disagree on a finding

When one reviewer supports a vulnerability claim and another rejects it, pause publication and compare the evidence at the same scope. Ask each reviewer to write the exact actor, route, revision, control and effect they assessed, plus the strongest evidence against their own conclusion. Separate disagreements about facts from disagreements about interpretation, then run one bounded check that could change the verdict. A majority vote or the highest proposed severity cannot substitute for that check.

The answer changes when the reviewers were looking at different routes, environments, revisions or definitions of the claim. Both may have accurately read their own slice of the system. If the decisive fact is unavailable, keep the claim unresolved with an owner and a deadline. NIST SP 800-30 Rev. 1 treats incomplete knowledge of controls and system dependencies as a source of uncertainty in risk assessment. Record that uncertainty rather than averaging two incompatible conclusions into a false precision.

Freeze the claim before debating it

The AppSec lead first pins the disputed statement. “Invitation privilege escalation” is too broad. A reviewable claim is: an invited member can submit requested_role=owner to the legacy acceptance route, and the server grants owner access instead of the role stored in the invitation. This identifies an actor, controlled input, entry, expected control and unauthorized effect.

Then ask both reviewers to use one comparison sheet:

Do this before debating a score. The OWASP Authorization Cheat Sheet distinguishes authentication from permission to perform an action and recommends validating permission on every request. That is the control question; “the user was logged in” does not resolve whether an invitee may choose the owner role.

Illustrative case: two invitation routes

In this fictional service, reviewer A inspects accept_legacy() and sees requested_role from the request passed to membership creation. Reviewer B inspects accept_v2() and sees the server load a stored invitation role, ignoring the client's requested_role. Both analyses concern the same repository revision, but they have silently evaluated different routes. No real service or exploit was tested for this example.

The adjudicator reconstructs the route and deployment map. The illustrative evidence reads:

The right outcome is not “A wins” or “B loses.” Reviewer B identified a real control in v2; reviewer A identified a different path through v1. Narrow the candidate to v1, preserve v2 as counterevidence against any claim that all invitation acceptance is vulnerable, and request independent verification of the complete v1 path. If the gateway manifest were unavailable, keep production applicability unresolved. If v1 were disabled, reject the claim for that deployment while considering removal of the unsafe legacy code as hardening.

When the route is the same and reviewers still disagree

Scope alignment solves the first disagreement. It does not settle a harder one: both reviewers now inspect v1, but B says the shared membership writer clamps the role. Give the adjudicator the exact competing implementations rather than two confidence scores. These are alternative pseudocode implementations, not both present in the same system:

Writer A:
  invitation = load_invitation(token)
  require_invited_subject(actor, invitation)
  insert_membership(actor, invitation.account, requested_role)

Writer B:
  invitation = load_invitation(token)
  require_invited_subject(actor, invitation)
  insert_membership(actor, invitation.account, invitation.role)

A token bound to the right invitee and account can be perfectly valid in Writer A. That control still does not restrict the role chosen by the invitee. In Writer B, the request's role never reaches the membership write. “The writer validates the invitation” is too vague to distinguish them.

The discriminating artifact is the deployed writer body and its call binding. The supplied exercise packet contains Writer A, with a database constraint allowing the role names viewer, editor and owner. B treated that constraint as authorization. It only rejects unrecognized names: owner remains a valid value, and the constraint does not compare it with the invitation's stored viewer role.

Adjudication record: retain the v1 candidate for independent approval. The active v1 route permits the invited actor; the writer takes their requested role; the role-name constraint does not enforce the invitation's grant. Reject the broader claim that v2 is affected because v2 supplies the stored role. Preserve B's constraint reference and explain precisely why it did not defeat v1. No owner account was created in this exercise.

If the packet had contained Writer B, reject the v1 role-selection claim for that implementation. If no build-to-writer binding were available, leave production applicability unresolved. The adjudicator should be willing to reach all three outcomes; otherwise the exercise is just a vote with more paperwork.

A concise real handoff should contain the competing claims, the one artifact that separated them, the selected conclusion and the evidence that would reverse it. OWASP's reporting guidance supports making scope and technical reasoning visible. A second review adds value when it challenges the relationship between evidence and effect, rather than repeating the original explanation.

Decide who acts while the claim is unresolved

Unresolved does not mean “ignore.” If the potential effect is serious and the preventive change is cheap, the engineering owner may choose to remove v1 or enforce the stored invitation role while review continues. Describe that as a precaution, not proof of the original vulnerability. The AppSec lead owns the security disposition; the platform owner supplies route and policy evidence; the developer owns the code change. Keep business priority, technical difficulty and evidence status distinct.

If the team cannot obtain a necessary artifact, state the consequence: the claim stays in investigation, its conditional impact is described plainly, and any temporary restriction has an owner and expiry. Do not send an unverified issue to a developer as a confirmed vulnerability solely to make a queue appear complete. Conversely, do not erase the source concern because one reviewer found a safe alternate route.

Resolve the decision that remains after the fact dispute

Even after both reviewers agree on Writer A, they may disagree about the response. One wants to disable the legacy route immediately; the other prefers a small stored-role patch. This is no longer the same technical disagreement. Write the agreed facts first, then compare the operational choices against those facts. Otherwise a debate about release disruption may be mistaken for evidence that the unauthorized role-selection path does not exist.

For the invitation exercise, a response comparison could be:

No option becomes effective because a reviewer prefers it. The engineering owner needs to establish its enforcement and its effect on legitimate use. If a small patch is already clear, that work can proceed while platform confirms deployment details. Keep the exposure verdict limited to what those details support.

A useful adjudication note separates responsibilities: the security reviewer explains the unauthorized grant; the invitation owner chooses and implements the repair; the release owner confirms which versions receive acceptance requests. One person may hold several of these roles in a small team. The important distinction is between questions, not mandatory headcount.

Prevent the disagreement from restarting at every update

Preserve the point of agreement and the exact unresolved question. A record saying “reviewers disagree about invitations” invites the next person to repeat the whole discussion. A record saying “both agree Writer A uses the requested role; current deployment binding is unavailable” directs the next person to a release artifact. Once that artifact arrives, update the record instead of keeping the stale disputed label.

Ask each reviewer to state what would reverse their present conclusion. For reviewer A, a mandatory stored-role assignment before every write could defeat the v1 claim. For reviewer B, proof that the database constraint only validates role names removes the claimed countercontrol. If neither can name changing evidence, the problem may be an unstated definition or risk preference, and the adjudicator should surface it explicitly.

Disagreement also deserves a proportionate time budget. A missing writer body may justify a short evidence request; a complex external policy may require a specialist. Agree on a checkpoint and choose a safe interim action where warranted. The checkpoint does not automatically convert the claim to confirmed or false. It identifies when the owner must decide whether to continue investigation, obtain different expertise or maintain a precaution.

At closure, send both reviewers the bounded conclusion and the reason, retaining useful counterevidence. Reviewer B's safe v2 route still matters: it gives the repair owner a relevant comparison and limits the public claim. Keeping that contribution visible encourages future reviewers to raise disconfirming evidence rather than treating review as a contest to defend the first verdict. The record should make it easy for a later reviewer to identify precisely what changed, not which person prevailed.

Next step: Use the comparison sheet on the oldest unresolved disagreement and request one discriminating artifact. Link the decision to the uncertainty record; if a developer supplies counterevidence to one path, preserve it using the dispute workflow.