Compare repositories using the same as-of date and finding-status rules, then show open verified risk beside what was actually assessed and what matters in each service. Do not rank repositories by raw alert count, findings per thousand lines, or a single scanner score. A repo with zero approved findings and little review is an unknown, not the safest repo. The useful output is a two-part decision: which known issue gets engineering capacity now, and which poorly understood scope needs more assessment.
For an AppSec lead, the answer changes when a repository has different production exposure, business impact, review depth or ownership. A public customer API and an internal build tool should not be compared as if they had identical attacker paths. Use a shared decision format, while preserving those differences.
Fix the comparison population first
Start with a dated inventory of repositories that contribute to the same product or portfolio. Name the deployed services or packages each repo produces, the owner, and the current production revision if known. A dormant archive, a shared library and a directly exposed API can all contain serious issues, but they need different exposure questions. If ownership or deployment is unknown, keep that as a visible gap.
Use one finding population: open, independently verified and approved findings at the as-of time. Keep unresolved scanner candidates, rejected claims and verified-fixed history in separate columns. Mark accepted risks as a subset of the unresolved verified findings, with an expiry and response owner; acceptance alone does not remove the technical defect from the open total. A rejected candidate must not inflate the risk count; an unresolved candidate should not silently disappear. A merged fix is still open for this comparison until the chosen positive reassessment or deployment closure rule has been met.
Next, fix the assessment denominator. For each repo, state the selected sensitive workflows or control obligations, how many were actually reviewed, and why the rest were not. File inventory, indexing and actual review should be distinguished. If the inventory itself is incomplete, write unknown total rather than a coverage percentage. OWASP SAMM's defect-management guidance recommends viewing defects together with verification activities, component coverage, accepted risk and exposure windows. That is a better basis for comparison than a finding count alone.
Keep four dimensions in separate columns
The comparison card should carry:
| Dimension | Evidence to record | What it must not imply |
|---|---|---|
| Harm if the path succeeds | Affected customer action, data or business operation | A proven loss amount or breach count |
| Practical attacker path | Required access, deployed entry, existing controls and uncertainty | That a live exploit was performed |
| Open finding state | Approved count by harmful outcome; age; owner; fix status | That candidates are confirmed vulnerabilities |
| Assessment coverage | Reviewed/selected workflows, inventory status, exclusions and as-of revision | That a reviewed workflow is vulnerability-free |
Technical severity can help describe a finding, but it is not a repository grade. FIRST's CVSS v4.0 specification separates Base, Threat and Environmental metrics. A general vulnerability score is therefore only one input when the local service, attacker access and controls differ. For published CVEs, FIRST's EPSS estimates exploitation in the wild over the next 30 days, while CISA's Known Exploited Vulnerabilities data identifies vulnerabilities with evidence of exploitation. Neither establishes that a particular repository's deployed version is affected or that a bespoke authorization issue has been exploited.
Worked comparison: three repositories, two different actions
Illustrative case; repository names, findings and numbers are invented. No exploit was performed. A B2B company reviews three production repositories as of 18 October. The denominator in the table is the number of selected sensitive workflows for each repo, not lines of code. The workflows were selected before reviewing findings. They differ in complexity, so 8/10 is a scope disclosure rather than a directly comparable security score.
| Repository and service | Approved open findings | Selected workflows reviewed | Exposure and key uncertainty | Decision |
|---|---|---|---|---|
checkout-api — customer-facing purchase service |
2 high-impact; 1 lower-impact | 8/10 | One approved issue has a plausible ordinary-customer path; two workflows remain unreviewed | Fund immediate fix and complete the two reviews |
admin-console — staff administration |
1 high-impact | 5/5 | A verified staff role reaches the path; how many people hold that role is unknown | Count role holders and schedule a bounded fix |
billing-jobs — background billing work |
0 approved; 1 unresolved candidate | 2/9 | Seven workflows unreviewed; worker trigger and deployment revision uncertain | Fund targeted assessment before calling it low risk |
There are three repositories in the comparison inventory and each has a named service. checkout-api has the clearest supported near-term harmful path, so its approved issue gets engineering time now. billing-jobs has the greatest knowledge gap; it gets assessment time, not a congratulatory “zero issues” badge. admin-console is not automatically safe because its selected workflows were all reviewed; the number of people with the affected staff role changes practical exposure. A security lead can explain these decisions without fabricating a combined risk score.
The table's denominators are local to each repo: 8/10, 5/5, and 2/9 selected workflows. They do not mean 80%, 100% and 22% of vulnerabilities were found. One broad workflow can contain more code and more controls than several narrow ones. If a team cannot define comparable workflows, replace the fraction with a list of examined and unexamined named surfaces. If billing-jobs inventory is later found incomplete, withdraw the 2/9 ratio and show an unknown denominator until the missing scope is enumerated.
Make the work fit an actual weekly limit
Now give the team a fictional budget of five person-days this week, shared across the engineering and AppSec work listed below. The estimate includes review, so it cannot also be booked as free work by another team. Assume the named people have the required skills; if they do not, split the capacity by role before allocating it.
| Work package | Estimate | What completion would establish |
|---|---|---|
| Correct the leading checkout boundary finding and reassess | 3 days | Whether the approved path is prevented in the new revision; production mapping still recorded |
| Review billing trigger, deployment and one affected workflow | 2 days | Whether the candidate path is supported or defeated; not completion of all seven gaps |
| Count admin role holders and review active grants | 1 day | Current population with the required privilege; not a repair of the approved defect |
| Correct the admin finding and reassess | 3 days | Whether the approved admin path is prevented |
There is no allocation that completes all four packages. Two feasible plans illustrate the choice:
- Plan A: checkout correction 3 days + billing assessment 2 days = 5. The admin defect and role-population uncertainty remain unresolved this week.
- Plan B: checkout correction 3 days + admin role review 1 day = 4, with 1 day explicitly reserved for checkout reassessment overruns or urgent new evidence. Billing's candidate and seven workflow gaps remain unresolved; the admin fix also waits.
The security lead initially selects B because checkout is the clearest customer-boundary path and admin role population can be settled quickly. The reserve is real capacity, not an additional committed task. The record names billing's deferred uncertainty and the next checkpoint; a zero finding count is not the reason for deferral.
Before work starts, platform evidence arrives: billing's worker can be triggered through a customer-facing upload route, and the current artifact matches the candidate's assessed revision. That establishes a broader entry path. It does not establish that downstream authorization fails or that the candidate is a vulnerability.
The new allocation is Plan A. Checkout retains its three days. The admin role review and reserve give way to the two-day billing assessment because its missing control evidence now bears on a reachable customer workflow. Management records that the approved admin issue remains open and that the identity population check moves to the next available slot. No finding is invented to justify investigating the unknown.
If the billing assessment instead requires three days after the first day of discovery, the team must replan. It cannot claim to fit six days into a five-day week by calling the last day “follow-up.” A useful checkpoint reports remaining effort and a decision, not only percent complete.
Count a shared defect once and its exposure everywhere
Suppose one of checkout's two high-impact findings is AUTH-7 in a shared authorization helper also used by admin. Admin's one high-impact row is the same finding, not a separate defect. The corrected portfolio view has three distinct approved findings, rather than the sum of repository counts, 3 + 1 = 4:
| Stable finding | Affected repositories | Portfolio count | Work ownership |
|---|---|---|---|
| AUTH-7 | checkout-api, admin-console | 1 | Shared-helper owner; both integrations and deployments need closure evidence |
| PAY-4 | checkout-api | 1 | Checkout owner |
| LOG-2 | checkout-api | 1 | Checkout owner |
Keep the earlier capacity exercise's three-day checkout package only if it includes the necessary shared-helper and admin integration work. If it covered checkout alone, update the estimate; a shared source fix does not make the second deployment free. The admin role-population question may remain useful even when the code correction is shared.
This is why a repository league table cannot allocate the work by itself. Ownership, affected deployments, shared causes and unreviewed paths determine whether one action reduces several exposures or merely changes one row.
Revisit the allocation when shared ownership changes the cost
AUTH-7 exposes a practical comparison problem: the repository with the most visible findings may not own the change that resolves them. Ask the shared-helper maintainer to estimate the implementation and ask each consumer owner to estimate integration and release evidence. Keep those estimates distinct until the work plan shows who has the required capacity. A single portfolio finding can still require several coordinated deployments.
If one consumer cannot adopt the shared correction promptly, record the residual path and any temporary restriction for that service. Do not close the portfolio issue because the package version exists, and do not keep a corrected consumer's work invisible. The comparison should show one defect with uneven exposure states, which is more useful than either duplicating it or collapsing every service to the least informative label.
At the next weekly cutoff, update the work allocation from new evidence. A rejected billing candidate may free investigation capacity without counting as a fix. A newly supported billing path may justify changing the repair order. A complete admin role inventory may narrow the starting population while leaving the defect unchanged. Each movement needs its own reason.
This lets the AppSec lead explain priority changes without treating the original ranking as a permanent score. The fair comparison is the current decision from the available evidence, the actual effort and the remaining knowledge gaps. It is not a competition for the smallest number beside a repository name.
Publish the limitation with the ranking
Give leadership a two-line summary: “Fix checkout-api first because the approved path could cross a customer boundary; assess billing-jobs next because only two of nine selected workflows have been reviewed.” Follow it with the as-of date, each denominator, the highest-impact unverified assumption and the next checkpoint. If a team asks for a league table, show the four separate dimensions and explain which decision they support.
For the next portfolio review, choose three repositories and fill the comparison table with actual approved finding status and named workflow denominators. Use the coverage-and-uncertainty template to repair weak denominators, then the prioritization guide to turn the highest supported issue into a funded work item.