Measure monorepo security coverage on two separate axes: what source was discovered, acquired and actually examined, and which specific security requirements were evaluated for which components. Pin both axes to an immutable revision and a declared scope policy. A percentage of files indexed or a count of completed jobs is not a percentage of vulnerabilities found. The answer changes when enumeration is incomplete: if the full population of files is unknown, report the known slice and the failure reason, not a repository-wide fraction. OWASP’s ASVS assessment guidance expects a clear scope, method and result; its requirements are not satisfied by counting files.
A platform engineer can make this measurable without pretending every line has the same security value. The goal is a ledger that tells AppSec what work finished, tells service owners what to supply next, and lets a CTO see what a clean result actually means.
Decide what the denominator means
Begin with the immutable Git commit or release manifest. Enumerate tracked source and configuration, including submodules or repositories referenced by the build, and record which were unavailable. Do not use the number of files returned by a capped API response as “all files.” The inventory method and its failures are part of the coverage statement.
Keep these counts distinct:
| Count | Meaning | Common mistake |
|---|---|---|
| Discovered | Paths and relevant objects found in the pinned inventory. | Treating an incomplete tree response as the whole repo. |
| Eligible | Discovered material within the declared review policy, after recorded exclusions. | Shrinking the denominator silently when a parser fails. |
| Acquired | Eligible content actually retrieved and bound to the revision. | Counting filenames as source read. |
| Indexed | Acquired content made searchable or segmented. | Saying an index means human or model examination. |
| Examined | Content read for a named security question to a stated depth. | Treating one referenced line as review of an entire file. |
| Requirement obligation | A versioned requirement applied to a component or shared control with dependent paths. | Marking a requirement “done” for the whole monorepo after one route. |
The “eligible” population is a policy choice, so publish it with exclusions. Generated code, tests, migrations, IaC and documentation can all contain evidence about a particular claim; none should be discarded solely because of the folder name. A file may be examined for one question but not another. For precise work, record relevant line ranges or segments and the specific obligation they support. File counts are a coarse progress signal, not detection coverage.
Illustrative ASVS review: trace one authorization boundary
Illustrative case; repository, revision and all numbers are invented. Atlas Workspace has a monorepo at commit A71C9. It contains a public API, web client, job workers, shared authorization library, Terraform configuration, migrations and generated clients. The review question is whether a member of customer A can read or export a workspace owned by customer B. It maps to ASVS 5.0.0 v5.0.0-8.2.2, explicit permission for specific data items, and v5.0.0-8.4.1, cross-tenant controls. Those are defined in the official Authorization chapter.
The platform engineer generates an inventory and asks the API and worker owners to confirm deployment mapping. The example ledger is:
| Inventory stage | Count | What the count supports |
|---|---|---|
Discovered at commit A71C9 |
12,400 files | Complete selected Git tree, but external service source is separate. |
| Deliberately outside policy | 2,800 files | 1,800 generated clients, 400 binary assets and 600 unrelated documentation files, each with a recorded reason. |
| Eligible in this source review | 9,600 files | 12,400 - 2,800; includes tests, IaC and migrations when relevant. |
| Acquired | 9,100 files | 500 known eligible files could not be fetched after a provider error. A separate referenced submodule remains an unknown external scope. |
| Indexed | 8,700 files | 400 acquired files were not indexed because of tool limits; still available for manual review. |
| At least partly examined | 1,250 files | Read for named obligations; this does not mean every line in those files was reviewed. |
The arithmetic is useful: 9,100 of 9,600 eligible files were acquired, while 500 were not. But “1,250 of 9,600 files partly examined” is not a valid “13% secure” score. The files have unequal importance, and examination depth differs. A single shared authorization function may govern many routes; one unreviewed worker may leave the main question unresolved.
Recompute a complete twelve-obligation ledger
Atlas names this synthetic baseline AUTH-12 v1. Every row is a selected obligation at A71C9, not a claim to cover an entire ASVS level. Requirement IDs below are prefixed v5.0.0-; abbreviations save table width. Progress states are exclusive: reviewed means the declared source-review task finished; blocked means a decisive input prevented that task; not started means no task work began. Evidence and control conclusions are separate columns.
| ID | Requirement / component question | Progress | Evidence boundary / control conclusion |
|---|---|---|---|
| O1 | 8.2.2 workspace detail | Reviewed | Source and caller binding support named source path; deployment pending |
| O2 | 8.2.2 workspace list | Reviewed | Scoped query/projection support named source path |
| O3 | 8.2.2 export admission | Reviewed | API decision supported; no worker inheritance assumed |
| O4 | 8.2.2 export worker read | Blocked | Matching worker artifact unavailable; not assessed |
| O5 | 8.4.1 search tenant boundary | Reviewed | Supplied synthetic evidence establishes a failed control; finding approval recorded separately |
| O6 | 8.4.1 object delivery | Blocked | Storage/delivery policy unavailable; not assessed |
| O7 | 8.3.1 shared authorization operation | Reviewed | Mandatory service operation supported in inspected implementation |
| O8 | 8.2.3 export field projection | Reviewed | Source reviewed; external serializer unresolved; inconclusive |
| O9 | 8.2.1 admin export function | Reviewed | Source reviewed; current role mapping missing; inconclusive |
| O10 | 8.3.3 originating authority at worker | Blocked | Delegation contract unavailable; not assessed; explicit Level 3 addition |
| O11 | 8.2.2 legacy download route | Not started | No evidence examined; not assessed |
| O12 | 8.4.1 cache/account binding | Reviewed | Source reviewed; effective cache namespace unknown; inconclusive |
The totals are now reconstructible: 8 reviewed + 3 blocked + 1 not started = 12. Among the eight reviewed tasks, four have source-scope support (O1/O2/O3/O7), one has the example's failed control (O5), and three remain inconclusive (O8/O9/O12). This is not eight passed controls. Even the supported source paths may need deployment or other evidence for a broader application conclusion.
O7 is not multiplied by the number of consumers. Its implementation receipt can support O1–O3 only alongside each caller's binding evidence. O4 cannot borrow that support while its worker is unavailable. Likewise, a partially examined file can support one completed question without making every function in that file reviewed.
Add scope without rewriting the baseline
During the legacy-route review, Atlas discovers a mobile export entry point that creates the same job through a separate adapter. Add O13: v5.0.0-8.2.2 mobile export admission, not started, to AUTH-12 v2 (the ledger name retains its baseline identity). The current plan has 13 obligations; the original plan remains 12 in its history.
The source-task progress moves from 8/12 to 8/13 solely because the denominator expanded. No completed task was lost. Report “8 original tasks reviewed; one newly discovered obligation added; current plan 13.” Do not hide O13 in an exclusion or silently redefine the original baseline.
The next decision is to obtain the worker/delegation evidence for O4/O10 because it can affect the central export boundary and connect several incomplete obligations. Retrying unrelated source acquisition may raise the file count without answering that question. A leadership report can consume this ledger, but the ledger itself must keep all rows, definitions and changes visible.
Build the ledger in seven passes
1. Freeze identity and inputs. Record repository, immutable revision, inclusion policy, inventory tool settings, language support and review objective. If a deployment artifact cannot be mapped to that revision, mark production applicability unknown. Output: an assessment binding.
2. Enumerate completely or say why not. Traverse the selected tree, record submodule pointers and external repos, and classify every discovered object. A timeout, permission loss or provider truncation becomes an explicit incomplete inventory event. Output: a manifest plus error log, not a guessed total.
3. Record state transitions by reason. For each eligible unit, log acquisition, indexing and examination state. Group similar exclusions but retain enough detail to retrieve individual paths. If a parser rejects a 20 MB configuration file, preserve it as acquired but unindexed, then decide whether a manual bounded read is needed. Output: state totals that reconcile.
4. Map requirements to components. Choose the ASVS version and target level. Name obligations where a requirement must be checked on a route, worker, shared control or configuration. A shared library review supports several obligations only if the callers and runtime use are established. OWASP says applicability can differ by technology; for example, unused frontend or protocol sections may be irrelevant with a documented rationale. See ASVS applicability guidance.
5. Attach evidence and counterevidence. A requirement row needs a method, reviewer, version, evidence pointer and result. If a scanner alerts on an unfiltered query but middleware may restrict the caller, investigate both. A line citation proves the line exists, not that the whole exploit path works. Runtime evidence must be labeled separately from source evidence.
6. Compute only meaningful fractions. acquired / eligible is an acquisition fraction if inventory is complete and eligibility is explained. examined selected obligations / planned selected obligations is a progress fraction if the plan and additions are visible. Neither is “percent of vulnerabilities detected.” Report failed, blocked, not applicable and unstarted obligations separately.
7. Reconcile and publish the boundary. Have the relevant owners reconcile revision mapping and obligation outcomes. Keep reviewer identity and method on the receipt: human review, rule execution and model-generated analysis are different activities, even if each produces a record. A stored model response alone is not independent verification. The final statement names reviewed components, requirements, methods, exclusions and unknowns. Do not roll an incomplete external service into “100% monorepo coverage.”
Copy a minimal measurement record
Unit: Atlas Workspace monorepo, commit
A71C9; selected Git tree enumerated completely, external service source separately pending.Source ledger: 12,400 discovered; 2,800 reasoned exclusions; 9,600 eligible; 9,100 acquired; 8,700 indexed; 1,250 files at least partly examined for named questions. Illustrative numbers.
Requirement ledger: ASVS 5.0.0 authorization obligations for workspace read/export and cross-tenant control; AUTH-12 v1 has 12 planned, eight source tasks reviewed, three blocked, one unstarted; one reviewed obligation has a failed control in this invented case. Version 2 adds O13, making 13 current obligations without changing the eight completed tasks.
Method boundary: Source review at pinned commit; no claim of complete runtime validation or full application compliance.
Next evidence: Worker deployment revision, provider retrieval retry, submodule inventory and storage policy, each with an owner and date.
If a reviewer cannot reconstruct those counts from the manifest and work ledger, the statement is too strong. Start by reconciling discovered, excluded and eligible counts for one pinned revision. Then use the exclusions report template for the missing units and the buyer-verifiable coverage statement to turn the internal ledger into an honest external explanation.