How to organize source evidence from a million-line repository

For a very large repository, keep two linked records: a coverage ledger showing what source was discovered, acquired, indexed, examined, and left out; and a claim record showing the exact evidence and counterevidence for each security conclusion. Organize both by application component and security question, while preserving immutable repository revision, path, hash, and line range. A searchable index helps reviewers find code; it does not mean every indexed line was assessed.

The approach changes when your file inventory is incomplete or a critical control is outside the repository. In that case, make the gap visible and plan a continuation. Do not call a capped scan “full” because it returned quickly or found nothing. The GitHub Trees API, for example, explicitly marks truncated recursive listings and directs callers to fetch subtrees to continue. This is an inventory constraint before anyone begins evaluating code.

Illustrative case: one million lines, one urgent decision

Illustrative case, with invented counts and names: Meridian's monorepo contains roughly 1.2 million eligible source lines across eight deployable services, shared libraries, infrastructure configuration, migration scripts, and generated clients. The AppSec lead has one week to decide whether the customer-report download workflow has a credible cross-account read risk. The repository also has other valuable components, but that single decision is the immediate assessment scope.

The team can inventory the repository in bounded batches. It can index paths, symbols, and relationships to navigate. It can read the report route, shared permission check, queue worker, and storage adapter at pinned commit M42. It cannot responsibly say that all 1.2 million lines were reviewed. The report should make the selected path and deferred scopes legible to the CTO, while preserving precise technical evidence for the developers.

The AppSec lead gives each layer a different job:

This structure keeps navigation data from being mistaken for a security conclusion.

1. Pin and enumerate without hiding a cap

Record repository identity, immutable commit, scope policy, listing method, and any provider cursor or truncation indicator. Git's ls-tree reference describes the tree objects, paths, modes, and object IDs available in a selected revision. If the repository is read through a provider API, record the API's limits as well. A recursive response marked truncated, a timeout, or a subtree that was never visited means the manifest is incomplete.

Use a row per file or bounded shard in the internal manifest. The minimum fields are revision, path, object ID or content hash, mode, size, retrieval state, eligibility class, and reason if excluded. A file may be discovered yet unavailable to read; a generated file may be acquired yet explicitly excluded from this question; a symlink or gitlink may point to content outside the authorized snapshot. These distinctions let the team make a reproducible claim without publishing private source.

Do not solve size by taking a random sample and silently treating it as coverage. Sampling can serve a stated exploratory purpose, but its denominator and selection rule must be visible. For a security question about customer reports, include likely entry points, authorization controls, storage reads, job consumers, and release configuration. If the component map suggests a shared permission helper, that helper is part of the question even if it lives far from the report directory.

2. Build a component map with confidence labels

A component is not merely a directory. Use manifests, route registrations, deployment records, imports, queue subscriptions, and owner input to identify deployable units and shared code. Keep relationships as separate entries with provenance. A static import is strong evidence that code can be included in a build; it is weaker evidence that a particular branch executed in production. A README diagram is a search hint until checked against source and deployment.

For Meridian, the map starts with reports-api → account-auth → export-jobs → blob-store. The AppSec lead labels the first two edges observed in pinned source, the worker edge observed in queue configuration but deployment unknown, and the storage policy external artifact requested. An inferred link never becomes “proved reachable” just because it appears in a graph. If a reviewer discovers an alternate route, add it as a new edge with its own status rather than overwriting the original path.

Keep the map small enough to support the current decision. A million-line codebase does not need a million-line diagram. It needs a way to find the next material control and say when that search is incomplete.

3. Keep one claim record per harmful action

Give each candidate a stable internal ID and pin it to a source snapshot. The record should separate the claim from evidence, counterevidence, and assumptions. For example:

E-0143 — Illustrative investigation: An ordinary Meridian customer might obtain another account's report. Actor/input: signed-in account A user supplies report ID. Entry: report download route at M42. Control: shared permission helper found, but its deployed package mapping is unresolved. Effect: storage adapter can return report bytes by ID in source. Counterevidence sought: object-owner check in helper or storage policy. Missing: active worker version and storage policy. Status: investigation; no confirmed cross-account read and no executed exploit.

The technical attachment links exact path and range, object hash, the version of each relevant external control, and the reviewer who checked it. If a tool exports SARIF, the SARIF 2.1.0 standard provides structured locations and code flows. A SARIF path is a transport format for evidence, not a substitute for checking the real caller, control, effect, and applicability.

Keep source excerpts in an authorized review environment. For broader reporting, a source-free summary, location and hash can identify the evidence without copying private code into tickets or decks. Access to the technical attachment may differ from access to the executive decision summary.

4. Make the coverage ledger answer hard questions

The ledger needs units and reasons, not just a progress bar. In this illustrative case, the inventory has 1.2 million eligible lines after a declared scope policy. Assume an owner-supplied inventory at M42 independently counts 1.2 million eligible lines, of which 1.17 million were acquired, 900,000 indexed for navigation, and 14,000 examined for this question. The 30,000-line acquisition gap is measurable here because that inventory records line counts for the missing regular files. An unavailable submodule with no content or trusted count would instead add an unknown scope, not an invented LOC total. The 270,000-line indexing gap remains visible regardless of search results. These numbers are invented to show the ledger, not a performance benchmark.

The AppSec lead should check that categories do not double count generated copies or submodule contents, that every denominator refers to the same revision, and that the “examined” number reflects actual read/reasoned review rather than text merely passed through an index. Counts alone do not measure review quality, but inconsistent counts reveal a misleading scope claim.

If the team maps work to the OWASP Application Security Verification Standard, record the version, selected level, applicable component and outcome per requirement. A requirement reviewed for one route does not silently pass for every service. OWASP itself recommends versioned requirement IDs because identifiers can change. “ASVS checked” without version, scope and evidence is not useful to the next reviewer.

5. Hand the claim to someone who did not collect it

A useful evidence system survives a second reader and a new revision. Meridian sends reviewer B a source-free index for claim E-0143 with restricted links to three attachments: L1, route/caller at M42; L2, helper at package H7; and L3, storage adapter at M42. The index records exact object identities and ranges, the proposed edges, access requirements and which conclusion each item supports.

Reviewer B does not approve the packet. L2's attachment has expired, and the diagram asserts that the route invokes the ownership check without a caller reference. The helper's existence is not evidence of invocation. The handoff returns “incomplete evidence: cannot reproduce route → ownership decision; restore L2 and supply caller edge.” This is a valuable result, not bureaucratic friction. A graph with unsupported edges can make an unreviewed assumption look settled.

The owner restores L2 under the approved access policy and supplies L4, the exact call site. L4 shows that the route invokes a membership check, not the object-owner helper shown in the diagram. The claim record is amended; the old graph is retained as superseded rather than silently rewritten. The storage policy and deployed package mapping still determine whether the broader cross-account read is supported.

Carry evidence across a rename without carrying a stale conclusion

Before the review finishes, M43 renames reports/download.ts to documents/fetch.ts without changing its blob. It also changes the helper from H7 to H8. These are different revalidation cases:

A stable claim ID survives these events; its evidence versions do not become interchangeable. The point of hashes is to detect what can be reused, not to skip the deployment and call-graph relationships around unchanged bytes.

Publish two views of the same record

The engineering view retains restricted source, exact locations, rejected links and revision history. The leadership view says: “The initial ownership edge was unsupported; corrected caller evidence shows membership only. Document/storage enforcement and deployed package mapping remain unresolved. M43 introduces H8 and requires revalidation before a current conclusion.” It links to evidence IDs and owners without copying private source into the report.

The handoff is complete when another authorized reviewer can reopen the governing evidence and explain which edge supports or defeats the claim. An inaccessible attachment, a moving branch link or a precise line number in the wrong package fails that criterion. None of this implies that all 1.2 million lines were assessed: the coverage ledger and the claim record continue to answer different questions.

For your next large assessment, create the snapshot manifest and the five-field coverage ledger before opening findings. Then use the cross-repository trace for each material handoff and the targeted-versus-full guide to phrase the result honestly for leadership.