Measure where PRs actually wait, then shorten the critical path without calling unexamined code clean. Separate queue time, setup or build time, scanner analysis, upload and human review. Run independent work in parallel, stop duplicate or superseded jobs when safe, and reserve broad analysis for a scheduled or post-merge lane where it can still generate owned findings. Keep a fast PR decision for the checks that must finish before merge. The answer changes when a scanner needs a complete build or cross-component context: a narrow diff run may be faster but may miss the path that matters, so its scope must be explicit.
GitHub's CodeQL troubleshooting guide identifies memory, language matrices, amount of code, schedules and query selection as analysis-time controls. Those are options to evaluate against detection needs, not a list of safe shortcuts to apply blindly.
Diagnose the delay before editing the workflow
Take a representative sample of recent PR runs, including small and large changes, each supported language, failures and reruns. For each run record the PR head revision, queue wait, setup, required build, each security job, upload/reporting, total time until the last required check, and whether a human decision followed. Report median and a slow-tail value such as the 90th percentile, with the sample size and percentile method. Use enough representative runs for the tail estimate; seven constructed runs below illustrate arithmetic, not a reliable production latency distribution. A five-minute scanner can still be the cause of a long merge delay if it starts after a 25-minute build; a 20-minute scanner may add no wait when it runs alongside a longer required job.
For each scanner, document what it needs: only lockfiles, source text, a full dependency install, a compiled build, a container image, or a live environment. Also record the exact scope and exclusions. A timing chart without the coverage description encourages optimizations that look fast because they stopped scanning the relevant code.
Work from complete runs, not a sum of medians
Constructed timing exercise: Atlas Pay runs a lockfile package scan and a compiled CodeQL analysis. All figures below are fictional teaching data, not executed benchmarks. Seven serial runs complete the same declared kinds of checks. Each row is a complete run. Stage columns are durations in minutes; the final column is elapsed time from the trigger. Queue time is separate from occupied runner time.
| Run | Queue | Checkout/build | Analysis | Package scan | Result upload | Last required result |
|---|---|---|---|---|---|---|
| S1 | 1 | 7 | 18 | 5 | 1 | 32 |
| S2 | 8 | 6 | 11 | 5 | 1 | 31 |
| S3 | 2 | 9 | 15 | 5 | 1 | 32 |
| S4 | 4 | 7 | 16 | 5 | 1 | 33 |
| S5 | 10 | 5 | 10 | 5 | 1 | 31 |
| S6 | 3 | 8 | 18 | 5 | 1 | 35 |
| S7 | 5 | 6 | 17 | 5 | 1 | 34 |
Sort the last column: 31, 31, 32, 32, 33, 34, 35. Its median is 32 minutes. The individual stage medians are 4, 7, 16, 5 and 1, which add to 33. Those are different statistics: each stage median can come from a different run. A useful end-to-end median must come from run-level end-to-end observations.
S4 alone provides a valid 33-minute critical-path example:
Serial S4: queue 0–4 | build 4–11 | analysis 11–27 | package 27–32 | result 32–33
Proposed: queue 0–4 | build 4–11 | analysis 11–27 | result 27–28
package job: queue 0–4 | setup 4–5 | scan 5–10 | result 10–11
Under unchanged queueing, the model's last required result is max(28, 11) = 28 minutes. This is a modeled S4 improvement, not a measured change in the cohort median. The package job needs only the lockfile in this example; if it needs the compiled artifact, the dependency graph must change.
Let the runner pool contradict the attractive model
Atlas's runner pool is constrained. Starting a second job competes with other PRs. The paired exercise below keeps each row's build and analysis duration unchanged but supplies new queue times. Package setup and result upload each take one minute; the scan takes five. Each job starts queueing at the PR trigger.
| Paired case | Analysis-job queue | Analysis result at minute | Package-job queue | Package result at minute | Last required result |
|---|---|---|---|---|---|
| P1, S1 inputs | 7 | 33 | 1 | 8 | 33 |
| P2, S2 inputs | 9 | 27 | 2 | 9 | 27 |
| P3, S3 inputs | 8 | 33 | 3 | 10 | 33 |
| P4, S4 inputs | 10 | 34 | 4 | 11 | 34 |
| P5, S5 inputs | 11 | 27 | 4 | 11 | 27 |
| P6, S6 inputs | 6 | 33 | 5 | 12 | 33 |
| P7, S7 inputs | 8 | 32 | 2 | 9 | 32 |
The final times sort to 27, 27, 32, 33, 33, 33, 34, giving a 33-minute median. Parallel execution did not improve the completed-run median in this constructed comparison. Across all seven cases, occupied runner time rises from 195 to 209 minutes: each parallel case adds one minute of package setup and a separate one-minute result upload. Queue time is not counted as occupied runner time here; this is a workload measure, not a provider billing quote.
The decision is to withdraw the blanket parallel rollout and first remove confirmed duplicate executions, then trial reserved capacity for the analysis job. Keep the same lockfile, intended compiled source, languages and query configuration in the comparison. A small, synthetic cohort establishes no universal performance claim; a real rollout needs representative completed runs and failure counts alongside them.
Account for cancellation without inventing a fast result
A later PR run for head H8 starts at 09:00 and is canceled at 09:12 after H9 arrives. H8's twelve elapsed minutes are not a successful twelve-minute assessment; its occupied runner time is recorded separately. The replacement H9 run starts queueing at 09:12 and completes at 09:46: 34 minutes from the new push, with its own examined-scope receipt. Attribute the canceled run's consumed compute to the same workflow policy when comparing cost; excluding it from the successful-run latency median does not make that compute free. The author has been waiting across changes for 46 minutes, which is a different metric again.
Keep these runs outside the seven-pair comparison unless the cohort definition explicitly includes them. Report H8 as superseded and H9 as completed; never improve the success median by counting cancellation as completion. PR concurrency groups must also avoid canceling independent release or scheduled assessments. GitHub documents workflow-specific concurrency grouping; the group is part of the evidence design, not merely a speed switch.
Change one bottleneck at a time
1. Draw the dependency graph. Put each job on a timeline with its inputs and output. Mark the latest required finish time. Move a job off the critical path only if it truly has no dependency on the preceding build or scan. Do not make the reporting job appear “successful” before its scanner has produced a result.
2. Remove duplicate execution. Compare PR, push and scheduled triggers against actual branch rules. A push to a PR branch can produce more than one workflow run depending on configuration. Decide which analysis is the authoritative PR result and which is the default-branch or schedule result. Keep an immutable revision and run ID in the output, so an older green result cannot be mistaken for a new one. GitHub's workflow configuration options distinguish these triggers.
3. Parallelize independent units. Package analysis based on a lockfile may run beside source analysis. Multiple language analyses may use a matrix rather than sequential initialization. GitHub states that its default CodeQL workflow uses a language matrix, while some advanced configurations analyze languages sequentially unless changed (analysis-time guide). Check whether jobs share a build or artifact before splitting them. Parallel work can reduce wall time while increasing total runner use; measure both.
4. Check resources and query cost. If analysis is CPU- or memory-bound, compare runner specification with the tool's guidance and profile the slowest language. A larger runner may help, but it has a cost. Review expensive custom queries for value and validity before dropping them. Do not turn off a class of checks solely because it produces alerts the team has not triaged.
5. Scope narrow runs deliberately. A PR lane can focus on changed components when the tool and risk model support it. Keep a broader run with a documented cadence to inspect cross-component behavior and untouched code. GitHub specifically advises maintaining at least one scheduled whole-code workflow when splitting CodeQL analysis, because some data flows become visible only in a complete build (analysis-time guide). Also note that a GitHub paths filter controls whether a workflow starts; it does not itself guarantee that only those files are analyzed, and a skipped required workflow can leave a PR pending (workflow triggers).
6. Cancel only superseded work. On platforms that support concurrency groups, group PR jobs by the PR or branch and cancel the old run when a new commit supersedes it. GitHub documents concurrency and cancel-in-progress, including the need for distinct groups when unrelated workflows must not cancel each other. Do not cancel a release or default-branch scan merely because a PR with the same branch-like name starts. Verify the new run actually completes before treating the PR as assessed.
7. Make failure visible. Record completed, partial, timed out, no source analyzed and skipped distinctly. A successful workflow process can still be useless if the scanner saw no intended source. GitHub's “No source code was seen during the build” guidance illustrates why build configuration belongs in the validity check. Route an incomplete required scan to a rerun or an explicit manual review, not a green check.
Keep a latency-and-coverage ledger
After each workflow change, inspect a new representative PR sample. Use a small ledger rather than a dashboard that shows only total duration:
| Metric | Meaning and decision use |
|---|---|
| Merge wait, median and slow tail | Did the change make developer wait shorter, including queue and reviewer time? |
| Runner minutes or compute cost | Did parallelism move work to more concurrent machines? |
| Intended versus examined scope | Did every configured language, component and input actually reach the scanner? |
| Inconclusive share | Are timeouts, skipped runs and build failures rising? |
| PR findings versus later broad findings | Are important issues appearing only after merge, suggesting the PR scope is too narrow? |
| Stale-result incidents | Did a check from an older revision appear to authorize the new one? |
For Atlas Pay, the first validation asks whether the package scan still sees the same lockfile and produces results for the new head revision. The second asks whether CodeQL still analyzes the compiled invoice service and all intended languages. Only then does the team compare wait times. If median improves but the broad scan later finds a newly exposed cross-service path, Atlas revisits its PR trigger and human-review criteria; the fast lane has an observable detection limit.
When speed conflicts with coverage
If the tool needs a full build and the build dominates PR time, choose explicitly among a faster build, more suitable capacity, a bounded PR scan plus comprehensive scheduled scan, or a human gate for sensitive changes. Document the risk of each. A diff-limited scan is not equivalent to a full assessment. A scan that omits generated routes, configuration, shared authorization or a downstream service may miss the decisive control or harmful effect. A clean report is only as broad as its recorded scope and method.
If a required scan cannot finish within the team's wait budget, do not silently mark it optional and call the same policy enforced. Change the merge policy in the open: state which checks still block, who reviews sensitive changes, when broader work runs, and how its findings return to owners. For the next step, instrument the slowest required job for one week before changing triggers. Use the PR-versus-scheduled assessment guide to place the work, then the PR security-check guide to write the merge rule.