Write the decision first, then show the few observations that could change it. A useful memo names the buyer's job, pilot scope and revision, pre-agreed hard requirements, observed results, missing evidence, operating cost, dissent, owner and next decision date. Use the same evidence rubric for each vendor: claim → artifact → observed pilot result → limit → decision. Keep vendor assertions separate from what your own team reproduced. If a mandatory trust or coverage requirement remains unknown, recommend a bounded extension or narrower use rather than converting the unknown into a passing score. If the decision changes your tool mix, also compare a platform with a focused code security tool.
The memo is a decision record, not a marketing comparison. NIST's Secure Software Development Framework gives purchasers a vocabulary for discussing secure development with suppliers. OWASP WSTG reporting guidance emphasizes scope, limitations and findings that both management and technical staff can understand. Carry those habits into the buying decision. A two-week pilot can justify a next step; it cannot prove the absence of undiscovered vulnerabilities or predict every future operating cost.
Agree the decision rule before showing results
Write hard gates before the pilot begins. Examples: approved source locations, acceptable repository permissions, a visible partial-scan state, an export path compatible with your workflow, and a minimum ability to substantiate findings in the critical service. These are organization-specific requirements, not universal vendor standards. Some can be verified by configuration and contract; others require a controlled run.
Then choose a small number of weighted preferences: reviewer effort, developer handoff quality, supported frameworks, scan speed, integration friction and expected cost at intended frequency. A weighted score can help within candidates that pass the hard gates. It must not turn a failed data boundary into a purchase because the UI was attractive. If a requirement was added after seeing a result, identify the change and get buyer agreement rather than quietly rewriting the rubric.
The security leader should appoint technical, platform, privacy and purchasing reviewers before the pilot. Give each a decision right for the evidence they own. The CTO or sponsor owns the final tradeoff. Record dissent that could change rollout conditions.
Copyable one-page memo
Copy this structure into your decision system and replace every bracket:
Decision and date: [Proceed / narrow / extend / reject] [candidate and use case] on [date]. Decision owner: [name or role]. Next checkpoint: [date and exact question].
Job and success rule: [What work should improve?] Hard gates: [list with pass/fail/unknown]. Preferences: [weights or ordering agreed before run].
Pilot protocol: [selected repository/service], [immutable revision], [deployment context], [time and spend cap], [allowed vendor help], [source and access boundary], [known exclusions].
Evidence table: For each material claim, record [claim], [artifact link or ID], [what our team observed], [limit or counterevidence], [pass/fail/unknown/outside scope], [reviewer]. Include rejected or unresolved results where present; otherwise state that the sample supplied none rather than inventing one.
Operating model: [who reviews alerts], [where issues go], [fix reassessment], [ongoing volume and cost assumption], [access revocation and data deletion].
Open questions and trigger: [missing artifact], [owner], [deadline], [what result changes the decision]. Dissent: [role and reason]. Approval: [sponsor and required control owners].
Do not collapse all evidence into “X alerts found.” If the selected repository was only partially assessed, say where. If the pilot used synthetic code, say how it differs from the intended rollout. A finding can have a severity score and still require technical verification; FIRST's CVSS v4 specification separates vulnerability characteristics from threat and environmental context, so do not let a score substitute for your business priority.
Turn mixed pilot notes into a short decision
Basin Analytics, candidates, artifacts, figures and outcomes in this example are invented. No vendor evaluation was run. Basin has one AppSec engineer and wants clearer authorization findings for a TypeScript API, with Go-worker coverage desirable but optional for the first rollout. The initial evaluation uses synthetic code at R1. Its hard gates were declared before results: selected-repository access, an approved processing route, visible exclusions, revision-bound findings and positive reassessment of a fix at a changed revision.
The raw notes arrive in different forms:
| Note received | What belongs in the memo |
|---|---|
| “A found the gateway issue. Looks critical.” | A-03 calls a policy-dependent path confirmed without the missing policy. Verification weakness, not a new confirmed vulnerability. |
| “B handled the safe example.” | B-03 contains the compulsory account-bound query and a rejection reason checked against R1. Useful evidence for the selected case. |
| “B supports Go; run timed out.” | B-07 shows indexing stopped and worker review incomplete. No Go adoption evidence. |
| “Everything stored in EU.” | Storage location is documented; actual inference destinations and fallback remain unanswered. Mandatory route gate unknown. |
| “Review took 90 minutes, so cost seems fine.” | Ninety active reviewer minutes for the prepared cases; future volume, fees and review capacity still need a budget. |
| “Fix looks good.” | No changed-revision reassessment was evaluated. Record the missing gate evidence, not the opinion. |
The editor removes promotional adjectives but preserves the disagreements and missing facts. The resulting first page is shorter than the notes:
Decision — 18 October 2026: Extend B for five working days using only the already-approved synthetic packet. No private-source or organization-wide installation. Owner: Head of Security. Decision checkpoint: 25 October.
Why: B's selected TypeScript outputs support one risky path and explain rejection of a nearby safe path (B-02/B-03). A's gateway-dependent conclusion lacks decisive policy evidence (A-03). This supports a narrow follow-up, not a general accuracy ranking.
Gates: selected-repository grant, revision identity and visible exclusions pass for B in the exercise. Processing route is unknown and blocks private source. Fix reassessment is untested. Go review is incomplete and excluded from any rollout proposal.
Evidence required: vendor supplies exact route/fallback and retention records; platform reconciles the configuration; AppSec assesses one changed revision; purchasing supplies written fees and a bounded operating budget. Named owners return artifacts by the checkpoint.
Decision rule: if the required data boundary fails, reject that hosted configuration. If the route and reassessment pass, consider only a 30-day TypeScript rollout within an approved budget. Go requires a separate completed evaluation. Dissent: Go owner rejects any implication that the partial run qualifies the worker.
The full source packet, grant capture and reviewer notes stay in an access-controlled annex. The first page contains only observations that determine the next commitment. A long catalog of demonstrated buttons would distract from the unanswered route and reassessment gates.
Keep a small annex that can change the decision
Use stable evidence IDs rather than mutable dashboard links alone. The relevant annex is:
| ID | Artifact and reviewer observation | Limit at first decision |
|---|---|---|
| B-02/B-03 | R1 source paths, output and independent reviewer notes for the risky and safe TypeScript cases | Synthetic selected cases; no runtime exploitation evidence |
| B-07 | Scope/indexing record marks Go incomplete | No conclusion about the unreviewed worker |
| B-08 | Actual selected-repository permission capture | Valid only for that installation/grant |
| B-09, requested | Model/service route, permitted destinations, fallback and retention terms tied to configuration revision | Not supplied yet |
| B-10, requested | Original claim compared with corrected R2 and complete changed-path scope | Not evaluated yet |
| B-11, requested | Fees, scan allowance, human review limit and stop/review triggers | Ninety pilot minutes do not predict portfolio cost |
A screenshot can support “the interface showed partial,” but not “all source was reviewed.” A model route diagram can show intended flow, but needs to match the actual selected configuration and accepted terms. Keep each artifact's role narrow enough that a later reviewer knows what would invalidate it.
Finish the extension with a different memo
At the checkpoint, suppose the fictional returned evidence establishes the following: B-09 identifies and bounds the approved route and fallback behavior, and the privacy/platform owners accept the corresponding retention and support terms for configuration P2; B-10 includes the corrected TypeScript path at R2 and a positive reassessment; B-07 remains incomplete for Go. Purchasing accepts a $600 fixed 30-day fee plus at most $200 usage, while the AppSec manager allocates eight active review hours. These are chosen exercise budget limits, not market prices or a claim that eight hours will suffice.
The new memo should make a commitment rather than repeat “conditional extension”:
Decision — 25 October 2026: Approve a 30-day rollout of B for the named TypeScript API only, under configuration P2 and its accepted source-handling terms. The repository grant is limited to that service's repository. Go remains excluded; its owner retains the existing review method.
Basis: B-09 resolves the route gate for P2; B-10 supplies positive new-revision evidence for the selected fix; B-02/B-03 remain the bounded usefulness evidence. No general detection rate is claimed.
Operating commitment: One AppSec owner maintains evidence-based dispositions and routes work to developers. Proportionate containment or defensive repair can proceed while a credible claim is still being checked; that does not turn it into a confirmed finding. Budget is $600 fixed plus a $200 usage ceiling and eight reviewer hours over 30 days. Review at four hours consumed or $100 variable usage to assess remaining workload; reaching a hard limit pauses new work until an explicit decision. Partial runs remain visible.
Revisit: At day 30, decide whether to continue using actual claim volume, supported outcomes, active effort and developer handoffs. Reassess immediately if the model route, grant, support access or required scope changes. A Go rollout needs a separate decision and evidence.
A product can be approved for one service while remaining unsuitable or unproven for another. The remaining Go gap is not a reason for an indefinite synthetic pilot, because Go was optional for the declared first step. If Go had been mandatory, this same packet would not support the rollout.
If B-09 had instead shown an unapproved route, the second memo would reject hosted use under that configuration, regardless of the TypeScript result. Write that alternative in the original decision rule so the final meeting cannot quietly turn a hard gate into a preference.
For standards-based requirements, retain the selected ASVS version and actual requirement evidence in the annex. A mapped finding is not application-wide conformance, and a scanner purchase does not satisfy an independently required pentest by implication.
Preserve the decision when the sponsor changes
The memo should remain usable if Basin's sponsor or AppSec owner leaves during the thirty-day rollout. Place the accepted configuration, budget limits, repository scope and open Go question beside the decision, with links to the evidence annex. A replacement owner should not have to reconstruct whether “approved” meant the selected TypeScript service or every repository in the organization.
Record which commitments require a new decision. Adding a repository, changing the processing route or raising the hard spend ceiling changes the approved arrangement. An ordinary review within the existing scope does not require reopening the purchase discussion. This distinction lets the operator act while preserving the limits the sponsor accepted.
At the checkpoint, compare actual work with the memo's assumptions and explain any difference. If review effort reaches the cap before the expected volume is processed, the team has learned something about capacity; it has not demonstrated a detection failure or justified silently increasing the budget. The next memo can narrow frequency, allocate more capacity or stop the rollout using that evidence.
Close the loop after the memo
Send the decision to the people who will live with it: AppSec, service owners, platform, privacy and the budget owner. Each open condition needs an owner, artifact and date. Update the memo when new evidence changes a gate, preserving the original decision and why it changed. When an extension ends, choose proceed, narrow or reject; avoid an indefinite “pilot” that quietly becomes production access.
For a reusable pilot protocol, start with the two-week scanner evaluation guide. Use the verification-claim checklist and the regional data-handling checklist to collect the two evidence sets most likely to change this memo.