The detection-evidence pack — how to prove your detection works

This playbook fires the moment someone asks you to prove your detection works — a DORA supervisor reviewing a threat-led penetration test, a NIS2 competent authority running a capability assessment, a cyber-insurance underwriter tightening a renewal, a TLPT blue-team reviewer, or your own board after a peer breach. The deliverable is a standing evidence pack you can hand over on demand, not a slide assembled the night before.

What is not in scope: buying detections, tuning rules, or choosing a SIEM. This playbook assumes you already run detection tooling; its job is to prove that the detections you have actually fire, against which adversary techniques, and how fast. A tooling inventory — the list of products you own — is explicitly not what this produces, and explicitly not what any of these audiences will accept as evidence.

The instinct when asked to prove detection is to answer with the stack: the EDR vendor, the SIEM, the number of enabled rules, last quarter's alert count. None of it answers the question. A rule that exists is not a rule that fires; an alert count measures volume, not coverage; and a product logo proves procurement, not detection.

Evidence is the opposite shape. It is built around the adversary, not the tool, and it has exactly three load-bearing columns:

• What — the adversary technique, named against a shared framework so the reviewer can map it to their own threat model. MITRE ATT&CK is the lingua franca every regulator, insurer and red team already speaks.
• Whether — did a detection actually fire when that technique ran, verified by test, not asserted from a rule name.
• How fast — the mean time to detect (MTTD) from technique execution to alert, because a detection that fires next morning is a post-mortem, not a control.

A pack that carries those three columns for the techniques that matter to your sector is evidence. Anything less is an inventory wearing an evidence label.

Goal: a technique-by-technique map of what you claim to detect, before you test any of it. Checklist:

• Pick the threat model, not the whole matrix. Enumerate the ATT&CK techniques that your sector's actual adversaries use — ransomware affiliates, the initial-access and lateral-movement techniques in your incident history, the tactics your TLPT threat-intelligence report profiled. Boiling the ocean across all 600+ techniques produces a map nobody maintains.
• Map each technique to the detection(s) expected to fire. One row per technique: the data source, the rule or model, and the layer (endpoint, network, identity, cloud).
• Grade coverage honestly — covered / partial / none — as a claim, not yet a proven fact. Partial means the rule exists but is untuned or noisy; none means you have no detection at all and are relying on a compensating control.
• Flag the blind spots by data source. A technique that only shows on the wire (lateral movement, C2 beaconing, exfiltration) is invisible to an endpoint-only stack no matter how many EDR rules you count.

The common trap here is rule-count coverage — reporting '80% coverage' because 80% of techniques have a rule. A rule is a hypothesis until a test fires it. Phase A produces the hypothesis; Phase B turns it into evidence.

This is the load-bearing phase, and the one most organisations skip. For every in-scope technique, execute it against the live stack and record what actually happened.

• Run the technique, do not model it. Atomic Red Team is the standard library of small, reproducible tests mapped to ATT&CK; a purple-team session or a TLPT red-team run produces the same ground truth at higher fidelity. The point is that an alert either fired or it did not.
• Record the outcome per technique: did an alert fire, at which layer, at what fidelity (high-signal vs buried in noise), and — critically — the MTTD, the wall-clock gap between technique execution and the alert.
• Distinguish tested from claimed. A technique marked 'covered' in Phase A that produces no alert under test moves to a gap. This reconciliation, done with dates, is the single most persuasive artefact in the whole pack.
• Timestamp everything. Execution time and alert time are the raw material for the MTTD column and the proof that the test actually ran on the date you claim.

MTTD under one hour for a high-severity technique says your detection can contain before serious damage; MTTD over 24 hours says it cannot. Reviewers know these thresholds — reporting the number honestly beats claiming a coverage percentage every time.

Turn the test results into a durable artefact, versioned and dated, not a one-off export:

• The coverage matrix: technique → detection → covered/partial/none, reconciled against the Phase B tests.
• Per-technique validation results: execution timestamp, alert timestamp, layer, fidelity.
• The MTTD distribution for high-severity techniques — a median and a tail, not a single average that hides the slow detections.
• An accepted-gap register: each technique you do not detect, why, and the compensating control you rely on instead.
• A change log: what moved since the last run, because coverage decays silently every time a log source drops or a rule is disabled.

Standing is the operative word. A pack assembled once for an audit is stale by the next deployment. Re-run the validation on a cadence — quarterly at minimum, and on any material change to the estate — so the pack you hand a supervisor reflects the stack you run today, not the one you ran at last year's assessment.

Ordered by what a reviewer opens first:

• The coverage matrix, dated — the one-page answer to 'what do you detect'.
• Per-technique validation results with execution and alert timestamps — the proof the matrix was tested, not asserted.
• The MTTD distribution for high-severity techniques — the number insurers and blue-team reports consume directly.
• The accepted-gap register — the honesty that makes the rest credible.
• The raw detection telemetry behind the 'when' column — the dated events proving each alert actually fired.
• The change log since the last run — proof the pack is standing, not a snapshot.

The hardest column to produce honestly is the 'detected / when' for the network-visible techniques — lateral movement, C2 beaconing, exfiltration — because they often leave no clean endpoint log and are exactly where a rule-count map overstates coverage. This is where Zero Hunt's AI Traffic Analysis contributes: an on-appliance deep-learning model with four parallel inference heads (suspicious traffic, malware classification, attack-type identification, application fingerprinting), trained on billions of PCAP sequences and running locally on the appliance GPU, produces timestamped detection events for those network-visible techniques as a byproduct of normal operation — so the 'detected / when' telemetry feeding your MTTD column already exists and is dated, rather than being reconstructed for the audit. It is the network-detection input to the pack, not the whole pack — the endpoint, identity and cloud rows are still yours to test.

Anti-patterns that turn a detection-evidence pack back into an inventory:

• Counting rules, not techniques. '80% of techniques have a rule' is a Phase A hypothesis reported as a Phase B result.
• Claiming coverage you never tested. The gap between 'we have a rule for that' and 'the alert fired under test' is where real incidents live.
• Coverage without timing. A matrix with no MTTD answers 'can it detect' but not 'in time to matter' — and every consuming audience asks the second question.
• The 100% coverage claim. A pack with no accepted-gap register reads as either dishonest or untested; a candid gap list is what makes the covered rows believable.
• The stale pack. Tested once at audit time and never re-run, so it certifies a stack that no longer exists after two quarters of change.
• The endpoint-only view. Assuming the EDR sees everything, so network-visible techniques are never measured and the pack silently overstates coverage on exactly the techniques attackers use to move and exfiltrate.

Build the pack once and export it per audience — the underlying evidence is the same, only the framing changes.

• DORA. The TLPT blue-team report under RTS (EU) 2025/1190 times what the defenders detected against what the red team actually did — a detection-evidence pack is the standing baseline that makes that report a comparison rather than a surprise. The published DORA TLPT engagement playbook names 'no measured detection baseline going in' as a top failure mode; this pack is the fix.
• NIS2. Article 21(2) of Directive (EU) 2022/2555 requires detection and incident-handling capability as a management-body duty, and competent-authority assessments increasingly ask for evidence of it, not a description.
• Cyber insurance. Underwriting has shifted from questionnaire checkboxes to evidence, and MTTD is the metric that translates a coverage matrix into the language a carrier prices — a tested pack is a renewal asset.
• Methodology. NIST SP 800-115 is the neutral reference for the security-testing methodology behind Phase B, and pairs naturally with a continuous threat-exposure management programme that keeps the pack standing rather than annual.