Create a control-evidence matrix for a security review

A policy says what should happen; evidence shows what did happen for a particular system and version. Security review becomes faster when risks, controls, tests and owners are connected in one matrix.

Define the measurable control

Start from the system threat model and applicable organizational requirements. For each risk, name the preventive or detective control, implementation location, owner, evidence id, build/version, collection date and retest cadence. Record gaps honestly and distinguish design evidence from operating evidence.

The AI reliability lab makes the decision reproducible with authored data:

from reliability_cases import control_evidence_case

result = control_evidence_case()
assert result["gaps"] == ["D-07"]
assert result["review_ready"] is False
assert result["framework"] == "NIST AI RMF mapping exercise"
assert result["certification_claim"] is False

Two fixture controls have current evidence. The deletion-reconciliation item D-07 is stale, so the pack is not review-ready. The framework field says this is a NIST mapping exercise and explicitly makes no certification claim.

Challenge the result

A screenshot without query, version or timestamp is weak evidence. A passing test can also be irrelevant if it exercises a mock while production uses a different path. Have reviewers sample evidence back to source and verify operating cadence.

Keep a reviewable evidence pack:

• risk-control-evidence-owner matrix
• immutable test and configuration references
• gap and compensating-control register
• review and retest record

This work aligns with the evaluation, security, cloud operations, reliability and FinOps sequence in the FDE for Professionals course. The linked course describes the learning pathway; this article’s numbers are synthetic and do not report a model, client, audit or production result.

Practice task

Build five rows from your threat model. Intentionally expire one item and show how the review status changes without changing the underlying policy.

Continue learning

This article is part of the Advanced AI reliability and assurance sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Run an AI incident tabletop exercise.
• Continue with Evaluate private inference against a managed model API.

Reference: NIST AI RMF Playbook.