Evidence Automation
Feynman Engine™ powered evidence collection
NIST SP 800-53A Rev 5 defines an assessment method as one of three actions: examine, interview, and test.
Evidence automation eliminates the manual burden of compliance documentation through intelligent collection from integrated systems. Powered by the Feynman Engine™, Thalorin automatically gathers, validates, and organizes compliance evidence with full lineage tracking from source to control.
Machine-readable evidence has stopped being a preference in federal work. FedRAMP 20x reached Phase 3 — wide-scale adoption — in the second half of fiscal 2026, and the programme's own account of what made its pilot reviews fast names consistent machine-readable schemas and assessor review of the validation code itself. NIST shipped OSCAL v1.2.3 on 7 August 2026, having added a Control Mapping model to the control layer in v1.2.0 the previous December. The artifact an assessor opens is becoming a file with a schema rather than a folder of exports.
Evidence is not requested against a control. NIST SP 800-53A Rev 5 decomposes each control into assessment objectives built from determination statements, and each statement is assessed by one of three methods — examine, interview, test — carrying depth and coverage attributes valued basic, focused, or comprehensive. A collector reporting that multi-factor authentication is enabled has answered a control heading. It has not answered a determination statement at comprehensive coverage, which is a question about which population was examined and how the sample was drawn.
Two of those three methods can be instrumented. The third cannot. Interview requires the person named as control owner to describe, in their own words, a process an integration now performs on their behalf — and the organisations that automate collection most thoroughly are often the ones whose control owners have least to say about it. Automating examine and test raises the stakes on the interview rather than lowering them, because it becomes the only point at which an assessor learns whether anybody understands the control or merely receives its output.
Collection is the easy half. The harder half is the join: each artifact carries the asset that produced it, the query and credential scope that retrieved it, the run that captured it, and the interval it covers rather than only the instant it was taken. SP 800-53A Rev 5 applies every method to an assessment object — a specification, a mechanism, an activity or an individual — so an artifact that cannot name its object has not been placed. Naming it is what lets one artifact answer a determination statement in one framework and a different statement in another.
Compliance operations need modern infrastructure
An empty result renders as a clean one
A collector whose credential has expired returns no findings, and no findings is exactly what a control operating without exception looks like. Telling them apart requires the pipeline to record what it expected to find, not only what it found.
Capture proves an instant, not a period
An API response or a screenshot demonstrates state at the moment of retrieval. Showing that a control operated throughout an observation window is a different claim, and it is the claim most test procedures make.
The interview method has no connector
SP 800-53A Rev 5 allows examine, interview and test. Automation reaches the first and the third. The second depends on a named control owner describing a process they may only ever see as a dashboard.
Crosswalks map requirements, not evidence
The PCI Security Standards Council's mapping of PCI DSS v4.0.1 to NIST CSF 2.0, published 23 July 2026, states that the two are not interchangeable and neither replaces the other. Mapped requirements can still demand different populations at different rigour.
How Thalorin helps
Automated evidence collection
Collection targets the determination statement rather than the control heading, so an artifact arrives carrying the assessment objective it answers and the depth and coverage it can support.
Source system integration
Connectors record the query issued, the credential scope it ran under and the response received, which is what makes an empty result distinguishable from a control with nothing to report.
Evidence lineage tracking
Every artifact resolves back through its collection run to the asset and system of record that produced it, so the assessment object a method was applied to is held as data rather than asserted in a filename.
Validation and quality checks
Schema and freshness checks reject an artifact before it reaches a package: an export dated outside the period it is filed against, a truncated response, a field the source system quietly stopped populating.
Temporal evidence management
Evidence carries the interval it covers, not only its timestamp of capture, so a control tested across an observation window is evidenced across that window rather than at one end of it.
Auditor-ready packaging
Packages emit through the OSCAL assessment layer — assessment plan, assessment results, plan of action and milestones — alongside a human-readable form, the shape FedRAMP 20x reviews have rewarded.
Evidence Automation: common questions
Can compliance evidence collection be fully automated?
No, and the limit is structural rather than technical. NIST SP 800-53A Rev 5 defines an assessment method as one of three actions: examine, interview, and test. Examine and test can be instrumented against source systems. Interview cannot, because it requires a named control owner to describe the process in their own words to an assessor. The right ambition is to remove collection labour so the effort left over goes into the one method no integration can serve.
What is OSCAL, and do we actually need it?
OSCAL is NIST's machine-readable format for control catalogues, profiles, system security plans and assessment artifacts, organised into a control layer, an implementation layer and an assessment layer. Release v1.2.3 landed on 7 August 2026; v1.2.0 in December 2025 added a Control Mapping model. You need it for FedRAMP, where machine-readable schemas are what fast reviews have in common. Outside federal work it is useful mainly as a discipline: it forces evidence to carry structure rather than filenames.
Why does automated evidence get rejected during fieldwork?
Usually for provenance or for period. An artifact with no verifiable link back to the system that produced it proves something was captured, not that your control operated, and an assessor cannot test a claim they cannot trace. The second reason is temporal: a capture taken during fieldwork shows the state during fieldwork, while the test procedure asks about the whole observation window. Both are cheap to fix at collection time and expensive afterwards.
Can one piece of evidence satisfy two frameworks at once?
Often, but a published crosswalk does not by itself establish it. When the PCI Security Standards Council mapped PCI DSS v4.0.1 to NIST CSF 2.0 on 23 July 2026, it said plainly that the two are not interchangeable and neither is a replacement for the other. Reuse holds where the underlying control is the same one and the more demanding framework's depth and coverage are met. Where either differs, you have a mapping between requirements, not shared evidence.
What do depth and coverage mean on an evidence request?
They are the two attributes that set how hard an assessment method is applied, and both take the values basic, focused, or comprehensive. Coverage addresses the scope or breadth of the assessment objects included; depth addresses the rigour and level of detail with which the method is applied. The same control can therefore generate very different requests: a basic examination of a configuration standard, or a comprehensive one that samples every host in the boundary and reconciles it to an asset inventory.
Regulatory state described as of August 2026. Requirements change; verify against the current rule before relying on any date above.
Talk to us about Evidence Automation.
See how one evidence artifact satisfies Evidence Automation requirements alongside every other framework you carry.