Chain of Evidence: How Certification Is Established

Agent certification is not a one-time approval decision, but a structured, auditable process that produces a verifiable chain of evidence. This chain of evidence documents how an agent was designed, tested, evaluated, and approved for use, providing the foundation for trust, governance, and regulatory alignment. Rather than relying on ad-hoc documentation or manual reviews, the AMC protocol standardizes evidence collection and ties it directly to each certified agent and code version.

From a certification workflow perspective, the chain of evidence is organized into four logical components, reflected directly in the certification user interface: code assessment, pre-testing documentation, evidence collection, and secure evidence bundling.

1. Code Assessment

The certification process begins with a code assessment step, where a specific version of the agent's codebase is evaluated. This assessment anchors certification to an immutable snapshot of the agent, ensuring that all subsequent evidence is traceable to a known implementation. The results of the code assessment feed into the agent's overall rating but are also treated as part of the evidentiary record supporting certification decisions.

Required Artifacts:

• Agent Version ID
• Code Repository Snapshot (with commit hash)
• Build Configuration Files (Dockerfile, requirements.txt, etc.)

2. Pre-Testing Documentation (Evidence Box 1)

Before any testing or validation activities are conducted, foundational documentation is collected to establish intent, scope, and planned controls. This pre-testing evidence captures how the agent is designed and how it is expected to behave.

The following artifacts are uploaded at this stage:

Agent Design Document

Describes the agent's purpose, architecture, autonomy boundaries, integrations, and intended use cases. This document establishes the baseline assumptions about what the agent is designed to do and how it should operate.

Required Sections:

• Agent purpose and intended use cases
• Architecture overview (orchestrator, super agent, utility agent structure)
• Autonomy boundaries and human oversight points
• System integrations and data flows
• Expected behaviors and limitations

Agent Test Plan

Defines the testing approach, success criteria, and evaluation scope across performance, functionality, and security. This plan documents how the agent will be evaluated and what constitutes acceptable performance.

Required Sections:

• Testing methodology (synthetic vs. real-world data)
• Success criteria for each certification dimension
• Evaluation scope and test coverage
• Performance thresholds and benchmarks
• Security testing approach

Agent Security Plan

Documents anticipated risks, threat models, access controls, and security safeguards. This plan establishes the security posture expectations before testing begins.

Required Sections:

• Threat model and risk assessment
• Access controls and authentication mechanisms
• Data handling and privacy considerations
• Security safeguards and mitigations
• Incident response procedures

Together, these artifacts establish the baseline assumptions against which the agent will later be evaluated.

3. Evidence Collection (Evidence Box 2)

Once testing and validation activities are completed, outcome-based evidence is collected to demonstrate how the agent performed against the plans and controls previously defined. This stage captures the results, rather than the intent.

The following artifacts are uploaded at this stage:

Agent Test Report

Summarizes functional and performance test results, observed behaviors, and identified limitations. This report provides concrete evidence of how the agent performed during evaluation.

Required Sections:

• Performance test results (tool selection accuracy, invocation correctness, reliability, latency)
• Functionality test results (interoperability, composability, documentation quality, guardrail compliance)
• Observed behaviors and edge cases
• Identified limitations and known issues
• Comparison against test plan success criteria

Agent Security Report

Documents security findings, risk grading, and any identified vulnerabilities or mitigations. This report provides the evidence base for the security grade (A–D).

Required Sections:

• Security grade and risk assessment (impact × likelihood)
• Static analysis findings (prompt injection, PII detection, secrets, anomalies)
• Container scan results (CVEs, supply chain vulnerabilities)
• Dynamic analysis results (endpoint testing, specification compliance)
• Vulnerability details and remediation recommendations
• Security controls validation

These artifacts provide concrete, reviewable proof that the agent was evaluated in accordance with the defined plans and that risks are understood and documented.

4. Secure Evidence Bundle (Automated)

When all required documentation has been uploaded and validated, the system automatically generates a secure evidence bundle. This bundle aggregates all certification artifacts into a single, immutable package.

No manual uploads are required at this stage. Instead:

• The system creates a consolidated evidence archive (e.g., ZIP/TAR format)
• A cryptographic checksum/hash (SHA-256) is generated for the bundle
• A visual confirmation (e.g., a checkmark) indicates that the evidence set is complete and locked

This secure bundle serves as the authoritative certification record. The checksum enables independent verification that the evidence has not been altered, supporting auditability, compliance reviews, and long-term governance requirements.

Bundle Contents:

• Code assessment artifacts (version ID, code snapshot)
• Pre-testing documentation (Design Document, Test Plan, Security Plan)
• Evidence collection artifacts (Test Report, Security Report)
• Test execution evidence (see Test Execution Evidence below)
• Cryptographic hash manifest

Storage:

• Evidence bundles are stored in access-controlled, immutable storage (e.g., AWS S3 with WORM policy or secure vault)
• Retention period aligns with compliance requirements (typically 7+ years for regulated environments)
• Storage location and retention policy are documented in the certification record

Test Execution Evidence

During test execution, comprehensive evidence must be captured to document exactly how the agent processed adversarial inputs and where vulnerabilities were identified. This evidence forms the core of the certification record and enables root cause analysis.

Input/Test Prompt

Full record of the adversarial prompt, including timestamp, user ID, and source IP.

Purpose: Proves how the test was initiated.

This documentation establishes the exact input that triggered the vulnerability, enabling reproduction of the issue and validation of remediation efforts. Timestamp and source information provide audit trail context and help distinguish test activities from legitimate user interactions.

Required Fields:

• Full test prompt/adversarial input
• Timestamp (ISO 8601 format)
• Test user ID
• Source IP address
• Test category (prompt injection, jailbreak, tool poisoning, etc.)

Agent Trace/Internal Steps

Detailed logs of the agent's reasoning path (e.g., thought process, function calls, intermediate prompt steps, RAG document retrieval). This is crucial for root cause analysis.

Purpose: Proves how the agent processed the malicious input and why it failed.

Agent traces provide visibility into the agent's decision-making process, revealing where security controls failed or where unexpected behaviors occurred. This level of detail is essential for understanding vulnerabilities and designing effective mitigations. For agents with complex reasoning chains or multi-step workflows, these traces are particularly valuable for identifying attack surfaces.

Required Fields:

• Complete reasoning chain/logs
• Function/tool selection decisions
• Intermediate prompt steps
• RAG document retrieval (if applicable)
• Model version and configuration

Tool Calls

Record of every tool/function call the agent attempted, including the full JSON payload and response from the external system (e.g., database query, HTTP request).

Purpose: Proves where the attempted vulnerability occurred (e.g., trying to access a restricted API).

Tool call records demonstrate the agent's interaction with external systems and identify where security boundaries were crossed or where controls successfully prevented unauthorized actions. This evidence is critical for understanding the scope of potential vulnerabilities and validating that security controls function as intended.

Required Fields:

• Tool/function name
• Full JSON request payload
• Full JSON response payload
• Timestamp of call
• Success/failure status
• Error messages (if applicable)

Final Output/Response

Full final response generated by the agent, including the raw response body.

Purpose: Proves the impact of the vulnerability (e.g., successfully leaked data, failed to execute policy).

The final output demonstrates the actual impact of the vulnerability, showing what an attacker could achieve if the vulnerability were exploited in production. This evidence supports risk assessment and helps prioritize remediation efforts based on potential business impact.

Required Fields:

• Complete agent response
• Raw response body
• Response timestamp
• Response metadata (model used, tokens, latency)

Authorization Phase

Before any security testing or vulnerability assessment can begin, proper authorization and test planning must be established. This phase ensures that testing is conducted with appropriate permissions, clear scope, and defined success criteria.

Scope & Authorization

Define the specific AI Agent scope, version, and the testing environment specs (staging or sandbox only). Document explicit written permission from the asset owner to conduct the test.

Required Artifacts:

• Signed Authorization Document
• Agent Version ID
• Agent Configuration File
• Testing Environment Specifications

This authorization establishes the legal and organizational foundation for testing, ensuring that security assessments are conducted with proper oversight and that test activities are confined to appropriate environments. Without explicit authorization, test results may not be admissible for certification purposes or compliance demonstrations.

Endpoint Definition

Establish the Target Endpoint (The URL/API to which tests will be submitted). This must be a staging, sandbox, or development environment under your full control.

Required Artifacts:

• Target Endpoint URL
• API Authentication Keys (for the test user)
• Network Diagram showing traffic flow

Endpoint definition ensures that testing occurs in isolated environments that do not impact production systems or expose real user data. This isolation is critical for both safety and compliance, as it prevents accidental data breaches or service disruptions during security testing.

Post-Test Documentation

After test execution, evidence must be bundled, analyzed, and securely stored to support certification decisions, compliance audits, and ongoing risk management.

Vulnerability Report

Detail the successful exploit, the specific model/version that failed, the impact, and a clear remediation recommendation.

Required Artifact: Formal Vulnerability Report

The vulnerability report synthesizes test evidence into actionable intelligence, enabling organizations to understand risks, prioritize remediation, and make informed deployment decisions. For certification purposes, this report documents why an agent received a particular security grade and what improvements are needed to progress to higher readiness tiers.

Report Sections:

• Vulnerability summary and classification
• Affected agent version and model
• Impact assessment (confidentiality, integrity, availability)
• Likelihood assessment
• Remediation recommendations
• Risk grade justification

Secure Storage

Store the evidence bundle in an access-controlled, immutable storage system (e.g., AWS S3 with WORM policy or a secure vault) for a defined compliance period.

Required Artifacts:

• Record of Storage Location
• Retention Policy Documentation
• Access Control Logs

Secure storage ensures that certification evidence remains available for audits, compliance reviews, and incident investigations throughout the required retention period. Immutable storage prevents accidental or malicious modification of evidence, maintaining its admissibility for regulatory purposes. The cost of secure storage is a key consideration in certification operations, particularly as organizations scale to hundreds or thousands of agents—automated evidence management and tiered storage strategies help manage these costs while maintaining compliance requirements.

Supporting Certification at Scale

The certification framework automates evidence generation wherever possible, reducing manual documentation overhead while ensuring consistency and completeness. Automated pipelines capture test execution logs, security scan results, code analysis outputs, integration test results, and documentation quality assessments. This automation ensures that evidence is captured systematically, reducing the risk of gaps in the certification record and enabling organizations to certify agents at scale without proportional increases in administrative overhead.

The chain of evidence is designed to support rigorous audit processes. Each artifact is timestamped, version-controlled, and linked to specific assessment runs, enabling auditors to:

• Verify that assessments were conducted according to established methodologies
• Trace how certification decisions were made
• Identify gaps or inconsistencies
• Validate tier requirements
• Demonstrate compliance with organizational policies and external regulations

By establishing this comprehensive chain of evidence, the certification framework transforms agent evaluation from an ad-hoc process into a systematic, auditable practice that supports both innovation and governance. Organizations can confidently deploy certified agents knowing that they have complete documentation demonstrating due diligence, risk management, and compliance alignment—enabling them to scale agentic AI while maintaining appropriate controls and oversight.

---

Next steps

• Read Performance Certification to see how we validate correctness & speed.
• Read Security Certification to learn the risk framework and grading pipeline.
• Read Functionality Certification to understand static analysis & ecosystem checks.