ADR-009: WORM Storage for Evidence Retention¶

Status¶

Deferred (January 2026)

Note: This ADR describes the managed WORM storage design. Per ADR-015, managed infrastructure is deferred to Phase 3. Phase 1 uses BYOS (Bring Your Own Storage) where users configure their own S3-compatible storage with Object Lock.

This design remains valid for when/if managed storage is implemented.

Context¶

Assay Evidence Bundles require immutable, tamper-evident storage for compliance with: - SEC Rule 17a-4: Broker-dealer recordkeeping requirements - CFTC/FINRA: Financial services compliance

The EU AI Act is not on that list. Article 12 asks that a high-risk system technically allow the automatic recording of events over its lifetime, and Articles 19(1) and 26(6) require providers and deployers to keep the logs under their control for a period appropriate to the intended purpose, of at least six months unless other Union or national law provides otherwise. None of them requires immutable or WORM storage; keeping logs in WORM storage is one way an operator may choose to hold them, not something the Act asks for (see the retention note under the Article 12 mapping below).

The Evidence Store MVP needs WORM (Write Once Read Many) storage to provide: 1. Immutability guarantees for audit trails 2. Regulatory compliance certification 3. Legal hold capabilities for investigations

Decision¶

We will use Amazon S3 Object Lock in Compliance Mode as the primary WORM storage backend.

Architecture¶

┌─────────────────────────────────────────────────────────────────┐
│                    Evidence Store Ingest API                     │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                      S3 Bucket Configuration                     │
│                                                                  │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │ Object Lock: ENABLED                                     │   │
│  │ Mode: COMPLIANCE (cannot be overridden by any user)      │   │
│  │ Default Retention: 90 days                               │   │
│  │ Versioning: ENABLED (required for Object Lock)           │   │
│  └─────────────────────────────────────────────────────────┘   │
│                                                                  │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │ Object Key Schema:                                       │   │
│  │ /{tenant_id}/bundles/{run_id}/{bundle_id}.tar.gz        │   │
│  │                                                          │   │
│  │ Metadata:                                                │   │
│  │ - x-amz-meta-run-id: {run_id}                           │   │
│  │ - x-amz-meta-bundle-id: {bundle_id} (sha256)            │   │
│  │ - x-amz-meta-tenant-id: {tenant_id}                     │   │
│  │ - x-amz-meta-ingested-at: {ISO8601 timestamp}           │   │
│  └─────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Retention Tiers¶

S3 Configuration¶

# CloudFormation / Terraform equivalent
EvidenceBucket:
  Type: AWS::S3::Bucket
  Properties:
    BucketName: assay-evidence-store-${Environment}
    VersioningConfiguration:
      Status: Enabled
    ObjectLockEnabled: true
    ObjectLockConfiguration:
      ObjectLockEnabled: Enabled
      Rule:
        DefaultRetention:
          Mode: COMPLIANCE
          Days: 90
    PublicAccessBlockConfiguration:
      BlockPublicAcls: true
      BlockPublicPolicy: true
      IgnorePublicAcls: true
      RestrictPublicBuckets: true
    BucketEncryption:
      ServerSideEncryptionConfiguration:
        - ServerSideEncryptionByDefault:
            SSEAlgorithm: aws:kms
            KMSMasterKeyID: !Ref EvidenceKMSKey

Legal Hold API¶

PUT /v1/bundles/{bundle_id}/legal-hold
Authorization: Bearer {token}
Content-Type: application/json

{
  "enabled": true,
  "reason": "Investigation case #12345",
  "requested_by": "legal@example.com"
}

Delete Semantics¶

API Behavior:

DELETE /v1/bundles/{bundle_id}

Important: In Compliance mode, even AWS root users cannot delete objects before retention expires. This is a regulatory requirement.

Alternatives Considered¶

1. AWS QLDB (Quantum Ledger Database)¶

Pros: - Native Merkle tree verification - Built-in cryptographic digest - SQL-like query interface

Cons: - Higher cost ($0.65/million requests vs $0.005/1000 requests for S3) - Limited ecosystem integration - AWS-only (no multi-cloud)

Decision: Not selected. S3 Object Lock provides sufficient guarantees at lower cost.

2. Azure Immutable Blob Storage¶

Pros: - Similar compliance certifications - Multi-region replication

Cons: - Vendor lock-in if we start with AWS - Different API semantics

Decision: Consider as future multi-cloud option.

3. Custom Merkle Chain on PostgreSQL¶

Pros: - Full control over verification logic - No cloud dependency

Cons: - Must build and certify compliance ourselves - Operational burden - No independent audit certification

Decision: Not selected. Regulatory certification is a hard requirement.

Compliance Certifications¶

S3 Object Lock has been independently assessed by Cohasset Associates for:

AWS provides contractual addenda for these requirements.

EU AI Act Article 12 Mapping¶

Note (retention, corrected 2026-09-24): Article 12 is a design duty on the provider: the system must technically allow automatic recording of events. Retention is a separate duty, set by Article 19(1) for the provider and Article 26(6) for the deployer, in each case for logs under their control, for a period appropriate to the intended purpose of at least six months unless other Union or national law provides otherwise. Financial institutions subject to internal governance requirements under Union financial services law keep them as part of the documentation kept under that law (Articles 19(2) and 26(6), second subparagraph). ADR-013 already records the provider half ("Log retention is governed by Article 19, not Article 12"). An earlier version of this ADR listed Article 12 as a reason for WORM storage in its Context and labelled the 90-day Standard tier "EU AI Act Article 12"; both were wrong on the article, and the tier also on the period. Other compliance wording in this ADR (for example "Regulatory compliance certification") predates ADR-042 and is not corrected here.

Note: Draft standard prEN ISO/IEC 24970 (AI System Logging) is expected in 2026 and may require adjustments.

Implementation Plan¶

Phase 1: MVP (Week 1-2)¶

• Create S3 bucket with Object Lock enabled
• Implement basic PUT endpoint for bundle upload
• Add retention period header handling
• Deploy to staging environment

Phase 2: Compliance (Week 3-4)¶

• Add legal hold API endpoints
• Implement retention tier selection
• Create compliance audit logs (CloudTrail)
• Document SEC 17a-4 procedures

Phase 3: Multi-Region (Q3)¶

• Cross-region replication for disaster recovery
• Multi-cloud support (Azure Immutable Blob)

Acceptance Criteria¶

• Bundles cannot be deleted before retention period expires
• Legal hold prevents deletion indefinitely
• All operations logged to CloudTrail
• Encryption at rest (KMS) and in transit (TLS 1.3)
• Cohasset compliance letter available for customers

Consequences¶

Positive¶

• Regulatory compliance out-of-the-box
• No custom verification logic needed
• Independent certification (Cohasset)
• Cost-effective ($0.023/GB/month for S3 Standard)

Negative¶

• AWS lock-in for initial implementation
• Cannot truly delete data during retention (even if requested)
• Storage costs accumulate over retention period

Neutral¶

• Versioning required (minor storage overhead)
• Must handle "object already exists" for idempotency

References¶

• AWS S3 Object Lock Documentation
• SEC Rule 17a-4 AWS Compliance
• EU AI Act Article 12
• Cohasset Associates Assessment
• Draft prEN ISO/IEC 24970 (AI System Logging)