If your SOC 2 Type II audit still runs on screenshots, spreadsheet trackers, and a frantic evidence scramble the week before fieldwork, the fix is soc 2 automation wired directly into your DevSecOps pipeline. Instead of manually gathering proof that controls operated over the audit period, you collect evidence continuously from the systems that already generate it: your CI/CD pipeline, cloud provider, identity provider, and ticketing system. This post walks through how I approach that automation in practice, including the control-to-evidence mapping, the tooling, and the pipeline changes that make continuous compliance a byproduct of how you already ship software.
Why Manual SOC 2 Evidence Collection Breaks Down
SOC 2 Type II differs from Type I in one expensive way: it tests whether controls operated effectively over a period, typically 3 to 12 months. That means a single screenshot of an access-control setting is not enough. You need to show the control held every day across the window.
Manual collection fails at this for predictable reasons:
- Point-in-time proof doesn't prove duration. A screenshot taken in March says nothing about February.
- Humans forget. Quarterly access reviews get skipped, and the gap only surfaces during fieldwork.
- Evidence goes stale. By the time you export logs, retention policies may have already rotated them out.
- Reviewer fatigue. When evidence collection is a person's side quest, quality degrades.
The core insight is that the systems enforcing your controls are also the systems that can prove them. You just have to capture that proof systematically.
Map Controls to Evidence Before You Automate Anything
Automation without a control mapping produces a pile of logs nobody can tie to an audit requirement. Start by mapping each applicable Trust Services Criteria control to a concrete, machine-collectable artifact.
Here is a representative slice of that mapping:
| Control area | Control intent | Automatable evidence |
|---|---|---|
| Logical access (CC6.1) | Access is restricted to authorized users | IdP group membership exports, MFA enforcement reports |
| Change management (CC8.1) | Changes are tested and approved | PR approval records, CI test results, deploy logs |
| Vulnerability management (CC7.1) | Vulnerabilities are identified and remediated | Scanner output, remediation ticket timestamps |
| Backup & recovery (A1.2) | Data is backed up and recoverable | Backup job logs, restore test results |
| Monitoring (CC7.2) | Anomalies are detected | Alert configurations, incident records |
Write this mapping down before touching code. It becomes the specification your automation targets. If you want help building the control catalog itself, our compliance and security capabilities cover the control-design work that precedes automation.
The DevSecOps Approach to SOC 2 Automation
The principle behind devsecops compliance is that compliance evidence should be generated, versioned, and stored by the same pipeline that builds and deploys your software. No parallel manual process.
I organize this into four layers.
1. Enforce Controls in the Pipeline
The strongest evidence is a control that cannot be bypassed. Enforce change-management rules as pipeline gates rather than policies people are asked to follow.
For example, require signed commits, mandatory reviews, and passing security scans before merge. In GitHub Actions:
# .github/workflows/compliance-gate.yml
name: compliance-gate
on:
pull_request:
branches: [main]
jobs:
security-checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run SAST
run: semgrep ci --sarif --output=semgrep.sarif
- name: Dependency scan
run: trivy fs --exit-code 1 --severity HIGH,CRITICAL .
- name: Upload evidence artifact
uses: actions/upload-artifact@v4
with:
name: security-scan-${{ github.sha }}
path: semgrep.sarif
retention-days: 400
Note the retention-days: 400. Evidence must outlive your audit window. A 365-day period plus fieldwork buffer means artifacts need to survive longer than a year.
Pair this with branch protection requiring at least one approval. The approval record itself becomes CC8.1 evidence.
2. Collect Evidence on a Schedule
Some controls are not pipeline events. Access reviews, backup verification, and configuration drift need periodic capture. Run scheduled jobs that export state into immutable storage.
# collect_iam_evidence.py
import boto3, json, datetime
iam = boto3.client("iam")
s3 = boto3.client("s3")
def collect_users_with_mfa():
report = []
for user in iam.list_users()["Users"]:
name = user["UserName"]
mfa = iam.list_mfa_devices(UserName=name)["MFADevices"]
report.append({
"user": name,
"mfa_enabled": len(mfa) > 0,
"created": user["CreateDate"].isoformat(),
})
return report
def store(evidence):
key = f"soc2/cc6.1/mfa/{datetime.date.today().isoformat()}.json"
s3.put_object(
Bucket="compliance-evidence-vault",
Key=key,
Body=json.dumps(evidence, indent=2),
# Object Lock enforces immutability for the retention window
)
if __name__ == "__main__":
store(collect_users_with_mfa())
Store evidence in a bucket with Object Lock in compliance mode so artifacts cannot be altered or deleted before the retention period expires. Auditors care about integrity; an immutable store removes an entire class of questions.
3. Centralize and Index
Scattered evidence is nearly as bad as no evidence. Land everything in one versioned repository or bucket with a consistent path convention keyed to control IDs. A predictable structure like soc2/<control-id>/<evidence-type>/<date> means your auditor can navigate without a translator.
4. Alert on Collection Failures
If an evidence job silently fails, you discover the gap during fieldwork, which is the worst possible time. Treat a missed collection as an incident. Wire job failures into the same alerting path as production outages.
# Fail loudly if evidence collection skips
- name: Verify daily evidence exists
run: |
TODAY=$(date +%F)
if ! aws s3 ls "s3://compliance-evidence-vault/soc2/cc6.1/mfa/$TODAY.json"; then
echo "Missing MFA evidence for $TODAY" >&2
exit 1
fi
Choosing Between Platform Tools and Building Your Own
You have two broad paths for soc 2 evidence collection, and most mature teams use a blend.
Compliance automation platforms connect to your cloud and SaaS accounts and continuously test configured controls. They are fast to stand up and come with pre-built control frameworks. The tradeoff: they cover the integrations they support, and anything bespoke still needs custom collectors.
Custom collectors, like the scripts above, give you complete control over exactly what evidence you capture and how. The tradeoff is maintenance.
My recommendation:
- Use a platform for the broad, well-supported surface area: cloud posture, IdP, HR system, device management.
- Build custom collectors for your proprietary systems and anything the platform maps poorly.
- Keep both feeding the same evidence vault with the same naming convention.
Requirements vary sharply by sector. Regulated environments in financial services or healthcare often need custom collectors to satisfy overlapping frameworks. If that is your situation, our work across regulated industries shows how overlapping control sets get reconciled into a single evidence pipeline.
Making Continuous Compliance Stick
Tooling gets you partway. Sustained continuous compliance depends on a few operational habits.
- Assign control owners. Every automated control needs a human accountable for it, even when collection is automated. Automation proves the control ran; the owner answers why when it didn't.
- Review the evidence, not just the collection. A job that runs successfully but collects the wrong data is a silent failure. Sample your own evidence quarterly.
- Version your control mapping. When a control changes, the mapping and the collector should change together, in the same commit, with a reviewer.
- Dry-run your audit. Before fieldwork, pull evidence for a random date in your window and confirm every control has coverage. This is the single highest-value rehearsal you can run.
The payoff of compliance automation is not just a smoother audit. It is a security posture you can actually trust, because you have daily proof your controls are operating rather than an annual hope that they were.
A Realistic Rollout Sequence
If you are starting from mostly manual processes, sequence the work so you get value early:
- Weeks 1–2: Finalize the control-to-evidence mapping. Stand up the immutable evidence vault.
- Weeks 3–5: Add pipeline gates for change management. These are high-value and relatively self-contained.
- Weeks 6–8: Deploy scheduled collectors for access, backups, and configuration state.
- Weeks 9–10: Add failure alerting and run your first evidence dry-run.
- Ongoing: Quarterly evidence sampling and mapping reviews.
You do not need everything automated before the first benefit lands. Each gate and collector reduces manual burden independently.
FAQ
How long should I retain SOC 2 evidence artifacts?
Retain evidence for at least the full audit period plus a buffer for fieldwork and any follow-up. For a 12-month Type II window, I configure retention for roughly 400 days or more so no artifact rotates out before the auditor reviews it. Use immutable storage with Object Lock to guarantee integrity.
Can I fully automate SOC 2 evidence collection?
Most of it, but not all. Technical controls like change management, access, and vulnerability scanning automate cleanly. Some controls still require human artifacts, such as policy acknowledgments, board oversight records, and vendor risk reviews. Aim to automate the high-frequency technical controls and streamline, rather than eliminate, the human ones.
Do I need a commercial compliance platform, or can I build everything myself?
Either works, and a blend is common. Platforms accelerate coverage of well-supported cloud and SaaS integrations. Custom collectors handle proprietary systems and multi-framework requirements. The key is funneling both into one consistently structured evidence store so auditors see a single source of truth.
How does DevSecOps improve SOC 2 outcomes specifically?
By enforcing controls as pipeline gates, you convert policies people might forget into mechanisms that cannot be bypassed. The pipeline that ships code also produces tamper-resistant, timestamped evidence as a byproduct, which is exactly the operating-effectiveness proof Type II requires.
What is the most common evidence gap auditors find?
Coverage gaps across time. Teams prove a control existed at one moment but cannot show it operated every day of the period. Scheduled collection with failure alerting closes this gap, which is why I treat a missed evidence job as an incident.
