SERIES: “The Death of Compliance Theater” — Part 2 of 3
Part 1 of this series made the case that a passing audit and a secure environment are two different things, answering two different questions. This post is about a specific, well-intentioned attempt to close that gap and why it mostly automated the wrong half of the problem.
Over the past several years, a wave of GRC (governance, risk, and compliance) automation platforms promised to modernize an obviously painful process: manual evidence collection, endless spreadsheets, and audit cycles that could stretch for months. Many of them delivered on that promise, connecting to cloud APIs, automating screenshot collection, and turning what used to be an eight-month audit slog into something closer to a four-week dashboard exercise.
That was real progress, and it deserves credit. But faster is not the same as different. Ask most compliance teams today whether their workload has actually dropped, and the honest answer is: not by nearly as much as they were promised.
Quick answer: GRC automation platforms genuinely improved the speed of collecting compliance evidence. What they mostly didn’t change is the underlying nature of that evidence a configuration check confirming a setting is correctly enabled, not a behavioral check confirming that setting is actually stopping anything right now. Faster point-in-time snapshots are still point-in-time snapshots, and a growing body of survey data suggests compliance teams are working just as hard as before, in some ways harder, despite years of automation investment.
Key takeaways:
Before looking at the gap, it’s worth being fair about the gains, because they were real. GRC automation platforms replaced spreadsheet trackers with structured control libraries. They connected directly to cloud providers, identity platforms, and HR systems, so a large share of routine evidence could be pulled automatically instead of requested by email. They gave auditors a shared workspace instead of a shared drive full of loosely named screenshots. And for fast-growing companies chasing their first SOC 2 or ISO 27001 certification, they turned an intimidating, months-long project into something a small team could actually finish.
None of that is trivial. For many organizations, it was the difference between having a compliance program and not having one. The problem isn’t that these platforms failed at what they set out to do. It’s that what they set out to do make evidence collection faster was only ever half of the problem.
Consider the most common thing a GRC automation tool actually checks: whether multi-factor authentication is enabled for every user in an identity provider. The tool queries the API, confirms the setting, and marks the control satisfied. It does this reliably, quickly, and at scale which is exactly what it was built to do.
What it cannot see is what happened after that setting was correctly configured. An adversary-in-the-middle phishing kit doesn’t need to defeat MFA it sits between the user and the real login page, harvests the session token the moment it’s issued, and replays it. No failed authentication. No MFA bypass alert. The configuration remains, accurately, “enabled.” The session sitting on top of that configuration may already belong to someone else.
This is the configuration-versus-behavior gap, and it’s structural, not a bug any individual GRC vendor forgot to fix. A tool built to verify settings against an API response was never going to be the tool that verifies whether those settings are functioning as intended against a live adversary. Those require fundamentally different data sources one reads configuration state, the other reads security telemetry such as authentication patterns, session activity, network flows, and endpoint events and most compliance automation platforms were built to specialize in the first.
MFA isn’t the only example. A GRC tool can confirm that logging is enabled on every production server, and it will be right. What it typically can’t confirm is that every one of those servers is still forwarding logs to the place anyone actually reviews them, or that nobody noticed when a critical system went quiet three weeks ago. It can confirm that encryption at rest is turned on for a storage bucket, and it will be right again while an attacker with a stolen API key reads the decrypted data through a perfectly legitimate interface. In each case the configuration check is accurate, and in each case it answers a narrower question than the one the business thinks it’s answering. That is how a compliance dashboard can be entirely green while an attacker is active inside the environment.
If your organization serves customers across healthcare, financial services, and Europe, compliance isn’t one project it’s a dozen overlapping ones, each demanding evidence for nearly identical underlying questions in slightly different vocabulary. A recent industry survey found that more than 88% of organizations maintain multiple compliance frameworks simultaneously, and despite years of automation tooling, 63.2% still manually remap the same evidence between frameworks, while another 25.9% re-document evidence separately for each one rather than reusing it at all.
Access control revocation, for instance, gets asked about in slightly different phrasing under SOC 2, ISO 27001, HIPAA, and half a dozen other frameworks a mid-sized enterprise might carry simultaneously. The underlying fact is the same every time: when someone leaves, do they lose access promptly? Yet in most organizations, someone still translates that one fact into each framework’s language, attaches it to each framework’s control, and tracks each one separately.
Automating the collection of the underlying evidence didn’t automatically solve the harder problem of mapping one piece of evidence to a dozen different frameworks’ worth of language, which is exactly the kind of manual translation work that same survey found most organizations are still doing by hand. And that work grows with every new framework a customer or regulator adds.
The same research found that while 80.6% of organizations now say they collect evidence continuously year-round, only 38.3% rely primarily on automated tooling to do it — the rest are running a continuous process with a heavily manual execution, which tends to produce exactly the kind of dedicated, high-effort review cycle GRC automation was supposed to eliminate. Roughly 43% of practitioners’ time on evidence work goes to detection and collection alone, before any of the harder work of validating findings or demonstrating that a fix actually held.
The single biggest audit bottleneck in that research, cited by 50.7% of practitioners, wasn’t documentation or policy at all. It was getting time from technical teams.
There’s a cultural cost to this pattern that’s easy to underweight. When engineers and IT administrators get pulled off product work to hunt down the same screenshots and policy acknowledgments they produced last quarter — even with a slicker dashboard collecting them — compliance keeps getting experienced as an administrative tax rather than a meaningful signal about the organization’s actual risk posture. Over time, that perception shapes behavior: teams do the minimum the audit requires, and the distance between documented controls and enforced controls quietly widens.
Perhaps the most concerning finding in recent survey data isn’t about effort at all it’s about belief. Just over half of practitioners surveyed, 51.2%, said they believe a certification reflects an organization’s security posture on an ongoing basis. Only 41.8% correctly recognized that a certification is really only accurate as of the moment the assessment concluded.
That’s a genuinely risky misunderstanding to have at scale, inside the very teams responsible for managing an organization’s risk. If half of the people closest to a compliance program believe a nine-month-old SOC 2 report still describes today’s environment, the gap between “compliant” and “secure” that Part 1 of this series described isn’t just a structural problem with audit methodology it’s actively being reinforced by how the people running the program understand their own tools.
The same misunderstanding tends to travel upward. Boards see a current certification and read it as current security. Customers see a trust page and assume continuous assurance. Insurers see a clean report and price risk accordingly. Every one of them is looking at a snapshot and treating it like a live feed.
If environments stood still, a quarterly configuration snapshot might be close enough. They don’t. Cloud resources are created and destroyed in minutes by automated pipelines. SaaS applications are adopted by business teams long before security reviews them. Non-human identities service accounts, API keys, automation tokens, and increasingly AI agents now carry their own permissions that drift over time, often without any human reviewing them between audits.
Every one of those changes is a moment where the configuration verified last month stops describing the configuration that exists today. A snapshot-based approach can only catch that drift at the next snapshot. Meanwhile, regulators are tightening the timelines they expect organizations to meet: the SEC requires public companies to disclose material cybersecurity incidents within four business days of determining materiality, and the EU’s NIS2 directive expects an early warning on significant incidents within 24 hours. Both assume an organization knows, in close to real time, what is actually happening in its environment which is exactly the question point-in-time evidence was never designed to answer.
Put plainly: GRC automation platforms got very good at answering “can we produce this evidence faster.” They mostly didn’t change what counts as evidence in the first place a configuration snapshot rather than a behavioral one — and they didn’t resolve the deeper structural cost of maintaining a dozen frameworks that ask the same questions in different words.
| Compliance problem | Did GRC automation fix it? | What still remains |
|---|---|---|
| Slow, manual evidence collection | Largely yes | Collection is faster through API connections |
| Configuration vs. behavior | No | Checks confirm a setting exists, not that it works against a live attack |
| Multi-framework mapping | Partly | 63.2% still remap evidence by hand; 25.9% re-document it per framework |
| The annual audit scramble | Partly | “Continuous” programs still depend on heavily manual execution |
| Certification overconfidence | No | Many practitioners treat a point-in-time report as ongoing proof |
That suggests a simple test for any compliance automation investment. Does the evidence come from a configuration snapshot, or from live telemetry that reflects actual behavior? Is each piece of evidence remapped by hand for every framework, or mapped once? Does the work still pile up before each audit, or happen continuously in the background? And is the vendor honest about which parts of a control its automation actually covers?
Answering those questions well requires a different kind of evidence pipeline. Instead of asking an API what a setting is, it asks the security telemetry what the control is doing. Instead of storing evidence per framework, it stores it once against the underlying requirement and lets every framework that asks the same question reuse it. And instead of treating compliance as a separate workflow that borrows engineering time before each audit, it treats compliance evidence as something the security program is already producing every day.
That’s the gap Part 3 of this series is about: what changes when compliance evidence is generated as a byproduct of live security operations telemetry — the same data answering “are we compromised right now” rather than as a parallel, API-driven snapshot process running alongside it. It looks at how Seceon aiCMX360 (aiCompliance CMX360™) takes that approach on the Seceon Open Threat Management (OTM) platform.
GRC automation tools mainly verify configuration state through cloud and SaaS APIs. They speed up evidence collection, but they can't confirm that a control is actually working against a live attack, and most still leave cross-framework evidence mapping to people.
They reduced the time needed to collect certain kinds of evidence, but survey data suggests overall workload hasn't dropped as much as the automation promise implied. In one recent industry survey, 80.6% of organizations said they collect evidence continuously, yet only 38.3% rely primarily on automated tooling meaning most of that "continuous" process still depends on significant manual effort.
Because a configuration check and a behavioral check answer different questions. Confirming multi-factor authentication is enabled tells you the setting exists; it doesn't tell you whether an attacker has found a way around it for example, by stealing an already-authenticated session token, which doesn't trigger a new MFA prompt at all.
Because most frameworks ask overlapping questions using different terminology, and most organizations still translate evidence between them manually. Industry survey data shows more than 88% of organizations maintain multiple frameworks simultaneously, and 63.2% manually remap the same underlying evidence across them rather than mapping it once.
No — a certification reflects a point-in-time assessment. Notably, survey data shows a meaningful split in understanding this: just over half of practitioners believe certification reflects posture on an ongoing basis, while a large minority correctly understand it's only accurate as of the assessment date. Treating a certification as an ongoing guarantee is a common but risky misunderstanding.
Continuous compliance monitoring verifies controls all the time using live operational data, instead of proving them once a year with screenshots and sampled evidence. It asks whether a control is working right now, not whether it passed when an auditor last looked.
Continuous compliance built on live security telemetry rather than configuration snapshots. Seceon aiCMX360, part of the Seceon Open Threat Management (OTM) platform, generates compliance evidence from real detections and responses and maps it once across 40+ regulatory frameworks.
This is Part 2 of a 3-part series on continuous compliance and security assurance. Read Part 1: Compliant and Breached, or continue to Part 3: How Seceon aiCMX360 Makes Compliance Continuous.
