Why AI Data Governance Is a Moving Target (And Where Most Policies Break)

Over the last year, organizations have encountered an intensifying wave of regulatory pressure, inquisitive shareholders, and heightened client demands regarding the oversight and governance of their AI implementations. Because it’s become increasingly clear that in order to maximize your organization’s gain from using AI, you need to give it access to the same valuable data you’re most obligated to protect. 

This is the bind that every CISO, CIO and Executive Team is now in. Business productivity goals demand AI agents integrated into all of your valuable enterprise data and resources. Compliance, privacy, and risk want to know what the agent touched, who authorized it, where the data went, and how to prove it after the fact. 

As a result, most enterprises now have some version of an “approved AI tools” policy—and many of them are either unrealistic or already out of date. 

The reasons behind the increasing regulatory pressure

Traditional software applications and IT systems were adopted and secured by the IT team, which structured frameworks for how, when, and where data flowed. Even innovations like Cloud Computing are often an IT / engineering aligned adoption strategy.  Today, AI adoption is not just focused on engineers. Everyone within the organization can access and utilize AI tools, many of whom are knowledge workers and may not understand the underlying tech and how or where the data is used. 

The uncertainty in how and where AI tools (Chat, Co-pilots, Agents or AI Applications) leverage an organization’s data is a real problem, especially when you consider that data is one of their most important and valuable resources. It’s an even bigger challenge in industries with strong regulator, compliance and brand equity risks. For many enterprise organizations, data privacy and data use regulations and restrictions – like Sarbanes-Oxley, HIPAA, and GDPR – could result in stiff penalties for organizations where AI leaks personally identifiable information (PII) or other sensitive data. 

Organizations in these industries, including financial services, healthcare, insurance, and even government agencies, might think they have their data locked down by simply denying access to AI tools or by implementing acceptable use policies for AI solutions. But history shows that restricting the use of AI solutions or simply putting policies on the books for how and when AI tools can and should be used isn’t enough to safeguard an organization’s data. 

The two core problems with AI governance 

Two problems remain largely unaddressed, and they're the ones that matter most going forward. The first is shadow AI. Employees use personal accounts on personal devices, or free-tier tools that train on inputs, and no network-level control catches this reliably. 

The second, and the one that matters more as adoption increases, is that nearly every control built so far focuses on the input side, stopping sensitive data from going in. Almost none of them address the output side — an AI system retrieving something it shouldn't surface, or an autonomous agent taking an action based on data it should never have touched in the first place. As agentic workflows become standard, this shifts from a theoretical risk to an operational one. The question stops being "what did the model see" and becomes "what did the model do with what it saw."

That second gap is architectural, not incidental. Most governance today sits at a single checkpoint, a gateway between the enterprise or user and the model provider. A checkpoint at the exit catches the last step and misses everything that happened before it — the paste into the prompt, the agent pulling records from a connected system, the moment two datasets merge and the combined output should inherit a sensitivity neither had alone.

Legacy Cybersecurity Solutions Won’t Cut It 

Beyond their AI usage policies, enterprises are relying on a patchwork of technical controls that don’t quite fill in all the cracks. Data loss prevention (DLP) systems inspect traffic before it reaches an AI tool, scanning for social security numbers, credit card formats, and other structured patterns. Enterprise AI deployments come with zero-data-retention agreements, so prompts aren't used for training. Prompt and output logging gives security teams an audit trail to review after the fact.

These controls work well for what they were designed to catch: someone pasting a spreadsheet of customer records into a chatbot. They work far less well for the more common case, which is an employee pasting a paragraph of unstructured text, a customer complaint, an internal strategy note, a snippet of proprietary code, that doesn't match any pattern a DLP rule is looking for. Semantic leakage doesn't look like exfiltration until it's too late to undo. 

Why this is where we’ve focused

We built Shield, Holokai’s policy engine, around a different premise: governance has to live at every point data moves through, not just the boundary or endpoint. Detection at the point of use, before anything enters a workflow. Policy decisions driven by the actual text of the regulations an organization carries, not a generic sensitivity label. Remediation that gives users a compliant path forward instead of a dead end. And an audit trail that's a byproduct of normal operation rather than a reconstruction project after something goes wrong.

The distinction we think matters most: a governance layer that can only say no gets routed around. One that can tokenize, mask, or route data appropriately turns "no" into "yes, under these conditions," which is the only version of governance that survives contact with how people actually work.

We go deep on how this works, the multi-layered  detection architecture, the regulation-aware policy engine, and what distributed enforcement looks like across employee chat and agentic workflows, in our white paper, Distributed Governance for Enterprise AI. If you're a CISO, security architect, or compliance lead trying to figure out where your current AI governance stack has blind spots, that’s your next read.