Safely Steering Agentic AI: How to Rely on Zero Trust and Compliance

AI agents/Agentic AI is revolutionizing the business world: from automation and data analysis to autonomous decision-making. But with the speed at which AI agents are being introduced, the risk is also growing: Who is actually controlling what these agents do? And how can companies ensure that AI does not become a compliance or security risk? In this article, you will learn how the principle of "In Dubio Pro Securitate" and modern governance approaches help to deploy AI agents safely and compliantly.

Wie entsteht die Sicherheitslücke von KI-Agenten?

Many companies deploy AI agents without truly knowing their access rights, behavior, or risks. Common mistakes:

• AI agents are granted excessively broad access rights ("least privilege" is ignored).
• There is no continuous monitoring or audit trails.
• Compliance requirements (e.g., ISO 27001, SOC 2) are not applied to AI agents.
• Security incidents are piling up: chatbots conclude contracts at absurd prices, agents install keyloggers, or leak internal data.
• Shadow AI arises, similar to Shadow IT, because nobody really has an overview of the agent fleet.

Agents must fundamentally be treated as insecure until their security is formally proven – because the cost of an allowed, but insecure AI action is far higher than blocking a useful, but unproven action.

Erik Meijer, speaker at the Enterprise AI Summit 2026 and internationally recognized computer scientist, software architect, and innovator in the field of programming languages and formal methods

Von Unschuldsvermutung zu Sicherheitsvermutung: 3 Sicherheitsprinzipien für Agentic AI

1. in dubio pro securitate

In the traditional legal system, the maxim is: "When in doubt, in favor of the accused." For AI agents, this principle must be reversed: When in doubt, in favor of security ("In Dubio Pro Securitate"). This means that every action of an AI agent is initially considered potentially insecure until the opposite is proven.

Why is this necessary?

• AI agents can make far-reaching decisions in a fraction of a second (e.g., conclude contracts, send data, configure systems).
• A single mistake can cause enormous financial, legal, or reputational damage.
• The speed and complexity of AI systems exceed human control capabilities.

2. deferral and formal verification as a protection mechanism

Deferral means that an AI agent does not act directly, but initially only generates a suggestion (plan, code, action). This suggestion is then checked before execution.

Formal verification is a mathematical procedure that checks whether the agent's proposal fulfills all defined rules and security requirements. This is done, for example, with so-called SAT solvers or SMT solvers, which automatically search for counterexamples.

Example: An AI agent proposes to sign a contract. Formal verification checks whether all contract terms, price floors and compliance requirements are met. If a rule is violated, the proposal is rejected and the agent must generate a new plan.

3. separation of evidence finding and evidence examination

A central principle for safe AI agents is the clear separation between Finding evidence and examination of evidence.

• Finding evidence: The AI model creates a proposal (e.g. code, plan, action) that is intended to fulfill the desired requirements. The model can act creatively and flexibly here, but errors or rule violations are possible.
• Evidence testing: An independent, deterministic system then checks whether the proposal really complies with all safety and compliance rules. The agent is only allowed to act if the check is passed.

Why is this important?

• Avoidance of „blind trust“:
  AI models are powerful, but not infallible. They can make mistakes, overlook security gaps or even - intentionally or unintentionally - circumvent rules. If the same system that makes the suggestion also decides on its security, there is a high risk of errors or manipulation.
• Independence and traceability:
  The proof check is independent of the AI model and works deterministically. This means that there is always the same test result for the same proposal. This creates transparency and traceability - important prerequisites for audits, compliance and trust.
• Efficiency through repetition:
  As the generation of suggestions by AI agents is very inexpensive, unsafe or incorrect suggestions can simply be discarded. The agent keeps generating new suggestions until one passes the test. This increases security without slowing down the speed of innovation.

Practical example: An AI agent proposes a code change. The code is only executed if a formal check system confirms that no compliance rules have been violated. Otherwise, a new proposal is generated.

This separation ensures that AI agents act innovatively, but never in an uncontrolled or unsafe manner. A must for any company that relies on AI.

Concrete steps towards secure AI agent governance

Zero Trust and Least Privilege for AI agents

A central principle of modern IT security is the Zero Trust Model. Applied to AI agents, this means that no agent automatically receives trust or far-reaching rights. Every access and every action of an agent must be explicitly permitted and traceable at all times.

The principle of the „Least Privilege“ complements this approach by giving each agent only the minimum rights it really needs for its task. This prevents an agent - such as a reporting agent - from accessing sensitive customer data or administration functions if it does not need them for its work.

In practice, all agent identities are managed centrally, for example with DevSecOps platforms such as Container8. The authorizations are regularly checked and adjusted to ensure that no unnecessary rights remain. If there is any suspicion of misuse or misconduct, the rights of an agent can be revoked immediately to prevent damage.

Operationalize formal verification

To further increase the safety of AI agents, each plan they propose undergoes formal verification before execution. This means that the proposal is checked against all relevant rules and compliance requirements using mathematical methods and automated tools such as policy-as-code or SAT solvers. Only if the plan passes this check and no rule is violated may the agent actually carry out the action.

An illustrative example is an AI agent that wants to trigger a payment: the verification checks whether the amount is within the permitted limits, all compliance requirements are met and there are no data leaks. Incorrect or risky actions are thus detected at an early stage and blocked before they can cause any damage.

Compliance „Shift Left“ - Governance as a code

Traditionally, compliance checks often only take place at the end of a development process. In the age of AI and DevOps, this is too late.

With the „Shift Left“ approach, compliance checks are integrated into the development process at an early stage and, ideally, automatically. This is achieved through „policy-as-code“: rules and guidelines are formulated as code and integrated directly into the CI/CD pipeline. Developers and agents receive immediate feedback as to whether their changes are compliant. This significantly reduces the effort required for manual audits and accelerates the market launch of new functions without jeopardizing security.

Continuous monitoring and audit trails

Another important component is the continuous monitoring of all agent activities. Every action of an AI agent is seamlessly logged, so that it can be traced at any time who did what and when.

Modern monitoring systems analyze these activities in real time and recognize unusual or unauthorized behavior immediately. For example, if an agent attempts to access data that is outside its area of responsibility, the system sounds an alarm and automatically blocks the action.

Central dashboards provide a complete overview of all agent activities and enable rapid escalation in the event of policy violations. This transparency is not only essential for security, but also for audits and compliance checks.

Peer review and manual control of critical actions

Automation alone is not enough, especially for particularly sensitive or financially critical actions. This is where the dual control principle comes into play: at least two people or systems must approve an action before it is carried out. This provides additional security and prevents a single agent - or a single person - from making critical decisions alone.

In practice, threshold values are defined above which a peer review is mandatory, for example for particularly high amounts or when accessing particularly sensitive data. This creates a balanced combination of automatic and manual control that ensures both efficiency and security.

Summary

• AI agents are a massive lever - but also a massive risk if governance is lacking.
• The „In Dubio Pro Securitate“ principle protects companies from costly mistakes and compliance violations.
• Formal verification, zero trust and policy-as-code are the keys to the secure operationalization of AI agents.
• Continuous monitoring and audit trails create transparency and traceability.
• Companies that act now will secure a decisive competitive advantage.

Do you want to know how to use AI agents in your company safely and compliantly?
Get in touch with our XALT AI experts: We analyze your agent landscape, identify compliance gaps and show you the way to zero trust governance.