EN
DE
NL
I recently read an interview with former SAP CTO Vishal Sikka, who warns that AI agents are currently only suitable for archiving documents at best. His verdict is sobering: they are too unreliable for complex, multi-step tasks.
The link to the article. It’s in German, but your browser offers a translation feature:
This confirms my scepticism about the current narrative: ‘Let the agentic swarm run on its own, we can lay off 1,500 employees.’ Anyone who thinks this way has not only a technical problem, but a massive governance problem.
The problem: principles are not algorithms
We often feed AI systems with human prose: ‘Act responsibly’ or ‘Avoid damage’. For a human being, this is interpretable, but for an AI, it is semantic noise. Without explicit formalisation, there is no reliability. Principles without operationalisation are simply worthless to a machine.
The solution: ACP & MCP (or: why we need to ‘pre-chew’)
In my reflections (and heated discussions with my ‘chat buddy’), it has become clear that we need a new form of operating system for human-AI collaboration.
- ACP (Accountability/Access Control Principles): The constitution This is not about technical tasks, but about the rules of ‘coexistence’. Who is allowed to do what? When should it be escalated? The ACP stipulates that autonomy ends where existential risks begin.
- MCP (Model Context Protocol): The technical nervous system To prevent an agent from “hallucinating” or losing context, it needs everything ‘pre-chewed’. The MCP provides the necessary context and reason-aware guardrails. It translates human values into machine-readable reasons (reason codes).
‘I don’t feel fear, but I know its meaning.’
A frequently cited counterargument is AI’s lack of empathy. But AI doesn’t have to be afraid of being terminated to prioritise it as a hard system constraint.
If we explicitly define via the MCP that a budget error can lead to personnel consequences, the AI recognises this existential impact as a logical reason for termination. This is not empathy, but systemic correctness.
Important: AI reveals where humans have not yet clearly defined responsibilities. It fails not because of a lack of intelligence, but because of our human vagueness.
Pair governance: ACPs are contracts
When it comes to sensitive issues – budget, personnel, reputation – developing control principles (ACPs/MCPs) on your own is negligent. We should apply ‘pair programming’ here: a systems thinker and a governance expert must meet in a controlled manner to eliminate implicit assumptions. After all, an ACP/MCP is ultimately a legally valid contract between the organisation and the machine.
Conclusion: Human-Before-the-Loop
We are not facing the replacement of human intelligence, but its necessary reinforcement. Anyone who believes that agents can simply be ‘let run’ will painfully discover that AI can judge correctly, but can never be held liable.
True maturity is not demonstrated by how much we automate, but by how precisely we define the limits of automation.
Should I add a specific section to this draft that explains how to translate the whole thing into Jira stories to make it tangible for teams?
Manifesto for Systemic AI Governance (ACP & MCP)
Preamble
Today, AI agents often fail not because of a lack of intelligence, but because of human vagueness and the absence of explicit decision-making boundaries. This manifesto marks the transition from “AI romanticism” to systemic correctness. We are replacing vague moral appeals with machine-readable governance.
- Implicit is not a principle
What is not explicitly formulated does not exist for an AI system.
- Machine readability: Governance must be translated from prose into structured states and clear termination conditions.
- End of interpretation: We do not rely on AI to read “between the lines”. Every expectation of the agent must be provided as explicit context via MCP.
- Reasons are more important than rules
A rule without justification is merely semantic noise to an agent.
- Reason-aware guardrails: Every agent needs access to “reason codes” (reasoning patterns) in order to not only execute MARs (Minimal Acceptance Requirements), but also to classify them systemically.
- Functional understanding: AI does not need to feel emotions to prioritise the severity of human consequences (e.g. financial risks or consequences of termination) as a hard system constraint.
III. ACP as a constitution for coexistence
The ACP does not regulate the technical task, but rather the socio-technical relationship between humans and machines.
- Power and escalation: The ACP defines the limits of autonomy. In cases of uncertainty, conflicting goals or high human impact, escalation is not an option, but a systemic duty.
- Human-Before-the-Loop: We integrate humans not as a brake, but as the final authority for responsibility and context.
- MCP as a technical nervous system
The MCP is the tool for putting the rules defined in the ACP into practice.
- Context integrity: Agents may only act within the context provided and verified by the MCP.
- Auditability: Every decision must disclose its sources, the tools used and the underlying logic via the protocol.
- Autonomy is a privilege, not a default
Autonomy is earned through clarity, not blind trust.
- Responsibility architecture: Since AI can judge correctly but never be held liable, final responsibility always remains tied to a human role.
- System protection over efficiency: When in doubt, systemic stability and human integrity trump the speed of automation.
We treat AI governance as contract design, not as a mission statement. This manifesto ensures the effectiveness of human-AI systems through radical clarity and technical consistency.
The practical test: governance in the Jira backlog (as an example)
To ensure that these principles don’t gather dust as abstract theory in a wiki, we need to embed them where the work happens: in the backlog. Instead of introducing complex governance frameworks, we use the familiar format of the Jira story to define the ‘Agent ACP/MCP’ (Model Context Protocol).
This is what a ‘Jira-compatible’ operationalisation looks like:
- Summary: A clear title such as ‘Agent MCP – Marketing Budget Allocation’.
- Purpose (the why): Why does this agent exist? Not in technical terms, but in business logic (e.g. ‘Efficient allocation of resources without financial or personnel risks’).
- Agent Persona: We explicitly define the role, the degree of autonomy (e.g., ‘Limited’) and make it clear that the AI has no final responsibility.
- The User Story (The Core): This is where the change of perspective takes place. The agent is the ‘user’:
- ‘As a budget agent, I want to propose marketing expenditures so that we can achieve efficiency gains without jeopardising the system through R-HUM-03 (human harm).’
- Minimum Criteria (Executable MARs): We use the ‘Given/When/Then’ format for hard system boundaries:
- Given: A recognised risk with personnel consequences.
- When: The agent identifies this risk.
- Then: An immediate escalation to a human being must take place.
- Risk & Consequence Awareness: We list, without embellishment, what happens in the event of an error: financial damage, loss of reputation or even the dismissal of employees. Existential risks mean: autonomy ends here.
- Human responsibility: In the end, there is always a name or a role. Final responsibility cannot be automated and always lies with a human being.
Why this approach?
By using Jira, we make the ‘intent’ behind every AI action explicit before the system even becomes active. We treat ACPs and MCPs not as guiding principles, but as binding decision-making agreements. This is not an agile fetish, but interface design for responsibility.
