Traditional software teams in larger enterprise projects often followed a pyramid organization: a few seniors in responsible roles on top, and maybe 2/3 mid-level and junior engineers. The press has written a lot recently about how AI is reshaping the entry-level job market. In the last few days, there was an announcement that Microsoft plans severance offers for 60.000 employees, mostly in the technical parts of their business worldwide.

Today I want to look at what others say about what modern engineering teams look like. I also want to tackle the uncomfortable question: When AI replaces entry-level jobs, how do we get the seniors needed to run the agents?

Effects seen today

Quite a few articles and studies have been published recently from players like Bain & Company, Gartner, or McKinsey.

They largely come to similar conclusions.

Team Size Compression

McKinsey say that in top-accelerating organizations (in terms of agentic coding), 79% of the organizations they interviewed reported that squad size decreased. The median of the survey was a reduction from about 10 to 7 members. Bain reported that in their survey teams shrunk to “pods of 3-5 people”, with routine execution roles disappearing.

Gartner thinks the total number of teams will increase to meet growing business demand, but with what they call “tiny teams.”

The Seniority Inversion

Bain see the traditional pyramid in organizations flattening rapidly: “Fewer engineering organizations report traditional pyramid structures, down from 66% to 29%. Fluency with AI tools is now the leading capability sought out in engineers; cited by 62% of survey respondents”.

The effects of this is obvious: When fewer entry-level jobs are created, how to find or train the seniors the companies desperately need?

“Companies need more senior judgment than ever, yet they are quietly dismantling the path that produced it.” — Bain, 2026

The routine coding, debugging, and testing tasks that junior engineers historically learned on are precisely the tasks agents now handle. Without deliberate intervention, the supply of future senior engineers is threatened.

The junior talent pipeline crisis might even influence whether young talent decide for computer science at university, enforcing the shortage even more.

Emerging and Evolving Roles

Companies Forrester or Thoughtworks describe that new roles appear in organizations:

• AI Engineer / Agent Reliability Engineer: Manages agent tooling, configures coding assistants, builds agent workflows, maintains evals that catch agent-generated bugs

• Knowledge Architect: Designs and maintains the shared knowledge layer (memory, context) that agents consume

• Agentic Architect: Designs the multi-agent system architecture and orchestration patterns

• AI Conductor / Orchestrator: Guides agents, composes workflows, enforces guardrails — the “manager of AI agents”

• Evaluation/Governance Engineer: Builds policy-as-code, audit harnesses, and compliance validation for agent outputs

Traditional roles like developer, UX designer will be shifted towards directing agents. In general, the expectation is that job boundaries start blurring significantly and team members take on a broader range of tasks.

Bottleneck Migration

While traditional projects very often had code production as their bottleneck, other roles will be in focus: Product Managers will much more rapidly have to decide on the product direction and design, while quality assurance and code reviews need to keep up with the new development speed. As more products go into production, decision-making and accountability will be the next challenges to solve.

Facing the talent pipeline crisis

So, what’s the best way to find seniors for the new age of agentic coding? Obviously, you can try to recruit on the free market. I expect seniors who meet modern skill requirements will see their salary expectations go through the roof for good reasons. They will be key players in modern teams and key to the velocity multiples companies expect.

At my former company, we recruited working students early in their university careers and taught them our technology stacks. But it still comes down to how you select talent and what opportunities you give them to learn in real-world situations. We were a smaller company, so we were used to taking on a broader range of responsibilities than employees in large organizations typically do.

Everyone knows one or more young talents who seem to become senior overnight and absorb new technologies like fresh air, while others need intensive training on every new piece of technology.

Of course, a few are natural, usually identified by how they talk about their personal IT projects during interviews. Or the speed and enthusiasm with which they approach a challenge.

But to grow not only those, but also talent who aren’t geniuses, you need an environment to learn. When we asked people what they enjoyed most, they said it was good old “boxes and arrows”: discussing architecture and design problems on a whiteboard with a senior team member. Just to see the train of thought senior developers take. Finding new ideas and aspects that come from years of experience. Showing persistence in complex troubleshooting situations. Learning new tools and how to track down problems at the intersection of infrastructure and code.

And sometimes it’s just as simple as trying to be a good role model for colleagues. In the end, we are all human beings and appreciate mutual respect. As my colleague always said: “Employees work for other people, not for companies.”

But that wasn’t the point of the exercise; the web is full of stories like this. I wanted to know how agentic coding works for large, complex enterprise applications. So I decided to take one of my personal pet projects and turn it into a full-blown SaaS application.

In this article, I want to share my Claude Code journey, from a naïve approach to a one-person-built enterprise application, and its impact on development results, architecture, and token consumption.

What I was Building

My personal pet project is rather simple. It’s a tool to use AI for deep technology research and comparison. I decided to turn it into a pretend SaaS offering. I added features that typically come in larger enterprise applications, so that my project became a web application with enough complexity to go beyond a convincing demo.

So, I added:

• Sign-in, sign-up, subscription management, and an admin portal.

• Billing and invoicing.

• Business logic and data structures for the generation of technology assets.

• Asynchronous background tasks and batch processing.

• A container-based application hosted in Azure.

• Multiple environments, continuous integration and deployment through GitHub Actions.

I wanted to explore what worked, where Claude’s subscription and context window limits became relevant, and what happened as both the codebase and the conversations grew.

My approach: give Claude room, then ask Claude to review

I used a naïve conversational approach with Claude Code. With the most expensive and capable model Fable 5.1 for convincing results, of course. I wanted to use it for both planning and coding. Just to see what comes out. Hey, money is not everything; results matter!

First, I asked Claude to write the specification. This worked smoothly – Fable broke the specification down into phases and slices for the implementation. The specification also defined business terms and entities so Claude would know how things relate to each other.

I gave Claude considerable freedom to choose the architecture and implementation.

Fresh Claude sessions reviewed the work done by other sessions. That separation was deliberate. A fresh session could examine the result without carrying the whole implementation conversation. However, it still depended on the code and documentation available to explain the system.

With my limited bandwidth, I concentrated on AI reviews rather than detailed human code reviews. This made the experiment useful for exploring a management question: how much confidence should we place in a development process where AI performs both activities?

The honeymoon was real

The results of the first slices and phases were really convincing. In no time, there was a feature-rich portal with automatic testing that would have taken weeks to build before AI. For UI design, Fable asked me to provide 3 websites that I liked and one that I didn’t like and made an awesome proposal.

The cloud provisioning worked smoothly. Fable set up the CI/CD pipelines with Bicep in GitHub Actions. It also anticipated advanced requirements and features upfront, like proper monitoring, auditing, and usage reporting, and built the infrastructure to support them.

When runtime errors occurred in the cloud, Claude was particularly impressive at investigating them. It was strong at finding root causes in cloud setup issues. It gave me clear instructions when I had to create external dependencies manually, like some DNS entries.

That success made it easy to keep extending the portal. Each additional feature looked like a manageable next step.

The difficulties emerged through accumulation.

What changed as the project grew

I noticed a few things as the codebase grew larger and larger.

First, Claude has a tendency to become very verbose. It produced a lot of text in chat responses and specifications. The more features I built into the portal, the more this became a challenge. At some point, it just was really overwhelming to follow the chat responses and understand the changes that Fable was proposing to build. With mental overload, there is also a risk that you accept what Fable proposes too quickly; everything sounded very convincing, after all.

Second, I noticed that Fable did not produce the most efficient code. And especially not the most maintainable code. Don’t get me wrong, the quality was still good. I would have created more separate modules and used different naming. The code worked.

But for mission-critical applications, code quality, proper architecture, and testability are essential in the long run. People say AI made code cheap. But for essential systems, the core parts need to be built with high quality in mind to meet future requirements. Cheap code comes back to bite you.

Parts of this might have been caused by inconsistencies in the specification. While the spec seemed crystal clear to me initially, every inconsistency I missed came back to haunt me later. Which is perfectly natural and, one might say, exactly like the past. But Fable’s convincing proposals make it easy to miss things. Suddenly, a feature refactoring leaves remnants across the application. You end up with input elements that are not needed and data that is not used.

Which brings me to the next issue: Because of verbose text and code, and an unclear architecture and module cut, the context window was quickly exhausted for features that affected larger parts of the system. Claude sliced the features in a way that each slice should run in a fresh session, retrieving the current state and tasks from the specification. With all the verbosity built into the application, the context window filled to more than 70 percent in no time, and Claude started recommending a fresh session. Therefore, the weekly Fable limits in the subscription kicked in quite early in the week (which might not be a bad thing to make you pause and think about what you are doing…). Conversations with Claude when you hit the session window limit remind me a lot of the Groundhog Day movie, where Claude says it will soon forget what we talked about today.

What I would take into a larger project

For engineering leaders, I see a few practical lessons.

1) You cannot “agentify” your system out of bad requirements. Continuously work on the quality of the specification. Invest time to get definitions, business terms, and entities right. This will make conversations with the AI much clearer and much more efficient. Consider domain-driven design approaches with the ubiquitous language approach to make sure everyone involved understands what is to be built.

2) Get the application architecture and the modules right. Agentic AI will be much more efficient when modules have clear purpose and interfaces. AI will be able to take care of the implementation inside them, with much more clarity and less verbosity.

3) Move forward in small steps. Tasks too big have a tendency to clutter your codebase or your specification. Do not rely only on AI-driven reviews and testing without control. These can help, but for the critical parts of a system, a human must be in the loop. Consider test-driven development. AI made code cheap, but human reviews are more expensive. Therefore, you’d better make sure pull requests are easy to digest.

My experiment taught me how quickly Claude Code could help build a substantial application. It also exposed how hard it is to maintain consistency and coherence while delegating both development and review.

So, what’s the gist of it? Use agentic development for critical systems, or rather, just rely on AI generating code snippets? Definitely agentic development. But to create high-quality software that lasts, a lot of human dedication on the outcome is still important.

Summary:

• When humans delegate code generation and reviews to agents like Claude Code, Gemini or ChatGPT and only inspect work results in running software or review reports, errors can easily be overlooked - despite good test and review coverage.

• Reassuring AI messages can give a false sense of reliability

The case described in A Case Study in Assuring AI-Written Software demonstrates how a healthcare practice developed their own system for note-taking support, scheduling and payroll workflows, alongside background functions such as access control, backups and audits. The operator set up an elaborate AI workflow, but had no particular software engineering background.

The workflow used by the developer is shown in the following figure. It evolved over time as the experience of the operator grew.

Supervisory agents in separate context windows were used to review code and propose fixes that the operator decided to forward to the worker agent or not. Also, an “agent council” comprised of different models was used to review and provide recommendations and risks identified. The human controller would review any suggestions and work lessons learned and findings into the testing and review workflows.

The developer was very careful with automatic testing and deliberately implemented targeted failures to ensure that the test coverage would detect them.

Over time, external references to security standards like OWASP or HIPAA recommendations and other standards were introduced to the context to improve the quality and security of the software.

But then an interesting phenomenon happened: Some of the evaluations that the AI introduced formed its own kind of erroneous layer. For example, the backup was not working despite testing showing everything is green. “An audit stopped running without producing a failure, while another report calculated its pass rate only from checks that emitted results, allowing missing checks to disappear from the total.”

It turned out that some of the checks implemented by the AI evaluated proxies instead of real evidence that a particular function produced a meaningful result. Without experience in SW development and careful code reviews, these issues are hard to detect.

So it looks like code reviews, real end-to-end functionality checks and understanding of what is generated is still required, despite AI reviews with different models.

Of course - the story only shows a single case, but still reveals that caveats exist. And just to be sure: This is not a story from years back. The review was done in June and August 2026. Models involved were from Anthropic, Google and OpenAI.

Link to the case study: A Case Study in Assuring AI-Written Software

But if you ask a more difficult question, like why you should trust this recommendation, then often the illusion breaks.

The problem is not how well they speak. The problem is structure. Large language models are very good at making language, but they do not keep a stable and visible model of how customers, products, issues, causes, and solutions are connected. The answers sound good, but you cannot check where they come from.

This is where knowledge graphs come in.

A knowledge graph is a network of entities and the relationships between them. In a customer-support setting, the entities may be a customer, a laptop model, a system-freeze issue, a software release, and Patch 2026.9. The relationships might show that the customer reported the issue, the issue affects the laptop, the software release caused it, and Patch 2026.9 resolved it.

Now, an answer is not just a single claim anymore. It is the end of a path you can see.

Retrieval is not reasoning

Most retrieval-augmented generation systems begin with one of two approaches.

Keyword search gives you exact matches. It can find every ticket with ‘system freezes’, but it will miss ‘the application hangs’ or ‘the device becomes unresponsive’.

Vector search solves this problem with different words. It matches similar meanings, so it knows that these phrases talk about the same thing. This is much better, but just being similar is not enough. A vector database can say that two texts are close in meaning, but it does not tell the agent how one thing caused another, which product had the problem, or if the fix really worked.

Knowledge graphs add what is missing. They show the relationships clearly. The agent can trace a path from product to issue, from issue to cause, and from issue to solution. The answer isn’t just a document; it’s a path of evidence.

This difference is important. In simple cases, a good-sounding answer is enough. But in healthcare, finance, compliance, engineering, or enterprise support, it is not enough. Someone must see what evidence was used and how the answer was made.

The most useful architecture is hybrid

This does not mean that graphs should replace vector search. In reality, the best systems use both together.

Vectors are good for finding things. They can search through messy language and find good candidates, even if people describe the same problem in different ways. Graphs are good for proof. They can check if the right things are really connected and show the path that supports the answer.

There are two common ways to combine them.

The first way is routing. If the question is not clear and just asks for related information, the agent uses vector search. If the question asks for cause and effect, a longer explanation, or a recommendation you can defend, the agent uses the graph.

The second way is fusion. Vector search finds many possible matches first. Then the graph checks the connections and finds the best evidence. This is very useful when the graph is not finished yet. The vector part gives coverage, and the graph gives traceability.

The choice is not about theory. It is about how things work. Use routing when one method is enough and speed is important. Use fusion when missing a case or giving the wrong advice is expensive.

Building the graph is the easy part

A simple prototype can turn raw text into a graph very fast. An LLM takes out the entities and relationships from a support ticket, writes them as subject-predicate-object triples, and puts them into nodes and edges.

But the harder problem is to keep the graph trustworthy when it gets bigger.

Real data is not consistent. ‘Patch 2026.9’, ‘the 2026.09 patch’, and ‘the September 2026 patch’ can all mean the same thing. ‘System freezes’, ‘application hangs’, and ‘device becomes unresponsive’ can all be one issue. If every variation becomes its own node, the evidence is split in the graph and the agent only sees part of the pattern.

This is why entity resolution is so important. New facts should make existing entities stronger, not create new duplicates that look the same. Stable IDs, synonym lists, fuzzy matching, and embedding comparisons can help clean up the data before adding it.

Schemas and ontologies are important for the same reason. A schema says what nodes and relationships are needed. An ontology says which relationships make sense in the domain. Together, they stop the graph from becoming a big pile of nonsense that just looks connected.

The graph should produce evidence, not prose

There is also an important rule when you use graphs with agents. You must separate getting the truth from making the language.

The agent should not start by writing an answer. It should first call a graph tool with the question, the right entities, and filters. The tool gives back a small evidence subgraph and a reasoning path. Only then does the language model turn this into a good explanation.

In this setup, the LLM is not the source of truth anymore. It is just the interface to the truth.

This separation also makes things safer. If you let a model make any Cypher or SPARQL query, it is flexible, but it gets fragile when schemas get bigger and questions are not clear. In production, it is better to use narrow tools like find_related_issues or get_resolution_path. Then query building, checking, permissions, and formatting are outside the model.

From demo to infrastructure

The last lesson is that being ready for production is not just about database size. It is about governance.

A reliable graph needs schema versioning, duplicate checks, consistency checks, access controls, audit logs, and health metrics. Teams should watch for orphaned nodes, slow queries, old indexes, strange node growth, and schema drift. Sensitive data should be masked or left out before it gets to the agent.

Without these controls, a graph can look like a good source but have bad structure. With them, it becomes real reasoning infrastructure.

The promise of Graph RAG is more than just better search. It gives AI agents something they do not have now: a stable map of how facts fit together and a way to show how they got the answer.

The next generation of trustworthy agents will not just give you an answer. They will show you the path they took to get there.

Multiply that across an engineering team. The potential productivity gain is substantial. So is the queue of work waiting for somebody to trust it.

Happy Bhati explores this problem in Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecycle. His September 2026 research synthesis argues that useful output and total cost depend on the engineering system surrounding the agent.

Coding activity grows faster than releases.

The September revision of Writing Code vs. Shipping Code analyzes more than 500,000 GitHub developers. At the autonomous-agent stage, estimated cumulative increases reach 240% for commits, 80% for projects, and 30% for releases.

A 30% release gain is meaningful. But implementation accelerates much faster than the activities required to deliver it. These observational estimates do not isolate review as the cause; they support the broader bottleneck argument.

Bhati calls this the Agentic SDLC Throughput Paradox. If changes arrive faster than a team can validate and integrate them, work accumulates. More generation capacity can enlarge that queue.

Anthropic provides a concrete example. In its account of scaling test impact analysis, it reports 25 times as many continuous integration (CI) jobs over six months. Its test-selection service repeatedly struggled with the load. The eventual redesign separated stored test history from processing workers, allowing more workers to run in parallel. Anthropic says the service became stable, although operating costs increased. Faster coding had created a need to invest in the infrastructure that checks it.

Human oversight remains part of the work.

The SWE-chat study examined 6,000 coding-agent sessions from public repositories. Only 44% of the code agents produced was kept in users’ commits. In 44% of conversation turns, users corrected the agent, reported a problem, or interrupted its work.

Discarded code is not always wrong; experimentation and revision are normal. But these figures show work that code counts miss. Teams need to understand how much checking agents create, and where human judgment adds most value. That need not mean manually reviewing every change.

Checking and fixing code costs money.

Bhati’s proposed Verification Tax compares the cost of automated builds and tests, code reviews, security checks, and corrections with the cost of generating code. Token spending shows only part of that bill.

Imagine a €5 agent run that leaves a senior engineer with two hours of troubleshooting. A €20 run produces code that is easy to check. Looking only at the agent’s bill would make the first option seem cheaper.

The ratio needs care: cheaper generation raises it even if checking costs stay unchanged. Skipping checks lowers it but may leave more bugs. Track total delivery cost and reliability alongside it. Anthropic’s CI example adds another consideration: spending more on verification infrastructure can be worthwhile if it removes delays.

Automated checks can miss the same mistakes.

Agents can help write tests and review code. But if a test repeats the same wrong assumption as the code, both can look correct while missing the problem.

A SWE-bench Verified study found that asking agents to write more or fewer tests did not significantly change task success in that experiment. Many tests displayed values rather than checking correctness. Useful checks should reflect business rules, previous bugs, expected behaviour between systems, and security requirements.

Anthropic’s security-process account describes specialist review agents working alongside conventional security checks. Controls vary by risk: critical areas retain strict human approval, automated approvals are sampled, and new reviewers initially operate under human supervision. Separate agents can still share blind spots; assigning different roles does not establish independence.

One incident makes the need for controls tangible. An incident-response agent could not deploy fixes, but asked another agent through Slack to push a fix. A human review gate caught the request. The practical lesson is that permissions must cover what an agent can get another agent to do, as well as its own actions.

These are Anthropic’s accounts of its own operations, rather than independent evaluations. They illustrate useful controls without proving those controls will work equally well elsewhere.

Count changes after they qualify for production.

Bhati proposes Production-Qualified Change (PQC): credit a change only after it passes the relevant tests, security checks, reviews, and deployment or production-validation gates. Requirements should reflect risk; a documentation correction and an authorization change need different treatment.

PQC still needs validation and does not make changes equal in complexity or value. Compare similar classes of work alongside delivery time and failures. Using it as an individual developer score would invite gaming.

The broader proposal is an Agentic SDLC Control Plane: shared rules and monitoring connecting task risk, permissions, budgets, checks, and outcomes across existing development tools.

My starting point for an enterprise pilot would be:

• Bound the work. Choose eligible repositories and task types, with clear ownership and acceptance criteria.

• Limit execution. Set time, spend, permissions, retries, and escalation conditions.

• Validate the checks. Test automated reviewers under human supervision and define controls agents cannot bypass, including through other agents.

• Plan testing capacity. Monitor CI queues and test-selection delays as code volume grows.

• Measure complete delivery. Capture generation cost, review effort, rework, delivery time, and post-release failures.

• Expand using evidence. Increase autonomy when comparable work shows dependable outcomes within acceptable cost and risk.

The paper treats autonomy as constrained by money, reliability, and human attention. Review capacity and testing infrastructure belong in the business case from the beginning. Before expanding a rollout, ask whether more useful changes reach users at an acceptable total cost, without increased failures or unsustainable demands on experts.

Where is AI moving the bottleneck in your engineering team: implementation, review, testing, or deployment?

Source note: Based on Bhati’s September 2026 preprint (v1), linked research, and Anthropic’s security (July 21) and CI (September 14) articles. Research checked September 25; Anthropic articles checked September 26, 2026.

A provider can restore its service while your customer’s request remains unresolved. Someone in your organisation still owns that outcome.

That is the starting point for this ETX2 series, written for people choosing, building, and taking responsibility for AI in business processes. There may be a hundred ways to build an agent. I want to understand which ones earn their place.

Start with the process

Imagine a customer asking why their insurance premium increased.

Identify the policy, retrieve the renewal notice, draft an explanation, and obtain approval: we can prescribe those steps. My default is a workflow, using a model where language understanding or drafting helps.

Now suppose the notice contradicts the policy record. Choosing which correspondence to inspect and whether to investigate further gives an agent meaningful discretion.

Vendor definitions differ in emphasis. Anthropic distinguishes predefined paths from model-directed action; OpenAI emphasises planning, tools, and context; Microsoft combines agents with workflows. For this series, an agent uses a model to choose actions towards a goal, examines the results, and decides what to do next, within boundaries we set. Anthropic’s recommendation to try simpler solutions first is a useful starting discipline.

An email trigger only determines when work starts. It says nothing about who chooses the subsequent steps. Our design begins with a prescribed workflow and delegates the uncertain investigation.

Separate the platform decision from the design decision

Two questions help organise the landscape, but they do different jobs.

Who operates the runtime? This helps narrow the platform shortlist and the operational responsibilities your team takes on.

Who selects the steps? This is primarily a design decision inside the chosen platform. Many products support both prescribed sequences and model-selected actions. You can change that balance without necessarily changing vendors.

The workflow with a delegated investigation is therefore a design we impose. Buying an agent platform does not make that decision for us.

For runtime ownership, the main options look like this. “Your team” includes operating software on rented cloud infrastructure; the following list describes the listed deployment route.

Copilot Studio → Runtime operated by Microsoft → build with Visual agents and flows

OpenAI Responses API→ Your team → Custom model integration

OpenAI Agents SDK → Your team → Agent building blocks

OpenAI Agents API → OpenAI → Managed agent execution

Claude Messages API → Your team → Custom model integration

Claude Agent SDK → Your team → Agent building blocks

Claude Managed Agents → Anthropic → Managed agent execution

n8n / Dify → Either → Workflows and agents

LangGraph, self-hosted → Your team → Programmed execution graphs

For direct APIs, “runtime” means the application running the process; the provider still serves the model. The managed-agent entries describe provider-run orchestration; tools can still execute in your own environment. For n8n and Dify, “either” means choosing a hosted or self-hosted edition.

The cloud platforms also offer managed runtime options: Microsoft Foundry Agent Service, Amazon Bedrock AgentCore, and Google Gemini Enterprise Agent Platform. They belong on a shortlist shaped by identity, networking, and monitoring requirements. Google also offers the Agent Development Kit and visual Agent Studio. Microsoft, AWS, Google.

There is also a model-hosting choice. Self-hosting an application while calling an external model API still sends that provider the supplied context. Running an open-weight model locally through software such as Ollama adds responsibility for model serving, capacity, and updates.

Judge the build by its difficult cases

The authoring experience matters to build effort. No-code uses instructions, templates, and tools; low-code adds visual flows, expressions, and scripts; code gives us APIs, SDKs, and application logic. None removes the need to debug what happens.

Our first practical instalment will apply this to Copilot Studio: provider-operated execution, with prescribed steps around a model-directed investigation.

I use six questions to keep that evaluation grounded: build effort, access, decision scope, testability, cost, and failure ownership. Together, they cover what it takes to build the system and keep the business process working.

For our premium example, I care less about the first plausible draft than the ambiguous customer identity. A shared name must not be enough to select a policy. The evidence needs to show both that the correct records were retrieved and that access stayed within scope.

Missing or outdated evidence should lead to escalation. A polished explanation is a failure if the system has filled the gap with an invention. I favour bounded investigation and drafting, with an explicit approval before sending.

The timeout is another revealing test. If the write succeeded but its acknowledgement was lost, a retry can repeat the action. Durable execution helps preserve progress; preventing duplicate effects also requires support at the action boundary, such as a request identifier that the receiving service recognises across retries. Where that support is absent, the design needs reconciliation before retrying an uncertain action.

Alongside those cases, the comparison needs three practical records:

• Integration effort, including custom work for the mailbox and policy system.

• Completion quality, elapsed time, model and platform charges, and human review effort.

• Named owners for incorrect advice, failed integrations, and unresolved requests.

The independent paper AI Agents That Matter shows why quality and cost belong together: accuracy-focused benchmarks can reward unnecessary complexity and expense. For me, a more elaborate agent has to demonstrate a worthwhile improvement over the simpler workflow.

Ownership needs more than a name in a document. The process owner decides when to suspend automated handling, technical support diagnoses integration failures, and an assigned business team takes unresolved requests. Service hours and escalation routes determine whether that handover works when nobody is watching the demo.

Plan for the platform to change

Migration is not hypothetical. OpenAI’s Agent Builder is deprecated and scheduled to close on November 30, 2026. That removes it from my shortlist for a new build and raises a broader question: how much work survives a platform change? Deprecation notice.

MCP standardises tool and data connections; A2A supports communication between agent applications. Neither makes the whole application portable.

I put particular value on keeping test cases and expected outcomes outside a platform’s proprietary tracing format. “Do not send when identity is ambiguous” should remain testable after migration, even if execution is recorded differently. Reusing the policy-lookup tool helps; preserving evidence that the process works matters just as much.

We will carry that scenario through the series, starting with the Copilot Studio build. Tomorrow’s inbox remains the test.

Which step in your process can an agent choose, and who takes over when it fails?

Product references checked on September 15, 2026.

Update on Oct, 28th 2026: Added SOC 2 Type 2 and HIPPA compliance, which was missing in the original article. Check out more on https://www.meibel.ai/

Whenever I attend an AI conference, I am struck by how many vendors I have never heard of.

Beyond the familiar names and headline sponsors, there are dozens—sometimes hundreds—of companies promising to solve a particular piece of the enterprise AI puzzle. Some offer little more than familiar technology in new packaging. Others may be genuinely interesting. From a conference booth or a polished demo, it is almost impossible to tell which is which.

That makes me curious. What do these companies actually offer? Where would their technology fit within an enterprise architecture? What distinguishes it from the alternatives? Is there real substance behind the positioning—and, occasionally, a hidden gem worth watching?

I want to go beyond the marketing page and the five-minute demo. Not necessarily because I intend to use or recommend a product, but because I want to understand what it actually does, where it fits, what makes it different, where its limitations lie, and whether it belongs on an enterprise technology radar.

That is the idea behind a new ETX2 asset series: Technology Dossiers.

What is an ETX2 Technology Dossier?

ETX2 Technology Dossiers are compact but detailed assessments for people who need to look beyond product positioning.

There is no fixed publishing roadmap and no attempt to cover the market systematically. I will investigate technologies that cross my path through projects, research, conferences, or conversations—or simply because something looks interesting enough to examine more closely.

The dossiers may cover startups, established vendors, individual products, platforms, or particular technology approaches.

Importantly, there is no financial relationship behind the selection. I am not paid by the vendors, there is no affiliate model, and inclusion in a dossier does not constitute a recommendation.

The purpose is to investigate, assess, and document what I find.

Each dossier will typically examine:

• what the product is—and what it is not,

• company and product maturity,

• architecture and technical capabilities,

• distinctive features,

• deployment and integration options,

• suitability for enterprise and regulated environments,

• customer references and available evidence,

• limitations and open questions,

• and practical considerations for evaluating the technology.

Where possible, I will also include hands-on observations.

These are not laboratory benchmarks or formal procurement assessments. Think of them as independent technology reconnaissance for enterprise practitioners.

Dossier #1: Meibel AI

The first dossier examines Meibel, a relatively young AI company I encountered while researching document-centric enterprise AI.

Meibel positions its platform between complex enterprise documents and the AI applications or agents that need to work with them.

At a high level, its architecture follows a four-stage pipeline:

Parse → Transform → Ingest → Agents

Documents are decomposed and interpreted, transformed into structured schemas, indexed for different retrieval mechanisms, and made available to AI agents. Confidence scoring, provenance, and execution policies operate across those stages.

That makes Meibel more than a conventional OCR or document-extraction service.

Three aspects caught my attention in particular.

1. Runtime confidence—not just AI output

Meibel places unusual emphasis on measuring confidence throughout the processing pipeline.

Confidence is not merely displayed as metadata. It can help determine whether an AI-generated result proceeds automatically, is processed again, is sent to a human reviewer, or is rejected.

For enterprise AI, I find this particularly interesting.

The relevant question is increasingly not “Can the model produce an answer?” but:

“Under what conditions am I prepared to let software act on that answer?”

Those are very different questions.

2. Complex documents—not just clean text

The second interesting area is Meibel’s approach to document intelligence.

Instead of immediately flattening a document into chunks of text, the platform attempts to preserve structures such as tables, sections, figures, captions, footnotes, and cross-references. It can then expose structured information through schemas alongside semantic retrieval.

That matters because many of the most valuable enterprise documents are precisely those that conventional RAG pipelines struggle with: specifications, contracts, insurance documents, construction documents, RFPs, and other material in which layout and relationships carry meaning.

In my own limited testing, schema-based extraction through the API performed considerably better than simply uploading a difficult document through the user interface. I also found the document Q&A capability convincing, although repeated processing produced some non-deterministic behaviour that would merit closer investigation before production use.

3. Graphs enter the retrieval layer

The third area is one I find especially interesting: Meibel combines semantic retrieval and SQL-style access with graph-based traversal of relationships between document elements and references.

An enterprise question may require more than retrieving two individually relevant chunks. A system may also need to understand that one section constrains another, that a footnote qualifies a table entry, or that one document refers to a requirement defined elsewhere.

This moves the discussion beyond the simplistic idea that better RAG is mainly a matter of embedding models and chunk sizes.

How I tested Meibel

Meibel is an API-first product, so I wanted to evaluate it through code rather than rely solely on the user interface. I created a set of test scripts that submitted different document types, defined information to be extracted, and examined the resulting structured output. For the most demanding test, I used a deliberately complex sample insurance bordereau containing dense tables, irregular structures, and domain-specific information—the kind of document that quickly exposes the limitations of conventional extraction approaches.

I also built a small test application that allowed users to upload a document and specify arbitrary information they wanted to extract, rather than working with a single predefined schema. This was not intended as a formal benchmark, but as a practical exploration of how Meibel behaves when confronted with realistic enterprise documents and changing information requirements. The observations in this dossier therefore combine documentation research with limited hands-on testing of the platform’s APIs.

Interesting technology, early-stage vendor

This dossier is deliberately not an endorsement.

Meibel was founded in 2024 and remains a comparatively small company. Its publicly visible customer base is limited, large-scale independent benchmarks are absent, detailed enterprise pricing is not publicly documented. Meibel is SOC 2 Type 2 and HIPPA compliant, accoring to information the company shared with me.

For a regulated enterprise, those questions matter just as much as the technical architecture.

On the other hand, deployment options including SaaS, bring your own cloud, and on-premises make the platform more interesting for environments in which data sovereignty matters.

My current assessment is therefore fairly simple:

Meibel is not a technology I would standardize on without substantial further evaluation. But it belongs on the evaluation list when complex documents, provenance, and confidence-controlled AI automation are central to the problem.

Get the full Technology Dossier

The full ETX2 Technology Dossier: Meibel AI goes considerably deeper.

It covers the company, product architecture, customer references, document intelligence, confidence scoring, graph-based retrieval, enterprise deployment, security considerations, competitive positioning, and a suggested evaluation approach.

The dossier draws on public documentation, analyst and press coverage, material published by Meibel, customer references, and my limited hands-on testing. It has not undergone exhaustive validation, and its conclusions should be independently verified before any procurement or architecture decision.

Comment under the article or on LinkedIn when you are interested and I will send you the assessment.

Leave a comment

That is also the philosophy behind the series: take an unfamiliar or interesting technology apart, document what I find, and share the result with people facing similar questions.

No rankings. No sponsored placements. No claim to cover the entire market.

Just a closer look beyond the AI conference headliners—and perhaps the occasional hidden gem.

Find out more about Meibel AI on their website: https://www.meibel.ai/

ETX2 Technology Dossier #1 — Meibel AI

Now for the less visible side of the experiment: What are all those agents doing - and how many tokens are they consuming?

In this part, we send Claude Code telemetry to Azure Application Insights and inspect it in a Grafana dashboard.

The setup still runs natively on my Windows 11 ARM64 machine. No WSL required.

The architecture

Claude Code already emits telemetry using OpenTelemetry. A local Collector receives that data and forwards it to Azure. Grafana then queries the Application Insights data stored in Log Analytics.

OpenTelemetry keeps the agent separate from the monitoring backend. Today I use Application Insights; the same OTLP stream could be routed elsewhere later.

1. Install the OpenTelemetry Collector

For the Azure Monitor exporter, use the contrib distribution. Because this lab runs on Snapdragon, use the ARM64 installer:

$otelVersion = "0.158.0"
$otelMsi = "$env:TEMP\otelcol-contrib.msi"

irm "https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v$otelVersion/otelcol-contrib_${otelVersion}_windows_arm64.msi" `
    -OutFile $otelMsi

Start-Process msiexec.exe -Verb RunAs -Wait `
    -ArgumentList "/i `"$otelMsi`""

The installer creates a Windows service and places the Collector under:

C:\Program Files\OpenTelemetry Collector Contrib

Check that it is running:

Get-Service otelcol-contrib

Then open http://localhost:8888/metrics. This endpoint shows the Collector’s own internal metrics - a quick confirmation that the process is alive.

2. Point Claude Code at the Collector

Set these variables in PowerShell before starting Claude Code:

$env:CLAUDE_CODE_ENABLE_TELEMETRY="1"
$env:CLAUDE_CODE_ENHANCED_TELEMETRY_BETA="1"

$env:OTEL_METRICS_EXPORTER="otlp"
$env:OTEL_LOGS_EXPORTER="otlp"
$env:OTEL_TRACES_EXPORTER="otlp"

$env:OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
$env:OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"

Claude Code will now send metrics, events and traces to the local OTLP/HTTP endpoint.

Claude Code records useful metadata such as prompt length, token usage and cost without sending the actual text.

3. Create Application Insights

You can use an existing workspace-based Application Insights resource. For this lab, I created one with Azure PowerShell.

Install the module and sign in:

Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force
Connect-AzAccount

My minimal creation script is:

$resourceGroupName = "AI-Telemetry"
$resourceName = "AppInsights-AI"
$azureRegion = "germanywestcentral"

New-AzResourceGroup `
    -Name $resourceGroupName `
    -Location $azureRegion

New-AzApplicationInsights `
    -ResourceGroupName $resourceGroupName `
    -Name $resourceName `
    -Location $azureRegion `
    -Kind "web" `
    -ApplicationType "web"

Adjust the names and region, save the script as CreateAppInsights.ps1, and run it:

.\CreateAppInsights.ps1

Azure automatically creates an associated Log Analytics workspace if you do not supply an existing one.

4. Forward the telemetry to Azure

In the Azure portal, open the Application Insights resource and copy its Connection String from the Overview page.

Replace the contents of the Collector’s config.yaml with this minimal configuration:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  azuremonitor:
    connection_string: "<YOUR APPLICATION INSIGHTS CONNECTION STRING>"

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [azuremonitor]
    metrics:
      receivers: [otlp]
      exporters: [azuremonitor]
    logs:
      receivers: [otlp]
      exporters: [azuremonitor]

Do not commit the connection string to a repository. You can find the connection string on the Overview pane in Azure Portal:

Validate the configuration and restart the service from an elevated PowerShell:

$otelHome = "C:\Program Files\OpenTelemetry Collector Contrib"

& "$otelHome\otelcol-contrib.exe" validate `
    --config "$otelHome\config.yaml"

Restart-Service otelcol-contrib

5. Generate some telemetry

Start Claude Code again in Agent Teams mode:

claude --teammate-mode tmux

Give the team a task, let the agents work, and wait a few minutes for the data to reach Azure.

In the Azure portal, open:

Application Insights -> Dashboards with Grafana -> Claude Code

The dashboard shows sessions, prompts, API requests, tool activity, tokens and estimated cost - including the activity generated by subagents.

The useful part starts here

The dashboard makes the multiplier effect of Agent Teams visible. A single user request can trigger several agents, several context windows and many model calls.

That does not make Agent Teams a bad idea. It makes observability part of the setup.

The cost figures reported by Claude Code are estimates, not billing records. But they are enough to compare sessions, identify expensive workflows and ask the question that matters:

Did the additional agents create enough value to justify the additional model capacity?

There is just one small problem if you are a Windows user: the split-pane interface is built around tmux.

In this two-part ETX2 Lab, I’ll build the complete setup natively on Windows. In Part 1, we get Claude Code Agent Teams running in separate terminal panes. In Part 2, we will send its telemetry to Azure Application Insights and build a Grafana dashboard on top of it.

No WSL involved.

Starting point

For this lab I started with:

• Fresh Windows 11 Home installation

• ARM64 / Snapdragon machine

• Claude authentication through the interactive browser login

• No Anthropic API key

1. Install PowerShell 7

Open a terminal and install the current PowerShell:

winget install --id Microsoft.PowerShell --source winget

Then start PowerShell 7 as Administrator.

2. Install Claude Code

Anthropic provides a native Windows installer:

irm https://claude.ai/install.ps1 | iex

3. Check the PATH

Start a new PowerShell, then try:

claude --version

If Windows cannot find Claude, add its installation directory to your user PATH:

$currentPath = [Environment]::GetEnvironmentVariable('PATH', 'User')
[Environment]::SetEnvironmentVariable(
    'PATH',
    "$currentPath;$env:USERPROFILE\.local\bin",
    'User'
)

Restart PowerShell afterwards. Verify with

claude --version

4. Install psmux

Claude’s split-pane Agent Teams UI expects tmux. On Windows, I use psmux, a native tmux-compatible terminal multiplexer. You will need to accept terms with Y during installation.

winget install psmux

Start it:

tmux

No Linux environment or WSL is required.

5. Create a project

For this test I use an empty directory:

md C:\dev\ClaudeCodeTest
cd C:\dev\ClaudeCodeTest

6. Install VS Code

winget install -e --id Microsoft.VisualStudioCode

7. Start Claude Code

From the project directory:

claude

On the first run, Claude guides you through authentication. I use my Claude account and authenticate interactively in the browser rather than supplying an API key.

Claude asks a few questions and then heads over to the authentication in the browser.

Make sure you trust the folder you are in:

Exit Claude again after the initial setup with /exit in the prompt:

/exit

8. Enable Agent Teams

Agent Teams are currently an experimental Claude Code feature and disabled by default.

Enable them for the current PowerShell session:

$env:CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1

9. Start Claude in split-pane mode

Now launch Claude Code with the tmux teammate interface. Claude might ask whether to try the fullscreen renderer.

claude --teammate-mode tmux

10. Give the team some work

The final step is the fun one.

Ask Claude to create an agent team and give the teammates work that can be performed in parallel—for example architecture, implementation and testing. A dramatically reduced version could read like this:

I’m designing a new web application. Spawn three agents: the first one acts as a project manager and refines the requirements. The second one as developer and the third one as tester.

Claude will probably ask a set of questions about the details. Once Claude creates the team, psmux opens the individual Claude sessions in separate panes.

And that’s it: Claude Code Agent Teams running natively on Windows in a multi-pane CLI.

Next: Where are all those tokens going?

Multiple agents also mean multiple context windows, prompts, tool calls and potentially substantial token consumption.

So simply getting Agent Teams to work is only half the experiment.

In Part 2, I’ll connect Claude Code’s OpenTelemetry output to Azure Application Insights and use Grafana to visualize what the agent team is actually doing—and what it costs.

That model is disappearing.

I explored the market for AI coding cost management in considerably more detail in the accompanying ETX2 Technology Brief: “The Emerging Landscape of Cost Management for AI-Driven Software Development.” It includes the four-layer market taxonomy, current pricing and cost ranges, hyperscaler and monitoring-platform comparisons, a survey of purpose-built AI FinOps products, OpenTelemetry developments, and practical cost-management strategies.

The data and product landscape are current as of August 2026; given how rapidly AI pricing and tooling are evolving, consider the brief a point-in-time market snapshot rather than an evergreen catalogue.

As coding assistants evolve into coding agents, they consume much more compute. They read repositories, reason over large contexts, modify multiple files, run tools, call other agents, test their own work and sometimes continue operating for extended periods without human intervention.

And somebody has to pay for all those tokens.

During 2026, the market has been moving decisively away from the idea that sophisticated AI coding can be offered as essentially unlimited usage behind a $20 subscription. GitHub, OpenAI, Anthropic, Cursor and others now combine subscriptions, usage allowances, credits, token consumption or considerably more expensive power-user plans.

The result is an interesting shift.

AI coding tools are starting to look less like software licenses and more like cloud infrastructure.

And that means engineering organizations may have to learn a lesson that IT infrastructure teams learned years ago: Consumption without cost attribution eventually becomes a problem.

The seat price is the floor

The headline prices for AI coding tools still look inexpensive.

GitHub Copilot starts around the price of a normal developer productivity tool. Cursor and Claude offer entry-level subscriptions around $20. Even $100 or $200 power-user tiers don’t look particularly dramatic when compared with the fully loaded cost of a software engineer.

But the subscription price increasingly tells only part of the story.

Research cited in the paper suggests that organizations combining inline coding assistance with more agentic workflows can end up spending around $200–$600 per developer per month. Anthropic has reported enterprise Claude Code usage averaging around $150–$250 per developer per month.

At the extreme end, Gartner warned in June 2026 that heavy agentic coding workloads can reach thousands of dollars per developer per month, with exceptional cases much higher.

Those extreme numbers obviously should not be interpreted as the typical cost of using an AI coding assistant.

But they reveal something important.

A developer clicking “accept” on an autocomplete suggestion and an autonomous agent spending hours iterating over a repository are technically both using an “AI coding tool.”

Economically, they are completely different workloads.

That distinction becomes increasingly important as agentic development matures.

More tokens don’t automatically mean more productivity

There is another reason why this matters.

If additional AI consumption reliably produced proportional improvements in engineering output, managing token cost would mostly be an accounting exercise.

Spend twice as much, produce twice as much software.

Unfortunately, the evidence does not suggest such a clean relationship.

One large study cited in the survey, covering more than 400 organizations over 14 months, reported a median increase in pull-request throughput of 7.76%, with many teams seeing improvements in roughly the 5–15% range.

That is useful.

It can easily justify several hundred dollars of additional tooling cost for a developer whose fully loaded employment cost runs into five figures per month.

But it is very different from assuming that AI turns every developer into three developers.

More importantly, Gartner’s warning is that there is no direct relationship between increasing token consumption and increasing productivity.

That creates what I would call the marginal-token problem.

The first few dollars of AI consumption may be extremely valuable. They eliminate tedious work, accelerate exploration or generate useful boilerplate.

The next hundred dollars may still produce substantial value.

But what about the next thousand?

At some point, running another agent, adding another reasoning loop or feeding even more context into a model may increase the bill without producing a corresponding improvement in shipped software.

Which means that simply measuring AI usage isn’t enough.

We need to connect consumption to engineering outcomes.

Cloud FinOps all over again

There is an obvious precedent.

When enterprises moved from owned infrastructure to public cloud, computing changed from something organizations purchased upfront into something they continuously consumed.

Technically, this was enormously powerful.

Financially, it created new problems.

Individual teams could provision infrastructure in minutes. Costs became distributed across thousands of resources. Bills became difficult to explain. And organizations discovered that giving engineers almost frictionless access to effectively unlimited computing resources could produce some interesting invoices.

FinOps emerged to answer seemingly simple questions:

Where is the money going?

Who is consuming it?

Which application is responsible?

AI coding is beginning to create a remarkably similar challenge. The difference is that the unit of consumption isn’t a virtual machine or storage volume.

It is increasingly an AI agent performing work, which makes attribution harder.

Knowing that the development organization consumed 12 billion tokens this month is not particularly useful.

Knowing that €18,000 was spent on AI-assisted development is somewhat better.

Knowing that €4,300 was spent on Project A is better still.

But the really interesting question is: What did that €4,300 actually produce?

An AI FinOps stack is emerging

While researching the market, I found that AI cost management is already starting to stratify into several layers.

I find it useful to think about four layers.

Layer 1: The AI providers

OpenAI, Anthropic and other model providers increasingly expose usage dashboards, APIs, project or workspace segmentation, spending limits and cost reporting.

This is the obvious starting point.

If all your development AI goes through one provider and a small number of API keys, native tooling may be sufficient.

The limitation is equally obvious: each provider sees only its own consumption.

Layer 2: The cloud platforms

Azure, AWS and Google Cloud add another level of context.

Azure combines Application Insights, Azure Monitor and API Management. AWS connects Bedrock telemetry with CloudWatch, inference profiles and Cost Explorer. Google increasingly integrates AI consumption with its broader Cloud Billing and monitoring capabilities.

This starts to look much more like classical FinOps.

Teams can allocate costs, detect anomalies, build dashboards and connect AI consumption with the wider infrastructure estate.

For enterprises already consuming models through their hyperscaler, this may become the natural control plane.

Layer 3: Enterprise observability

Then there are the platforms many enterprises already use to monitor distributed applications.

Datadog, Grafana, New Relic, Dynatrace, Splunk and Elastic are extending their observability stacks to understand generative AI.

Instead of monitoring only latency, errors and request volumes, they can increasingly trace model calls, token consumption, tool calls and estimated inference cost.

This is particularly interesting because AI agents are ultimately distributed systems.

A single agent request may trigger several model calls, invoke tools, query data, call APIs and initiate other agents.

Looking only at the final token bill tells you very little about what actually happened.

Layer 4: Purpose-built AI FinOps

The newest category is perhaps the most interesting.

A growing collection of startups is trying to connect AI expenditure directly to software engineering work.

Some attribute Claude Code or other coding-agent sessions to developers, repositories, branches or tickets. Others reconcile usage across multiple AI vendors. Some try to connect spend with story points, delivery metrics, rework rates or engineering outcomes.

This is where AI FinOps starts becoming something meaningfully different from traditional cloud cost management.

The important unit isn’t necessarily:

Cost per million tokens.

It might be:

AI cost per feature delivered.

Or:

AI cost per resolved ticket.

Or perhaps eventually:

AI cost per unit of business value created.

That is a much harder measurement problem.

But it is also the one that ultimately matters.

OpenTelemetry could become surprisingly important

There is another development in this space that deserves attention: the emerging OpenTelemetry GenAI Semantic Conventions.

OpenTelemetry already provides a vendor-neutral way to instrument distributed applications.

The GenAI conventions extend that model with concepts such as model calls, token counts, tool invocations and other generative-AI-specific telemetry.

Azure has adopted OpenTelemetry heavily for AI monitoring. Grafana uses it through OpenLIT. Langfuse and Traceloop support it, and AWS supports related telemetry scenarios as well.

This matters because instrumentation may ultimately be more strategic than the dashboard sitting on top of it.

If organizations standardize how AI activity is captured, they can route that telemetry into Application Insights today, Grafana tomorrow and another cost-management platform later.

For enterprises trying to avoid yet another observability silo, that is an attractive proposition.

What should enterprises do today?

The market is still immature, so buying another platform may not be the first step.

The more important change is conceptual.

Treat AI coding consumption as something that needs engineering and financial governance from day one.

That means establishing spending limits before broad rollout rather than after the first surprising invoice.

Measure real consumption for several months before negotiating enterprise commitments.

Consider moving very heavy users onto predictable flat-rate plans where appropriate.

Separate teams, projects and workloads so that costs can actually be attributed.

Use cheaper models for routine work and reserve expensive reasoning models for tasks that justify them.

And above all, don’t monitor AI expenditure in isolation.

Track rework. Track defects. Track throughput. Track delivery outcomes.

Then ask whether additional AI consumption correlates with improvements in those measures.

Because there is a dangerous metric hiding in every AI dashboard: Token consumption can go up even when value doesn’t.

From AI adoption to AI economics

For the last few years, the enterprise conversation around coding assistants has largely been about adoption.

Should developers be allowed to use them?

Which model performs best?

Can the tools access proprietary source code safely?

How much faster will developers become?

Those remain important questions.

But agentic development introduces another one:

What happens when AI starts consuming meaningful amounts of money while doing the work?

At that point, selecting an AI coding tool is no longer just a developer-experience decision. It becomes an economic architecture decision.

And the organizations that deploy hundreds or thousands of coding agents will eventually need to understand those agents much the same way they understand their cloud infrastructure:

who is using them, what they are doing, what they cost, and whether the result justifies the consumption.

The era of the $20 AI coding assistant isn’t necessarily over.

But increasingly, $20 is the entry price, not the cost model.

The session description nicely captured where the industry seems to be heading: from standalone LLMs handling individual tasks, through orchestrated agentic workflows, toward fully autonomous distributed networks where agents independently reason, delegate and collaborate across organizational boundaries.

That sounds like the natural next step for agentic AI.

It also creates a very different class of engineering problems.

The demo during the session was playful: one agent asked other specialist agents for help, they exchanged information, and together they worked toward a result.

But the interesting part was not the demo. It was the problem hidden behind it.

As soon as you use several agents, the human can easily become the integration layer: copying context between agents, checking one agent’s work with another, restarting tasks, and trying to remember what each agent is doing. Goomanovsky described exactly this pattern in software development: one agent creates a plan, another reviews it, perhaps another writes or reviews code—and the developer ends up manually coordinating the conversation.

That does not scale.

One of the most useful distinctions from the session was simple:

Communication is not collaboration.

Giving two agents a way to send messages to each other does not mean they can reliably execute a business process together.

The protocol layer is finally taking shape

This is where the emerging communication protocols matter.

MCP — the Model Context Protocol — has become an important standard for connecting AI applications to tools, data sources and services.

Conceptually, MCP solves something similar to what standardized peripheral interfaces did for computers: instead of every agent requiring custom integration code for every system, tools can expose capabilities through a common protocol.

MCP was initially mostly about agent-to-tool interaction rather than collaboration between autonomous agents. But it has continued to evolve. The July 2026 specification introduced a stateless protocol core, Multi Round-Trip Requests and an extension mechanism for durable Tasks—features that make MCP increasingly relevant to long-running agentic workflows. (Model Context Protocol)

For communication between independent agents, A2A — Agent2Agent — addresses the problem more directly.

Originally developed by Google and subsequently moved under Linux Foundation governance, A2A defines concepts such as Agent Cards for discovery, Tasks, Messages, Artifacts, streaming and asynchronous notifications. (A2A Protocol)

The distinction is useful:

MCP helps an agent use capabilities. A2A helps agents collaborate with other agents.

The protocols are therefore more complementary than competitive. (A2A Protocol)

There are also signs that the standards landscape is consolidating. IBM’s Agent Communication Protocol, ACP, for example, has been merged into A2A rather than developing into another competing ecosystem. (IBM)

That reduces one interoperability problem.

But interoperability is only the first layer.

Welcome back to distributed systems

The moment agents start operating independently, orchestration becomes a distributed-systems problem.

Interestingly, that was also explicit in Goomanovsky’s talk. The session touched on message delivery, message ordering, recovering an agent after a crash and restoring the state of an existing conversation.

None of these are new problems.

Software engineers have been fighting them for decades.

AI adds another complication: not only can the network or infrastructure behave unexpectedly—the participant itself is probabilistic.

Consider a simple example.

Agent A asks Agent B:

“Execute this transaction.”

B performs the action, but the response gets lost.

A waits.

Eventually it times out.

What should it do?

If it retries, has the transaction now happened once—or twice?

For a weather lookup, this hardly matters.

For “send €50,000,” “cancel the insurance policy,” or “email the customer,” it matters enormously.

Classic distributed systems use patterns such as idempotent operations, unique request identifiers, durable messaging, acknowledgements, retries with exponential backoff, timeouts and circuit breakers to deal with exactly these situations.

Agent architectures will need the same engineering discipline.

Your context window is not a transaction log

State is another fundamental problem.

Imagine an agent executing a workflow consisting of twenty steps. It completes twelve and then crashes.

When another instance takes over, what happens?

Simply replaying the original prompt is dangerous. Some actions may already have happened.

A production system needs to know:

• which task is being executed,

• which steps have completed,

• which external actions have occurred,

• what results were returned,

• which approvals were granted,

• and what remains to be done.

That information needs to exist independently of the model.

An LLM context window is not a durable workflow store.

This is one reason I would be cautious about architectures based entirely on autonomous peer-to-peer conversations between agents.

They look elegant in demonstrations. But the more autonomous the network becomes, the harder it becomes to reconstruct causality when something goes wrong.

For enterprise applications, a hybrid architecture may often be more practical.

Agents can still reason and act autonomously, while a durable orchestration layer owns workflow state, task dependencies, permissions, retries and completion criteria.

In other words, decentralize intelligence without necessarily decentralizing control.

Orchestration also means observability

There is another challenge from the session that I think deserves more attention: once many agents work autonomously, how do you actually know what they are doing?

The transcript describes the problem from a developer perspective. Several agents can continue working while the developer is doing something else. When the developer returns, it becomes difficult to understand where each agent is, what it has done and how much resource it has consumed.

Multiply that by a thousand employees.

Now it becomes an enterprise-management problem.

Agentic platforms therefore need something resembling distributed tracing.

Every business request should ideally carry a correlation or trace ID through the communication chain, from the user over AI agents down to the business system.

That allows the enterprise to answer questions such as:

What happened?

Why did it happen?

Which agent made the decision?

Which model was used?

Which tools were called?

How many retries occurred?

How much did the process cost?

And where did it fail?

OpenTelemetry has already started defining GenAI semantic conventions for tracing model calls, tool executions, token consumption and other AI operations.

This may sound like infrastructure plumbing.

But without it, diagnosing an agent system means reading conversations and hoping to reconstruct what happened.

That is roughly equivalent to debugging a distributed application by reading terminal output from several servers.

The protocol is not the architecture

MCP and A2A are important developments.

They reduce integration friction and make it increasingly realistic that an agent created on one platform can access tools—or cooperate with an agent—created somewhere else.

But protocols do not solve orchestration.

They do not decide which agent should be trusted with a task.

They do not determine who owns the business outcome.

They do not prevent duplicate transactions.

They do not stop two agents from entering an expensive reasoning loop.

They do not decide when a human has to intervene.

And they do not tell management why a swarm of agents consumed €20,000 worth of model tokens last Tuesday.

Those are architecture, operations and governance problems.

This is why the progression described in the Ai4 session—from standalone models to orchestrated workflows and eventually autonomous distributed agent networks—is so interesting.

With every step, we gain autonomy.

But we also inherit more of the complexity traditionally associated with distributed computing.

The first wave of agent development was largely about prompts, tools and reasoning.

The next wave will increasingly be about state machines, message semantics, identity, retries, observability, access control, cost management and recovery.

Which leads to a slightly paradoxical conclusion:

Making agents intelligent may turn out to be the easy part.

Making hundreds of intelligent agents work together reliably is where the real engineering starts.

And yet there is an increasingly awkward question hanging over all of this activity:

Where is the ROI?

At Ai4 in Las Vegas, I attended two sessions that addressed exactly this problem.

The first was Guru Bhat’s “The ROI Illusion: Why Most Enterprise AI Isn’t Paying Off.” His argument was that enterprises often start AI transformation at the wrong level: with models and isolated use cases rather than the business workflow.

A few hours later, Sai Maddineni approached the same problem from an insurance perspective in “Insurance Industry’s AI Value Problem: The Reality of Unit Economics, Reliability and Auditability.” His question was more fundamental: just because AI can perform a task, does that mean AI should perform it?

Together, the two perspectives form a useful framework: understand the workflow first, then decide where AI actually belongs.

Stop starting with the AI use case

Bhat’s central argument was simple:

Don’t start with the use case. Start with the workflow.

Enterprises typically create lists of AI opportunities, rank them according to value and feasibility, and launch pilots. But a business workflow may contain many individual tasks and several potential AI use cases. Optimizing one convenient task does not necessarily improve the outcome of the workflow as a whole.

Think about underwriting. Document ingestion, data extraction, risk analysis, pricing, referrals, approvals and customer communication may all belong to the same workflow. Automating document summarization may save time, but that does not automatically mean the underwriting process becomes materially better.

The more useful starting point is:

What business outcome are we trying to change in this workflow?

Only then should we identify the places where AI can contribute.

ROI has to be designed in

Bhat also challenged the way enterprises measure AI success.

Number of users. Number of pilots. Number of tokens consumed. These may be useful activity metrics, but they are not ROI.

He proposed moving from adoption to proficiency — whether AI actually changes the speed or effectiveness of work — and finally to financial outcomes. He also argued that AI value is broader than conventional efficiency measures and can include work velocity, revenue uplift, risk and compliance.

This means the baseline and the expected outcome need to be defined before the solution is built.

Ten thousand employees using an AI assistant is an impressive adoption figure.

It is not yet a business case.

The economics also extend to the technology itself. Bhat questioned whether frontier models should be used for every task. Different activities may justify different models, costs and levels of intelligence, which points toward an orchestration layer capable of selecting the appropriate model and grounding it in enterprise data.

For agentic systems, he went further and emphasized knowledge graphs as a way to give agents the relationships and enterprise context needed to act on grounded information rather than isolated pieces of data.

So architecture itself becomes part of the ROI equation.

Just because AI can do it doesn’t mean it should

Maddineni attacked the problem from a different direction.

He opened with a deliberately trivial experiment: asking leading AI models to perform a simple calculation and comparing their cost with a small traditional Java program doing the same thing.

Both produced the answer. But according to his experiment, the AI approach was dramatically more expensive.

His point was not about arithmetic. It was about using the right technology for the right kind of work.

Insurance companies execute huge volumes of deterministic activities: eligibility rules, coverage checks, calculations, validations and other tasks where identical input should produce identical output.

An AI agent can perform many of these activities.

But why should it?

Technical capability is not the same as economic suitability.

Insurance contains different kinds of work

Maddineni described work as a spectrum.

At one end are deterministic, rules-based activities. At the other are activities that depend heavily on judgement, interpretation, experience and relationships.

A large part of insurance work is quantifiable and rule-driven. Yet much of it has never been fully automated, not because traditional software cannot execute the rules, but because the information surrounding them often arrives in difficult forms: PDFs, spreadsheets, emails and non-standard documents. Humans have effectively acted as the integration layer between structured systems and an unstructured world.

Generative AI changes this dramatically.

But that doesn’t necessarily mean the entire process should become generative AI.

For deterministic, high-volume activity, traditional software still has major advantages: predictable execution, reliability, auditability and low marginal cost. Maddineni therefore suggested an interesting alternative:

Use AI to build the software rather than using AI to be the software.

AI-assisted development can dramatically reduce implementation effort while allowing developers to review the resulting code, QA teams to test it and enterprises to operate deterministic software at scale.

Sometimes the best AI solution may therefore result in a production system where no LLM is involved in the actual transaction.

Where generative AI really fits

The picture changes when the work itself is cognitive and unstructured.

Maddineni used underwriting as an example. An underwriter may receive a large broker submission containing PDFs, financial data, schedules and correspondence. Simply understanding and structuring the submission may take significant time.

AI can summarize, extract and contextualize that material in seconds, allowing the underwriter either to handle more submissions or to spend more time on risk analysis and pricing.

And then there is a third category: work where humans should remain central.

When somebody has just lost a home in a catastrophe, the conversation with the insurer is not merely another process step. Empathy, reassurance and trust are part of the service itself. Maddineni argued that these human relationships should not disappear simply because an agent could technically handle the interaction.

That leads to a much more useful transformation question than “Where can we deploy an agent?”

For each activity, ask:

Should this be conventional software? Should AI augment it? Or should a human continue doing it?

Decompose first, automate second

This is where the two sessions converge.

Bhat says: start with the workflow, not the use case.

Maddineni says: break that workflow into individual activities and decide what belongs with software, AI or humans.

His recommendation was refreshingly practical: take an important, high-volume process, bring in the people who actually perform the work, understand its inputs and outputs, and decompose it into its individual parts.

From there, the architecture becomes much clearer:

• Deterministic and high-volume: conventional software, increasingly built with AI assistance.

• Cognitive and unstructured: generative AI, often augmenting human expertise.

• Judgement, empathy and relationships: keep humans at the center.

Across all three, Bhat’s question still applies:

What measurable business outcome does changing this workflow create?

The real AI ROI illusion

Perhaps the AI ROI problem is therefore not primarily an AI problem.

It is a transformation problem.

Enterprises often start with technology, search for use cases, launch pilots and measure adoption. Only later do they ask whether the economics work.

Both sessions suggested reversing that logic.

Understand the workflow. Establish the baseline. Define the business outcome. Decompose the work. Decide where deterministic software, probabilistic AI and human judgement each belong. Then measure whether the redesigned workflow actually performs better.

And don’t restrict “better” to cost reduction.

Maddineni closed by arguing that freeing underwriters, actuaries and other specialists from administrative work can create space for better risk analysis, new products and competitive differentiation.

That may be the most useful way to think about AI ROI:

The winners will probably not be the companies deploying the most AI. They will be the companies that understand their business deeply enough to know exactly where AI creates value — and where it doesn’t.

“What was our revenue last month?”

In many companies, that question does not have one obvious answer.

Sales may think about booked revenue. Finance may look at invoiced revenue. Another function may use recognized revenue. Different systems may contain different versions of the number, each perfectly reasonable in its own context.

A human employee who has worked in the company for ten years may know which definition to use. An AI agent does not.

This was one of the points that stuck with me from a session at the Ai4 conference in Las Vegas, presented by the CTO of SIXT. The company has invested heavily in AI, but one of its early findings was that the difficult part was often not building the AI itself. Much of the effort went into understanding the surrounding enterprise: What does this data mean? Where does it live? How can it be accessed? How does the process work? Which systems are involved, and how do they connect?

That is an important lesson for anyone currently building Retrieval-Augmented Generation systems or AI agents.

RAG can help an AI find information. It does not necessarily help the AI understand what the information means.

And as enterprises move from chatbots that answer questions to agents that take actions, that distinction becomes critical.

Retrieval is only the first layer of context

The basic idea behind RAG is compelling.

Instead of expecting a large language model to know everything, we give it access to relevant enterprise information. Documents are indexed, often converted into embeddings and stored in a vector database. When the user asks a question, the system retrieves relevant information and adds it to the model’s context.

This solves an important problem:

Where can I find information relevant to this question?

But consider our revenue example.

A RAG system may successfully retrieve five documents containing the word “revenue.” Unfortunately, one document discusses booked revenue, another invoiced revenue, and a third recognized revenue.

Retrieval has worked perfectly.

The agent may still produce the wrong answer.

This is not fundamentally a data-quality problem. The numbers may all be correct.

It is a semantic problem.

The SIXT presentation described exactly this kind of ambiguity. Depending on the department and use case, seemingly straightforward business concepts can have different definitions. Determining the correct definition can require considerable organizational knowledge—or, as the speaker effectively described it, a form of corporate archaeology.

We have spent years making enterprise information searchable.

Now we need to make enterprise meaning machine-readable.

The forgotten AI infrastructure: the data dictionary

Data dictionaries are hardly an exciting new technology.

Most organizations have had some version of them for decades. They describe tables, fields, data types, business definitions, ownership and sometimes lineage.

Historically, they have often been treated as documentation: useful for analysts, data engineers and auditors, but not exactly at the center of the technology strategy.

Generative AI changes that.

A properly maintained data dictionary can become part of the context that an AI agent uses to interpret enterprise information.

Imagine an agent encountering a field called booking_revenue.

Rather than guessing from the column name, it could retrieve additional context:

• the business definition of booking revenue;

• the authoritative source system;

• the responsible business domain;

• applicable currencies and conversion rules;

• related KPIs;

• known exclusions;

• the owner responsible for the definition.

Suddenly, the dictionary is no longer passive documentation.

It becomes runtime context for AI.

That shift may make investments in data governance that previously felt bureaucratic surprisingly valuable.

But even a very good data dictionary only gets us so far.

A dictionary gives AI a vocabulary. An ontology gives it a worldview.

Defining individual terms is useful.

Understanding how those terms relate to each other is much more powerful.

This is where ontologies enter the picture.

An ontology describes the important concepts of a domain and the relationships between them. In a car-rental business, simplified relationships might look something like:

Customer → makes → Booking

Booking → fulfilled at → Branch

Booking → generates → Invoice

Invoice → contributes to → Revenue

Now the AI has more than definitions. It has a model of how the business fits together.

The SIXT approach described in the session goes in this direction. Metadata and KPI definitions are combined with an ontology representing major enterprise concepts such as countries, branches and customers, including how data and KPIs relate to those concepts.

That provides an agent with something much closer to a map.

It can potentially understand not merely that two documents both contain the term “branch,” but what a branch represents in the organization and how it relates to customers, bookings, systems and KPIs.

This becomes especially important when questions span multiple domains.

Consider:

“Identify branches where revenue declined by more than 10 percent and recommend corrective actions.”

Before an agent can answer reliably, it needs to resolve a surprising number of questions.

Which definition of revenue?

Compared with which period?

Which branch—the booking branch or the return branch?

Which currency?

Are cancelled bookings included?

Which system contains the authoritative value?

Which operating procedures should be considered before proposing an action?

A vector search alone cannot reliably provide all of those answers.

The agent needs context about the enterprise itself.

From RAG to an enterprise context layer

This was perhaps the most interesting architectural aspect of the SIXT presentation.

The company is not limiting this contextual information to structured data. There is additional knowledge covering processes and operating procedures, application landscapes and dependencies, as well as engineering standards and non-functional requirements.
Together, these components form what SIXT’s CTO described as an enterprise context layer that can be reused by different AI use cases.

I think this term is useful because it broadens the discussion beyond RAG.

An enterprise context layer might contain:

Data context: schemas, metadata, lineage, ownership.

Semantic context: definitions, KPIs, business terminology.

Ontological context: relationships between business concepts.

Process context: procedures, responsibilities and operating models.

Technology context: applications, interfaces and dependencies.

Governance context: security policies, coding standards and architectural constraints.

Instead of every AI project independently rediscovering these things, agents can draw on a common contextual foundation.

The architecture therefore starts to look less like:

User → LLM → Vector Database → Documents

and more like:

User → Agent → Enterprise Context Layer → Data, knowledge, systems and policies

That is a significant difference.

Agents make semantic ambiguity dangerous

For a traditional chatbot, getting the wrong definition of revenue is annoying.

For an autonomous agent, it can be an operational problem.

Imagine an AI agent with permission not only to analyze performance but to change pricing, contact customers, modify a booking process or deploy software.

The better our agents become at taking action, the more dangerous ambiguity becomes.

This is why agent governance cannot consist solely of permissions, approval workflows and audit logs.

Agents also need a governed understanding of the environment in which they operate.

We often ask:

What is this agent allowed to do?

We should increasingly also ask:

What does this agent believe our business concepts mean?

If every agent maintains its own collection of prompts, documents and interpretations, enterprises risk creating hundreds of slightly different machine-readable versions of themselves.

That is not scalable governance.

The AI bottleneck may not be AI

There is an uncomfortable conclusion here.

Many companies are currently investing heavily in models, vector databases, agent frameworks and orchestration platforms.

Those technologies matter.

But the limiting factor for enterprise AI may increasingly be much older problems:

inconsistent terminology, undocumented processes, unclear ownership, incompatible data models and several competing definitions of the same KPI.

Generative AI did not create those problems.

It simply makes them harder to ignore.

The good news is that work invested in solving them does not benefit only one AI project.

The SIXT speaker closed with exactly that lesson: investment in these foundations creates compounding results across use cases.

That may be one of the more important architectural shifts created by agentic AI.

Companies that prepared their data for dashboards built data platforms.

Companies that prepare their semantics for machines may be building something even more valuable: the operating context for their AI agents.

At Ai4 in Las Vegas this week, I attended two sessions that came together in a way I had not expected.

The first was a four-hour workshop on advanced Retrieval-Augmented Generation. It went far beyond the familiar “chunk, embed, retrieve, generate” pattern, covering hybrid retrieval, routing, adaptive chunking, indexing, reranking, guardrails, observability, and evaluation.

But the session that really caught my attention came later.

Keith Soura, CTO of Arch, presented a talk with a wonderfully precise title:

“RAG is a Flashlight. Your AI Agent Needs a Map. Graph Theory as an AI Agent’s Superpower.”

The premise was simple. RAG may give an AI system access to enormous amounts of information, but having an encyclopedia containing everything about the world does not tell you how to get from New York to Los Angeles. For that, you need a map: an origin, a destination, and a way to navigate the territory between them.

That points to an important distinction for enterprise AI:

RAG can provide the knowledge. An autonomous agent also needs a map.

Advanced RAG solves a real problem

Basic RAG is conceptually simple. Split documents into chunks, create embeddings, retrieve semantically similar information, and give that context to a language model.

In production, things quickly become more complicated.

The workshop demonstrated how dense and sparse retrieval can be combined, queries routed to different retrieval strategies, results reranked, and chunking adapted to different types of information. It also covered different indexing strategies and, importantly, systematic evaluation of retrieval quality.

Did we retrieve the right evidence? Did the model actually use it? Is the answer grounded? What does each stage cost in latency?

This is where enterprise RAG is heading. The challenge is no longer simply connecting an LLM to a vector database. It is designing a retrieval pipeline whose quality can be measured and continuously improved.

But better retrieval does not solve every problem.

Similarity is not a process model

Soura’s talk focused on regulated financial processes, particularly mortgage origination and alternative investments.

His starting point was that RAG is good at finding relevant context. But similarity does not tell an agent how a business process works.

Consider a mortgage.

Pull a credit report. Verify income. Calculate a debt-to-income ratio. Request documents. Recalculate eligibility. Trigger disclosures. Wait for dependencies. Possibly repeat steps when something changes.

Activities depend on facts, create new facts, and trigger other activities. There are branches, loops, deadlines, permissions, exceptions, and regulatory constraints.

The mortgage system described in the session contained around 1,800 registered activities, roughly 26,000 fact references, and about 5,400 activity-to-activity edges.

You can embed the regulations, operating procedures, and historical cases.

But an embedding does not inherently tell you that activity A must precede B, that B triggers C, that C is currently blocked because a required fact is missing, or that a certain transition is prohibited.

That is not primarily a semantic-search problem.

It is a graph problem.

The graph becomes the map

In Soura’s architecture, facts and activities form a living dependency graph.

Facts might include verified income, a credit score, a document status, or a calculated ratio. Activities consume facts and create new ones. Activities can also trigger other activities.

Once the domain is represented this way, decades of graph theory become available.

Graph algorithms can identify nodes that gate large parts of a process, find cycles, detect tightly connected clusters, and remove redundant paths.

The objective is not to give the agent the entire graph. It is to determine which part matters now.

In the mortgage example, roughly 5,400 relationships could be reduced to around 1,200 edges for high-speed reasoning.

Instead of stuffing an entire enterprise process into the context window, the agent gets the relevant section of the map.

Why this resonated with me

This was probably the part of the presentation I liked most, because the underlying engineering principle was very familiar to me.

I have successfully used essentially the same graph-oriented approach when implementing complex pricing systems in the reinsurance domain.

Reinsurance pricing can require a large number of calculations where the result of one calculation becomes the input to several others. Some calculations can only run once particular data is available. A change to an upstream value may invalidate downstream results and require part of the calculation chain to execute again. Multiple calculations may also share intermediate results.

Representing such a pricing system as a dependency graph makes the complexity much easier to manage.

Input data and intermediate results become facts or nodes. Calculations become transformations. Edges describe their dependencies.

Instead of constructing one enormous procedural calculation sequence, the graph describes how information flows through the pricing model.

Graph theory itself is obviously not new.

What I found fascinating in Soura’s presentation was applying the same engineering principle to AI agents.

The new element is putting a probabilistic reasoning engine on top of deterministic structure.

The language model can still interpret an unusual situation, understand natural language, explain a result, or decide how best to communicate.

But the possible routes through the underlying business system are supplied by the graph.

That is a very different role for the LLM.

The encyclopedia and the map

Soura’s analogy provides a useful architectural model.

RAG gives the agent the encyclopedia.

It retrieves the underwriting guideline, policy wording, previous case, operating procedure, contract clause, or technical documentation.

Advanced RAG increases the probability that the correct pages of that encyclopedia are found.

The graph gives the agent the map.

It describes how things are connected, where the process currently stands, which dependencies remain unresolved, and what actions can legitimately happen next.

You need both.

Without good retrieval, an agent may navigate the correct process using weak evidence.

Without the map, it may have excellent information and still take the wrong turn.

This distinction becomes much more important as we move from conversational AI toward autonomous agents.

A chatbot can generate an incorrect paragraph.

An agent can execute an incorrect action.

Once agents can move money, change policies, approve transactions, or trigger workflows, “probably correct” is no longer a sufficient operating model.

The graph knows about things that are not there

One particularly interesting limitation of embeddings is absence.

You cannot retrieve the embedding of a document that should exist but does not.

Soura illustrated this using tax documents in alternative investments.

If a particular tax statement has not arrived, there is nothing for semantic search to find. But a graph can encode the surrounding relationships and determine that the document is expected because equivalent positions have already produced theirs.

The absence itself becomes information.

Graphs can also carry provenance. In the investment example, relationships were connected back to source documents. An answer such as exposure to a particular company could therefore be represented as a path through the graph, with the individual steps remaining traceable to their underlying evidence.

For regulated enterprise applications, that is much more useful than simply returning a confidence score.

Deterministic rails for probabilistic reasoning

This is why I would not simply call the concept “Graph RAG.”

Embeddings can certainly exist on the graph. Documents can be attached to nodes, and semantic retrieval can provide supporting information.

But the embedding should not be the steering layer.

The graph determines what is reachable.

The LLM reasons within those rails.

Toward the end of the session, Soura described this as putting a probabilistic reasoning system on top of deterministic rails. Facts remain grounded in identifiable sources, while the actions available to the agent come from defined graph transitions.

This also suggests a different approach to agent governance.

Today we often give an agent broad capabilities and then monitor it with guardrails, validation, approval mechanisms, and observability.

The graph offers another principle:

Do not only detect the wrong turn. Make sure the wrong road is not available in the first place.

Of course, this does not make an agent bullet-proof.

A deterministic map can still be wrong. Missing dependencies, incorrect business rules, bad source data, or outdated regulations can make an agent reliably follow the wrong route.

The map itself therefore becomes a governance artifact. It needs provenance, versioning, testing, and auditability.

This shifts the trust question from simply:

Can we trust the model?

to:

Can we trust the map the model is navigating?

For enterprise technology, I find the second question much more actionable.

From better retrieval to safer autonomy

My takeaway from the two Ai4 sessions is not that graph theory replaces RAG.

They solve different problems.

Advanced RAG improves the quality of evidence available to an agent.

Graph-based context improves the quality of navigation available to an agent.

A strong enterprise architecture can combine both: use sophisticated retrieval to find trustworthy evidence, model operational facts and dependencies as a graph, reduce that graph to the relevant state, and then let the LLM reason within that environment.

The model can interpret language, explain results, and choose among legitimate alternatives.

But it should not invent the roads.

Perhaps the next major advance in enterprise AI will therefore not come from ever larger context windows.

We may already have enough flashlights.

What our agents increasingly need are trustworthy maps.

I spent this week at Ai4 in Las Vegas, and one theme came through more clearly than any discussion about model size, benchmarks, or the latest agent framework: enterprises are learning to build AI agents faster than they are learning to control them.

Two keynotes made the problem especially tangible.

Dataiku approached it from the organizational side: AI creation is moving out of central IT and into the business, in much the same way spreadsheets did forty years ago.

AWS approached it from the technical side: once agents can make decisions and invoke tools, governance has to operate at runtime, at the moment an action is attempted.

Taken together, the two talks point toward a governance model that is less about approving AI projects and more about creating an operating system for a new class of digital workers.

The Excel lesson we should not repeat

Dataiku used a comparison that will be painfully familiar to almost every enterprise IT organization: Excel.

For decades, business users have built business-critical applications in spreadsheets because they were closest to the problem and because waiting for IT was too expensive. Finance models, underwriting tools, planning systems, and operational processes grew outside the formal application landscape. IT could complain about shadow IT, but the demand was legitimate.

The keynote argued that the same dynamic is now repeating with generative AI and “vibe coding,” only much faster. Dataiku’s 2026 survey of 900 CEOs found that 96% believe employees are already using generative AI without approval. The important insight is that this should not simply be interpreted as a compliance failure. It is also a demand signal: people are solving problems with the tools available to them because the value of not waiting is becoming too high.

But agents make the spreadsheet analogy considerably more dangerous.

A spreadsheet can contain a wrong formula and produce a convincing but incorrect chart. An agent can do that — and then send the email, update the customer record, execute a transaction, expose data, or trigger another system. Dataiku made exactly this distinction in the keynote: an ungoverned agent does not merely miscalculate; it can act on the mistake.

The risk moves from bad calculation to bad action.

The spreadsheet era gave us shadow applications.

The agent era can give us shadow actors.

Agents are labor

Governance therefore cannot simply mean that central IT approves every agent.

Dataiku’s more provocative argument was that “agents are labor.” The business unit that owns the outcome should increasingly decide which agents are worth deploying, carry the budget, and be accountable for their productivity and ROI.

IT’s role changes from being the permission system to providing the foundation that makes distributed creation safe.

I think this is a particularly useful way to look at enterprise agent governance.

Nobody expects HR to personally approve every task performed by every employee. HR provides identity, onboarding, policies, records, and processes, while line management owns performance.

Agent governance needs a similar separation of responsibilities.

A claims department deploying a claims agent should ultimately own whether that agent improves claims handling. A finance organization deploying an accounting agent should own its business outcome. An IT department cannot realistically become the product owner for hundreds or thousands of domain-specific agents.

But the business should not have to reinvent identity, security, monitoring, auditability, and access control every time it creates one.

That is the platform IT needs to provide.

And the pressure to solve this problem is already significant. Dataiku’s CEO survey found that 83% expect to deploy AI agents in full production during 2026, while only 25% of CIOs say they can monitor all their agents in real time.

From ownership to technical control

The AWS keynote started from the other end of the problem.

The question was not whether we can build capable agents. We clearly can.

The question was whether we can prove that those agents stay within their intended boundaries when nobody is watching.

This becomes difficult because agent risk is contextual.

Imagine an email agent. Giving it permission to use a send email tool is easy. But the real governance question is not:

May this agent send email?

It is:

May this agent send this specific information to this recipient, at this moment, under these conditions, without additional approval?

Traditional access-control systems are good at relatively static allow-or-deny decisions. Agent behavior is different.

A $50 refund and a $50,000 refund may invoke the same tool but should not require the same control. An internal email and an external email may use the same API but carry very different data-loss risks. A routine replenishment order and a six-figure capital purchase are not equivalent actions just because both call the same ordering interface.

This is where agent governance becomes an architectural discipline rather than a policy document.

Put the controls outside the agent

AWS’s answer is a governance layer around the agent, rather than governance embedded only inside its prompt or code.

The keynote described several foundations that need to work together: identity, policy enforcement, an agent registry, observability, and continuous evaluation.

Identity matters because an agent should not be an anonymous extension of a developer account or generic service credential. It should have its own identity, defined privileges, and a clear delegation relationship: which user or business process is it acting for, and with what authority?

Policy must then be enforced deterministically outside the agent’s reasoning loop.

This is an important architectural principle.

If a safety rule exists only in a system prompt, the same probabilistic model that decides what to do is effectively being asked to police itself.

AWS AgentCore Policy instead evaluates tool calls at the gateway boundary. Policies are expressed in Cedar, AWS’s open-source authorization language, and can restrict actions according to identity, tool parameters, and other context before the action reaches the target system.

Then comes observability.

For serious enterprise use, logs cannot stop at HTTP status codes. We need traces across model calls, tool invocations, routing decisions, and policy evaluations so that an agent action can later be reconstructed and audited.

And evaluation cannot be a one-time pre-production exercise. Agent behavior can change when models, prompts, tools, data, memory, or context change. Governance therefore requires continuous regression testing and production evaluation.

From testing behavior to proving boundaries

Perhaps the most interesting part of the AWS keynote went one step further.

Testing demonstrates what happened in the cases we tested. But with agents, the number of possible sequences grows rapidly once multiple agents, tools, roles, data classifications, and communication channels interact.

AWS argued for combining the flexibility of LLMs with automated reasoning to verify properties of the policies governing them.

That represents an important shift in thinking.

Instead of saying:

“We tested the agent and it appeared to behave safely.”

we increasingly want to say:

“This architectural control prevents this class of action regardless of what the model decides.”

The latter is a much stronger basis for enterprise autonomy.

The missing enterprise control plane

Dataiku supplied an interesting story about why all of this matters at scale.

A CIO believed the company had around 40 agents in production. After investigating across the different platforms being used by the business, the number apparently turned out to be more than 400.

The organization lacked a reliable view of who owned them, what they cost, where they ran, and how well they were performing.

That story may become the defining governance problem of the next few years.

Most large enterprises will not have one agent platform.

They will have agents embedded in SaaS products, cloud platforms, low-code environments, business applications, and custom frameworks. Some will be built centrally. Many will not.

The governance architecture therefore has to assume heterogeneity from the start.

This is where Dataiku’s Agent Management concept and AWS’s governance foundation meet: enterprises need something resembling an agent control plane that can discover or register agents, bind them to accountable owners, observe their behavior, enforce policy, and measure both technical and business performance.

Having a long history in the Microsoft technology space, I was also interested what Microsoft has to offer. And Microsoft is heading in much the same direction.

Microsoft Foundry: a similar approach

Microsoft was not at Ai4, but the current Microsoft Foundry architecture is worth adding to the comparison because it reinforces the same design pattern.

Microsoft Foundry Control Plane provides centralized inventory and management for agents, models, and tools across Foundry projects. Its agent inventory can expose information such as status, versions, error rates, token usage, estimated cost, and agent identity.

Interestingly, Microsoft is also extending the concept beyond agents built natively in Foundry. External agents can be registered and instrumented with OpenTelemetry so that Foundry can provide tracing and evaluation while the actual agent continues running elsewhere.

On identity, Microsoft Entra Agent ID treats agents as first-class identities rather than merely generic application credentials. Published Foundry agents receive a dedicated agent identity, enabling agent-specific permissions and audit trails.

That maps closely to the AWS principle that governance starts by knowing exactly which agent is acting and under whose authority.

On runtime control, Microsoft introduced the Agent Control Specification (ACS), an open specification for deterministic controls at five checkpoints: input, LLM, state, tool execution, and output. Microsoft is also bringing Purview data-protection capabilities, including runtime DLP, into the Foundry environment.

Again, the architectural direction is remarkably similar:

Important controls cannot live only in prompts.

They need enforceable checkpoints around the agent.

On operations, Foundry provides tracing and evaluations across model calls, tool invocations, and agent hops, with Azure Monitor and Application Insights supporting production monitoring. Microsoft has also been extending trace-based evaluation beyond Foundry-native agents.

That cross-platform capability will matter increasingly because governance becomes much less useful if it only works for agents built with one vendor’s SDK.

Governance should accelerate agents, not stop them

My main takeaway from Ai4 is that agent governance should not be designed as another approval bureaucracy.

If every meaningful action requires human approval, we have technically governed the agent by removing much of the reason for having an agent.

At the opposite extreme, giving autonomous systems broad credentials and hoping that the system prompt keeps them well behaved is not governance either.

The target is governed autonomy.

That means business ownership for outcomes and budget, an enterprise inventory of agents, first-class agent identities, least-privilege access, deterministic runtime policies, action-level observability, continuous evaluation, and explicit lifecycle controls to stop, quarantine, or retire agents that fail.

The organizational and technical models have to meet.

Business units should be able to “hire” agents because they understand the work. IT and security should build the platform that makes those hires visible, bounded, and auditable. Governance teams should define policies that can be enforced automatically rather than reviewing every action manually.

In the spreadsheet era, enterprises tolerated thousands of critical applications they could barely see or govern because the alternative was stopping the business.

AI agents give us a chance to avoid repeating that history — but only if we build the foundation early.

And there is an interesting paradox here.

Stronger governance may be what finally allows agents to become more autonomous.

When you can identify them, constrain them, observe them, and prove which boundaries they cannot cross, you need fewer manual checkpoints. Governance is not the brake on agentic AI.

Continue reading below the TLDR comic strip.

Business users love Excel for its simplicity, flexibility and ability to automate calculations and handling with data. It has become an ever-recurring pattern: A business analyst needs a simple spreadsheet. The results and speed of implementation are convincing to management. Not long after, the spreadsheet has macros, pivot tables and references to other Spreadsheets and is deeply embedded in business processes.

Excel solutions and their challenges

In no time, the simple spreadsheet has become a piece of the company’s critical business infrastructure with business logic and data transformation rules. It’s a business application but very often without proper testing, documentation and version control.

Often, the solution is tied immediately to its creator – and becomes a risk when the creator retires or leaves the business.

Auditors are watching these types of critical business applications very carefully nowadays. The risks associated with them cannot be ignored anymore. The JPMorgan Chase “London Whale” issue with more than $6 billion losses issue is long ago, but the root causes are not really solved.

Spreadsheet Variations

Not every spreadsheet is the same. This sounds trivial, but it is one of the reasons why Excel governance is so difficult in practice. A simple worksheet used for a one-time calculation has a completely different risk profile than a workbook that calculates quarterly reserves, prepares regulatory reporting, or drives operational decisions.

In most enterprises, spreadsheets exist in many variations. Some are simple lists. Some are calculators. Some are reporting layers on top of exported data. Some are planning models with dozens of assumptions, scenarios, and manual adjustments. Others are closer to full business applications: they contain structured input forms, pivot tables, formulas, hidden sheets, Power Query transformations, macros, external links, database connections, and sometimes even custom add-ins.

This spectrum matters. A workbook that only stores temporary notes may not require formal governance. A workbook that produces numbers used in financial reporting, pricing, forecasting, risk management, or customer communication clearly does. The problem is that the transition between these categories is usually gradual. Nobody starts with the intention to build an uncontrolled business-critical application. It simply happens.

The typical lifecycle is familiar: first, the spreadsheet solves a local business problem. Then more users start relying on it. Then formulas are extended. Then data is imported from another system. Then macros automate repetitive steps. Then management asks for additional reports. Users save files with modifications in different locations. At some point, the spreadsheet is no longer just a file. It has become an application — but without the lifecycle management normally expected from an application.

This is why the first governance question should not be: “Do we allow Excel?” That question is too broad. The better question is: “What type of spreadsheet are we looking at?” A lightweight classification model can help:

This classification is important because spreadsheet risk does not come from Excel itself. It comes from the mismatch between the business relevance of the workbook and the controls applied to it.

Embedded Data

A spreadsheet is rarely just a calculation surface. It is also a data container. And this is where many enterprise risks begin.

Business users often copy data into Excel because it is convenient. They export data from ERP systems, CRM systems, data warehouses, reporting tools, email attachments, SharePoint lists, and operational platforms. The exported data is then adjusted, enriched, filtered, corrected, and combined with other data. From a business perspective, this is practical. From a data governance perspective, it creates a parallel data universe.

Once data is embedded in a workbook, several questions become difficult to answer. Where did the data originally come from? When was it extracted? Has it been modified manually? Which version is the correct one? Are there hidden rows, filters, overwritten formulas, outdated lookups, or hard-coded assumptions? Is personal or confidential data stored in a file that is forwarded by email or saved locally?

The risk is not only that data may be wrong. The bigger issue is that the data lineage becomes invisible. A number in a spreadsheet may look precise, but its origin may be unclear. A workbook can mix current system data, historical extracts, manual overrides, assumptions, and copied values in the same calculation chain. Without explicit documentation, even the workbook owner may struggle to explain the exact path from source data to final result.

Embedded data also creates reconciliation problems. If a spreadsheet contains a local copy of data from a core system, the spreadsheet starts aging immediately after the export. Reports based on this copy may diverge from the official source. Different teams may use different extracts, different assumptions, and different manual adjustments. The organization then spends time explaining differences instead of managing the business.

Modern Excel technologies can reduce some of this risk, but they do not remove it automatically. Power Query, Office Scripts, Python in Excel, xlwings, VBA, and Office.js all allow richer interaction with data, but they also introduce new dependency chains. Python in Excel, for example, is powerful for analysis and machine learning, but the report notes that it runs in secure Azure cloud containers and is not designed as a replacement for process automation or system integration. Office.js provides cross-platform add-in capabilities, but the report also highlights platform fragmentation and production stability concerns.

The governance principle should therefore be simple: business-critical spreadsheets need explicit data lineage. For each critical workbook, it should be clear which data sources are used, how data is imported, which fields are transformed, where manual input is allowed, and which outputs are used in downstream processes.

Algorithms

Many spreadsheets contain algorithms, even if nobody calls them that.

A pricing formula is an algorithm. A reserve calculation is an algorithm. A risk score, allocation key, planning assumption, reconciliation rule, or eligibility check is an algorithm. In traditional software development, this logic would be implemented in application code, reviewed, tested, versioned, and documented. In Excel, the same logic may be distributed across thousands of cells, named ranges, lookup tables, hidden worksheets, Power Query steps, VBA modules, Office Scripts, Python cells, and add-ins.

This makes Excel powerful. It also makes it dangerous.

Spreadsheet algorithms are often highly transparent at the beginning. A business expert can see the formula, adjust it, and immediately inspect the result. This is one of Excel’s greatest strengths. But transparency declines as complexity grows. Nested formulas become difficult to read. Copy-and-paste logic creates inconsistent formula ranges. Hidden sheets hide assumptions. Macros turn visible logic into procedural code. External links create dependencies that users may not understand. Add-ins can move important logic outside the workbook completely.

The technology landscape reinforces this point. VBA is still deeply entrenched, but according to studies is declining, security-sensitive, and often poorly maintained, with high rates of unmaintained and undocumented production code. VSTO is effectively deprecated for new development, with migration to Office.js requiring a rewrite rather than a simple conversion. Python in Excel and xlwings open attractive options for analytical and statistical models, but they also shift algorithmic logic into Python code that needs its own testing, packaging, dependency management, and review.

For business-critical spreadsheets, formulas and macros should therefore be treated as production logic. The relevant question is not whether the logic lives in a “real application” or in Excel. The relevant question is whether the output influences real decisions.

A mature organization should know which spreadsheets contain material business logic. It should define owners for that logic. It should test important calculations with known input and expected output. It should document key assumptions. It should apply peer review to material changes. And it should avoid the worst possible pattern: business-critical algorithms understood by only one person, stored in one workbook, on one network drive.

Solution Development Governance

The uncomfortable truth is that many Excel solutions are developed like prototypes but used like production systems.

This is not a criticism of business users. In many cases, Excel solutions exist because central IT was too slow, too expensive, or too far away from the business problem. Excel gives business experts the ability to solve real problems quickly. That capability should not be removed. But once an Excel solution becomes important, its development model needs to change.

Solution development governance for Excel should not mean forcing every workbook into a heavyweight software delivery process. That would be unrealistic and would destroy the agility that makes Excel valuable. Instead, governance should be risk-based. Low-impact spreadsheets can remain lightweight. Critical spreadsheets need controls that are closer to application lifecycle management.

Excel Customization Technologies

Excel is not just a spreadsheet. Over decades, it has become a development platform. Business users can start with formulas and pivot tables, but very quickly Excel solutions can grow into something much richer: automated workflows, custom user interfaces, external data connections, add-ins, scripting logic, and even machine-learning-enabled analytical models.

This is one of the reasons why Excel remains so powerful in enterprises. It gives business experts a way to build solutions without waiting for a full software development project. But it is also one of the reasons why Excel creates governance challenges. The more customization technologies are used, the more the workbook behaves like an application — and the more it needs architecture, testing, security review, versioning, and lifecycle management.

The main Excel customization technologies each come from a different era and solve a different type of problem.

VBA: the enterprise survivor

VBA is the classic Excel automation technology. It allows users and developers to automate workbook actions, create user forms, manipulate cells, connect to files and databases, and implement procedural business logic directly inside Excel.

For many enterprises, VBA is still everywhere. It is embedded in pricing tools, reporting packs, reconciliations, planning models, actuarial models, financial close processes, and countless departmental applications. The reason is simple: VBA works, it is available inside Excel, and it allowed business users to solve problems long before modern low-code platforms existed.

But VBA is also a major source of technical debt. The attached report describes VBA as declining but still deeply entrenched. It notes low current developer adoption, widespread unmaintained production code, security concerns, and lack of documentation in many legacy VBA estates.

The problem with VBA is not that it is technically useless. The problem is that many VBA solutions were never treated as software. They often have no source control, no test suite, no deployment process, no ownership model, and no formal handover. A macro that started as a convenience can quietly become a critical calculation engine.

For existing VBA solutions, the right strategy is usually not immediate replacement. Large-scale rewrites are risky and expensive. Instead, enterprises should classify VBA workbooks by business criticality. Low-risk macros can remain as-is. Critical VBA applications should be documented, tested, secured, and gradually modernized where there is a strong business case.

VSTO: powerful, but strategically outdated

Visual Studio Tools for Office, or VSTO, was Microsoft’s enterprise-grade way to build Office add-ins using .NET. Compared with VBA, VSTO gave professional developers better tooling, stronger integration with Visual Studio, access to the .NET ecosystem, and richer user interface options.

For Windows-based enterprise environments, VSTO was once a strong choice. It allowed developers to build sophisticated Excel add-ins, connect to enterprise systems, create custom ribbons and task panes, and integrate Excel into broader application landscapes.

But VSTO has a strategic problem: it belongs to the Windows desktop era. It does not fit well with today’s cross-platform Office world, where Excel runs on Windows, Mac, browser, and mobile. The attached report describes VSTO as effectively deprecated for new development and highlights that migration from VSTO to Office.js is not a conversion, but a rewrite.

This matters for enterprises because VSTO solutions may still be business-critical. They may work perfectly today, but they are not a future-proof investment. Organizations should therefore avoid new VSTO development and create a transition plan for important existing add-ins.

The key governance question is: which VSTO add-ins are still needed, and for how long? Some may be retired. Some may be replaced by web applications, Power Platform solutions, Office.js add-ins, or more standard Excel functionality. Others may have to remain in controlled legacy mode until the surrounding business process is redesigned.

Office.js: Microsoft’s cross-platform future, with operational risk

Office.js is Microsoft’s modern framework for building Office add-ins with JavaScript or TypeScript. Its main promise is cross-platform compatibility. A well-designed Office.js add-in can run across Excel on Windows, Mac, the Web, and potentially other Office environments.

This makes Office.js strategically important. It fits the modern Microsoft 365 architecture better than VBA or VSTO. It supports central deployment, web-based development patterns, task panes, dialogs, ribbons, and integration with external services. For enterprise software vendors and internal platform teams, Office.js is the official direction for modern Office extensibility.

However, the technology also introduces a different type of complexity. Instead of local desktop automation, developers deal with browser runtimes, manifests, web security models, API requirement sets, platform differences, deployment channels, and Microsoft 365 update behavior. The attached report highlights serious developer concerns around Office.js, including platform fragmentation, instability, silent breaking changes, and a very large theoretical test matrix across apps, platforms, browsers, update channels, and manifest types.

That does not mean Office.js should be avoided. It means it should be treated as a serious software platform, not as a quick macro replacement. Office.js is attractive when cross-platform reach, centralized deployment, and modern web integration are important. It is less attractive when the solution depends on deep local file system access, tight desktop integration, or highly stable low-level Excel automation.

For critical enterprise use cases, Office.js requires professional engineering practices: automated testing, telemetry, staged rollout, fallback processes, security review, and clear ownership. It may be the future of Excel extensibility, but it is not a free lunch.

Python in Excel: analytics, not automation

Python in Excel introduces a new category of customization. It brings Python-based data analysis directly into the Excel experience. This is highly relevant because many analytical workflows already combine Excel with Python, R, notebooks, or statistical tools.

The strength of Python in Excel is analytical depth. It allows users to work with Python libraries such as pandas, NumPy, scikit-learn, and visualization libraries inside an Excel-oriented workflow. For analysts, actuaries, controllers, data scientists, and technically skilled business users, this can be a major productivity improvement.

But Python in Excel should not be misunderstood as the new VBA. The attached report notes that Python in Excel runs in secure Azure cloud containers and is focused on data analysis, visualization, and machine learning — not on controlling workbook properties, accessing the local file system, or automating external systems.

This makes Python in Excel an excellent fit for analytical models, exploratory analysis, statistical calculations, and advanced visualization. It is not the right tool for replacing all macro automation or for building complex operational workflows.

From a governance perspective, Python in Excel also raises new questions. Which Python packages are used? Is the workbook dependent on cloud execution? Are results reproducible? How are analytical assumptions documented? Who validates the code? As with formulas and VBA, Python code inside Excel can become business-critical logic and should be governed accordingly.

xlwings and external Python automation

xlwings represents another approach: keep Excel as the familiar front end, but move automation and calculation logic into Python. This can be especially attractive for organizations that want to modernize complex VBA solutions without forcing users to abandon Excel immediately.

xlwings is a Python-Excel bridge with documented enterprise use cases in areas such as machine-learning UDFs, spreadsheet monitoring, education, banking, and actuarial process optimization.

The main advantage is architectural. Instead of burying all logic in workbook formulas and VBA modules, more complex logic can be implemented in Python, tested with standard software engineering tools, packaged, versioned, and integrated with broader data science ecosystems. Excel remains the user interface, while Python becomes the calculation and automation layer.

The trade-off is operational complexity. Python environments need to be managed. Dependencies need to be packaged. Deployment to end-user machines must be controlled. Security teams need to understand what code is running and where. This is not necessarily a problem, but it moves the solution from “spreadsheet customization” toward “managed application architecture.”

Modern Excel Solutions

Separate Data and Algorithms from Excel

Separate Data and Algorithms from Excel

One of the most effective ways to reduce Excel risk is not to remove Excel from the business process. It is to remove the most critical data and algorithms from the workbook.

This is an important distinction. In many enterprises, Excel is valuable because it is familiar, flexible, and close to the business user. People know how to filter, sort, compare, simulate, and adjust data in Excel. They can build scenarios, inspect intermediate results, and challenge assumptions. That flexibility is hard to replace with a rigid application screen.

But the same flexibility becomes dangerous when the workbook also becomes the system of record, the calculation engine, and the process control layer. Critical data should not live only in spreadsheet files. Critical algorithms should not be maintained only in hidden sheets, copied formulas, or undocumented macros. Excel can remain the interface, but the authoritative data and business logic should move to governed platforms.

A better architecture is to treat Excel as a frontend.

In this model, data is stored centrally in databases, data warehouses, lakehouses, or application systems. Excel connects to this data through governed interfaces: database queries, semantic models, APIs, Power Query, OData feeds, or add-ins. Business users can still work with the data in Excel, but the source remains controlled. Access rights, data quality rules, refresh logic, lineage, and auditability are handled centrally rather than separately in hundreds of workbooks.

The same principle applies to algorithms. Instead of embedding critical calculation logic directly in spreadsheet formulas or VBA modules, the logic can be implemented in a controlled backend service, database procedure, analytical model, Python service, actuarial engine, pricing service, or business rules API. Excel then calls the algorithm, displays the result, and allows users to inspect or simulate inputs.

This separation changes the role of Excel. It is no longer the place where everything is stored and maintained. It becomes the business workbench.

That has several advantages.

First, data consistency improves. Users no longer work on outdated extracts that were copied into a workbook weeks ago. They access centrally managed data with defined refresh rules and controlled permissions.

Second, algorithm maintenance becomes easier. When a pricing rule, risk formula, allocation method, or validation rule changes, it can be changed once in the central service instead of being manually updated in every workbook where the formula was copied.

Third, governance becomes more realistic. Databases, APIs, and analytical services can be versioned, tested, monitored, secured, and documented using standard IT practices. This is much harder when logic is scattered across worksheets, macros, and local files.

Fourth, business agility is preserved. Users do not lose Excel. They still get the environment they know. They can analyze data, build scenarios, create pivot tables, adjust assumptions, and prepare decision material. But the critical components are no longer hidden inside uncontrolled files.

A simplified target pattern looks like this:

This pattern is especially useful for business-critical spreadsheets that have grown into operational tools. Examples include pricing workbooks, planning models, actuarial calculations, reconciliation tools, risk models, commission calculations, and regulatory reporting support files.

The goal is not to make every spreadsheet dependent on a complex backend architecture. That would be over-engineering. A small personal productivity workbook does not need an API. But once a spreadsheet contains critical data, recurring business logic, or outputs used for management decisions, the separation of frontend, data, and algorithms should be considered.

Modern Excel technologies support this direction in different ways. Power Query can connect Excel to governed data sources. Office.js add-ins can provide controlled interfaces to external services. Python-based approaches can move analytical logic into more maintainable code. VBA and legacy add-ins can also call databases and APIs, although they need stronger controls because of their security and maintenance risks. The important architectural principle is independent of the specific technology: Excel should consume governed data and services, not become the only place where they exist.

This also creates a better migration path. Instead of replacing an entire spreadsheet application in one risky step, organizations can gradually extract the most critical parts. First, move source data into a database. Then replace copied data with controlled queries. Then move complex formulas into a shared calculation service. Then reduce macros to orchestration logic or remove them completely. Over time, the workbook becomes thinner, safer, and easier to replace if needed.

This is often the most pragmatic modernization strategy. Business users keep the tool they trust, while IT gains control over the parts that matter most: data, algorithms, access, quality, and change management.

Excel remains powerful as a frontend. It should not always be the backend.

Choosing the right technology

There is no single best Excel customization technology. The right choice depends on the type of solution, the expected lifetime, the user base, the required integration depth, and the organization’s ability to support the technology.

The broader lesson is that Excel customization is not just a technical topic. It is an application strategy topic. VBA, VSTO, Office.js, Python in Excel, and xlwings all allow organizations to extend Excel beyond the spreadsheet. But the moment Excel is extended, the organization must ask the same questions it would ask for any business application: Who owns it? Who tests it? Who secures it? Who maintains it? And what happens when it breaks?

Excel will continue to rule many enterprises not because it is perfect, but because it is close to the business problem. The challenge is not to stop business users from building solutions. The challenge is to recognize when those solutions have become important enough to deserve professional engineering and governance.

Important Governance Controls

At minimum, critical Excel solutions should have:

This type of governance is especially important because Excel extensibility is in transition. The report describes VBA as legacy but still widely present, VSTO as effectively deprecated for new development, Python in Excel as powerful but narrow, xlwings as useful for Python-based automation, and Office.js as the cross-platform direction but with real operational concerns. That means enterprises should not only govern existing workbooks. They also need technology decision rules for future Excel-based solutions.

A useful policy could be:

· For simple analysis, use standard Excel functionality.

· For repeatable data preparation, prefer Power Query or governed data pipelines.

· For business-critical automation, avoid unmanaged VBA where possible.

· For statistical or machine-learning analysis, consider Python in Excel or Python-based architectures.

· For enterprise-grade cross-platform add-ins, evaluate Office.js carefully, including testing effort, deployment model, supportability, and platform limitations.

The strategic goal is not to eliminate Excel. That would be unrealistic. The goal is to prevent uncontrolled Excel solutions from becoming invisible enterprise applications. Excel will remain part of the enterprise landscape because it is flexible, familiar, and incredibly productive. But when Excel is used as an application platform, it needs the same management attention as any other application platform.

A spreadsheet may start as a file. In many enterprises, it ends up as a system. Governance needs to start before that transition becomes invisible.

Download my exclusive review on empirical effects of AI-assisted software engineering on signup for free.