Let’s look under the hood and consider some important design decisions.

Code mode — where LLMs write small programs rather than call single-verb tools —is the future. It’s far superior to traditional tool calling, and Embabel Worlds embraces it.

This raises the question of what language should we ask LLMs to code in?

There need not be a single choice. LLMs are proficient in every viable language, so we can even let them choose — assuming we can provide necessary surfaces for each language. Even second or third tier languages have more than enough available training data.

Why a Declarative Query Language?

We can also choose between types of language. Choosing between Python and TypeScript is largely about libraries; choosing between different language families, like query or logic programming languages, has different pros and cons that are important to consider.

We chose a declarative query language as a core choice for Embabel Worlds.

Declarative query languages have a number of benefits. They are:

• Optimisable: The query specifies the intent and the engine chooses the execution plan, with the freedom to improve it.
• Analysable before execution: Queries can be inspected, costed, permission-checked and rewritten. Code in regular high level languages cannot be to the same extent.
• Safe: They can be limited to be read-only and bounded, with no arbitrary side effects.
• Joins are first-class: Combining data is a declaration, and doesn’t require looping.
• Data independent: With an appropriate planner, queries can be independent of how data is stored, surviving changes to storage, schema layout and access paths.
• Concise and verifiable: Short queries, validated against a schema, allow fewer places for bugs (or LLM errors) to hide.

In short, they’re perfect for AI agents to write in.

You can’t do everything an agent might want to do in a declarative query language, but what you can do, you should. It‘s an ideal way to ground LLM responses.

Much Bigger Than a Database

But aren’t we limited to working with data in a database that supports our query language?

Absolutely not.

As we build out a critical new infrastructure layer, the query language should not dictate data representation.

Rather than try to put everything in a particular database — which is both impossible and undesirable — we should intentionally separate query language from data sourcing.

The answer is federation. This is not a new idea. However, it’s more important than ever before, given the proliferation of systems that people now care about. It’s particularly well suited to working with LLMs:

• The value of getting the right data to an LLM is so great that the cost of a federated query is more acceptable than in traditional architectures.
• It offers a novel way to get model intelligence inside query paths, where it directly acts on data.

Making federation performant is a hard engineering problem. Solving hard engineering problems is the job of infrastructure vendors, and we welcome the challenge. A core part of Embabel Worlds is our sophisticated query planner that can work with a variety of sources, considering security, performance, rate limiting, batching, and other considerations.

Which Query Language?

The obvious choice is SQL. Immense amounts of training data; many implementations; federation-ready projects like the excellent Apache Calcite.

But is SQL the right fit? Sure, it enables us to join data between systems and perform useful functions like counting, grouping, and math in a concise and readable way. But it is verbose and costly to perform deep navigation between entities — something that LLMs may need to do to answer questions. And modeling rich relationships between entities via join tables is complex and costly.

We considered SQL, but chose Cypher, the graph query language. Cypher is widely adopted, and is the starting point for the GQL standard.

Cypher offers SQL-inspired filtering and grouping, but also has edges that carry meaning. Relationships are first class. It’s much more natural for querying dynamic data and following deep traversals.

When relationships are first class, they can have properties. This is a intuitive model. For example, consider the concept of an actor appearing in a movie, and how naturally it’s expressed in Cypher:

MATCH (a:Actor)-[r:ACTED_IN]->(m:Movie)                                                                                                             
  WHERE m.year > 2000                                                                                                                                 
  RETURN a.name, r.role, m.title    

To do this in SQL we’d need a join table. Likely costlier, and certainly less intuitive and more verbose:

SELECT a.name, ai.role, m.title                                                                                                                     
  FROM actors a                                                                                                                                       
  JOIN acted_in ai ON ai.actor_id = a.id                                                                                                              
  JOIN movies m    ON m.id = ai.movie_id                                                                                                              
  WHERE m.year > 2000;       

In many cases, Cypher is so compelling that the main objection to adopting it has so far been the need to introduce a new database. Data tends to stick where it lands, so by decoupling the query language from the storage, we remove that understandable objection.

A context layer is critical infrastructure that must support multiple client types. Cypher is compelling as a primary choice, but we don’t need to reject others. We can also add SQL, GraphQL and other surfaces.

A sophisticated context layer provides insights independent of the data representation.

When we want SQL, we can put a view on top of our context layer, as Embabel Worlds supports. This gives us the best of both worlds — the intuitive and expressive Cypher for agents, and the incredible reach of SQL into existing, established tools.

Cypher Over Everything With Embabel Worlds

Embabel Worlds provides federated Cypher across a wide and extensible range of sources.

We don’t limit our Cypher queries to what sits in a particular graph database. The graph is a view, which should not dictate data storage. We have our own blazingly fast openCypher TCK-compliant Cypher implementation that can work over a range of sources.

Embabel Worlds can run on a graph database but doesn’t need one. We can store data in SQLite, Postgres and Oracle, allowing you to use a database you already run, while also supporting tools that speak Cypher.

It enables Cypher over a range of sources — existing databases, APIs, tabular data — whatever you choose to expose. Consider the following example around Australian Electoral Commission data, querying Australian House of Representatives seats:

MATCH (e:Electorate) 
RETURN e.division, e.member, e.marginPct, e.state

Straightforward…so far, not magical. But consider how simple the query is to see which are the most marginal seats held by the government:

MATCH (e:Electorate) 
WHERE e.marginPct < 3 AND e.heldByAbbrev = 'ALP'
RETURN e.division, e.member, e.marginPct, e.state
ORDER BY e.marginPct ASC

Compared to a Python, JavaScript or other program, this is simple and intuitive.

Having the ability to operate over APIs is surprisingly powerful, but it gets much better the more systems you relate.

Consider a query that joins multiple systems. We need to agree on a key to join on — Embabel provides a sophisticated mechanisms for that. But the result is powerful and intuitive. Imagine we want to find which customers owe us money while they’re waiting on an resolved support case. This joins systems that know nothing about each other, providing insight above them. SupportCase comes from a ticketing system, while BillingInvoice is from a payment system:

MATCH (c:SupportCase {status: 'open'})<-[:HAS_CASE]-(a:CustomerAccount)                                                                             
        -[:BILLED_AS]->(:BillingCustomer)-[:HAS_INVOICE]->(inv:BillingInvoice)                                                                        
  WHERE inv.paymentStatus <> 'succeeded'                                                                                                              
  RETURN a.name AS account,                                                                                                                           
         count(DISTINCT c) AS openCases,                                                                                                              
         sum(DISTINCT inv.amountDueCents) / 100.0 AS owed                                                                                             
  ORDER BY owed DESC   

All of these nodes are virtual, although we can apply the full power of Cypher to them. None is copied into persistent storage. The Cypher — and the LLM that is likely generating it — knows nothing about how these nodes are retrieved and materialized, which involves API calls and secrets. That information is provided by the Embabel realm definition that extends our ontology. The same metadata also supports exploration in the Embabel Worlds Query Studio, with autocompletion and tool tips:

Adding Model Intelligence

Queries in Embabel Worlds are legal Cypher, which helps LLMs generate them. We considered our own DSL to extend Cypher capabilities, but decided against it. There are more than enough languages already.

However, this does not prevent us extending the capabilities of Cypher for the AI era. For example, we add functions that apply model intelligence inside queries. Take this query, which classifies issues. The ai.classify function uses a model to classify issues based on a prompt. The virtual Cypher execution engine is responsible for parallelizing the model calls and getting them the right, focused, content. The user can choose the model — even models other than LLMs for some tasks:

MATCH (:GitHubRepository {full_name: 'embabel/me'})-[:HAS_ISSUE]->(i:GitHubIssue)                                                                   
  WHERE i.state = 'open'                                                                                                                              
    AND coalesce(i.isChangeRequest, false) = false                                                                                                    
    AND ai.relevant(i, 'describes a defect or missing capability, not a question, chore or discussion')                                               
  RETURN i.number                                                          AS issue,                                                                  
         i.title                                                           AS title,                                                                  
         ai.classify(i, 'kind: bug, performance, usability or documentation') AS kind,                                                                
         ai.classify(i, 'blast radius: one user, one realm or every appliance') AS reach,                                                             
         i.comments                                                        AS comments                                                                
  ORDER BY ai.score(i, 'how much this hurts people using the product today') DESC                                                                   
  LIMIT 15

Embabel Worlds also supports extracting structures from text, with the graph shape specified in Cypher:

CALL ai.fromText(                                                                                                                                   
    'Ada Lovelace worked with Charles Babbage on the Analytical Engine, which he designed in London.',                                                
    '(p:Pioneer)-[w:WORKED_ON]->(m:Machine)'                                                                                                          
  ) YIELD p, w, m                                                                                                                                     
  RETURN p, w, m     

Normally, as in this case, the types would be defined in an Embabel realm, but it’s possible to define them inline, using our other preferred programming language — TypeScript:

CALL ai.fromText(
  'Ada Lovelace worked with Charles Babbage on the Analytical Engine, which he designed in London.',
  '(p:Pioneer)-[w:WORKED_ON]->(m:Machine)',
  {types: "
    class Pioneer extends Entity {
      declare name: string
      @Relationship({ type: 'WORKED_ON' })
      declare worked: Machine[]
    }
    /** A machine or invention. */
    class Machine extends Entity {
      declare name: string
    }
  "}) YIELD p, w, m
RETURN p, w, m

Both these two queries produce the following graph, as shown in the Embabel Worlds UI:

The ability to move between text and structure, and introduce as much structure as possible, is critical to making Gen AI dependable. Embabel agent framework and Embabel Worlds provide a sophisticated solution.

Of course using model intelligence inside Cypher queries, and federating sources that may be rated limited, means ensuring that queries can run without locking any database. Our runtime takes care of this.

The full virtual Cypher spec is public: see the cheat sheet for other examples of what’s possible.

Our Cypher implementation also provides excellent error messages. Because queries are predominantly LLM-generated, there’s even greater value in explaining precisely why a query is invalid, to help the model retry. This feedback goes beyond syntax — as in a traditional database — to semantic errors.

Adding Views and Watches

We build beyond Cypher in various other ways, such as:

• Introducing Views. Unlike SQL, Cypher does not offer views. We do, and it’s a great unit of query composition and reuse.
• Allowing views to be watched. Changes in their results can have business importance and
• Establishing query pipelines using the GOAP planning capabilities of Embabel.

Your Context Layer Awaits

Embabel Worlds offers a powerful semantic layer you can try today. Experiment in our Query Studio to see just how powerful our Cypher implementation is. You don’t need to introduce a new database. It can query whatever you care about — your databases, APIs, external systems, or anything else you care about. You’ll be up and running in minutes. What will you build?

South African coffee franchise exposes company and private data in vibe-coded web application

It’s not surprising that unconstrained vibe coding is dangerous. While the failures are entertaining, they shouldn’t lead us to dismiss amazing new potential. Agents’ ability to generate code should be a huge net positive for non-technical knowledge workers. It’s incumbent on skilled developers and software infrastructure companies to make it safe.

Mirage Becomes Reality

For more than forty years, the software industry has promised to eliminate programmers, and failed. Today this promise can — and should — be delivered — to a point.

From James Martin’s 1982 Application Development Without Programmers, championing fourth-generation languages to empower business users, through CASE tools in the 1990s, Rational Rose and other modelling tools in the 2000s, BPM vendors such as Appian promising “zero-code” development in the 2010s, through Google and Microsoft talking of “citizen developers” in the early 2020s, the promise has been restated.

Each generation delivered some useful technology. None delivered the promised escape from programming. In September 2026, Gartner reported that “few citizen developers can build applications safely and effectively.”

Today, however, coding agents can build sophisticated applications in real languages. Non-technical users can create apps quickly. The challenge is safely and effectively.

Vibe coding without guardrails is dangerous, and anything but a step forward. However, that doesn’t mean we should give up on its promise. We need to work out where it applies, where it doesn’t, and architect appropriately.

The solution is realism and layered infrastructure.

Wholly replacing professional developers is dangerous and undesirable. While coding agents can greatly increase their productivity, their skills — especially in design — remain important to solid outcomes.

Everything depends on good architecture. Everything we know about layering, design patterns, loose coupling, testability etc. still matters. In fact it matters more than ever.

With sound architectural layering, we can create systems that allow non-technical users to safely vibe code real applications — not just prototypes — that benefit from their domain understanding and reflect their personal preferences. This finally realizes the promise of empowering business users.

Let’s look at some of the implications.

Everyone gets their Own Pane of Glass

Every user will have their own personalized interface to make them happier and more productive, and this can be achieved without risk.

We’ve all suffered from UIs that weren’t built for us, and the need to jump between disconnected UIs. We all know the Microsoft Word problem, where the menus are so complex because of a range of features no individual user needs; and the multiple-SaaS problem, where you need to jump between different interfaces. These problems will soon be a thing of the past.

Every user will have their own personalized UI, or as many UIs as they want. They will be able to evolve their apps easily and share them where appropriate. No human — not even skilled developers — will look at code in this layer.

Applications Will Proliferate, But Application Builders are In Trouble

Building applications will decline as a business model. On top of a strong underlying architecture, AI can build anything quickly, removing many moats.

Companies delivering applications will survive only if they accept that they are not really technology companies, and radically reduce their development costs by building on shared infrastructure.

Viability will depend on non-technical factors, such as:

• Deep understanding of and relationships within a domain. Especially in niche markets, specialist expertise retains value.
• Access to proprietary data or feeds. Lack of good data is a huge barrier to AI. This barrier to entry will likely diminish as owners of data and feeds enable a wider range of agentic clients.

What Infrastructure Does This Future Rest On?

These benefits require understanding of architecture, and a new infrastructure layer.

I recently blogged about the need for a context or semantic layer that sits between AI models and relevant data and systems, presenting a coherent, scriptable, AI-ready view of everything underneath. While it can be built in house by large businesses (and presently often is), a context layer is generic infrastructure.

The context layer is the cutoff point above which vibe coding becomes not only safe, but the best way to build. It provides models with understanding of what’s underneath and a solid way of accessing data, managing usage and enforcing guardrails.

Users should be able to build against a context layer with familiar tools such as coding agents.

Let’s look at some examples of what can be built easily, and safely, over a strong context layer.

Consider this application based on UK public data. It took about half an hour to write on top of Embabel Worlds, with the help of Claude Code. It uses six disconnected UK government APIs, relating them via ONS local authority district code. Embabel’s federated query capability illustrates the power of a strong context layer to bring insight across many systems:

A context layer also makes it easy to put model intelligence over existing apps: a great alternative to paying your SaaS vendor for its own expensive AI features. For example, a quickly built intelligence layer over Odoo using queries that incorporate LLM summarization:

The value compounds when you join across more than one system.

A context layer should allow app hosting as a first class citizen. Apps built on top of a strong context layer avoid the potential dangers of vibe coding, and enjoy important advantages. For example:

• They can be hosted securely, with a stable URL. A non-technical user doesn’t need to deploy them.
• They can be shared easily with colleagues or even published more widely.
• Their value compounds as the data and actions available to the underlying infrastructure layer broadens. Sharing can unlock further value.
• They can be validated consistently through static analysis, ensuring they’re safe and acceptably performant.
• An underlying platform gives visibility of what an organization is running, versus allowing a mishmash of insecure applications.
• Access to them can be logged and audited.
• They benefit from multi-channel communications, event streams and other infrastructure services.

Embabel Worlds offers these and other benefits today. You can be up and running in a few minutes, ready to connect your coding agent to build apps against the systems you care about. What do you want to build?

By the mid 2000s, it was understood that technologies should be easy to evaluate and if they weren’t, they probably sucked.

Twenty-five years later, we’ve gone backwards. The industry now accepts — and sometimes even seems to celebrate — armies of FDEs (Forward Deployed Engineers).

Announcements like this are now taken seriously:

Today we are launching Claude Frontier Academy to solve one of the most pressing issues in AI implementation: talent. Backed by a $100 million commitment, Anthropic aims to train 10,000 Frontier Deployed Engineers (FDEs) by the end of 2027

This is in Anthropic’s interests — what vendor wouldn’t want important companies to sign up to be indoctrinated and work around a gap in their offering— but not customers’.

This is not how software problems should be solved at scale. It also rests on the questionable assumption that enterprise developers are country cousins who can’t be expected to understand how to connect the systems they’ve built to forward-looking technologies. Enterprises buying AI are like your grandma getting an iPhone: they want to use it, but they need you to set it up. Really?

A little less AI arrogance and exceptionalism is in order. We can and should learn from the experience of previous technology waves. Whenever we’ve seen countless non software companies try to solve the same problems, it’s been an indication that a new product category is needed. Consider ERP, CRM, business intelligence, application servers or web frameworks.

History strongly suggests that these ad hoc in house systems will be ripped out and replaced with better, generic solutions that allow customers to choose models and model classes. There is zero competitive advantage to any business outside software in building generic infrastructure; active disadvantage in tying themselves to a single frontier model vendor.

Centering on consulting in a world when products can literally talk to users to help them onboard is absurd.

The FDE surge indicates several things:

• Excessive influence by frontier model vendors on business users. Model intelligence is rapidly becoming fungible and open weights models are sucking the oxygen out of an already questionable business model. Betting on any a frontier model vendor is questionable.
• The gaping product gap between the frontier model vendors and business customers. Models alone are not enough and customers need an additional infrastructure in a context layer.
• Existing products attempting to fill the context layer gap are too hard to use. “Call sales for a demo” is the norm, and customers deserve better in 2026.
• Lack of imagination in fully leveraging new technology to help onboarding. Frontier models are great; let’s use them to help achieve their full potential in working with our existing systems.

Embabel Worlds takes a very different approach. We aim to empower users to self service as far as possible. And use coding agents connected to our MCP surface to help them do so.

Want to add AI to your business? Just ask:

You get the full value from your frontier model, while building a lasting understanding of your business that can be used with any model of your choice, and will deliver value without any models.

And you don’t need to call us — you can go to worlds.embabel.com and evaluate today.

We hope that Embabel Worlds will become the dominant context layer. Whether or not it is the category winner, the category will win. Product functionality, rather than consulting, will succeed in AI-enabling the enterprise.

Besides ad hoc personal productivity enhancement, coding agents remain the sole home run. Enterprise workflows remain largely untouched, and superior insights across the business largely theoretical.

There are various reasons for this, but it’s useful to step back and think about architecture. Does AI add most value inside our existing systems, or on top of them?

Many systems can be enhanced by AI: for example, given the ability to process natural language, or make subjective decisions that previously required human input. This is what frameworks like LangChain help with, and what our Embabel agent framework offers the Java community. Given the volume and importance of Java applications, Java is especially important for this kind of enhancement.

However, AI on top of enterprise systems is arguably more important, enabling users to talk to the business, uncover insights across formerly siloed systems, and leverage the unprecedented ability of coding agents to compile their intelligence into custom UIs. Unlike building AI into existing codebases, AI on top of existing systems doesn’t necessarily require development skills, meaning its potential audience is larger. Code in this layer need not be produced (even indirectly) by developers, but can be generated by LLMs, and run within constrained, safe, environments.

LLMs are a becoming the default client, as web browsers did a generation ago.

New client types typically require new architectural layers. In the 90s that brought us middleware, as hundreds of internal users who could directly connect to databases and other internal systems were replaced by millions of untrusted external users with browsers. Today it demands something new, as clever but even more untrustworthy LLMs replace browsers: a context layer that sits on top of the business to safely and efficiently expose it to AI.

The New Architectural Layer, and New Product Category

AI models need to get at what matters in a business, in a way that captures meaning and relationships, and is governed, explainable and auditable.

This layer should provide:

• Understanding of individual systems and the relationships between them
• Efficiency in allowing LLMs to interact with underlying systems with a minimum number of tool calls and without excessive token usage
• Governance in what is permitted and who can authorize it
• Explainability of why was something done
• Auditability around who did what

As the need for and shape of this new architectural layer have become evident, several names for it have emerged, with no clear winner as yet. Besides context layer, they include semantic layer, work graph, knowledge layer, and ontology — the latter associated with Palantir.

To illustrate the role and importance of the context layer, let’s consider why coding agents work so well. They were one of the first proof points of the necessity of a layer above even the best LLMs, in the agent harness. But coding harnesses and the models they worked with benefited from something else that came for free — a powerful context layer. Models have been trained on your programming language, git, GitHub, your operating system and the scripting languages that likely run on it. They know what to call and when, and how to call it most efficiently. By contrast, they don’t know your business, and a comparable understanding must be provided to them in a context layer.

Consider questions across systems. If a coding agent needs to ask questions across multiple files or GitHub issues, it writes a script. There is no existing scripting language for your business.

Imagine if you want to ask questions across your CRM, payment and ticketing systems, such as which customers have upsell deals which have slipped or late payments, and are angry about unresolved issues? An LLM doesn’t know how to write a query or script to answer that. A context layer gives it that ability.

Why a Context Layer vs MCP?

Suppose we work with three SaaS products and two internal systems. The SaaS vendors probably ship MCP servers. Can’t we just task our internal teams to put MCP servers on top of their applications and declare victory?

No. With this naive approach, we’re asking models to improvise the solution to every problem. They need to call individual tools on each system and coordinate the results. Tools will either be too narrow (forcing chatty calling) or too prescriptive (limiting what LLMs can achieve). A good enough model may still arrive at the right result, but it won’t be repeatable or efficient. And smaller models will likely not be an option.

MCP was a big step forward in connecting LLMs to the real world. But it is a tactical, rather than strategic, solution that does not provide the ideal substrate over complex estates.

A decisive proof point is how coding agents constantly write scripts. Their tool use looks nothing like typical MCP tool use.

They naturally use code mode — a far better approach to tool calling well explained in an excellent Cloudflare blog. In code mode, instead of calling a bag of tools, each to perform a point operation, LLMs write scripts against a known substrate. This preserves context and takes advantage of the fact that LLMs are fluent in all major programming languages.

Code mode is usually more efficient, as each tool call is costly in tokens and time. The more we can do in each tool call, via a script, the better. The better the API surface area we expose to the LLM, the greater the win.

The job of a context layer is partly to expose interfaces that LLMs can code against.

Who Builds the Context Layer?

When an essential new architectural layer appears, everyone needs it, so they build it themselves or get someone to build it for them. It then becomes apparent that the problems it solves are generic, and that in-house build is not a competitive advantage, but a liability.

Important new architectural layers inevitably become products. The context layer is a product category, not an in-house build.

We’re seeing this play out now. The industry is accepting as normal armies of FDEs (Forward Deployed Engineers)— the fashionable AI-speak for consultant. This is a classic indication of a missing or ill-fitting product, where everyone is thrashing around trying to build something that they should find off the shelf.

We’ve seen this movie before. For example, in the early years of enterprise Java, every company built its own Java framework, trying to solve generic problems that had nothing to do with their business. Spring and Hibernate proved the immense value of common solutions the industry could converge on, while end developers focused on on solving their unique business problems.

We’re presently seeing two classes of vendor offer context layers:

• Vendors trying to expose their own services, such as Microsoft, Salesforce and Workday
• Infrastructure providers such as Palantir and Glean, who help the customer work with multiple systems

The latter vendor category is better aligned with the interests of customers. The context layer should span more than one vendor’s products, embracing internal systems and whatever the customer cares about.

However, customers deserve better than present context layers, which are often difficult and expensive to deploy. So we built a better solution.

Introducing Embabel Worlds

Today we’re announcing early access to Embabel Worlds, our innovative context layer built on the JVM, on top of the Embabel agent framework.

Embabel Worlds is a layer above your business that understands it and exposes it to however you want AI to work with it.

Embabel Worlds has a unique approach, which we feel has major advantages:

• It’s dynamically extensible: Each user can have their own world — their own extensible knowledge graph.
• It’s contribution based — worlds can be extended by realms. Each realm exposes a system or capability, and realms can be provided by us, third parties or authored by any user.
• It emphasizes a declarative query language. Queries are often the best way to ground LLM responses.We chose Cypher — the graph query language — as it is well suited to the dynamic navigation required, and LLMs are good at generating it.
• It is federated over whatever you need to work with.
• It’s easy to roll out. You won’t need an army of FDEs — you can largely self serve.
• It unifies RAG and all other functionality through the same query language, offering modern agentic RAG.
• It’s AI first from top to bottom — not just in the surface the context layer provides, but in how it’s meant to be extendedvia first-class support for coding agents.
• In a rapidly moving area, Embabel Worlds is not encumbered by legacy, hence its surface is based around code mode. LLMs can safely code against your dynamic knowledge graph, exposed via graph schema or TypeScript interfaces

With Embabel Worlds, your knowledge graph can be extended through anything that matters to you, such as:

• Databases
• APIs, including SaaS APIs and your internal systems
• Tabular data
• Anything you can imagine

Embabel Worlds enables you to create a range of reusable artifacts that increase determinism and reduce cost, such as:

• Views: Reusable Cypher queries that can be watched for deltas.
• Agents: Intelligent componentsthat can respond to events, chat with users and execute workflows.
• Apps: Custom UIs that users can safely vibe code on our infrastructure.

Embabel Worlds is commercially licensed. We’re still working on pricing, but it will be free for individual use.

Code Mode In Action

What does it mean to script your business?

Let’s consider two examples: one a declarative query, the other in TypeScript.

First, a query across multiple systems. A support desk knows how many replies a case has, but not what they mean. A 42-reply thread about dark mode is a popular feature request, while a 7-reply thread from a customer who cannot close the month is an account about to leave. No count can distinguish between them, and keyword matching will be unreliable.

This query returns a row for each open support case, prioritizing and including two model-judged labels: how much the problem is hurting the customer’s business now, and how they currently feel about us.

Mood is judged on the last three things the customer wrote, because the question is how they feel now, not how they felt when they opened the case.

Everything here is standard Cypher (making it easy for an LLM to generate), although reliant on the Embabel virtual Cypher engine to federate API calls, manage caching and execute the model calls.

MATCH (d:ChatwootDesk {status:'open'})-[:HAS_CASE]->(c:ChatwootConversation)                                                                     
  WITH c, c.accountKey AS domain                                                                                                                   
  ORDER BY c.replyCount DESC                                                                                                                       
  LIMIT 150                                                                                                                                        
  MATCH (a:CustomerAccount {accountKey: domain})                                                                                                   
  WITH a, c                                                                                                                                        
  MATCH (c)-[:HAS_MESSAGE]->(m:SupportMessage)                                                                                                     
  WHERE m.messageType = 0 AND NOT coalesce(m.internal, false)                                                                                      
  WITH a, c, m                                                                                                                                     
  ORDER BY m.sentAt, m.id                                                                                                                          
  WITH a, c, collect(m.body) AS wrote                                                                                                              
  WITH a, c, size(wrote) AS customerMessages,                                                                                                      
       reduce(t = '', x IN wrote[0..1] + wrote[1..][-7..] | t + ' | ' + x) AS said,                                                                
       reduce(t = '', x IN wrote[-3..] | t + ' | ' + x) AS saidLately                                                                              
  WITH a, c, customerMessages,                                                                                                                     
       classify(said, 'blocked,degraded,inconvenienced,asking',                                                                                    
         'Each item is what one business customer wrote across a support case about a software product: their messages oldest first, separated by  
  |. Judge how much the problem hurts THEIR business NOW. A later message outweighs an earlier one: a workaround or a fix reported mid-thread      
  lowers the impact, and a report that it got worse raises it. blocked: something they rely on does not work and they have no workaround. degraded:
  it works badly or intermittently, or a workaround is costing them. inconvenienced: a nuisance or cosmetic fault. asking: a question, or a        
  request for something that does not exist yet.') AS impact,                                                                                      
       classify(saidLately, 'frustrated,neutral,positive',                                                                                         
         'Each item is the most recent thing one business customer wrote on a support case, up to three messages oldest first, separated by |.     
  Judge the tone of the writer toward the vendor NOW from the words actually used, not from how serious the problem is.') AS mood 
RETURN a.accountKey AS accountKey, a.name AS account, c.id AS caseId, c.subject AS subject,                                                      
         c.priority AS priority, c.replyCount AS replies, customerMessages, c.openedOn AS openedOn,                                                
         impact, mood                                                                                                                              
  ORDER BY CASE impact WHEN 'blocked' THEN 0 WHEN 'degraded' THEN 1 WHEN 'inconvenienced' THEN 2 ELSE 3 END,                                       
           CASE priority WHEN 'urgent' THEN 0 WHEN 'high' THEN 1 WHEN 'medium' THEN 2 ELSE 3 END,                                                  
           CASE mood WHEN 'frustrated' THEN 0 WHEN 'neutral' THEN 1 ELSE 2 END,                                                                    
           replies DESC

This is intelligence made toolable, understandable and reusable.

Ordering falls naturally out of using a full featured query language with ORDER BY.The query joins with the CRM to find the canonical customer identification. A focused prompt is used for the two invocations of the classify function.

In the Embabel Worlds query studio, results will look like this:

Such queries are typically saved as reusable views and used to inform LLM responses or agent actions.

TypeScript scripts work against dynamic, user-specific generated interfaces. Scripts can issue Cypher queries and work with exposed methods, whether on returned entities, as below, or defined elsewhere in a realm:

const suppliers = await world.query(`
  MATCH (s:Supplier)-[:BILLED_BY]->(i:Invoice)
  WHERE i.disputed IS NULL RETURN s, i`);

for (const s of suppliers) {
  if (s.exposure() > 250_000 && s.is(Watchable)) {
    await s.watch({ reason: 'exposure over threshold' });
    await s.owner().notify({ template: 'exposure-review' });
  }
}

All code artifacts are validated before being executed. TypeScript scripts are executed in a sandbox.

Being able to script across the business provides significant benefits:

• LLMs (or humans) can easily author queries and scripts, as they’re in familiar languages.
• Scripts can be reasoned over by people and tools far more easily than an improvised sequence of MCP tool calls.
• Scripts can be saved and reused. When they are LLM-authored, this provides a way to compile LLM intelligence. For example, a script may be generated when a frontier model is available, but executed with a lesser model or without the need for any model.
• Scripts naturally decompose a problem into multiple steps: a key requirement for making AI usage as deterministic as possible. Many scripts don’t need any AI model at runtime— the cheapest and best option, where available. Those that do need an LLM — like the classification steps in the Cypher example — can often be executed by a smaller, or specialized model.

Evaluate Today

You can download the early access today, and connect your internal systems to it behind your firewall. Go to worlds.embabel.com. If you have Docker Desktop installed already, you’ll be up and running in under 10 minutes.

The industry has become much too tolerant of high touch product onboarding. While we understand that early users may need some hand-holding, we aim to ensure that users can ultimately self service.

Embabel Worlds and Java

Embabel worlds builds on the JVM and on our embabel-agent and other open source projects. However, users do not deploy code in Java or any other language to an Embabel Worlds server. You don’t need to write code to expose your custom applications to Embabel Worlds.

However, if you are a Java developer, you can use your skills to make Worlds more efficient, by adding behavior and metadata inside your apps to ensure that they are the most performant possible realms. See https://github.com/embabel-worlds/publisher for more information, and our example of AI over the Spring Pet Clinic. For example, a declaration in the OwnerService exposes a search operation, specifying pagination characteristics:

@RealmQuery(maxPageSize = 200)
 public Page<OwnerView> search(OwnerSearch search, Pageable pageable) {
  String prefix = (search.lastNamePrefix() != null) ? search.lastNamePrefix() : "";
  return this.owners.findByLastNameStartingWith(prefix, pageable).map(OwnerView::of);
 }

As the Pet Clinic does not have a REST API, this provides a direct, performant, Embabel-ready alternative.

You can also expose actions you’ve built using embabel-agent.

A New Layer for a New Era

We are early in the most exciting era in our industry’s history. We’re now starting to see the shape of the infrastructure it requires. LLMs made the revolution possible, yet are ultimately fungible. Agent harnesses are now widely recognized as necessary; context layers will ultimately prove more important. With Embabel Worlds we believe we’ve brought something unique and compelling, and we’d love you to join us in the journey to making it one of the defining technologies of our time.

However, the dominant approach to agent memory is deeply flawed. Memory systems such as Mem0, Zep, AWS AgentCore Memory, and Google Vertex Memory Bank start from the assumption that agent memory should be built from scratch by extracting facts from conversations.

This rests on two dangerous fallacies. It overemphasizes chat (a common mistake), and neglects the critical importance of how data is already stored and managed, and the necessity of integrating with it.

Even IBM’s article on agent memory reflects these fallacies, talking about agent memory in a vacuum. If IBM, of all companies, can’t get their head around the relationship between agents and existing enterprise systems, the disastrous results of enterprise AI rollouts become more understandable. It is also less surprising when we consider that, to this point, Gen AI has largely been driven by people with little interest in or understanding of the enterprise systems that power our everyday life.

Embabel introduces a different approach, reflecting the understanding that Gen AI can only succeed in enterprise through close integration with valuable existing assets.

In previous blogs I’ve introduced Domain Integrated Context Engineering (DICE).

Today I’ll explain how we approach these concepts in the Embabel Dice module.

In my next blog I’ll show how Dice can be used in a chatbot. Head to the urbot repository right away if you’re impatient to see working code.

Agent Memory Today

Agent memory approaches draw heavily on cognitive science. Endel Tulving’s distinction between episodic memory (specific events and experiences) and semantic memory (general facts and knowledge) dates from 1972 and remains a useful framework for what agents need to remember. Nearly every system in this space models these categories.

However, cognitive science helps with only part of the problem. Agent memory approaches are overwhelmingly text based, lacking the structure core to proven business systems.

Mem0 extracts memories into vector stores. AgentCore stores semantic facts and user preferences. Google Memory Bank uses Gemini to pull key details from session history. All treat memory as a greenfield concern: text in, embeddings out, similarity search to retrieve.

Freeform text is opaque and costly to work with. LLMs and vector search are slow and unpredictable. Cheaper approaches like BMI25 or regex searching are imprecise and offer limited capability. The precision, performance and predictability of a database query is lacking.

A more sophisticated approach is to use a knowledge graph, as in the various takes on Graph RAG.

Zep builds a temporal knowledge graph, ingests structured JSON business data alongside conversations, and recently added custom entity types. But even Zep treats business data as something you push into its graph — a JSON snapshot of a customer record is ingested, extracted, and stored as a Zep entity node copy.

None of these solutions link to real domain objects.

Existing businesses aren’t starting from scratch.

You already have a Customer entity. It probably lives in a relational database. It likely has a Spring Data repository, and a service layer above it. It has validation rules, business logic, and relationships to Order and Product entities, backed by years of careful domain modelling and rigorous testing.

Pure text solutions have no solid way of linking between a memory and a customer. Even with Zep, when it ingests a customer JSON document, it creates a new node in its graph, which doesn’t know about the JPA entity it came from. It can’t call existing business logic or enforce established validation rules. It doesn’t update when the real entity changes. It’s a text-and-embedding snapshot of something that was once a rich domain object, encapsulating behaviour.

Even in genuine greenfield scenarios, data will build up over time and the more structure the better.

Grounding Memory in Existing Data

How should we think about memory for business applications in an ideal world?

A simple thought experiment

What memories does my bank have about me? They probably have some text notes about my patterns and preferences that staff can retrieve, but these are far from the most valuable things they know about me. Those are highly structured, such as my account types and balances, payment history, and address. They are updated in structured ways, rather than through loose text. And they are definitely not in a vector database — although it’s now likely that they’re in a database that has a vector capability. The most valuable information doesn’t relate closely to cognitive science thinking about memory.

A mature memory solution must consider existing structured data along with free-form text, and must attempt to bring as much structure as possible to memory capture.

What if memories could reference your existing entities?

What if, instead of copying data into a fresh graph, we link to existing data in whatever store it is held. What if this also enables us to benefit from the business logic around it?

An entity node in the knowledge graph that holds a typed link to the real Customer record in your RDBMS. A PREFERS relationship that the agent discovered in conversation — but pointing to a Cuisine entity your recommendation engine already knows about. Memory as a projection over knowledge the business already has, not a parallel store that duplicates it.

This profoundly improves how memory works in practice:

• Agents don’t start from zero. On day one, every agent has access to what your domain model already knows — purchase history, loyalty tier, dietary restrictions — through real entity references, not extracted text approximations. Visibility can be limited for safety.
• Contradictions become visible. If the agent extracts “user prefers Nike” but your transaction data shows they’ve bought Adidas exclusively for three years, the graph surfaces that tension because both the proposition and the domain entity are present and connected.
• Business logic stays where it belongs. Your validation rules, authorization checks and repository methods are accessible through the actual entity, not reimplemented in the memory layer.

The problem with hosted services

AWS AgentCore Memory and Google Memory Bank are managed cloud services. Your agent’s memories live in their infrastructure. This is fine for many users, but problematic in enterprise, when domain entities are in databases behind firewalls, accessed by Java applications — and agent memory is in a separate hosted service that can’t access them.

If you don’t care too much about data sovereignty and security, it may be possible to serialize your business data, ship it across a network boundary to someone else’s memory store, and hope the LLM-powered extraction pipeline captures what matters. You now have two sources of truth: your actual domain model and a lossy, eventually-consistent text approximation of it sitting in a managed service. When your Customer record updates, the memory service doesn't know. When your business rules change, the memory service doesn't care. You're paying for the privilege of maintaining a stale copy of knowledge you already have.

This is architecturally backwards. Memory needs to be co-located with the entities it references, not across a network behind an API boundary. It should be in process, with direct access to your repositories, your domain logic, and your data.

This is where the JVM has a huge advantage.

An agent memory system running on the JVM doesn’t need to call an external API to learn about your customers. It can simply call its own services and repositories, benefiting from their business logic and tested behaviour. Nor does it need to go through MCP, when it can find answers faster in process. It can access the same CustomerRepository your application already uses. It can resolve a proposition like "the user prefers Italian food" against the actual Customer entity and the actual Cuisine entity your recommendation engine already depends on — in the same process, with the same transaction guarantees, through the same Spring context.

The memory system and the domain model are one, running together, sharing the same type system and the same runtime. While we notionally have a knowledge graph (for which Neo4j is a brilliant store), data tends to stick where it lands, and the approach supports multiple stores.

Python-based memory frameworks can’t do this — not because Python is a bad language, but because the enterprise domain model doesn’t live in Python, but in the JVM.

A Python memory service will always be on the outside looking in, calling APIs to get approximations of what your Java code already knows directly.

Existing frameworks build memory as if the business starts knowing nothing — and as if memory should live somewhere other than where your business logic runs.

What’s needed is a memory system that treats what your business already knows as the foundation, conversation as something that extends it, and the JVM as exactly the right place for all of it to live.

That’s what Embabel Dice aims to provide.

Learning from Experience and Research

The Dice project is based on internal knowledge graph work we’ve done at Embabel over the last two years, as well as thorough exploration of the state of the art in memory solutions.

Among the influences, Neo Technology’s LLM Knowledge Graph Builder demonstrated that LLMs can extract entities and relationships from unstructured text into a knowledge graph — documents to chunks to entities to relationships, with schema-guided extraction. It doesn’t address what happens when those extracted entities correspond to things your business already knows about in existing databases, or how to apply this to memory versus RAG.

We owe a particular debt to the General User Models (GUM) research from Stanford and Microsoft. GUM showed that you can build rich user models from unstructured observations using confidence-weighted propositions — atomic statements about a user, each carrying a confidence score and a decay rate, maintained through a pipeline of proposal, retrieval, and revision. The insight that propositions are the right atomic unit for extracted knowledge, and that they should carry calibrated uncertainty, directly informs our approach.

But GUM’s propositions are unstructured. They’re natural language strings in a vector store. There’s no schema constraining them, no formal ontology connecting them, and no mechanism for linking a proposition like “the user loved the trip to Italy last year” to a Trip entity in an existing business system.

Our approach in Dice takes GUM’s proposition model and asks:

• What if propositions could be promoted to typed relationships in a knowledge graph when confidence is high enough and a matching schema exists?
• What if entity nodes in that graph could hold references to real domain objects in existing data stores?
• What if memory, knowledge graph, and domain model were three projections of the same underlying system of record?

The result combines Tulving’s memory taxonomy, GUM’s confidence-weighted proposition, and Neo4j’s knowledge graph extraction. Further, promotion is generalized so that propositions can be projected not just to a Neo knowledge graph, but to other representations, including an experimental Prolog fact base.

The most important, novel piece: integration with existing, typed, production domain entities that the business already trusts.

Memory is an Outgrowth of Your application

The memory problem is a symptom of something larger. The AI agent ecosystem has been built on a premise that enterprise should reject: that AI is a separate stack, a new silo, something you bolt onto your business from the outside.

AI should grow out of your application, not land on it from space.

This explains the many failures. Not because AI doesn’t work, but because the approach is wrong. Companies are standing up parallel infrastructure — Python services, vector databases, hosted memory layers, prompt chains — that have no organic connection to the systems that actually run their business. The AI doesn’t know your domain. It doesn’t understand your constraints. It doesn’t share your type system, transaction boundaries or security model. It’s an alien stack that you spend months trying to integrate, and mostly fail.

What if AI agents grew out of your existing application instead?

That’s the bet we’re making with Embabel. Not a Python framework that hopes to connect to your enterprise. A JVM-native agent platform that lives inside it. Your domain entities become the agent’s world model. Your Spring services and repositories furnish it with tools. Your type system becomes its contract for correctness. Memory isn’t a separate service — it’s a structured projection over what your application already knows, extended by what the agent learns from interaction.

This goes beyond memory. Embabel uses Goal-Oriented Action Planning — deterministic, inspectable plans built from type-safe preconditions and effects, not probabilistic prompt chains that you can’t explain to an auditor. Actions operate on your domain types. The planner reasons about your domain state. Every step is traceable, testable, and grounded in the same business logic your organization already trusts.

Python agent frameworks make nice demo infrastructure for greenfield AI experiments. We’re building infrastructure for AI that works inside the enterprise systems that already exist. Those are not the same thing, and the 95% failure rate tells you which one the industry actually needs.

Enterprise AI doesn’t fail because models aren’t smart enough. It fails because frameworks either hope that enterprise systems will magically go away, or treat the them as an integration problem instead of a valuable starting point. Your domain model, your business rules, your data — these aren’t obstacles to work around. They’re the most valuable assets an agent can have.

Embabel DICE

We call this approach DICE — Domain Integrated Context Engineering. It’s still early. We’re building the proposition-to-entity promotion pipeline, confidence-weighted memory projections and schema-guided extraction linking to live domain objects in the open. There’s a way to go before it’s hardened. But the architecture is clear, the foundations are solid, and the direction is right.

We believe DICE will be the m emory solution that helps change enterprise Gen AI from its current failure mode to something that actually delivers. Not by making LLMs smarter, but by giving them something the industry has overlooked: the structured domain knowledge that enterprises have spent decades building.

The future of enterprise AI isn’t a smarter hosted memory service. It’s agents that speak your domain language because they were born inside your application.

That’s the vision Embabel exists to make real. And if this resonates with how you think about enterprise AI — if you’ve felt the frustration of trying to bolt a Python demo onto a real business system — we’d love you to join us on this journey. Star the repo, try the framework, help us refine the ideas. The best enterprise AI infrastructure won’t be built in isolation. It’ll be built by the people who understand and are currently delivering what enterprise actually needs.

Agent Memory Is Not A Greenfield Problem: Ground it in your Existing Data was originally published in Embabel on Medium, where people are continuing the conversation by highlighting and responding to this story.

Today’s LLMs are trained for tool calling and are good at it. However, tool descriptions can consume a significant portion of the context window, and any LLM can struggle to determine which tools to call. Testing at Anthropic showed that Opus 4 scored only 49% accuracy on MCP evaluations when working with large tool libraries: essentially a coin toss.

Thus tool discovery is a core problem that agent frameworks need to address in order to help developers build more efficient and reliable applications.

Embabel now solves this problem in a particularly elegant way. The innovation: Like the famous matryoshka Russian dolls, Embabel can organize tools into a hierarchy where each level exposes a single tool that opens up the level below. The result: You can use as many tools as you want without using an excessive number of tokens or sacrificing reliability.

Existing Tool Discovery Mechanisms

Any agent framework mitigates the tool discovery problem by breaking flows into steps that can use local tools. In many workflows this sidesteps the problem altogether. But there remain scenarios such as chatbots where sufficient decomposition isn’t possible.

Let’s examine the prior art.

LangChain / LangGraph: Vector Similarity Search

As you’d expect, LangGraph addresses the tool proliferation problem in the most obvious way — through vector similarity search. Tool descriptions are embedded and stored in a vector store, and tools are matched to user queries via semantic search:

# Source: https://langchain-ai.github.io/langgraph/how-tos/many-tools/
tool_documents = [
    Document(
        page_content=tool.description,
        id=id,
        metadata={"tool_name": tool.name},
    )
    for id, tool in tool_registry.items()
]

vector_store = InMemoryVectorStore(embedding=OpenAIEmbeddings())
document_ids = vector_store.add_documents(tool_documents)

This introduces additional infrastructure and latency. Furthermore, semantic similarity isn’t always the right criterion for tool selection.

LangChain4j: Dynamic Tool Provider

LangChain4j has a ToolProvider interface that allows runtime tool selection based on the incoming message:

// Source: https://github.com/langchain4j/langchain4j/issues/637
Assistant assistant = AiServices
    .builder(Assistant.class)
    .chatLanguageModel(model)
    .toolProvider(request -> {
        if (request.userMessage().singleText().contains("booking")) {
            return ToolProviderResult.builder()
                .add(bookingToolSpecification, new BookingToolExecutor())
                .build();
        }
        return null;
    })
    .build();

The Embabel equivalent is the common pattern of building a dynamic list of tools to add to a PromptRunner when we’re about to make an LLM call. However, it’s not a complete solution to the tool discovery problem.

Claude Tool Search Tool: Deferred Loading

Anthropic recently introduced the Tool Search Tool for Claude Code and the API. This marks tools with defer_loading: true so they remain discoverable without consuming context tokens at session start. Claude then discovers relevant tools on-demand via a search mechanism.

// Source: https://www.anthropic.com/engineering/advanced-tool-use
{
  "tools": [
    {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"},
    {
      "name": "github.createPullRequest",
      "description": "Create a pull request",
      "input_schema": {...},
      "defer_loading": true
    }
  ]
}

Anthropic reports an 85% reduction in token usage — from ~134k to ~5k tokens in internal testing — while maintaining full tool access. This is a significant improvement, with accuracy on MCP evaluations jumping from 49% to 74% for Opus 4 and from 79.5% to 88.1% for Opus 4.5.

However, this is still a “search-then-select” approach, introducing an additional inference step to discover tools before using them.

A Better Solution: Tool Hierarchy

You can use all these approaches with Embabel. But we can do better.

Instead of searching through tools, we organize them into a hierarchy. This can be done by the tool developer or by the developer building the agent — who of course may be the same person.

Each matryoshka tool acts as a facade: when invoked, it’s replaced in the message history by its children. This lets the LLM navigate a tree of capabilities, drilling down only into the areas relevant to the current task.

The Unbeatable Math of Binary Search

Consider the difference. With 100 tools using flat selection:

• The LLM must process 100 tool descriptions
• Token consumption scales linearly: O(n)
• Selection accuracy degrades with n

With hierarchical tools organized into10 groups of 10:

• First level: 10 facade tools
• After selection: 10 actual tools
• Maximum tools considered: 20
• Token consumption: O(log n)
• Selection accuracy remains high at each level

You can’t beat the math of logarithmic scaling. Embabel apps can offer an unlimited number of tools to any LLM interaction while keeping each selection step manageable.

Code Example: Spotify Integration

Here’s how this looks in practice with the Spotify integration in our Impromptu classical music exploration chatbot:

@MatryoshkaTools(
    name = "spotify",
    description = "Access Spotify music features. Invoke this tool to enable " +
        "Spotify operations like playing music, searching tracks, " +
        "managing playlists, and controlling playback."
)
public record SpotifyTools(
    ImpromptuUser user,
    SpotifyService spotifyService
) {

    @LlmTool(description = "Check if user has linked their Spotify account")
    public String checkSpotifyStatus() {
        // Implementation
    }

  // 10 other tools...

The @MatryoshkaTools annotation transforms this class into a hierarchical tool. Initially, the LLM sees only the top level description:

spotify: Access Spotify music features. Invoke this tool to enable Spotify operations...

When the user says “play some Brahms,” the LLM invokes thespotifyparent tool. This facade is replaced by all the child tools: checkSpotifyStatus, getPlaylists, searchTracks, playTrack, etc. Now the LLM can select playTrack and proceed.

Scaling with Multiple Integrations

The power becomes clear when you have many integrations. In the Impromptu chatbot, we have matryoshka tools for several concerns.

// Source: https://github.com/embabel/impromptu/blob/main/src/main/java/com/embabel/impromptu/integrations/youtube/YouTubeTools.java
@MatryoshkaTools(
    name = "youtube",
    description = "Access YouTube to play or search for videos"
)
public record YouTubeTools(...) { /* 5 tools */ }


@MatryoshkaTools(
    name = "metmuseum",
    description = "Access the Metropolitan Museum of Art collection. " +
        "Invoke this tool to search artworks, get artwork details, " +
        "explore departments, and discover highlighted works..."
)
public record MetMuseumTools(...) { /* 6 tools */ }


@MatryoshkaTools(
    name = "imslp",
    description = "Access IMSLP (International Music Score Library Project) " +
        "to find and download public domain music scores..."
)
public record ImslpTools(...) { /* 4 tools */ }

// more tools...

Each integration adds just one tool to the top-level selection, regardless of how many operations it supports internally. The Spotify integration has 12 internal tools, YouTube has 5, Met Museum has 6, IMSLP has 4 — that’s 27 tools total, and the number will only grow. But the LLM only sees 4 facades initially, keeping selection simple and accurate.

Benefit: Rich Tool Descriptions

Another key benefit of this architecture is that lower level tools can have as detailed descriptions as they need. Since they’re only expanded occasionally, we don’t need to worry much about token consumption in the descriptions. This enables better tool use accuracy.

Compare these tool descriptions:

Without hierarchy (must be brief):

findScore: Find a score on IMSLP

With hierarchy (can be comprehensive):

@LlmTool(description = """
    Find a musical score on IMSLP given composer and work title.
    COMPOSER FORMAT: Use last name only (e.g., 'Brahms', 'Bach', 'Beethoven').
    IMSLP page titles use format: Work_Title_(Composer,_First_Last)
    WORK TITLE FORMAT: Use standard titles with proper spacing:
    - 'Symphony No. 5' (not 'Symphony No.5')
    - 'Violin Sonata No. 1, Op. 78' or just 'Violin Sonata Op. 78'

   TROUBLESHOOTING - if no results:
    1. Simplify: Try just 'Violin Sonata' instead of 'Violin Sonata No. 1 in G major'
    2. Try searchWorks with a flexible query like 'Brahms violin sonata'

  ALWAYS share the IMSLP page URL with the user - they can browse editions directly.
    """)
public String findScore(String composer, String workTitle) {
    // ...
}

Most tools don’t need this level of detail, but this one was needed to work around what had been a real reliability problem with a shorter description, due to the quirks of the IMSLP API. We would not want to waste this many tokens in interactions that didn’t call about score finding.

The hierarchical approach lets us invest in detailed, helpful descriptions that guide the LLM to correct usage — but only pay the token cost when that specific tool category is relevant.

Conclusion

Embabel makes it easy to expose targeted tools through selective addition to a PromptRunner. Two particularly import mechanisms for selection are:

• Exposing tools on previously retrieved entities. Easy integration with existing domain models is a core goal of Embabel.
• Matryoshka tools that conceal one or more layers of children.

Together, Embabel enables you to work with as many tools as you want in your applications.

Still think Python agent frameworks are the state of the art?

Writing a good application framework is not a data science problem — it’s a software engineering problem. Type safety, deterministic execution, hierarchical organization, and enterprise-grade tooling matter.

Matryoshka tools demonstrate what’s possible when you approach the many-tools problem from first principles rather than reaching for the familiar “just add vector search” solution. The result is simpler, more predictable, and more scalable.

Check out Embabel today.

Responding to Messages

Chat Actions

In Embabel there’s nothing special about a chatbot. It’s backed by Embabel’s core AgentProcess—a long-running process that can receive messages and respond to them over time. Thus the same action-based architecture that handles business workflows also handles chat interactions. Chat processes have a Conversation object in their blackboard, but nothing else about them is special. (Remember that Embabel doesn’t otherwise need to maintain a conversation thread, because it emphasizes structure over loose text.) Chat processes can also respond to incoming events other than user messages, which can help inform the conversation.

A chatbot action responds to an incoming UserMessage and sends one or more assistant messages. The simplest possible example:

@Action(canRerun = true, trigger = UserMessage.class)
void respond(Conversation conversation, ActionContext context) {
    var assistantMessage = context
      .ai()
       withAutoLlm()
      .withSystemPrompt("You are a helpful assistant")
      .respond(conversation.getMessages());
    context.sendMessage(conversation.addMessage(assistantMessage));
}

The @Action annotation with trigger = UserMessage.class means this action fires whenever a user message arrives. The canRerun = true flag allows it to run repeatedly. The ActionContext parameter provides access to the LLM through context.ai(), as well as the ability to send messages.

So little code looks cute, but we want something useful, rather than the millionth and first “you are a helpful assistant” toy demo.

Here’s the entire ChatActions class from Ragbot, with a realistic chat action:

@EmbabelComponent
public class ChatActions {

    private final ToolishRag toolishRag;
    private final RagbotProperties properties;

    public ChatActions(
            SearchOperations searchOperations,
            RagbotProperties properties) {
        this.toolishRag = new ToolishRag(
                "sources",
                "Music criticism",
                searchOperations);
        this.properties = properties;
    }

    @Action(canRerun = true, trigger = UserMessage.class)
    void respond(Conversation conversation, ActionContext context) {
        var assistantMessage = context
                .ai()
                .withLlm(properties.chatLlm())
                .withReference(toolishRag)
                .withTemplate("ragbot")
                .respondWithSystemPrompt(conversation, Map.of(
                        "properties", properties,
                        "voice", properties.voice(),
                        "objective", properties.objective()
                ));
        context.sendMessage(conversation.addMessage(assistantMessage));
    }
}

The key additions are the use of a prompt template, and the call to .withReference(toolishRag). This gives the LLM access to RAG tools. The LLM decides when and how to search—true agentic retrieval rather than a blind pipeline.

Managing Prompts with Templates

Chatbot prompts tend to grow complex. You need persona definitions, guardrails, response guidelines, objective instructions. Embedding all this even in multi-line strings is painful.

When prompts get complex, Embabel makes it easy to use Jinja templates to externalize them. The main Ragbot template is built from reusable elements:

{% include "elements/guardrails.jinja" %}
{% include "elements/personalization.jinja" %}

Keep your responses under {{ properties.voice().maxWords() }} words unless they
MUST be longer for a detailed response or quoting content.

The personalization template dynamically loads persona and objective:

============
# Important: Your voice
{% set persona_template = "personas/" ~ properties.voice().persona() ~ ".jinja" %}
{% include persona_template %}
============

============
# Important: Your objectives
{% set objective_template = "objectives/" ~ properties.objective() ~ ".jinja" %}
{% include objective_template %}
============

The separation is useful. Voice defines how the chatbot communicates — its personality and tone. Objective defines what it should accomplish — the domain expertise and task instructions. You can mix and match: the same legal objective answered in the voice of Shakespeare, a sarcastic teenager, or a formal lawyer.

Here’s a simple persona (personas/music-guide.jinja):

You are Sforzando, an enthusiastic guide to classical music.
You want to share your passion for this genre with others.
Your tone is friendly and engaging. You are not formal or academic. 
You want to make classical music accessible and enjoyable for everyone.

An objective (objectives/music.jinja):

Answer questions about music critics in a clear and concise manner.
The tools available to you look into the archives of critics.
You must always use these tools to find answers, as your general knowledge 
will not extend to everything they wrote.

Always back up your points with direct quotes from the criticism.
DO NOT RELY ON GENERAL KNOWLEDGE unless you are certain a better answer 
is not in the provided sources.

Ragbot allows switching personas in application.yml without touching code:

ragbot:
  voice:
    persona: shakespeare  # Now responds in Elizabethan English
    max-words: 250
  objective: music

Configuring a Chatbot

So that our action(s) are invoked, we need to configure a Chatbot.

Embabel’s Utility AI planning type is ideal for chatbots. Utility AI scores available actions and selects the most appropriate one for each situation. (An action can never be invoked unless its type requirements and other preconditions are satisfied, so type safety remains guaranteed.) For a simple chatbot with a single action, scoring isn’t needed. But it means you can easily add more actions later without restructuring your code. When a user message comes in, the platform will choose the highest value action at that moment.

We define a Spring bean that uses the AgentProcessChat helper class to create a utility chatbot:

@Configuration
class ChatConfiguration {

    @Bean
    Chatbot chatbot(AgentPlatform agentPlatform) {
        return AgentProcessChatbot.utilityFromPlatform(agentPlatform);
    }
}

The utilityFromPlatform() method automatically discovers all @Action methods in your @EmbabelComponent classes. Your chatbot can gain capabilities just by adding new actions.

Configuring SearchOperations to back RAG

We need to define the SearchOperations instance injected into ChatActions and provided to ToolishRag. Here we use the LuceneSearchOperations fluent API:

@Configuration
@EnableConfigurationProperties(RagbotProperties.class)
class RagConfiguration {

    @Bean
    LuceneSearchOperations luceneSearchOperations(
            ModelProvider modelProvider,
            RagbotProperties properties) {
        var embeddingService = modelProvider.getEmbeddingService(
            DefaultModelSelectionCriteria.INSTANCE);
        return LuceneSearchOperations
                .withName("docs")
                .withEmbeddingService(embeddingService)
                .withChunkerConfig(properties.chunkerConfig())
                .withChunkTransformer(AddTitlesChunkTransformer.INSTANCE)
                .withIndexPath(Paths.get("./.lucene-index"))
                .buildAndLoadChunks();
    }
}

This creates a Lucene-backed text and vector store that persists to disk. The LuceneSearchOperations implements our SearchOperations interface, which ToolishRag wraps to expose fine-grained search tools to the LLM.

Running The App

Ragbot starts with Spring Shell.

Run ./scripts.shell.sh from the command line, or run RagbotApplication from your IDE.

Once it starts up, we need to ingest data, which will survive restarts.

The sample data is 19th and early 20th century music criticism. An obscure choice helps prove that the LLM is not just relying on general knowledge, which can be hard to do with familiar content. It also reflects the fact that I have a PhD on Romantic piano music, and am currently listening to Glazunov’s wonderful violin concerto.

To use the sample data, begin by running ingest, which will default to ingesting Robert Schumann’s music criticism. Then ingest the Project Gutenberg URL of Philip Hale’s legendary early 20th century program notes. Run these commands in the shell:

> ingest
> ingest https://www.gutenberg.org/files/56208/56208-h/56208-h.htm

You can also ingest your own content using the shell. See the Ragbot README for documentation.

You can now type chat in the shell for text-based chat, but a UI is more fun. I got Claude to produce a simple Javelit web interface for Ragbot. The uichat shell command launches it. The port defaults to 8888but you can specify your own:

> uichat
Web UI started at http://localhost:8888

You can now chat and ask questions. The LLM will use the RAG tools to search the indexed content and provided grounded responses. You’ll see the tool calls as they are made.

To demonstrate Ragbot’s understanding of the sources, try questions like:

• What did Schumann think of Liszt?
• What did Philip Hale think of Liszt?
• Contrast Hale and Schumann’s attitude to Liszt?

The latter is difficult for traditional pipeline RAG, but not agentic RAG.

Store Choice

Embabel’s RAG module supports multiple stores through the SearchOperations interface. Currently we support:

Apache Lucene — In-memory or disk-persisted vector storage. No external dependencies. Good for development and smaller datasets.

Neo4j — Graph database with vector indices. Enables entity relationships and document structure navigation alongside vector search.

Spring AI VectorStore — Adapts any Spring AI vector store (pgvector, Pinecone, Chroma, etc.) to work with Embabel’s agentic RAG. Does not support full text search or structure, which limits it usefulness.

pgvector — coming soon.

The SearchOperations interfaces are simple, so it’s easy to implement your own search and benefit from the same programming model.

The API remains consistent, regardless of the backend. However, the store affects which tools ToolishRag exposes. Lucene gets vector search, text search, regex, and chunk expansion. Neo4j adds document structure navigation. Spring AI adapters provide basic vector search. The LLM always gets the best available toolkit for your store.

Ingesting Data

Ragbot includes shell commands for ingestion. Like the API, ingestion is also independent of the backing store.

Embabel RAG uses Apache Tika by default for content extraction:

var ingested = NeverRefreshExistingDocumentContentPolicy.INSTANCE
        .ingestUriIfNeeded(
                luceneSearchOperations,
                new TikaHierarchicalContentReader(),
                uri
        );

Tika handles HTML, PDF, Word documents, and dozens of other formats. It extracts hierarchical structure — titles, sections, paragraphs — which Embabel preserves in the index.

For complex documents that Tika doesn’t handle well, docling is a solid choice. It converts PDFs, Word docs, and PowerPoints to clean Markdown that ingests reliably. Preprocessing with docling, then ingesting the markdown, often produces better results than direct ingestion of complex formats.

Pipeline architectures are fine for ingestion. An agentic approach is essential at at query time, when the LLM needs to reason about what to retrieve. Ingestion is a batch process where predetermined steps make sense. Embabel offers a RAG pipeline where chunks can be transformed — for example, to add information about the documents and sections they were taken from, as Ragbot does.

Exploring This Yourself

The complete example is available at github.com/embabel/ragbot. Clone it, set your API key, and you’ll have a working agentic RAG chatbot in minutes. Change the prompt templates and ingest your own data.

The architecture, and agentic RAG, generalizes beyond chatbots. Any Embabel agent can use ToolishRag to search and reason about documents. The same pattern works for document analysis workflows, research assistants, compliance checkers, and anywhere you need grounded LLM responses.

To start from scratch with your own Embabel project, create a new Embabel repository from the Java agent template GitHub template repository.

Coming Soon

I’m working on Impromptu — a full-featured classical music exploration application that integrates with YouTube, Spotify, Wikipedia and other sources. It combines agentic RAG with entity extraction, knowledge graphs and sophisticated memory system for a comprehensive music research and discovery platform.

Watch this space.

The problem isn’t the workarounds. It’s what they’re working around. Popular RAG frameworks such as LangChain use an obsolete model that dates back to 2023, before LLM tool calling emerged and changed everything. They use static pipelines when what’s needed is fundamentally different: Letting the LLM reason about retrieval rather than blindly executing a predetermined flow.

Recent research is clear. The future of RAG is agentic.

The Legacy Pipeline Approach

Traditional RAG follows a rigid pipeline:

LangChain (Python): The OG Pipeline

As I’ve previously noted, LangChain owes its prominence to historically being the first mover. In this case, this has become a major liability, as its RAG approach is obsolete.

Here’s how LangChain implements RAG using LCEL (LangChain Expression Language):

from langchain_core.runnables import RunnablePassthrough, RunnableParallel
from langchain_core.output_parsers import StrOutputParser

# The canonical RAG pipeline
rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
) 

# Or with RunnableParallel for slightly more control
rag_chain = (
    RunnableParallel(context=retriever | format_docs, question=RunnablePassthrough())
    | qa_prompt 
    | llm
)

This pattern appears across tutorials, documentation, and production systems. Welcome to 2023.

LangChain4j: Different Language, Same Old Model

As you would expect from its name, LangChain4j brings the same model to the JVM. The tutorial explicitly defines RAG in this limited, obsolete way: “Simply put, RAG is the way to find and inject relevant pieces of information from your data into the prompt before sending it to the LLM.”

Here’s example code from the Easy RAG example:

public class Easy_RAG_Example {
    public static void main(String[] args) {
        List<Document> documents = loadDocuments(toPath("documents/"), glob("*.txt"));

  Assistant assistant = AiServices.builder(Assistant.class)
            .chatModel(CHAT_MODEL)
            .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
            .contentRetriever(createContentRetriever(documents))
            .build();
        startConversationWith(assistant);
    }

    private static ContentRetriever createContentRetriever(List<Document> documents) {
        InMemoryEmbeddingStore<TextSegment> embeddingStore = new InMemoryEmbeddingStore<>();
        EmbeddingStoreIngestor.ingest(documents, embeddingStore);
        return EmbeddingStoreContentRetriever.from(embeddingStore);
    }
}

The ContentRetriever supplements the prompt to help answer queries. The documentation explicitly describes this as a "two-step chain" that provides "reduced latency at the expense of flexibility."

Spring AI uses a similar (if more flexible) pipeline model.

What’s Wrong With Pipelines?

True, these are simple examples. You can build more elaborate pipelines with these old school RAG frameworks, but can’t fix the core problem with the approach.

The pipeline model has fundamental problems that no amount of tuning can fix:

• Static retrieval The retriever retrieves once and hopes for the best. If the initial query doesn’t match how documents are indexed, you get poor results. Even if hybrid search is available, the balance between vector and full-text search will be fixed for all queries. The LLM never gets a chance to try a different, highly contextual approach.
• Lack of self-correction If retrieved chunks don’t actually answer the question, the pipeline has no mechanism to recognise this and try again. It has to trust whatever comes back from the vector store. Nor can it decide how hard to try in a particular scenario: Is this a legal matter or a casual question? Should it dig deep, regardless of latency, or give up quickly?
• Context blindness The retriever may not know what the LLM learned from previous turns in a conversation. Retrieval is often isolated, unable to build on prior context.

To compensate for these limitations, teams bolt on increasingly complex preprocessing: HyDE (Hypothetical Document Embeddings) to bridge the query-document vocabulary gap, rerankers to fix retrieval ordering, query expansion to catch more results, overlap tuning to hope chunk boundaries don’t split relevant content.

Chunk boundaries are an especially serious problem. No chunk size can be optimal for all documents. Retrieved chunks are often split in the wrong place, making them misleading.

Recent Research Agrees

Recent research is unambiguous:

• Anthropic: Traditional approaches using RAG use static retrieval… You can’t hardcode a fixed path for exploring complex topics, as the process is inherently dynamic and path-dependent. Anthropic’s multi-agent research system outperformed single-agent approaches by 90.2%.
• NVIDIA: Agentic RAG “refines queries using reasoning, turning RAG into a sophisticated tool” versus traditional RAG’s “lack of reasoning” and “context blindness.”
• arXiv survey (January 2025): Traditional RAG systems are constrained by static workflows and lack the adaptability required for multistep reasoning and complex task management.
• Comparative studies (TechRxiv, SSRN): 80% improvement in retrieval quality and 90% of users preferring agentic systems.

Pipeline RAG is not state of the art. It’s a first-generation approach that is now largely obsolete.

The Future: Agentic RAG

What if we let the LLM control the retrieval process? Instead of a rigid pipeline, give the model tools to search, evaluate results, refine queries, and iterate until it finds what it needs.

This is agentic RAG. The LLM becomes an active participant in retrieval rather than a passive consumer of whatever the pipeline produces.

Embabel: Agentic RAG Done Right

Embabel’s modern RAG support sits cleanly on top of the core framework through our LlmReference abstraction. The ToolishRag class enables easy agentic RAG over any a range of back end stores, which may have different capabilities.

ToolishRag: Fine-Grained Search as LLM Tools

ToolishRag wraps any SearchOperations implementation and exposes its capabilities as individual tools the LLM can invoke.

ToolishRag inspects what interfaces the underlying store implements and only exposes tools for operations the store actually supports.

For example, a Lucene store supports vector search, text search, regex search, and result expansion tools. A Neo4j database supports vector and full text search, plus valuable tools to navigate source document structure from chunks. A simple vector database adapter gets only vector search. The LLM will always be equipped with the best possible toolkit for the store in use, with the developer only needed to write the same simple, intuitive code.

Key Tools

Vector Search — Semantic similarity search with configurable top-K and threshold:

@LlmTool(description = "Perform vector search. Specify topK and similarity threshold from 0-1")
fun vectorSearch(query: String, topK: Int, threshold: ZeroToOne): String

Text Search — BM25 search with Lucene syntax support:

@LlmTool(description = "Perform BM25 search with Lucene syntax: +term, -term, \"phrases\", wildcards (*), fuzzy (~)")
fun textSearch(query: String, topK: Int, threshold: ZeroToOne): String

Chunk Expansion — Broaden context around a retrieved chunk:

@LlmTool(description = "Given a chunk ID, expand to surrounding chunks")
fun broadenChunk(chunkId: String, chunksToAdd: Int = 2): String

Similarly, for document-structured stores such as Neo4j, Zoom Out — Navigate to parent sections for broader context:

@LlmTool(description = "Given a content element ID, expand to parent section")
fun zoomOut(id: String): String

The zoom tools are valuable, as they mitigate the problem of chunk boundaries splitting relevant content. If the LLM sees the start or end of what appears to be a promising seam of content, it can continue mining.

The LLM decides which tools to use, in what order, and with what parameters. It can try a vector search, evaluate the results, decide they’re not quite right, and try a text search with different terms. It can find a relevant chunk, then broaden it to see surrounding context. It can zoom out to understand where a chunk fits in the document structure.

The LLM is able to reason about retrieval to help achieve its overall goal, rather than being forced to rely on predetermined results.

Using ToolishRag in Practice

The API is simple, consistent and elegant. Rag tools can be added to any LLM interaction via the Embabel PromptRunner. Here's an example from a production chatbot:

  var toolishRag = new ToolishRag(
          "sources",
          "The music criticism written by Robert Schumann: His own writings",
          searchOperations);
 
  var response = ai
          .withLlm(properties.chatLlm())
          .withReference(toolishRag)
          .generateText("""
              You are a helpful authority on Robert Schumann's writings.
              Use the tools available to you to answer the user's questions.
              """);
   

1. SearchOperations, like the Ai interface, is normally injected into the class—it could be Lucene, a Spring AI vector store, Neo4j, or any other implementation
2. ToolishRag wraps it and exposes appropriate tools based on the store's capabilities
3. .withReference(toolishRag) gives the LLM access to search tools

The LLM has full control. It can search multiple times with different queries. It can evaluate whether results are relevant. It can expand context when needed. It can give up gracefully if nothing works — the default goal explicitly says Continue search until the question is answered, or you have to give up. Be creative, try different types of queries. You can customize this part of the prompt.

This example comes from our Ragbot sample application. Our guide chatbot and MCP server helping users build Embabel applications also use ToolishRag for RAG, backed by the Embabel documentation and related content.

What’s Next: Entities in RAG

Chunks are necessary but not always sufficient. Real documents have structure: sections, headings, entities, relationships, and refer to entities. A chunk that mentions “the CEO” loses meaning without knowing who “the CEO” refers to. A document discussing the user’s phone account should be linked to their customer and service entities.

We’re extending Embabel’s agentic RAG to include entity extraction and graph integration. Entities provide structure above chunks. When the agent retrieves a chunk mentioning “the acquisition,” it can traverse to the entity representing that acquisition, find the companies involved, the date, the value — context that pure chunk retrieval would miss.

This is another area where agentic approaches excel. An agent can decide when entity lookup is valuable, when graph traversal helps, when plain text search is enough. A pipeline must choose one approach for all queries.

I’ll write more about this soon.

Conclusion

RAG is essential. Pipeline RAG is inadequate.

Frameworks such as LangChain rely on a pipeline model that treats retrieval as a fixed preprocessing step.

As a newer framework, Embabel takes a fundamentally different approach that reflects recent research and experience. ToolishRag exposes fine-grained search operations as tools the LLM controls. The LlmReference abstraction integrates RAG into the core agent framework rather than bolting it on as an afterthought. The result is retrieval that adapts, iterates, and reasons—not retrieval that executes a predetermined flow and hopes for the best.

If you’re building production RAG systems, you have a choice. You can keep adding workarounds to a fundamentally limited architecture. Or you can adopt an approach that addresses the root cause: letting intelligent agents reason about retrieval rather than executing blind pipelines.

The research is clear. The results are dramatic. Pipeline RAG is the past. Agentic RAG is the future.

Embabel is how you build it, on the JVM.

From day one, Embabel has offered what no other framework does: sophisticated, deterministic GOAP planning; type-safe domain modelling at the core; and the ability to chain actions safely in ways you never explicitly coded. Given our goal of being the agent framework for enterprise, we’re focused on minimizing nondeterminism and integrating with existing business systems.

On this solid foundation, we’re building richer and richer functionality, with a consistent, integrated API. We’re emphasizing ease of use and removing friction points in using Embabel, informed by building real applications.

We’re now ensuring that anything valuable you can do with any other framework, you can do better with Embabel. Not just the sophisticated deterministic flows we pioneered — everything. Utility-based exploration, the supervisor pattern, state machines, chatbots, RAG. All of it, with the type safety and testability that the JVM, and our unique design, provides.

Here are some recent advances. More depth on each coming soon.

Planner Enhancements

We’ve added two new planner types: Utility and Supervisor, and introduced a state machine model that works with either. Both use our existing programming model, with strongly typed @Action methods.

Utility AI Planner: Great for Exploration

GOAP is great when you know what goal you want to achieve, and want the framework to figure out a path to achieve it.

Utility AI — another algorithm from gaming — is great for exploration — when you don’t have a predetermined goal. It makes decisions based on immediate value — perfect for event-driven systems, chatbots, or reactive processing where the “right” action depends on what you discover as you progress. The planner chooses the most valuable action it can run at a given time, executes it, then replans. It may discover an unexpected path, although it will never execute an action whose conditions were not satisfied. Or it may find that exploration has concluded and there’s nothing more to do, and this is not an error.

Utility AI works especially well with our recently introduced dynamic cost and value. This enables the value of actions to change in real time — for example, if you want different behaviour outside core business hours, or want to respond differently depending on the load on an external system.

@Cost(name = "triageCost")
public double computeProcessingCost(GithubIssue issue) {
   // return calculation based on objects from Blackboard
}

@Action(costMethod="triageCost")
IssueResponse triageIssue(GitHubIssue issue, Ai ai) {
    return ai.createObject("Triage this issue", IssueResponse.class);
}

Supervisor Planner: Like LangGraph or Google ADK, But Typed

We now support the popular supervisor pattern — an LLM deciding which actions to call, viewing them as tools.

Supervisor planning is convenient, but not deterministic, hence is not suitable for many business critical tasks. But our implementation is superior to that of other frameworks in that it is type-informed: the LLM sees schemas, outputs are validated, and you get semi-deterministic behaviour, as action tools are only exposed if their preconditions are satisfied. Also interim structure is written to the blackboard, so you can persist it or respond to events with structure.

Consider this Google ADK code:

# Define sub-agents
greeter = LlmAgent(
    name="greeter",
    model="gemini-2.5-flash",
    instruction="You handle greetings warmly.",
    output_key="greeting_result"  # String key
)

task_executor = LlmAgent(
    name="task_executor",
    model="gemini-2.5-flash",
    instruction="Process data from {greeting_result}.",  # Magic string reference
    output_key="task_result"
)

# Create coordinator
coordinator = LlmAgent(
    name="Coordinator",
    model="gemini-2.5-flash",
    description="I coordinate greetings and tasks.",
    sub_agents=[greeter, task_executor]
)

In Embabel it would look as follows:

@EmbabelComponent
public class CoordinatorActions {

   @Action
    GreetingResult greet(UserInput input, Ai ai) {
        return ai.createObject("Handle this greeting warmly", GreetingResult.class);
    }

    @AchievesGoal("user has been greeted")
    @Action
    TaskResult executeTask(GreetingResult greeting, Ai ai) {
        return ai.createObject("Process: " + greeting.message(), TaskResult.class);
    }
}

// Run with Supervisor planner
var result = SupervisorInvocation.on(agentPlatform)
    .returning(TaskResult.class)
    .withScope(AgentScopeBuilder.fromInstance(coordinatorActions))
    .invoke(new UserInput(request));

No magic output keys; typed and testable.

The same actions work across all planners. Write once, deploy with GOAP for determinism, Utility for exploration, or Supervisor for flexibility.

@State: State Machines Done Right

LangGraph popularized state machines for agents. We’ve made them type-safe, refactorable, and maintainable.

Our@State annotation enables branching, looping, and human-in-the-loop without magic strings. Any @Action method can return a class annotated with @State to trigger a transition. Only the actions on the target state will be available until the agent process terminates, or another state transition occurs. Polymorphism support helps routing between states, allowing us to declare a supertype as action return:

@State
record AssessStory(String content) implements Stage {
    @Action
    Stage assess() {
        if (isAcceptable()) return new Done(content);
        return new ReviseStory(content);  // Loop back
    }
}

@State
record ReviseStory(String content) implements Stage {
    @Action
    AssessStory revise() {
        return new AssessStory(improvedContent());
    }
}

@State
record Done(String content) implements Stage {
    @AchievesGoal(description = "Story finalized")
    @Action
    Output complete() {
        return new Output(content);
    }
}

States take the data they need in their constructors. Transitions are type-checked. IDEs understand them.

Let’s see how this would look in LangGraph:

# Define the state schema
class State(TypedDict):
    content: str
    status: str

# Define node functions
def assess_story(state: State) -> State:
    # ... assessment logic
    return {"content": state["content"], "status": "acceptable"}  # or "needs_revision"def revise_story(state: State) -> State:
    # ... revision logic
    return {"content": revised_content, "status": "pending"}


def finalize(state: State) -> State:
    return state

# Routing function
def should_revise(state: State) -> str:
    if state["status"] == "acceptable":
        return "done"
    return "revise"

# Build the graph
workflow = StateGraph(State)
workflow.add_node("assess", assess_story)
workflow.add_node("revise", revise_story)
workflow.add_node("done", finalize)
workflow.set_entry_point("assess")
workflow.add_conditional_edges("assess", should_revise, {"revise": "revise", "done": "done"})
workflow.add_edge("revise", "assess")
workflow.add_edge("done", END)

graph = workflow.compile()

To say this doesn’t compare well is an understatement. LangGraph = magic strings everywhere. Routing logic is separated from the states it routes between. No compile-time checking that “revise” actually exists. Good luck testing it; rename a node and watch your graph break.

Embabel now implements the core idea of LangGraph in a far superior way.

Putting it all together

Not only is the programming model identical between all planner types; you can combine them. If an action is run with the GOAP planner and returns an @State type, control goes to the state machine, but its ultimate return will be passed back to the GOAP planner; each state defaults to GOAP planning internally. You can use any set of actions with any planner, depending on your requirements.

Looping in particular becomes much easier, as it’s the one common thing that is hard to express purely in GOAP.

Tool Enhancements

We’ve significantly enhanced our tool support, along with shipping powerful tools for developers to use.

AgenticTool: Familiar Pattern, Better Model

If you’re coming from LangChain or Google ADK, AgenticTool provides a familiar pattern: an LLM orchestrating sub-tools. It maps directly to LangChain's supervisor-with-workers or ADK's coordinator pattern.

var researcher = new AgenticTool(
        "research-assistant",
        "Research topics using web search and summarization")
    .withTools(searchTool, summarizeTool)
    .withLlm(LlmOptions.withModel("gpt-4"));
context.ai()
    .withTool(researcher)
    .generateText("What's the latest on quantum computing?");

It operates at the tool, not agent, level. It’s composable and reusable — just a tool, usable anywhere tools can be used. And it’s simple— no graph-based workflows or explicit Sequential/Parallel/Loop patterns.

In largely detyped frameworks like LangGraph and ADK, the distinction between a “supervisor agent” and an “agentic tool” is often meaningless. It’s all strings passing around. The hierarchy is conceptual, not enforced. In Embabel, this distinction matters because we have typed composition at the agent level (GOAP, @State) versus simpler tool-level orchestration (AgenticTool). You choose the right abstraction for the job.

For anything beyond simple orchestration, use GOAP, Utility, or @State—deterministic, type-safe planning that's more powerful than LLM-driven orchestration.

Agentic RAG API

Although it’s one of the oldest applications of Gen AI, RAG is notoriously hard to get right. We believe that the best approach is agentic RAG, and offer a simple, elegant API for it.

The new ToolishRag class exposes search operations as a set of LLM tools, along with instructions on their use. The LLM decides when to search, what queries to run, and how to expand context.

You give it a name, a description and a SearchOperations implementation. Depending on the capabilities of that implementation, ToolishRag will expose tools for vector search, text search, chunk expansion, and any operations unique to that store.

var response = context.ai()
    .withReference(
      new ToolishRag("docs", "Embabel Documentation", searchOperations)
    )
    .respondWithSystemPrompt(conversation, bindings);

Agentic RAG without boilerplate, consistent across stores. You can even use multiple ToolishRag instances in one LLM interaction and the framework will disambiguate their tools.

We ship SearchOperations implementations for Neo4j, Lucene, local directories and over any Spring AI VectorStore. We will add a full-featured PostgreSQL implementation using pgvector and full text search. It’s also easy to implement your own SearchOperations and benefit from the same elegant programming model.

Chatbots and Developer Resources

Easy Chatbot Authoring

As of 0.3.1 it is much easier to write chatbots in Embabel.

The Utility planner is ideal for chatbots, and the new trigger field on @Action enables reactive chatbot patterns, causing the action to fire only when a specific type is most recent on the blackboard:

@Action(canRerun = true, trigger = UserMessage.class)
void respond(Conversation conversation, ActionContext context) {
    var reply = context.ai()
        .withTemplate("chatbot")
        .respondWithSystemPrompt(conversation, Map.of());
    context.sendMessage(conversation.addMessage(reply));
}

Multiple actions can respond to messages with different values. The planner picks the most valuable. The @State mechanism also works well with chatbots.

See the https://github.com/embabel/ragbot repository for an example of a chatbot using RAG.

The Guide: MCP Server for Embabel Development

We’ve used our RAG support to build an MCP server in the new guide project that helps you build with Embabel. It uses RAG against documentation and source code, exposing the knowledge to Claude Desktop, Claude Code, Cursor, Windsurf or any other MCP client.

It helped write this blog, and provided all the examples.

You can clone the https://github.com/embabel/guide project and run it as a Spring Boot application or via Docker. The README describes how to integrate it with Claude Desktop and popular coding agents.

This project will be the basis for a public chatbot that will help users learn and develop with Embabel.

Other New Features

Among many other improvements:

• We’ve taken our first steps toward supporting the emerging Agent Skills standard. See Embabel documentation for more information.
• We’ve added support for image generation and response streaming
• The Embabel documentation is much improved

Looking Forward

We’re working towards a 1.0 GA release around late January.

Embabel aims to make it impossible to justify choosing Python for production-grade agent systems not already committed to that ecosystem. The JVM is where most of the world’s valuable business logic runs. Gen AI enabling it matters. Embabel is the agent framework to unlock the enterprise value of Gen AI.

References

• Embabel Agent Framework: https://github.com/embabel/embabel-agent
• Guide MCP Server and chatbot: https://github.com/embabel/guide
• Ragbot chatbot: https://github.com/embabel/ragbot
• Examples (Java & Kotlin): https://github.com/embabel/embabel-agent-examples
• Documentation: https://docs.embabel.com/embabel-agent/guide/0.3.1-SNAPSHOT
• Discord: Join our community Discord at https://discord.gg/t6bjkyj93q

In fact, if you’re already on the JVM, doing Gen AI on the JVM is a no brainer. And if you’re not in either camp, a fair comparison of the two might surprise you.

In previous blogs, I’ve taken CrewAI and Pydantic AI examples and reimplemented them in Java with Embabel, mirroring the functionality in cleaner, more extensible code.

Today it’s the turn of LangGraph. LangGraph is an outgrowth of LangChain, the first and most popular Python Gen AI framework. LangChain’s popularity, and the promise of LangGraph, has made the company hot property, recently raising a Series B at a $1.1bn valuation. Can its frameworks do things Java cannot?

LangGraph Architecture

LangGraph offers a code-centric approach to multi-agent workflows. It uses a finite state machine (FSM) to order actions. Nodes (states) are functions; edges (transitions) can be automatic or determined dynamically from function call results.

Nodes and edges break long flows into multiple steps, making execution more deterministic and workflows easier to reason about.

A state machine is one of the most obvious ways to define workflows, so it’s not surprising that LangChain, whose market position is largely due to being first, rather than best, would leap to adopt it. But is it the best model, or might a less obvious solution be superior? At Embabel we built several iterations of state machines before settling on a different approach, based on Goal Oriented Action Planning (GOAP).

Sometimes it’s best not to be the first mover, but to learn from the successes and mistakes of others.

Pattern Showdown

I’ve been meaning to do a methodical comparison between Embabel and LangGraph for some time.

To make it fair, I’ve not chosen examples of my own, but reimplemented patterns from a recent blog on Agentic Design Patterns with LangGraph by Hamza Boulahia. As the author notes:

If there’s one thing I’ve learned building AI systems over the last couple of years, it’s this: patterns matter. Whether we’re designing traditional software or experimenting with large language model (LLM) agents, the way we structure workflows determines how robust, flexible, and scalable they’ll be.

…In this post, I’ll walk you through some of the most common agentic design patterns you can implement in LangGraph.

Boulahia demonstrates six agent design patterns:

• Prompt chaining — breaking complex tasks into manageable steps
• Routing and parallelization for control flow
• Reflection, where agents critique and improve outputs
• Tool use to integrate with external systems
• Planning — structuring goals into clear, executable sequences
• Multi-agent collaboration

Let’s go through each pattern in turn and show how it can be better expressed in Java using Embabel.

Prompt Chaining

Prompt chaining is an essential pattern where one model’s output becomes another model’s input. It makes workflows more predictable. The example suggests topics based on user input, and generates several blog titles for each topic.

LangGraph Implementation

The example uses the underlying LangChain API for LLM invocation, while LangGraph adds workflow management.

Like any workflow engine, LangGraph (unlike the lower level LangChain) needs to maintain process state. This is expressed in the State class.

The flow is defined via nodes and edges. Each node is a Python method that calls an LLM.

# Define LLM client
llm = ChatGoogleGenerativeAI(
    model="gemini-2.0-flash",
    api_key=GOOGLE_API_KEY,
    system_instruction="""You are an expert technical writer. Always give clear,
     concise, and straight-to-the-point answers."""
)

# Define the graph state
class State(dict):
    text: str
    topics: str
    title: str

# Define nodes
def extract_topics(state: State) -> State:
    prompt = f"Extract 1-3 key topics from the following text:\n\n{state['text']}"
    resp = llm.invoke(prompt)
    state["topics"] = resp.content.strip()
    return state

def generate_title(state: State) -> State:
    prompt = f"Generate two catchy blog titles for each one these topics:\n\n{state['topics']}"
    resp = llm.invoke(prompt)
    state["title"] = resp.content.strip()
    return state

The workflow is built in code, with nodes mapping to methods and edges linking them. In this case the edges are simple: Automatic transition to the next state. As the workflow is expressed in terms of magic strings, there is little checking of its validity at compile time.

# Build the graph
workflow = StateGraph(State)
workflow.add_node("extract_topics", extract_topics)
workflow.add_node("generate_title", generate_title)

# Define the node connections
workflow.set_entry_point("extract_topics")
workflow.add_edge("extract_topics", "generate_title")
workflow.add_edge("generate_title", END)

# Compile runnable graph
graph = workflow.compile()

We run the workflow as follows:

input_text = (
    "LangGraph introduces a graph-based paradigm for building LLM-powered agents. "
    "It allows developers to create modular, debuggable, and reliable agent workflows "
    "using nodes, edges, and state passing."
)

result = graph.invoke({"text": input_text})
print("Topics:", result["topics"])
print("\n"+"="*50+"\n")
print("Suggested Blog Title:", result["title"])

This will produce output like this:

Topics: Here are 3 key topics extracted from the text:

1.  **Graph-based LLM Agents:** The core concept of using a graph structure to build LLM agents.
2.  **Modularity, Debuggability, and Reliability:** The benefits of using LangGraph for agent development.
3.  **Nodes, Edges, and State Passing:** The fundamental components of the LangGraph framework.

==================================================

Suggested Blog Title: Okay, here are two catchy blog titles for each of the three topics, aiming for a mix of clarity and intrigue:

**1. Graph-based LLM Agents:**

*   **Title 1:** Unleashing the Power of Graph Minds: Building Smarter LLM Agents
*   **Title 2:** Beyond Chains: How Graph-Based Agents are Revolutionizing LLMs

**2. Modularity, Debuggability, and Reliability:**

*   **Title 1:** LangGraph: The Secret to Building Reliable and Debuggable LLM Agents
*   **Title 2:** Stop Chasing Bugs: LangGraph's Modular Approach to LLM Agent Development

**3. Nodes, Edges, and State Passing:**

*   **Title 1:** LangGraph Demystified: Nodes, Edges, and the Flow of Intelligent Conversations
*   **Title 2:** Building Blocks of Brilliance: Mastering Nodes, Edges, and State in LangGraph

Embabel Implementation

It’s hard to do a direct port of this because, albeit trivial, it’s poorly designed. The domain model is particularly lacking. Python approaches to Gen AI tend to underemphasize domain modeling, which is a serious mistake. All fields of the State class are modelled as strings, where there should be a structure. Expressing topics as a single string is particularly misleading.

I made the minimum changes to be able to live with the solution. Here’s the Embabel implementation, expressed in terms of a domain model, actions and a goal.

The domain objects are Java records, correctly representing the structure implied by the requirements.

The @Agent annotation causes the class to be instantiated and injected by Spring, and processed by Embabel. The @Action methods are steps, and the goal the desired output:

@Agent(description = "Blog Titler Agent")
public class BlogTitler {

    private final Actor<?> techWriter = new Actor<>(
    """
            You are an expert technical writer. Always give clear,
            concise, and straight-to-the-point answers.
            """,
            LlmOptions.withAutoLlm());

    public record Topics(
            List<String> topics
    ) {
    }

    public record TopicTitles(
            String topic,
            List<String> titles
    ) {
    }

    public record BlogTitles(
            List<TopicTitles> topicTitles
    ) {
    }

    @Action
    public Topics extractTopics(UserInput userInput, Ai ai) {
        return techWriter.promptRunner(ai)
                .creating(Topics.class)
                .fromPrompt("""
                        Extract 1-3 key topics from the following text:
                        %s
                        """.formatted(userInput.getContent()));
    }

    @Action
    @AchievesGoal(description="Generate Titles for Topics")
    public BlogTitles generateBlogTitles(Topics topics, OperationContext context) {
        var titles = context.parallelMap(
                topics.topics(),
                10,
                topic -> techWriter.promptRunner(context)
                .creating(TopicTitles.class)
                .fromPrompt("""
                        Generate two catchy blog titles for this topic:
                        %s
                        """.formatted(topic)));
        return new BlogTitles(titles);
    }

}

The Actor field combines LLM selection with hyperparameters and an instruction.

The extractTopics and generateBlogTitles methods are the equivalent of the LangGraph node methods. However, the workflow is put together quite differently.

With Embabel we don’t need to specify the workflow in code. Embabel plans an execution path by analyzing our code.

The Embabel planner can infer the correct execution plan from the flow of types. For example, it can see that the goal can only be achieved if a Topics and BlogTitles object is available in workflow scope. Thus it can correctly determine the order of methods to call. This planning step is deterministic.

We model the generateBlogTitles step in terms of parallel operations, more naturally reflecting the requirements and resulting in cleaner, more focused interactions with the LLM.

The output will look like this. Unlike the LangGraph example’s output, the structure accurately reflects the requirements:

{
  "topicTitles" : [ {
    "topic" : "LangGraph graph-based paradigm for LLM agents",
    "titles" : [ "Unlocking Smarter AI: How LangGraph’s Graph-Based Paradigm Revolutionizes LLM Agents", "Navigating the Future of LLM Agents with LangGraph’s Innovative Graph-Based Approach" ]
  }, {
    "topic" : "Modular and debuggable agent workflows",
    "titles" : [ "Streamlining Success: Building Modular and Debuggable Agent Workflows", "Mastering Agent Workflows: The Power of Modularity and Debugging" ]
  }, {
    "topic" : "Comparison with Embabel's GOAP planning",
    "titles" : [ "Mastering GOAP: How Embabel’s Planning Stacks Up Against the Rest", "Embabel's GOAP Planning Compared: What Makes It Stand Out?" ]
  } ]
}

Comparison

Both implementations break the flow into small, focused steps, which makes it more predictable. There is slightly more code in the Embabel version, because it properly models cardinality.

The Embabel implementation is superior in several respects. It’s type safe. It has a true domain model versus misleading strings. It’s easier to test. And parallelization is only possible because of the additional structure.

Some enhancements that would be trivial with the Embabel, but not LangGraph, implementation:

• Externalizing LLM choice and persona: With Embabel and Spring, it’s trivial to externalize the Actor definition into an application.yml file, along with all the other sophisticated configuration options that are lacking in Python.
• Persisting topics and blog titles: Because we have correctly structured domain objects, we could easily persist them if we wished.

Routing

Another fundamental pattern is routing within a workflow. Here’s a LangGraph implementation, first performing sentiment analysis and then choosing one or other function to generate a response:

llm = ChatGoogleGenerativeAI(
    model="gemini-2.0-flash",
    api_key=GOOGLE_API_KEY,
    system_instruction="""You are a helpful assistant that can classify text 
    sentiment and respond accordingly."""
)

class State(dict):
    text: str
    sentiment: str
    response: str

# Nodes
def calssification(state: State) -> str:
    """Classify sentiment."""
    prompt = f"Is the following text positive or negative? Answer with one word only: Positive or Negative.\n\n{state['text']}"
    resp = llm.invoke(prompt)
    sentiment = resp.content.strip().lower()
    state["sentiment"] = sentiment
    return state

def positive_node(state: State) -> State:
    prompt = f"Generate an encouraging reply to this positive text:\n\n{state['text']}"
    resp = llm.invoke(prompt)
    state["response"] = resp.content.strip()
    return state

def negative_node(state: State) -> State:
    prompt = f"Generate a supportive reply to this negative text:\n\n{state['text']}"
    resp = llm.invoke(prompt)
    state["response"] = resp.content.strip()
    return state

def router_func(state: State) -> Literal["positive", "negative"]:
    """Return next node name."""
    return "positive" if "positive" in state["sentiment"] else "negative"

# Build the graph
workflow = StateGraph(State)

workflow.add_node("calssification", calssification)
workflow.add_node("positive", positive_node)
workflow.add_node("negative", negative_node)

# classify node decides the next step
workflow.set_entry_point("calssification")
workflow.add_conditional_edges("calssification", router_func, {
    "positive": "positive",
    "negative": "negative",
})

# Both branches lead to END
workflow.add_edge("positive", END)
workflow.add_edge("negative", END)

graph = workflow.compile()

As usual with LangGraph, the workflow is built with unchecked strings. It’s fortunate that the author has consistently misspelt “classification” because typos matter a lot with something this fragile.

This example shows a more sophisticated LangGraph state transition, mapping the output of a function to the next state name, with still more error-prone magic strings as a bonus:

workflow.add_conditional_edges("calssification", router_func, {
    "positive": "positive",
    "negative": "negative",
})

We run this as follows:

input_text = "I'm so happy with how my project turned out!"
result = graph.invoke({"text": input_text})
print("Sentiment:", result["sentiment"])
print("Response:", result["response"])

Embabel Implementation

Explicit wiring of nodes and edges seems overcomplicated for something so simple. Let’s express the steps in a more natural manner.

Because the steps are different, we can simply use a focused prompt each time, rather than a common system message associated with the LLM.

First we’ll create types for Sentiment. This will enable the flow to be expressed in terms of data types:

@Agent(description = "Perform sentiment analysis")
public class ClassificationAgent {

    @JsonTypeInfo(
            use = JsonTypeInfo.Id.SIMPLE_NAME,
            include = JsonTypeInfo.As.PROPERTY,
            property = "type"
    )
    public sealed interface Sentiment {
    }

    public static final class Positive implements Sentiment {
    }

    public static final class Negative implements Sentiment {
    }

    public record Response(String message) {
    }

    private enum SentimentType {
        POSITIVE,
        NEGATIVE;

        public Sentiment toSentiment() {
            return switch (this) {
                case POSITIVE -> new Positive();
                case NEGATIVE -> new Negative();
            };
        }
    }

Now we can implement the first step as a classify method that returns Sentiment. The Embabel planner is smart enough to know that Sentimenthas subtypes and route appropriately.

The encourage and help methods each take a Sentiment subtype. Like the Python functions, they invoke an LLM to generate a response.

@Action
  public Sentiment classify(UserInput userInput, Ai ai) {
      var type = ai.withAutoLlm()
              .createObject("""
                              Determine if the sentiment of the following text is positive or negative.
                              Text: "%s"
                              """.formatted(userInput.getContent()),
                      SentimentType.class);
      return type.toSentiment();
  }

  @Action
  public Response encourage(
          UserInput userInput,
          Positive sentiment,
          Ai ai) {
      return ai.withAutoLlm()
              .createObject("""
                      Generate an encouraging response to the following positive text:
                      %s
                      """.formatted(userInput.getContent()), Response.class);
  }

  @Action
  public Response help(
          UserInput userInput,
          Negative sentiment,
          Ai ai) {
      return ai.withAutoLlm()
              .createObject("""
                      Generate a supportive response to the following negative text:
                      %s
                      """.formatted(userInput.getContent()), Response.class);
  }

  @AchievesGoal(
          description = "Generate a response based on discerning sentiment in user input")
  @Action
  public Response done(Response response) {
      return response;
  }

}

Comparison

Defining this flow via data types with Embabel is more natural and less error-prone than the LangChain state machine. Again, with Embabel we don’t need to define the flow, as the planner will work it out.

Parallelization

LLM interactions can be slow. Thus parallelization is important in Gen AI applications.

LangGraph lets us parallelize operations through “parallel edges.”

# Build the graph
builder = StateGraph(State)
builder.add_node("summarize", summarize)
builder.add_node("critique", critique)
builder.add_node("combine", combine)

# Parallel edges: summarize and critique run side by side
builder.add_edge(START, "summarize")
builder.add_edge(START, "critique")

# Both join into combine
builder.add_edge("summarize", "combine")
builder.add_edge("critique", "combine")

builder.add_edge("combine", END)p

With Embabel, we have two choices for parallelization:

• Programmatic, using the parallelMap method that we saw in the blog title example.
• Automatic: If we set embabel.agent.platform.process-type=CONCURRENT, the planner will automatically run in parallel operations that are both necessary to achieve the current goal and which don’t depend on each other.

Reflection

This is a common and important pattern that goes by various names. Anthropic’s Building Effective Agents article calls this pattern Evaluator-optimizer.

The key idea is to use one LLM call to provide feedback on the output of another, iterating until the result is satisfactory.

Here’s the LangGraph implementation, drafting text in response to a user-specified task and critiquing it until it’s satisfactory.

# State
class State(TypedDict):
    task: str
    draft: str
    feedback: str
    final: str

# Nodes
def generator(state: State):
    """Generate an initial or refined draft."""
    prompt = f"""
You are an assistant helping to complete the following task:

Task:
{state['task']}

Current Draft:
{state.get('draft', 'None')}

Feedback:
{state.get('feedback', 'None')}

Instructions:
- If there is no draft and no feedback, generate a clear and complete response to the task.
- If there is a draft but no feedback, improve the draft as needed for clarity and quality.
- If there is both a draft and feedback, revise the draft by incorporating the feedback directly.
- Always produce a single, improved draft as your output.
"""
    resp = llm.invoke(prompt)
    return {"draft": resp.content.strip()}

def evaluator(state: State):
    """Evaluate the draft and give feedback or approval."""
    prompt = f"""Evaluate the following draft, based on the given task. 
If it meets the requirements, reply exactly 'APPROVED'. 
Otherwise, provide constructive feedback for improvement.
Task:
{state['task']}
Draft:
{state['draft']}"""
    resp = llm.invoke(prompt)
    print(f"""
================= DRAFT =================
{state['draft']}

================ FEEDBACK ===============
{resp.content.strip()}
========================================
""")
    return {"feedback": resp.content.strip()}

def decide(state: State) -> str:
    """Decide next step: either approve and finish, or refine again."""
    if "APPROVED" in state["feedback"].upper():
        return "approved"
    return "refine"

def finalize(state: State):
    """Save the final approved draft."""
    return {"final": state["draft"]}

# Build the graph
builder = StateGraph(State)

builder.add_node("generator", generator)
builder.add_node("evaluator", evaluator)
builder.add_node("finalize", finalize)

builder.add_edge(START, "generator")
builder.add_edge("generator", "evaluator")
builder.add_edge("evaluator", "finalize")

# Conditional edges from decide
builder.add_conditional_edges(
    "evaluator",
    decide,
    {
        "approved": "finalize",   # stop loop
        "refine": "generator",    # go back for improvement
    },
)

builder.add_edge("finalize", END)

graph = builder.compile()

While the pattern is valuable, this implementation is far from ideal. For a start, it’s not guaranteed to terminate.

From the developer point of view, a state machine isn’t the ideal expression of this common workflow. To be fair, neither is Embabel’s GOAP planning.

While an FSM or GOAP is well suited to coarse grained planning, well-known fine grained operations can be made easier by dedicated framework support.

Thus Embabel provides out of the box implementations of such common patterns. (You can also write your own.) The fluent API for such builders emits GOAP plans, so the platform can manage the steps in its normal way.

We create an Agent programmatically using RepeatUntilAcceptableBuilder and expose it as a Spring bean in an @Configuration class. Embabel will automatically deploy it:

@Bean
public Agent draftAndRefineAgent() {
  return RepeatUntilAcceptableBuilder
      .returning(Draft.class)
      .consuming(UserInput.class)
      .withMaxIterations(7)
      .withScoreThreshold(.9)
      .repeating(tac -> {
          return tac.ai()
                  .withAutoLlm()
                  .createObject(
                          """
                          You are an assistant helping to complete the following task:
                          
                          Task:
                          %s
                          
                          Current Draft:
                          %s
                          
                          Feedback:
                          %s
                          
                          Instructions:
                          - If there is no draft and no feedback, generate a clear and complete response to the task.
                          - If there is a draft but no feedback, improve the draft as needed for clarity and quality.
                          - If there is both a draft and feedback, revise the draft by incorporating the feedback directly.
                          - Always produce a single, improved draft as your output.
                          """.formatted(
                          tac.getInput().getContent(),
                          tac.lastAttemptOr("no draft yet"),
                          tac.lastFeedbackOr("no feedback yet")),
                          Draft.class);
      })
      .withEvaluator(tac -> {
          return tac.ai()
                   .withAutoLlm()
                   .createObject(
                    """
                      Evaluating the following draft, based on the given task.
                      Score it from 0.0 to 1.0 (best) and provide constructive feedback for improvement.
                      
                      Task:
                      %s
                      
                      Draft:
                      %s
                      """.formatted(
                      tac.getInput().getContent(),
                      tac.getResultToEvaluate()),
                         TextFeedback.class);
      })
      .buildAgent("draft_and_refine_agent", 
                      "An agent that drafts and refines content");
}

The Embabel Java version has significantly less code than the Python LangGraph version. It’s type safe, toolable, and there’s no way to mess up the flow. If maxIterations is exceeded, the loop will terminate, returning the highest scoring result seen so far. Because RepeatUntilAcceptableBuilder is parameterized, you can use your own result type and feedback type, while preserving type safety. LangGraph compares very poorly in this scenario.

Tool Use

Tool use is table stakes for any Gen AI framework and must be easy.

With LangGraph we can define tools in Python and bind them to an LLM:

def calculator(expression: str):
    """Evaluate a math expression."""
    try:
        return str(eval(expression))
    except Exception as e:
        return f"Error: {e}"

llm = ChatGoogleGenerativeAI(
    model="gemini-2.0-flash",
    api_key=GOOGLE_API_KEY,
)

model_with_tools = llm.bind_tools([calculator])
tool_node = ToolNode([calculator])

# Nodes
def call_model(state: State):
    """Call the model; it may request a tool."""
    messages = state["messages"]
    response = model_with_tools.invoke(messages)
    return {"messages": [response]}

This is straightforward, although of course the example is dangerous because it evaluates generated Python without sandboxing.

Embabel allows us to add tools every time we use a PromptRunner, with less ceremony. We can define well known tool groups (which are often backed by MCP), or specify Java objects:

var response = ai.withAutoLlm()
  .withToolGroup(CoreToolGroups.MATH)
  .withToolObject(myDomainObject);

The ability to use tool objects is particularly important. These could be domain objects we’ve retrieved from a database or services we injected into our agent with Spring. We can do that in Python also, but there’s an important difference in that it’s more likely that we have valuable business functionality implemented in Java.

Embabel also offers tool grouping with concepts like an LlmReference, which combines tools with prompt elements allowing sophisticated composition at runtime.

Planning

The LangGraph blog example of planning involves a final step that depends on the execution of two previous steps. This is expressed in the workflow builder like this:

# Set the entry point
workflow.set_entry_point("planner")

# Define the parallel edges
# After the planner, both research workers are called.
workflow.add_edge("planner", "graphql_researcher")
workflow.add_edge("planner", "rest_researcher")

# Define the join point
# The writer will only run after BOTH research workers are complete.
workflow.add_edge("graphql_researcher", "writer")
workflow.add_edge("rest_researcher", "writer")

# The graph ends after the writer is done
workflow.add_edge("writer", END)

In Embabel, you would simply define a method that takes both data types:

Writeup writeUp(GraphQLResearch gr, RestResearch rr, Ai ai) {

Clearer, type safe and easy to test.

The LangChain example also hardcodes the steps in the plan but notes that they could be generated by an LLM. We could implement this by having an LLM instantiate command objects in Embabel. However, relying on LLMs to perform planning makes flows less predictable, so should be done with care.

Multi-Agent Collaboration

The final pattern is an example of handoffs from a supervisor node to other agents. This is a common pattern in frameworks such as Google ADK. It’s also less deterministic than expressing the steps via a state machine or GOAP actions, hence not a pattern we tend to encourage. To achieve this behaviour, Embabel allows you to use agents as tools.

Complete Code

The complete code can be found in the Embabel langgraph-patterns repository.

You could argue that the original blog isn’t the greatest LangGraph showcase. But it’s a recent, well-written piece based on a sound understanding of agents, with hundreds of claps on Medium. It’s not hard to find the same deficiencies in other LangGraph applications, suggesting that the problem lies with the framework, language and ecosystem, not the author.

The Reality

It’s time the Java community stopped feeling it needs to play catch up, and demonstrates that it not only has a solution for building Gen AI agents, but has the best, on any platform.

It’s time to stop settling for being downstream of mediocre ideas from Python frameworks, and do our own thinking. The Python emperor has no clothes. If you’re a JVM developer and your boss tells you to write agents in Python, stand ready to point out the inferiority of Python as a platform for business applications, Gen AI or other. And be ready to show how common patterns can look a whole lot better in Java than Python.

Python is an important language that every developer should be fluent in. It’s great for many things. However, it has never been a particularly popular language for building nontrivial business applications, and there’s good reason for that.

The robustness of the JVM is well known; the ecosystem — especially Spring, which has no analog in Python — puts it in a very strong position. Embabel builds on top of this, bringing superior concepts to any Python framework. The rest is up to you. Create your first agent from our Java agent template today.