LLM-based Multi-Agent Systems to Replace Traditional Software
0 likes232 views
The potential of LLMs is severely underutilized as they are much more capable than generating completions or summarizing content. LLMs demonstrate remarkable capabilities in reaching a level of reasoning and planning comparable to human abilities. Microsoft CEO Satya Nadella revealed the shocking future of traditional software as we know it as being replaced by intelligent agents. In this session we introduce how would that be possible, what are the benefits and challenges and make our steps into the concept with a practical implementation.
![● Search Service by function
○ Web search documentation
○ Direct link
● Get Service endpoint
● Generate code to integrate the Service
● Do
○ Compile/Interpret
○ Run
○ Detect errors
○ Improve code
○ Map output to JSON
○ Validate output
While [hasErrors]
● Enrich the current context from Service
Agent Dynamic Integration Flow](https://image.slidesharecdn.com/multiagents-longliveintelligentagents-250315124732-09fd3efa/85/LLM-based-Multi-Agent-Systems-to-Replace-Traditional-Software-13-320.jpg)
![● AI Agent
○ Brain – an AI Model (usually LLM), performing reasoning and planning
○ Thoughts – internal action-observation loop
○ Body – capabilities and tools the agent is equipped with
○ Action – interaction with the environment using tools
● Thought Types (LLMs can only input/output text. Multimodal use tools for images)
○ Planning, Analysis, Decision making, Optimization
○ Self-reflection, Prioritization, Goal Setting
● Tool Types
○ Search, Image generation, Information retrieval, API call
○ Attributes: [Description], [Endpoint], [Typed Arguments], [Output]
● Observation Types
○ Feedback from environment, API/Query responses, Sensor readings, Events
What is an AI Agent
Describe the
Tool to Agent
Prompt the
Agent
Agent
recognizes
the Tool
Agent
generates
code for Tool
Tool outputs
to Agent
Agent
generates
response](https://image.slidesharecdn.com/multiagents-longliveintelligentagents-250315124732-09fd3efa/85/LLM-based-Multi-Agent-Systems-to-Replace-Traditional-Software-17-320.jpg)
LLM-based Multi-Agent Systems to Replace Traditional Software
- 1. Global AI Bootcamp Bulgaria 2025
- 2. Our Proud Sponsors 2025- Bulgaria
- 5. Long Live Intelligent Agents End of Traditional Software is Near Agent Era is About Redefining Digital Work & Assistance
- 6. • Solution Architect @ • Microsoft AI & IoT MVP • External Expert Eurostars-Eureka, Horizon Europe • External Expert InnoFund Denmark, RIF Cyprus • Business Interests • Web Development, SOA, Integration • IoT, Machine Learning • Security & Performance Optimization • Contact • [email protected] • www.linkedin.com/in/ivelin • www.slideshare.net/ivoandreev About
- 7. TAKEAWAYS ● Satya Nadella BG2 Podcast: Agents Will Replace ALL Software ● https://youtu.be/9NtsnzRFJ_o?t=2808 ● Transcript - https://app.podscribe.ai/episode/118164535 ● Large Language Model based Multi-Agents: A Survey of Progress and Challenges ● https://arxiv.org/html/2402.01680v2 ● Hugging Face AI Agents Course ● https://huggingface.co/learn/agents-course/en/unit0/introduction ● Azure AI Agent Service ● https://learn.microsoft.com/en-us/azure/ai-services/agents/overview ● Multi-agent Multimodal Sample with Azure Assistant API ● https://github.com/Azure-Samples/azureai- samples/blob/main/scenarios/Assistants/multi-agent/ ● Multi-agent Creative Writer App (Advanced Sample) ● https://github.com/Azure-Samples/contoso-creative-writer/
- 8. Microsoft CEO with Bold Statement on Agents
- 9. The AI tier will become the place where all the logic is, people will start replacing backends. (Dec 2024) Interview Highlights IT Business Challenges… ● Traditional tools are uncapable of maintaining context and handling complex, continuous interactions well. ● Manual repetitive tasks handling reduces productivity and efficiency. ● Inefficient fragmentation of data and functionality across systems ● Users require increasingly personalized experiences and better interaction ● Processing vast amounts of data manually is challenging ● Organizations need to quickly adopt innovative solutions
- 10. Agents Early Phase Adoption ● Customers service – early area of agent adoption for cost and efficiency ● Agents will orchestrate business apps ● Agents will take over logic, interact with backends, replace traditional apps ● Agents will help automation of repetitive tasks ● Microsoft is integrating AI agents via connectors o i.e. with Adobe, SAP. Dynamics Open Issues ● Business model is uncertain and to be decided ● Demand for compute resources will grow exponentially Satya’s Predictions
- 11. Whoever could do Lots of Compute is a Big Winner
- 12. ● World’s first scalable quantum processor powered by topological qubits ● Topoconductor o Hardware-protected (a.k.a. topological) qubit o New state of matter that previously existed only in theory o Recognized from DARPA (Defense Advanced Research Projects Agency) Majorana 1, Feb 2025 Company Qubit Type Qubit Count Error Resistance Scalability Google (Sycamore) Superconducting ~72 Low Medium IBM (Eagle) Superconducting ~433 Low Medium IonQ, Quantinuum Trapped Ions ~32 Medium Low Microsoft (Majorana 1) Topological 8+ (scalable to 1M) High High
- 13. ● Search Service by function ○ Web search documentation ○ Direct link ● Get Service endpoint ● Generate code to integrate the Service ● Do ○ Compile/Interpret ○ Run ○ Detect errors ○ Improve code ○ Map output to JSON ○ Validate output While [hasErrors] ● Enrich the current context from Service Agent Dynamic Integration Flow
- 14. What is the geo location of a vessel with IMO number 9644342? Note: Licensing issues may apply when reproducing data from other sources Reasoning is no Longer Sci-Fi
- 15. AI Agents … • Are Autonomous • Have Objectives • Interact with Environment & Act
- 16. LLMs can do much more than prompt-completion: 1. Text understanding 2. Logical reasoning 3. Code generation and debugging 4. Data extraction 5. Multimodal (images, video, audio) – generate/understand 6. Chatbots, recommenders 7. Autonomous multi-agent collaboration • Orchestrate multiple AI systems to collaborate • Automate workflows • Planning and resource allocation LLMs’ Potential is Underutilized
- 17. ● AI Agent ○ Brain – an AI Model (usually LLM), performing reasoning and planning ○ Thoughts – internal action-observation loop ○ Body – capabilities and tools the agent is equipped with ○ Action – interaction with the environment using tools ● Thought Types (LLMs can only input/output text. Multimodal use tools for images) ○ Planning, Analysis, Decision making, Optimization ○ Self-reflection, Prioritization, Goal Setting ● Tool Types ○ Search, Image generation, Information retrieval, API call ○ Attributes: [Description], [Endpoint], [Typed Arguments], [Output] ● Observation Types ○ Feedback from environment, API/Query responses, Sensor readings, Events What is an AI Agent Describe the Tool to Agent Prompt the Agent Agent recognizes the Tool Agent generates code for Tool Tool outputs to Agent Agent generates response
- 18. Chatbot • Conversation with limited knowledge of context from window AI Agent • AI does not just talk, it takes actions on behalf of users • LLMs that combine state, tools, autonomous execution. • Combines strategic capabilities with autonomy • Highest level safety, security and responsibility Agent Stack • Involves retaining message history & multiple LLM calls. Chats vs. AI Agents
- 19. Agent Stack • Model & Storage • Tools • Deployment & Serving
- 20. Model Serving Layer • The core component is built on LLM (behind a paid API) • Closed Model Providers (OpenAI-GPT and Anthropic-Claude) • Open Model Providers (Together.ai, vLLM) Storage Layer • Persisted state required by stateful agents • Used for conversation history and external data (RAG) • Vector database or RDBMs with vector search Agent Stack Structure
- 21. Tools & Libraries • Enhance AI agent capabilities • Call tools via structured outputs (i.e. JSON) • Ecosystem of tool providers growing • Tool usage is a primary distinction of agents from chatbots Agent Frameworks • Support multi-agent communication and memory management • Orchestrate LLM calls, context window and state Agent Stack Structure
- 22. Deployment and Serving • Current state – mainly Jupyter and Python • Future state - services via REST APIs • Goal is automated, easier and secure deployment • I.e. OpenAI Assistants API Future Evolution • Rapidly evolving landscape • Tool ecosystem growth • Optimized deployment workflows • Improved tool execution integration and memory management Agent Stack Structure
- 24. From Agents to Multi-Agents • Collaboration • Typical Challenges
- 25. ● Involves multiple intelligent agents interacting and collaborating. ● Capable of decisions and action in typically human domains ● Benefits • Complex problem-solving • Coordination • Scalability ● Azure Assistants API ○ Easy to use framework for creating multi-agent systems ○ Persistent muti-agent systems, virtually indefinite context ○ Access and process files, code and functions Multi-Agent Systems
- 26. Communication is critical for collective intelligence ○ Cooperative ○ Debate ○ Competitive ● Communication Structures ● Capability Acquisition ○ Environment feedback ○ Interaction feedback ○ Human feedback How Agents Collaborate?
- 27. Collective Intelligence ● Traditional MA Systems use reinforcement learning to learn from offline data ● LLM-MA learn from instant feedback Scalability ● Each LLM-based agent is based on an LLM (i.e. GPT4) and requires huge resources ● Coordination complexity raises significantly with number of agents Multi-Modal Environment ● LLM agents are focused on text-based environments ● Integration in multimodal environment (sensors, video, audio) is a huge challenge Hallucinations Challenge ● Generating a factually incorrect output of an LLM is a huge problem ● The problem could be multiplied in MA scenario LLM-MA are Different from Traditional
- 28. Azure AI Foundry – Agent Service • Assistants vs. AI Agents • Service Highlights • Agentic Tools
- 29. ● A fully managed service to build, deploy and scale AI agents (Preview Dec 2024) ● Other Frameworks: smolagents, LlamaIndex, Langgraph Azure AI Foundry - Agent Service Assistant AI Agent Definition AI model assisting end-users Smart autonomous microservice w/o interactive UI Function Accurate response on context Planning and reasoning Use Cases Routine tasks • Chatbot, guidance Complex problems • Automation, workflows Trigger Reactive, on request Proactive, works autonomously Interaction Text-based, user-driven Action-oriented Memory Retain within a session Maintains indefinite context in time Complexity One-step: Request – Response Multi-step: Decision - Action
- 30. ● Your data are: ○ NOT available to OpenAI and not used to improve OpenAI models ○ NOT used to train or improve OpenAI Service models ○ NOT used to improve Microsoft or 3rd party products ○ NOT available to other customers ● Deployment Model ○ Deployment location – Global ($), Data Zone ($+10%), Regional ($+100%) ○ Usage - Standard, Provisioned, Batch API ● Billing ○ Charged by the usage of the base model of each agent ○ File search is billed by vector storage ■ 0.1 $/GB of vector storage per day (first 1GB free) ○ Agent Service ● Limits - 2’000’000 tokens/min You may Want to Know
- 31. ● Knowledge Tools ○ Grounding with Bing Search – real-time web data ($35/1000 requests) ○ File Search – augment the model with data from files ■ Vector Store: Max 10’000 files, Max 512MB/file, Max 5’000’000 tokens/file ■ File Types: docs (.docx, .pdf, .md, .pptx, .txt, .tex, .html), code (.c, .cs, .py, .sh, .css, .java) ■ Source: Local files or Azure Blob ○ Azure AI Search – AI-powered information retrieval ■ Skills – Image processing, translation, Natural Language Processing ● Action Tools ○ Function Calling – describe and run external functions ○ Code Interpreter – interpret and rune Python code in isolation ○ OpenAPI – connect to external API for interoperability ○ !!! Azure Functions – triggers and bindings, allow serverless scalability Agent Tools
- 33. ● Clone the repo ○ In PowerShell navigate to target root folder ● Check Python requirements (3.8+) ● Create a virtual environment for dependency isolation ● Activate (switch Python to) the virtual environment ● Install requirements (55 Python packages) ● Install Jupyter Lab in the environment ● Create Azure OpenAI service ● Create deployments of required OpenAI models (Azure AI Foundry > Deployments) Step-by-Step - Prerequisites git clone https://github.com/Azure-Samples/azureai-samples.git cd azureai-samples/scenarios/Assistants/multi-agent python --version python -m venv env envScriptsActivate pip install -r requirements.txt pip install jupyterlab
- 34. ● Configure the environment variables in the .env file ● Get Endpoint, Key, API Version, Deployment ○ GPT4_DEPLOYMENT_NAME (gpt-4o model) ○ DALLE3_DEPLOYMENT_NAME (dall-e-3 model) ○ GPT4VISION_DEPLOYMENT_NAME (gpt-4o model) ● Start Jupyter Lab from the project folder and environment ● Load the environment explicitly Step-by-Step - Configuration jupyter lab load_dotenv(dotenv_path="sample.env")
- 35. THANK YOU