Interfacing Large Language Models with ROS 2
This chapter will explore the possibilities of using GenAI in robotics by interfacing different large language models (LLMs) into ROS 2 by building AI agents. LLMs are a subset of GenAI mainly focused on generating text. So why does interfacing LLMs and AI agents with ROS 2 robots matter? Using LLMs and agents, robots can understand user input and generate an intelligent response. Users can use text, images, and voice as input, making the robot capable of answering questions intelligently. Other than interaction, LLM-based AI agents can control a robot by generating corresponding actions for the robot based on user input.
In this chapter, we’re going to cover the following main topics:
- LLMs for robotics
- Integrating ROS 2 robots with LLM agents
- Creating ROS 2 AI agents
- Popular ROS 2 AI agent projects