Applications of Robotics

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data.  2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro  -> RoboCasa produces N (varying visuals)  -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are creating tools to enable everyone in the ecosystem to scale up with us: - RoboCasa: our generative simulation framework (Yuke Zhu). It's fully open-source! Here you go: http://robocasa.ai - MimicGen: our generative action framework (Ajay Mandlekar). The code is open-source for robot arms, but we will have another version for humanoid and 5-finger hands: https://lnkd.in/gsRArQXy - We are building a state-of-the-art Apple Vision Pro -> humanoid robot "Avatar" stack. Xiaolong Wang group’s open-source libraries laid the foundation: https://lnkd.in/gUYye7yt - Watch Jensen's keynote yesterday. He cannot hide his excitement about Project GR00T and robot foundation models! https://lnkd.in/g3hZteCG Finally, GEAR lab is hiring! We want the best roboticists in the world to join us on this moon-landing mission to solve physical AGI: https://lnkd.in/gTancpNK

The global economy has an impending problem. While AI is compounding its ability at a historic rate, an aging population and declining fertility rates are already causing labor shortages. These trends, combined with declining costs of robotics hardware, underpin a compelling case for humanoid robots and physical AI. According to Morgan Stanley, the humanoid robot market is set to exceed $5 trillion by 2050. Even in 2025, the larger robotics space saw $21 billion of VC capital invested. And with a steady increase in patent activity mentioning “humanoid” over the past few years, these machines are already walking onto factory floors. For most of human history, productive output was a function of human muscle. Agriculture, manufacturing, logistics, and construction were all built around the physical limits of the human body. Because humans did the work, the built world standardized around human form: doorways, staircases, countertops, and tools are all designed for two arms, two legs, and hands that grip. Redesigning every factory, warehouse, and home around task-specific machines would be unfeasible. A humanoid robot that fits into existing infrastructure doesn’t need the world to change around it. Near-term use cases focus on structured, predictable settings, enabling a robot to learn quickly, make mistakes cheaply, and improve rapidly. My research team at Social Capital concluded that humanoid Robots will have the highest impact in these 7 areas: 1. Domestic Assistance: Supporting mobility needs, handling household chores, and providing medication reminders. 2. Manufacturing: Assisting assembly tasks, moving tools and parts, inspecting finished products. 3. Security & Monitoring: Patrolling facilities, investigating alerts, and assisting in emergencies. 4. Customer Service & Reception: Greeting and directing visitors, answering questions, and managing check-ins or bookings. 5. Facility Maintenance: Conducting routine inspections, performing minor repairs, cleaning, and sanitizing spaces. 6. Healthcare: Assisting nurses, delivering supplies or meals, monitoring patients. 7. Warehouse and Logistics: Picking and packing items, loading and unloading goods, and moving inventory in warehouses. By 2050, Morgan Stanley estimates that more than 1 billion humanoid robots could be working globally, with a market size of over $5 trillion. This is one of the biggest opportunities in the AI era.

Production changes everything. What worked in a demo starts breaking at scale. That’s where real AI systems are tested. Here are the concepts that actually matter 👇 - Prototype vs production A demo works in controlled conditions, while production systems deal with scale, failures, and messy edge cases. - Training vs inference Training happens occasionally to build the model, while inference runs continuously to serve real users. - Batch vs real-time inference Batch is cost-efficient for large workloads, while real-time is critical when user experience depends on instant responses. - Accuracy vs reliability Accuracy looks good on test data, while reliability shows consistent performance under real-world conditions. - Guardrails vs validation Guardrails prevent unsafe outputs, while validation ensures correctness. Both are needed for safe and dependable systems. - Offline vs online evaluation Offline testing uses past data, while online evaluation measures real user impact. One doesn’t guarantee the other. - Data drift vs model drift Data drift changes inputs, while model drift shows performance degradation. Detecting this early avoids silent failures. - Monitoring vs observability Monitoring tracks known issues, while observability helps you understand unknown failures and system behavior. - Model hosting vs model serving Hosting deploys the model, while serving handles scaling, routing, and real-time requests. This is where complexity grows. - RAG vs fine-tuning RAG brings in fresh external knowledge, while fine-tuning embeds knowledge into the model. One adapts, the other is fixed. - Latency vs throughput Latency is response speed, while throughput is volume. Systems often fail because latency becomes too high. - Prompting vs fine-tuning Prompting shapes behavior through instructions, while fine-tuning changes model weights. Many real systems rely more on prompting. Understanding these trade-offs is what makes AI systems actually work. Which of these has been the toughest in your production setup?

Could AI Robots Help Fill the Labor Gap? As a futurist field, embodied AI—also known as humanoids—is captivating. Labor shortages spurred by long-term demographic shifts, coupled with advances in generative AI, are accelerating the commercialization of robots designed to emulate human behavior. The global economy faces labor shortages due to demographic trends that may hinder growth for years. Concurrently, advancements in large language models and generative AI are poised to drive transformative innovations across various industries, from healthcare to manufacturing. These trends are likely to fuel the development of humanoids—advanced robots equipped with limbs and AI-powered "brains." The adoption of these humanoid robots might outpace that of autonomous vehicles, presenting significant opportunities for investors in companies developing these robots and their components, and industries integrating them into their workforce. Its worth noting that Adam Jonas, Head of Global Autos and Shared Mobility research at Morgan Stanley, notes the adaptability of humanoids: "Consider the vast array of tasks humans perform using just our hands or tools, and the numerous machines tailored for human dexterity. As the growth of the working-age population in advanced economies continues to decline, humanoids could become essential for industries struggling to attract sufficient labor to maintain productivity." Morgan Stanley analysts project that by 2040, the U.S. alone could have 8 million working humanoid robots, impacting wages by $357 billion. By 2050, this number could rise to 63 million, potentially affecting 75% of occupations, 40% of employees, and approximately $3 trillion in payroll. "The commercialization of humanoid robots will encounter significant challenges, particularly in gaining social and political acceptance, given their potential to disrupt a large portion of the global workforce," says Jonas. He highlights that up to 70% of construction jobs and 67% in farming, fishing, and forestry could be impacted. "While they may not be the ideal solution, they are an increasingly necessary one for a world facing significant longevity challenges." #HumanoidRobots #AILaborSolutions #FutureOfWork #LaborShortage #GenerativeAI #RoboticsInnovation #AIInvestment #EconomicGrowth #TechTrends #WorkforceTransformation #futures

These students were challenged to build a robot capable of scaling a vertical wall in record time, a task that mirrors real engineering problems faced by aerospace, manufacturing, and autonomous robotics teams worldwide. Will you be able to win? To succeed, each group had to master a full engineering cycle: 🔹 Mechanical design: calculating torque, motor ratios, surface grip, and center of gravity 🔹 Material selection: optimizing weight-to-strength ratios (aluminum, carbon fiber, 3D-printed composites) 🔹 Control algorithms: PID tuning, sensor feedback loops, and stability control 🔹 Energy efficiency: maximizing battery output and motor load under vertical stress 🔹 Failure analysis: testing, measuring, iterating, and rebuilding And this isn’t just academic. Challenges like this reflect real-world robotics breakthroughs: 📌 NASA’s Valkyrie robot uses similar balance and grip logic for climbing unstable surfaces in disaster response missions. 📌 Boston Dynamics spent over 10 years perfecting the control systems students experiment with on a smaller scale. 📌 Industrial robots used in warehouses face the same physics constraints — friction, payload, torque, and trajectory planning. 📌 Spacecraft design teams use identical modeling principles to ensure robots can maneuver on asteroids with extremely low gravity. And student innovation is accelerating fast: 🚀 University robotics teams report up to 40% faster prototype cycles thanks to rapid 3D printing. 🚀 High-school robotics programs now routinely use LIDAR, machine vision, and ROS, tools once limited to major research labs. 🚀 Over 90% of global robotics firms hire from hands-on competition pipelines like FIRST, VEX, and Eurobot. 🚀 The educational robotics market is growing 17% annually, driven by demand for engineers who can build, code, and troubleshoot under real conditions. Competitions like this create the mindset industry needs: not memorization, but building, breaking, fixing, optimizing — the same loop that drives innovation at the world’s leading tech companies. One student prototype at a time, the future of automation, AI, and robotics is already climbing upward. 🚀🤝 #Engineering #Robotics #STEM #Innovation #Education #AI #Automation #FutureOfWork #NextGenTech

The past 10 years have seen the United States move from a laggard to a leader as it pertains to the adoption of industrial robots (HS code 84.7950.000). With 2023’s trade data finalized, I thought it would be interesting to show both the trends in terms of unit imports of robots as well as the price per robot. Two charts below. Thoughts: •Top chart shows the number of industrial robots imported from the USA from all over the world. Prior to 2011, that number never crossed 10,000. However, it took off starting in 2015 (where it almost reached 50,000). 2023 is the second highest year ever, with ~128,000 industrial robots brought into the USA. This slightly outpaces what we saw in 2020 and 2022 (note, I’m guessing 2021 was lower due more to supply side issues [e,g., shortfall of semiconductors and other components] as opposed to less demand). •Bottom chart shows the average price per unit. Fun pattern here: note that price per unit is much lower in 2017, 2020, 2022, and 2023 (corresponding the years for the most imports). This does suggest the highest years are being driven by unobserved differences in the types of robots being brought into the US. •Given the labor market remains quite tight, I don’t see this trend reversing anytime soon. This points to the need for developing more workers with the skills to not only work with industrial robots, but also to be able to repair them. •As technology continues to become more flexible, I expect we will continue to see industrial robots find applications outside of traditional key sectors for their use like motor vehicle production, chemical manufacturing, fabricated metal product manufacturing, plastics manufacturing, and both basic metals and fabricated metal product manufacturing (see https://lnkd.in/dDmuan3P). Implication: I expect industrial robots will continue to become an increasingly common sight in American factories. It will be critical for firms to adapt their workforces to best leverage these technologies. #supplychain #supplychainmanagement #manufacturing #economics 

Physical AI is the next step forward for manufacturing performance. At WNE, I met "Hoxo", the humanoid robot developed by Capgemini and Orano, shows what the future could look like. Deployed at the Orano Melox École des Métiers in the Gard region of France, Hoxo is the first intelligent humanoid robot in the nuclear sector, able to replicate human movements and work safely alongside teams. With real-time perception, autonomous navigation, execution of technical gestures, and sophisticated interaction, stepping forward will be the least of its capabilities. This project, led by our AI Robotics & Experiences Lab with the expertise of Orano's on-site teams, embodies the convergence of robotics, artificial intelligence, computer vision, and digital twins to offer a scalable robotic platform to enhance industrial performance and potentially support operators through robotic assistance. Watch the full video below to discover why this is a major step forward for a strategic industry that has long been a pioneer in innovation. Pascal Brier

Robots eating ocean plastic. By copying fish that did it for 25 million years. 3.8 billion years of R&D. Zero patents. Free to copy. The numbers proving nature is the mastermind of sustainable design: ↳ 20,000,000 kg ocean plastic removed ↳ 160 liters waste per silent trip ↳ 2 million people living with limb loss Think about that. THE OCEAN CLEANUP copied manta rays: Opens wide. Glides forward. Plastic trapped, water flows through. Boyan Slat watched rays filter-feed. Built the same system. 20 million kg removed. WASTESHARK copied whale sharks: Silent. Autonomous. 160 liters per trip. Richard Hardiman saw these giants feed without disrupting anything. Built one in his garage. Now cleaning harbors worldwide. SOFTFOOT PRO copied human feet: 26 bones. Perfect energy return. Waterproof. Engineers studied barefoot mechanics. Amputees walking naturally. Swimming again. But here's what stopped me cold: Every breakthrough already existed. We just had to notice. Traditional Innovation: ↳ Years in labs ↳ Millions in R&D ↳ Fighting physics Biomimicry Reality: ↳ Solutions proven over millennia ↳ Physics already solved ↳ Free blueprints The Multiplication Effect: 1 observation = 1 breakthrough 10 biomimetic designs = industries transformed 100 applications = sustainable future At scale = Nature teaching humility Watch Rotterdam's harbors getting cleaner. See amputees running on beaches. Humans excel at observation. At adapting nature's genius to solve human problems. The future of innovation is recognising we're students in a 3.8-billion-year classroom. And class is always in session. Follow Dr. Martha Boeckenfeld for innovations nature already tested. ♻️ Share if you believe the best R&D lab is the world around us. #Biomimicry #Innovation #Sustainability #NatureInspired

Two words bedevil manufacturers looking to transform their shop floor: “conveyor belts”. The sprawling circulatory systems of the factory, any major changes must fit around them. If not, they must be altered or replaced, which is costly and time-consuming 💸 Autonomous Mobile Robots (AMRs) offer an alternative. These agile little robot couriers can do lots of the transportation tasks conveyors do, but have the advantage of modularity and flexibility, meaning they can be reconfigured again and again to accommodate new factory layouts, at a fraction of the cost and effort that it would take to transition a conveyor belt line. It’s just one of the reasons they are booming. By 2030, mobile robots could represent up to 46% of all robot revenue, and the industry is predicted to have a CAGR of 12% over the next five years. They’ll also no doubt feature somewhere at HANNOVER MESSE ! A great way of simplifying AMR integration in the production line is through #Virtualtwins, creating real time digital factories where operators can manage variables like robot speed and route to deliver the most streamlined and efficient AMR-enabled workflows. (More on this here: https://go.3ds.com/BrF) This is just one of the myriad ways AMRs can revolutionize factories. Where do you see the big applications in your industry? Let me know in the comments! #Robotics #HM24

Yesterday, we explored Synthetic Interoception and how robots might gain self-awareness. Today, we shift focus to physical intelligence: how robots can achieve the touch and finesse of human hands. Rigid machines are precise but lack delicacy. Humans, on the other hand, easily manipulate fragile objects, thanks to our bodies' softness and sensitivity. Soft-body Tactile Dexterity Systems integrate soft, flexible materials with advanced tactile sensing, granting robots the ability to: ⭐ Adapt to Object Shapes: Conform to and securely grasp items of diverse forms. ⭐ Handle Fragile Items: Apply appropriate force to prevent damage. ⭐ Perform Complex Manipulations: Execute tasks requiring nuanced movements and adjustments. Robots can achieve a new level of dexterity by emulating the compliance and sensory feedback of human skin and muscles. 🤖 Caregiver: A soft-handed robot supports elderly individuals and handles personal items with gentle precision. 🤖 Harvester: A robot picks ripe tomatoes without bruising them in a greenhouse, using tactile sensing to gauge ripeness. 🤖 Surgical Assistant: In the OR, a robot holds tissues delicately with soft instruments, improving access and reducing trauma. These are some recent relevant research papers on the topic: 📚 Soft Robotic Hand with Tactile Palm-Finger Coordination (Nature Communications, 2025): https://lnkd.in/g_XRnGGa 📚 Bi-Touch: Bimanual Tactile Manipulation (arXiv, 2023): https://lnkd.in/gbJSpSDu 📚 GelSight EndoFlex Hand (arXiv, 2023): https://lnkd.in/g-JTUd2b These are some examples of translating research into real-world applications: 🚀 Figure AI: Their Helix system enables humanoid robots to perform complex tasks using natural language commands and real-time visual processing. https://lnkd.in/gj6_N3MN 🚀 Shadow Robot Company: Developers of the Shadow Dexterous Hand, a robotic hand that mimics the human hand's size and movement, featuring advanced tactile sensing for precise manipulation. https://lnkd.in/gbpmdMG4 🚀 Toyota Research Institute's Punyo: Introduced 'Punyo,' a soft robot with air-filled 'bubbles' providing compliance and tactile sensing, combining traditional robotic precision with soft robotics' adaptability. https://lnkd.in/gyedaK65 The journey toward widespread adoption is progressing: 1–3 years: Implementation in controlled environments like manufacturing and assembly lines, where repetitive tasks are structured. 4–6 years: Expansion into dynamic healthcare and domestic assistance settings requiring advanced adaptability and safety measures. Robots are poised to perform tasks with unprecedented dexterity and sensitivity by integrating soft materials and tactile sensing, bringing us closer to seamless human-robot collaboration. Next up: Cognitive World Modeling for Autonomous Agents.