Sitemap
A list of all the posts and pages found on the site. For you robots out there, there is an XML version available for digesting as well.
Pages
Posts
publications
Large Language Model-Based Robot Task Planning from Voice Command Transcriptions
Published in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025
One of the primary challenges in building a General Purpose Service Robot (GPSR), a robot capable of executing generic human commands, lies in understanding natural language instructions. These instructions often contain speech recognition errors and incomplete information, complicating the extraction of clear goals and the formulation of an efficient and effective action plan. This work presents an end-to-end pipeline that leverages a Large Language Model to directly translate instruction transcripts into coherent action plans. Furthermore, the pipeline integrates environmental context into the model’s input, allowing for the generation of more efficient and context-aware plans. The system’s performance was evaluated using a simulator based on Generalized Stochastic Petri Nets, achieving a success rate of around 55% on the ALFRED dataset, even in unseen environments. The entire pipeline was also successfully deployed at RoboCup 2024 in Eindhoven, where it secured second place in the GPSR task. The code, dataset and models are available here.
Recommended citation: A. Certo, B. Martins, C. Azevedo and P. U. Lima, "Large Language Model-Based Robot Task Planning from Voice Command Transcriptions," 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hangzhou, China, 2025, pp. 21018-21023, doi: 10.1109/IROS60139.2025.11246378.
Download Paper
AI-Driven Task & Motion Planning for Service Robots with Oracle Cloud Infrastructure
Published in Oracle AI & Data Science Blog, 2025
During the 2024 edition of JEEC (Electrical and Computer Engineering Week) – an event organized by students from the Technical Superior Institute (IST) in Lisbon, Portugal, Oracle has agreed to undergo a pilot project with SocRob@Home, a research unit from the Institute for Systems and Robotics (ISR-IST). SocRob@Home, is an initiative of ISR’s Intelligent Robots and Systems group, that focuses on solving key challenges in domestic robotics, such as human-robot interaction, sensor fusion, and decision-making under uncertainty. SocRob@Home team is currently participating in notable EU-funded projects like euROBIN and international competitions like RoboCup@Home, shaping their vision to push the boundaries of domestic robotics. The joint pilot project aimed to enhance task and motion planning for domestic robots by integrating Oracle Cloud Infrastructure (OCI) technology with custom Generative AI tools. To support its AI/ML workloads, including training large language models (LLMs) and AI models for robot decision-making, SocRob@Home evaluated OCI’s AI infrastructure and services. The pilot tested the effectiveness of OCI for LLM fine-tuning, action tools for task execution, and GPU-powered instances for advanced AI modeling. This collaboration aimed to enhance the interactivity, adaptability, and precision of SocRob@Home’s robots, revolutionizing the scalability and efficiency of domestic robotic systems.
Recommended citation: P. Bordalo-Machado, A. Certo, R. Serra, R. Coimbra, H. Jain, and D. Soni. "AI-Driven Task & Motion Planning for Service Robots with Oracle Cloud Infrastructure," Oracle AI & Data Science Blog. (Dec. 2025), url: https://blogs.oracle.com/ai-and-datascience/ai-driven-task-motion-planningservice-robots-with-oci.
Download Paper