Towards Seamless Human-Robot Dialogue through a Robot Action Ontology

This research paper introduces a novel methodology enabling the Zora humanoid robot to effectively engage in dynamic interactions by responding to user queries and complementing its responses with appropriate gestures. Notably, these inquiries may extend beyond mere questions to encompass action commands articulated by the user, which the robot proficiently recognizes and executes. The integration of a Large Language Model enhances the system's capabilities, particularly in the domain of questionanswering. To bolster the recognition and execution of action commands, we have employed a robot action ontology established in previous research endeavors. This ontology defines relevant classes and individuals, forming the basis for a nuanced understanding of user-inputted action commands. Further refinement involves the generation of succinct three-word strings for each action, ensuring semantic alignment with the user's verbal instructions. Importantly, our system operates in two distinctive modes: STATELESS and STATEFUL. In STATEFUL mode, the robot possesses awareness of its present posture, allowing it to execute action commands only when they align with its current state. This adaptive feature enhances the overall effectiveness of the system, catering to the dynamic nature of human-robot interactions and promoting a seamless and contextually aware dialogue between the NAO humanoid robot and its users.