Knowledge acquisition from parsing natural language expressions for humanoid robot action commands

In this paper we propose an approach that allows the NAO humanoid robot to execute natural language commands spoken by the user. To provide the robot with knowledge, we have defined an action robot ontology. The ontology is fed to an NLP engine that performs a machine reading of the input text (in natural language) given by a user and tries to identify action commands for the robot to execute. The system can work in two modes: STATELESS and STATEFUL. In STATELESS mode, each human expression correctly interpreted by the robot as an action command is performed by NAO which returns in its default posture afterwards. When in STATEFUL mode, the robot has knowledge of its current posture and performs the command only if it is compatible with its current state. In this mode, the robot does not return to its default posture. For example, if the user had told the robot to stand on its right leg in a first command, the robot cannot perform a following command stating to stand on its left leg as the two actions (raise left leg and raise right leg are incompatible). For each action that the robot can perform we modeled a corresponding element in the ontology that also includes a list of associated compatible and non-compatible actions. Our system also handles compound expressions (e.g., move your arms up) and multiple expressions (different commands within one sentence) that the robot understands and performs.