Implementing Ask Behavior and OpenPose in the Golem Service Robot

The Golem-III project is a service robot presented internationally at RoboCup 2016 Leipzig, Germany, where the 6th place was obtained in the category @Home. It has also participated at RoboCup Nagoya 2017 and the RoboCup German Open 2018 and 2019, where the 3rd and 5th places were obtained respectively.

The Golem Group develops service robots based on Artificial Intelligence and Human-Robot Interaction to assist people in daily life activities. Our group comprises theoretical research, hardware and software development, and the construction of physical robots. In this project, the robots Golem (2002-2008), Golem en Universum (2019-2010), Golem-II+ (2010-2014) and Golem-III (2014 to the date) have been developed. Our robots are regularly presented at Museo de Ciencias Universum and Biblioteca Central of UNAM. The Golem group participates consistently at Torneo Mexicano de Robótica (TMR) and the international RoboCup competition in the category @Home. Our group has several ongoing projects.

The Golem Group is part of the Departamento de Ciencias de la Computación at Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas (IIMAS) including colaborators and students from Centro de Investigaciones en Diseño Industrial (CIDI) at Facultad de Arquitectura, Facultad de Ingeniería and FES Aragón and FES Cuautitlán of the Universidad Nacional Autónoma de México (UNAM).

Project Details

My participation in this project is mainly divided in three parts. The first one consisted of reviewing the requirements to implement the OpenPose software for the detection of people, as well as for the integration of this system with the other components of the project. This both at the hardware level to use the sensors that has the Kinect that has incorporated the robot and to capture the information with the highest quality and coverage possible. At the software level, it was necessary to review the appropriate conditions for the correct operation of OpenPose with the operating system, development tools and programming languages with which the robot operates. After this review the human detection system was successfully implemented, giving the robot a greater ability to perform tasks involving recognition and interaction with people.

The second part of my participation in the project consisted of reviewing and coordinating tests for two tasks performed by the robot in the RoboCup@home competition. These are the General Porpuse Service Robot (GPSR) and Extended Enhanced General Porpuse Service Robot (EEGPSR). In these tests, the robot performs general human assistance tasks, such as answering questions or performing relatively simple physical tasks to support the user, such as bringing or carrying things, arranging objects, etc. During these tests, the performance of the robot is measured in several aspects, such as the understanding, processing and response to the user’s indications. The accuracy, efficiency and time taken to complete individual tasks and the test as a whole were also important factors in the testing. In the case of simple corrections, they were applied to the robot, but for major changes, the corresponding sub-team was instructed depending on the work to be done for the correction.

The third part of my involvement in the project was the implementation of a general behavior in the robot. This process was a sub-routine that was used in more complex scenarios and routines for the robot to perform its tasks. This is how the ‘ASK’ behavior was improved. The main objective of this enhancement was to increase Golem-III’s ability to have an efficient Human-Robot interaction. Before the enhancement, Golem-III did not have many alternatives for an efficient interaction with the user in several scenarios, especially in cases where the user was not proactive in such interaction. To give a solution to this problem we studied the way of human interaction and some strategies to make the human being proactive and communicative enough when interacting with the robot. The proposed solution included three main aspects of communication: regular mode, cooperative mode and contextualized mode. The three levels are used in different situations depending on the user’s attitude towards the robot. Where the highest level of proactivity involves movements and displacements to attract the user’s attention.

Project Requirements & Specifications

This work is a research project, more details of the project can be found on the project website and in this article. Details on ask behavior can be found in this article presented at CEIAAIT 2019.