METHOD OF MULTIMODAL CONTACTLESS CONTROL OF MOBILE INFORMATION ROBOT Russian patent published in 2020 - IPC G06K9/00 G06F3/488 G06T7/20 

FIELD: data processing.

SUBSTANCE: invention relates to the field of artificial intelligence. Invention comprises a method for multimodal noncontact control of a mobile information robot, consisting of capturing three-dimensional space, gesture recognition, characterized by that combined processing of video and audio information from a user, serial processing of video information from user, formation of areas with people on each three-dimensional (3D) frame of depth map, calculation of 3D 25-point models of human skeletons, tracking the nearest person, transformation of 3D 25-point skeleton model of the nearest person into 2D 25-point skeleton model, determining a face and hand shape graphic region within the formed rectangular area with a human, computation of visual signs of gesture at a certain moment of time, serial processing of audio information from user, calculation of acoustic features, recognition of speech commands.

EFFECT: technical result consists in expansion of functionality of mobile information robot due to use of multimode human-machine interaction, which is understood as control of mobile information robot through combination of audio and video modalities.

1 cl, 1 dwg