Second place at MiGA 2025: recognising micro-gestures and emotions from video

Can AI recognise a person's emotions from barely noticeable gestural markers, without a single word? We have an answer.

For the international MiGA 2025 challenge hosted at IJCAI we developed two systems that analyse video and recognise:

• micro-gestures — the smallest movements of the hands and fingers (32 gesture classes);

• emotion from behaviour — a positive or negative state inferred from body movement and facial expression.

Where is the difficulty? Ordinary models see either the image or the human skeleton, and work with one of the two. We taught our system to combine both streams of information: the video and the three-dimensional coordinates of the joints. For this we developed a dedicated fusion module that finds connections between visual features and body pose at every moment in time. The model also accumulates a "memory" of typical representatives of each gesture class and refines its predictions by comparing new examples with those already learned.

For emotion recognition, the system simultaneously analyses the overall context of the video and the details of the face through separate streams that gradually exchange information with each other — much as different regions of the brain work together when we read someone else's feelings.

We took second place in the emotion recognition track among all participants of the challenge.

Technologies like these open the way to systems that can understand a person's emotional state in a wide range of real scenarios: from clinical diagnostics to adaptive interfaces and feedback systems.