MiEmo: A multi-modal platform on emotion recognition for children with autism spectrum condition

Introduction: This study presents MiEmo, a multi-modal digital platform designed to improve emotion recognition
in children with Autism Spectrum Condition (ASC). The platform integrates serious games with music and color
as feedback mechanisms to strengthen emotion understanding in addition to traditional visual interventions such
as pictures and videos. The study aims to assess the usability and potential effectiveness of MiEmo in supporting
therapy for children with medium- and high-functioning ASC.
Methods: A pilot usability study was conducted in two rehabilitation centers involving 19 children, 8 with me-
dium and 11 with high functioning ASC. Participants engaged with six training activities, or exergames, on the
MiEmo platform that implemented multi-modal feedback (music pieces and colored animations associated to the
emotion). The System Usability Scale (SUS) and qualitative feedback from therapists were used to evaluate the
platform’s usability.
Results: The average SUS scores were 86.88 for children with medium-functioning ASC and 96.75 for those with
high-functioning ASC, indicating positive usability. Therapists noted that while the platform was well-received,
further updates are needed for better adaptation to medium-functioning children. Multi-modal feedback,
particularly music and color, was found to enhance emotion recognition, with children responding well to the
integration of these sensory cues.
Conclusion: The study demonstrates that MiEmo has significant potential as a tool for socio-emotional training,
particularly for high-functioning children. However, limitations such as the small sample size, short intervention
duration, and lack of a control group suggest that future studies with larger participant groups are necessary to
validate these findings and assess long-term effects.