EmoLoop: a bi-directional system for emotion-driven interaction between remote audiences and live performers

This paper presents EmoLoop, a bi-directional interactive system designed to create real-time emotional feedback loop between performers and a distributed audience during music or theatre events. The proposed system collects emotional data from the audience using different feedback methods, such as EEG signals from wearable headbands, facial expression analysis through VR headsets, and interaction with smartphones. These emotional states, expressed as valence/arousal values or emotion categories, are used to control various parameters of the performance, like spatial audio, lighting, and scenographic elements, both on stage and in virtual environments. EmoLoop also includes a real-time feedback system that gives performers useful cues based on the audience’s emotions. Communication between components is managed with protocols like OSC, MQTT, WebSockets, MIDI, and ArtNet to ensure low-latency transmission. The architecture is modular and flexible, and the paper explains its components, integration, and possible use cases. EmoLoop offers a new approach to participatory performance, where emotions play a central role in shaping the artistic experience, in line with current research in the Internet of Sounds (IoS) and Internet of Musical Things (IoMusT).