Abbo, Giulio AntonioGiulio AntonioAbboPinto Bernal, Maria JoseMaria JosePinto BernalCatrycke, MartijnMartijnCatryckeBelpaeme, TonyTonyBelpaeme2026-07-272026-07-2720262296-9144https://imec-publications.be/handle/20.500.12860/60002<jats:p> Multi-party open-ended conversation remains a major challenge in human–robot interaction, particularly when robots must recognise speakers, allocate turns, and respond coherently under overlapping or rapidly shifting dialogue. This paper presents a multi-party conversational system that combines multimodal perception (voice direction of arrival, speaker diarisation, face recognition) with a large language model for response generation. Implemented on the Furhat robot, the system was evaluated with 30 participants across two scenarios: (i) parallel, separate conversations and (ii) shared group discussion. Results show that the system maintains coherent and engaging conversations, achieving high addressee accuracy in parallel settings <jats:inline-formula> <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m1"> <mml:mrow> <mml:mo stretchy="false">(</mml:mo> <mml:mrow> <mml:mn>92.6</mml:mn> <mml:mi>%</mml:mi> </mml:mrow> <mml:mo stretchy="false">)</mml:mo> </mml:mrow> </mml:math> </jats:inline-formula> and strong face recognition reliability <jats:inline-formula> <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m2"> <mml:mrow> <mml:mo stretchy="false">(</mml:mo> <mml:mrow> <mml:mn>80</mml:mn> <mml:mtext>–</mml:mtext> <mml:mn>94</mml:mn> <mml:mi>%</mml:mi> </mml:mrow> <mml:mo stretchy="false">)</mml:mo> </mml:mrow> </mml:math> </jats:inline-formula> . Participants reported clear social presence and positive engagement, although technical barriers such as audio-based speaker recognition errors and response latency affected the fluidity of group interactions. The results highlight both the promise and limitations of LLM-based multi-party interaction and outline concrete directions for improving multimodal cue integration and responsiveness in future social robots. </jats:p>engJournal article10.3389/frobt.2026.1766383WOS:001751944600001MEDLINE:42063572