What are you looking for today?

Biswas, Mriganka and Murray, John (2025) Does Embodiment Still Matter? Comparing User Experience with LLM-Powered Agents. In: UNSPECIFIED Springer nature. (In Press)

Document

  • [thumbnail of Does Embodiment Still Matter? Comparing User Experience with LLM-Powered Agents]
    Does Embodiment Still Matter? Comparing User Experience with LLM-Powered Agents
    csce25_springer_v2.pdf
    Restricted to Repository staff only

Abstract

Abstract

Conversational Artificial Intelligence (AI) is increasingly integrated into various aspects of human life, yet creating truly natural and engaging interactions remains a challenge. The role of physical embodiment in shaping user experience, particularly when coupled with advanced AI capabilities, requires further investigation.
This study aimed to investigate the impact of embodiment on user experience by comparing interactions with an embodied conversational agent (Pepper robot) versus a non-embodied agent (Laptop AI), both powered by the same sophisticated Large Language Model (LLM), Ollama Llama 3.2 7B. A within-subjects experiment (N=32) was conducted where participants interacted with both the embodied and non-embodied agents in a counterbalanced order. The agents utilised the Ollama Llama 3.2 7B model and Google Text-to-Speech. Post-interaction questionnaires assessed user experience on 5-point Likert scales. Results revealed significantly higher user ratings for the embodied Pepper robot across multiple dimensions, including perceived naturalness (p=.001), conversation flow (p=.010), understanding of agent responses (p=.005), relevance of re-sponses (p=.027), user engagement (p<.001), perception as a social entity (p=.002), sense of connection (p<.001), and comfort (p=.024). Participants ex-pressed a unanimous and strong preference for the embodied agent. Crucially, self-rated tech-savviness did not significantly correlate with these core interaction metrics for either agent type. Furthermore, the embodied agent met user expectations for naturalness, whereas the non-embodied agent did not (p=.002). Findings demonstrate that physical embodiment, when combined with an advanced LLM, substantially enhances user experience, fostering more natural, engaging, and socially resonant interactions compared to an equivalent non-embodied system, largely independent of user tech-savviness.

Item Type: Book Section
Additional Information: Computer Science, Computer Engineering and Applied Computing (CSE) 2025, July 21-24 2025, Las Vegas, USA https://american-cse.org/index.html/
Related URLs:
Depositing User: Mriganka Biswas
URI: https://sure.sunderland.ac.uk/id/eprint/19624

Further Details

More Information

Additional Information
Computer Science, Computer Engineering and Applied Computing (CSE) 2025, July 21-24 2025, Las Vegas, USA https://american-cse.org/index.html/
Depositing User

Identifiers

Item ID
19624

Users with ORCIDS

ORCID for John Murray

Catalogue record

Date Deposited
10 Nov 2025 09:23
Last Modified
10 Nov 2025 09:27

Contributors

Author
Author
Mriganka Biswas

University Divisions

Subjects

Actions (login required)

View Item (Repository Staff Only)

Downloads per month over past year