Abstract

Chatbots have emerged as intelligent conversational computer programs that simulate human conversation, processing user input and generating relevant responses. They find applications across diverse fields, offering support, assistance, and entertainment to users. Recent advancements in Artificial Intelligence and Natural Language Processing have led to the widespread adoption of chatbots, driven by increased computational power and the availability of open-source technologies. However, challenges remain in improving contextual understanding, emotional responsiveness, and addressing gender biases in chatbot interactions. Despite their prevalence, existing chatbot models often rely on a next-step approach, lacking the ability to consider the broader conversational context and underlying information shared among participants.

This thesis investigates the impact of contextual embedding information on transformer architectures for dialogue modelling tasks. Through a series of experiments, various transformer architectures were evaluated, and an innovative architectural approach called the Reencoder model was developed. A key feature of this new architecture is the inclusion of an additional reencoding step. This reencoding process enhances the model's capability to effectively capture and incorporate contextual information from previous turns in the dialogue history. It was consistently observed that such models exhibited superior performance and greater consistency compared to those employing alternative embedding strategies. This study sheds light on the mechanisms underlying the enhanced performance of contextual embedding layers and explores factors contributing to their effectiveness in dialogue modelling tasks. To strengthen the validity of the presented results, the thesis also presents enhanced algorithms that combine textual and audio embeddings that further enhance contextual understanding in dialogue modelling. The findings contribute to the ongoing research efforts aimed at improving chatbot implementations and evaluation methodologies, addressing critical challenges in human-chatbot interaction and advancing the field of conversational AI.