Close menu

SURE

Sunderland Repository records the research produced by the University of Sunderland including practice-based research and theses.

CAM: A Combined Attention Model for Natural Language Inference

This is the latest version of this item.

Gajbhiye, Amit, Jaf, Sardar, Al-Moubayed, Noura and Bradley, Steven (2018) CAM: A Combined Attention Model for Natural Language Inference. In: Proceedings 2018 IEEE International Conference on Big Data. IEEE, Seattle, USA, pp. 1009-1019. ISBN 9781538650349

Item Type: Book Section

Abstract

Natural Language Inference (NLI) is a fundamental step towards natural language understanding. The task aims to detect whether a premise entails or contradicts a given hypothesis. NLI contributes to a wide range of natural language understanding applications such as question answering, text summarization and information extraction. Recently, the public availability of big datasets such as Stanford Natural Language Inference (SNLI) and SciTail, has made it feasible to train complex neural NLI models. Particularly, Bidirectional Long Short-Term Memory networks (BiLSTMs) with attention mechanisms have shown promising performance for NLI. In this paper, we propose a Combined Attention Model (CAM) for NLI. CAM combines the two attention mechanisms: intra-attention and inter-attention. The model first captures the semantics of the individual input premise and hypothesis with intra-attention and then aligns the premise and hypothesis with inter-sentence attention. We evaluate CAM on two benchmark datasets: Stanford Natural Language Inference (SNLI) and SciTail, achieving 86.14% accuracy on SNLI and 77.23% on SciTail. Further, to investigate the effectiveness of individual attention mechanism and in combination with each other, we present an analysis showing that the intra- and inter-attention mechanisms achieve higher accuracy when they are combined together than when they are independently used.

[img]
Preview
PDF
Combined Attention Model for Natural Language Inference.pdf - Accepted Version

Download (587kB) | Preview
[img] PDF (Administrator use only)
10478.pdf - Published Version
Restricted to Repository staff only

Download (540kB) | Request a copy

More Information

Additional Information: [Conference] 2018 IEEE International Conference on Big Data (Big Data), 10-13 Dec. 2018, Seattle, WA, USA
Depositing User: Sardar Jaf

Identifiers

Item ID: 10478
Identification Number: https://doi.org/10.1109/BigData.2018.8622057
ISBN: 9781538650349
URI: http://sure.sunderland.ac.uk/id/eprint/10478
Official URL: https://ieeexplore.ieee.org/document/8622057

Users with ORCIDS

ORCID for Sardar Jaf: ORCID iD orcid.org/0000-0002-5620-0277

Catalogue record

Date Deposited: 11 Mar 2019 10:45
Last Modified: 28 Jan 2021 16:30

Contributors

Author: Sardar Jaf ORCID iD
Author: Amit Gajbhiye
Author: Noura Al-Moubayed
Author: Steven Bradley

University Divisions

Faculty of Technology

Subjects

Computing > Data Science
Computing > Artificial Intelligence
Languages > Languages
Computing > Software Engineering
Computing

Actions (login required)

View Item (Repository Staff Only) View Item (Repository Staff Only)

Available Versions of this Item