TagRec++: Hierarchical Label Aware Attention Network for Question Categorization

Venktesh Viswanathan, Mukesh Mohania, Vikram Goyal

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Online learning systems have multiple data repositories in the form of transcripts, books and questions. To enable ease of access, such systems organize the content according to a well defined taxonomy of hierarchical nature (subject - chapter -topic). The task of categorizing inputs to the hierarchical labels is usually cast as a flat multi-class classification problem. Such approaches ignore the semantic relatedness between the terms in the input and the tokens in the hierarchical labels. Alternate approaches also suffer from class imbalance when they only consider leaf level nodes as labels. To tackle the issues, we formulate the task as a dense retrieval problem to retrieve the appropriate hierarchical labels for each content. In this paper, we deal with categorizing questions and learning content. We model the hierarchical labels as a composition of their tokens and use an efficient cross-attention mechanism to fuse the information with the term representations of the content. We also adopt an adaptive in-batch hard negative sampling approach which samples better negatives as the training progresses. We demonstrate that the proposed approach TagRec++ outperforms existing state-of-the-art approaches on question and learning content datasets as measured by Recall@k. In addition, we demonstrate zero-shot capabilities of TagRec++ and preliminary analysis of it's ability to adapt to label changes.

Original languageEnglish
Pages (from-to)3529-3540
Number of pages12
JournalIEEE Transactions on Knowledge and Data Engineering
Volume36
Issue number7
DOIs
Publication statusPublished - 2024

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Attention
  • contrastive learning
  • Deep learning
  • dynamic triplet sampling
  • hard-negatives
  • Self-supervised learning
  • Semantics
  • Tagging
  • Task analysis
  • Taxonomy
  • Training
  • transformer

Fingerprint

Dive into the research topics of 'TagRec++: Hierarchical Label Aware Attention Network for Question Categorization'. Together they form a unique fingerprint.

Cite this