Nessy: A Neuro-Symbolic System for Label Noise Reduction

Alisa Smirnova, Jie Yang, Dingqi Yang, Philippe Cudre-Mauroux

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)
23 Downloads (Pure)


Noisy labels represent one of the key issues in supervised machine learning. Existing work for label noise reduction mainly takes a probabilistic approach that infers true labels from data distributions in low-level feature spaces. Such an approach is not only limited by its capability to learn high-quality data representations, but also by the low predictive power of data distributions in inferring true classes. To address those problems, we introduce Nessy, a neuro-symbolic system that integrates deep probabilistic modeling and symbolic knowledge for label noise reduction. Our deep probabilistic model infers the true classes of data instances with noisy labels by exploiting data distributions in an underlying latent feature representation space. For data instances where inference is not reliable enough, Nessy extracts symbolic rules and ranks them according to several utility metrics. Top-ranking rules are injected into the deep probabilistic model via expectation regularization, i.e., via a posterior regularization term constraining the class distribution in the objective function. In a real deployment over multiple relation extraction tasks, we demonstrate that Nessy is able to significantly improve the state of the art, by 7% accuracy and 10.7% AUC on average.

Original languageEnglish
Pages (from-to)8300-8311
Number of pages12
JournalIEEE Transactions on Knowledge and Data Engineering
Issue number8
Publication statusPublished - 2022

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Data mining
  • Data models
  • Deep learning
  • Deep probabilistic model
  • distant supervision
  • Feature extraction
  • neuro-symbolic systems
  • Noise measurement
  • noise reduction
  • Probabilistic logic
  • relation extraction
  • Training data


Dive into the research topics of 'Nessy: A Neuro-Symbolic System for Label Noise Reduction'. Together they form a unique fingerprint.

Cite this