Robust Gram Embeddings

Taygun Kekec, David Tax

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

Word embedding models learn vectorial word representations that can be used in a variety of NLP applications. When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data. We propose a regularized embedding formulation,
called Robust Gram (RG), which penalizes overfitting by suppressing the disparity
between target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlates
well to human similarities in a set of word similarity tasks.
Original languageEnglish
Title of host publicationProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
PublisherAssociation for Computational Linguistics
Pages1060-1065
Number of pages6
Publication statusPublished - 2016
EventEMNLP 2016: Conference on Empirical Methods in Natural Language Processing - Austin, TX, United States
Duration: 1 Nov 20165 Nov 2016

Conference

ConferenceEMNLP 2016
CountryUnited States
CityAustin, TX
Period1/11/165/11/16

Cite this