Robust Gram Embeddings

Taygun Kekec, David Tax

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

2 Citations (Scopus)


Word embedding models learn vectorial word representations that can be used in a variety of NLP applications. When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data. We propose a regularized embedding formulation,
called Robust Gram (RG), which penalizes overfitting by suppressing the disparity
between target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlates
well to human similarities in a set of word similarity tasks.
Original languageEnglish
Title of host publicationProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
PublisherAssociation for Computational Linguistics
Number of pages6
Publication statusPublished - 2016
EventEMNLP 2016: Conference on Empirical Methods in Natural Language Processing - Austin, TX, United States
Duration: 1 Nov 20165 Nov 2016


ConferenceEMNLP 2016
Country/TerritoryUnited States
CityAustin, TX


Dive into the research topics of 'Robust Gram Embeddings'. Together they form a unique fingerprint.

Cite this