Abstract
Word embedding models learn vectorial word representations that can be used in a variety of NLP applications. When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data. We propose a regularized embedding formulation,
called Robust Gram (RG), which penalizes overfitting by suppressing the disparity
between target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlates
well to human similarities in a set of word similarity tasks.
called Robust Gram (RG), which penalizes overfitting by suppressing the disparity
between target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlates
well to human similarities in a set of word similarity tasks.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing |
Publisher | Association for Computational Linguistics |
Pages | 1060-1065 |
Number of pages | 6 |
Publication status | Published - 2016 |
Event | EMNLP 2016: Conference on Empirical Methods in Natural Language Processing - Austin, TX, United States Duration: 1 Nov 2016 → 5 Nov 2016 |
Conference
Conference | EMNLP 2016 |
---|---|
Country/Territory | United States |
City | Austin, TX |
Period | 1/11/16 → 5/11/16 |