Abstract
End-to-end learning from crowds has recently been introduced as an EM-free approach to training deep neural networks directly from noisy crowdsourced annotations. It models the relationship between true labels and annotations with a specific type of neural layer, termed as the crowd layer, which can be trained using pure backpropagation. Parameters of the crowd layer, however, can hardly be interpreted as annotator reliability, as compared with the more principled probabilistic approach. The lack of probabilistic interpretation further prevents extensions of the approach to account for important factors of annotation processes, e.g., instance difficulty. This paper presents SpeeLFC, a structured probabilistic model that incorporates the constraints of probability axioms for parameters of the crowd layer, which allows to explicitly model annotator reliability while benefiting from the end-to-end training of neural networks. Moreover, we propose SpeeLFC-D, which further takes into account instance difficulty. Extensive validation on real-world datasets shows that our methods improve the state-of-the-art.
Original language | English |
---|---|
Title of host publication | IJCAI-20 |
Subtitle of host publication | Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence |
Publisher | International Joint Conferences on Artifical Intelligence (IJCAI) |
Pages | 1512 - 1518 |
Number of pages | 7 |
ISBN (Electronic) | 978-0-9992411-6-5 |
DOIs | |
Publication status | Published - 2020 |
Event | The Twenty-Ninth International Joint Conference on Artificial Intelligence - Yokohoma, Japan Duration: 7 Jan 2021 → 15 Jan 2021 Conference number: 29th https://ijcai20.org/ |
Conference
Conference | The Twenty-Ninth International Joint Conference on Artificial Intelligence |
---|---|
Country/Territory | Japan |
City | Yokohoma |
Period | 7/01/21 → 15/01/21 |
Internet address |