Robust Learning via Golden Symmetric Loss of (un)Trusted Labels

Amirmasoud Ghiassi*, Robert Birke, Lydia Y. Chen*

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

20 Downloads (Pure)

Abstract

Learning robust deep models against noisy labels becomes ever critical when today's data is commonly collected from open platforms and subject to adversarial corruption. The information on the label corruption process, i.e., corruption matrix, can greatly enhance the robustness of deep models but still fall behind in combating hard classes. In this paper, we propose to construct a golden symmetric loss (GSL) based on the estimated corruption matrix as to avoid overfitting to noisy labels and learn effectively from hard classes. GSL is the weighted sum of the corrected regular cross entropy and reverse cross entropy. By leveraging a small fraction of trusted clean data, we estimate the corruption matrix and use it to correct the loss as well as to determine the weights of GSL. We theoretically prove the robustness of the proposed loss function in the presence of dirty labels. We provide a heuristics to adaptively tune the loss weights of GSL according to the noise rate and diversity measured from the dataset. We evaluate our proposed golden symmetric loss on both vision and natural language deep models subject to different types of label noise patterns. Empirical results show that GSL can significantly outperform the existing robust training methods on different noise patterns, showing accuracy improvement up to 18% on CIFAR-100 and 1% on real world noisy dataset of Clothing1M.

Original languageEnglish
Title of host publication2023 SIAM International Conference on Data Mining, SDM 2023
PublisherSociety for Industrial and Applied Mathematics
Pages568-576
Number of pages9
ISBN (Electronic)9781611977653
Publication statusPublished - 2023
Event2023 SIAM International Conference on Data Mining, SDM 2023 - Minneapolis, United States
Duration: 27 Apr 202329 Apr 2023

Publication series

Name2023 SIAM International Conference on Data Mining, SDM 2023

Conference

Conference2023 SIAM International Conference on Data Mining, SDM 2023
Country/TerritoryUnited States
CityMinneapolis
Period27/04/2329/04/23

Keywords

  • Deep learning models
  • Noisy labels
  • Robust training
  • Symmetric loss function

Fingerprint

Dive into the research topics of 'Robust Learning via Golden Symmetric Loss of (un)Trusted Labels'. Together they form a unique fingerprint.

Cite this