Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech

YuanYuan  Zhang; Aaricia  Herygers; Tanvina Patel; Zhengjun Yue; Odette Scharenborg

doi:10.1109/ASRU57964.2023.10389756

Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech

YuanYuan Zhang, Aaricia Herygers, Tanvina Patel, Zhengjun Yue, Odette Scharenborg

Multimedia Computing

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

1 Citation (Scopus)

17 Downloads (Pure)

Abstract

Automatic speech recognition (ASR) should serve every speaker, not only the majority “standard” speakers of a language. In order to build inclusive ASR, mitigating the bias against speaker groups who speak in a “non-standard” or “diverse” way is crucial. We aim to mitigate the bias against non-native-accented Flemish in a Flemish ASR system. Since this is a low-resource problem, we investigate the optimal type of data augmentation, i.e., speed/pitch perturbation, cross-lingual voice conversion-based methods, and SpecAugment, applied to both native Flemish and non-native-accented Flemish, for bias mitigation. The results showed that specific types of data augmentation applied to both native and non-native-accented speech improve non-native-accented ASR while applying data augmentation to the non-native-accented speech is more conducive to bias reduction. Combining both gave the largest bias reduction for human-machine interaction (HMI) as well as read-type speech.

Original language	English
Title of host publication	Proceedings of the 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)
Publisher	IEEE
Number of pages	8
ISBN (Electronic)	979-8-3503-0689-7
ISBN (Print)	979-8-3503-0690-3
DOIs	https://doi.org/10.1109/ASRU57964.2023.10389756
Publication status	Published - 2023
Event	2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) - Taipei, Taiwan Duration: 16 Dec 2023 → 20 Dec 2023

Workshop

Workshop	2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)
Country/Territory	Taiwan
City	Taipei
Period	16/12/23 → 20/12/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Speech recognition
bias mitigation
non-native accents
data augmentation
voice conversion

Access to Document

10.1109/ASRU57964.2023.10389756

Exploring_Data_Augmentation_in_Bias_Mitigation_Against_Non-Native-Accented_SpeechFinal published version, 446 KB

Cite this

@inproceedings{a2c96cbd32a947e696a84a216a843187,

title = "Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech",

abstract = "Automatic speech recognition (ASR) should serve every speaker, not only the majority “standard” speakers of a language. In order to build inclusive ASR, mitigating the bias against speaker groups who speak in a “non-standard” or “diverse” way is crucial. We aim to mitigate the bias against non-native-accented Flemish in a Flemish ASR system. Since this is a low-resource problem, we investigate the optimal type of data augmentation, i.e., speed/pitch perturbation, cross-lingual voice conversion-based methods, and SpecAugment, applied to both native Flemish and non-native-accented Flemish, for bias mitigation. The results showed that specific types of data augmentation applied to both native and non-native-accented speech improve non-native-accented ASR while applying data augmentation to the non-native-accented speech is more conducive to bias reduction. Combining both gave the largest bias reduction for human-machine interaction (HMI) as well as read-type speech.",

keywords = "Speech recognition, bias mitigation, non-native accents, data augmentation, voice conversion",

author = "YuanYuan Zhang and Aaricia Herygers and Tanvina Patel and Zhengjun Yue and Odette Scharenborg",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) ; Conference date: 16-12-2023 Through 20-12-2023",

year = "2023",

doi = "10.1109/ASRU57964.2023.10389756",

language = "English",

isbn = "979-8-3503-0690-3",

booktitle = "Proceedings of the 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)",

publisher = "IEEE",

address = "United States",

}

Zhang, Y, Herygers, A, Patel, T , Yue, Z & Scharenborg, O 2023, Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech. in Proceedings of the 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Taipei, Taiwan, 16/12/23. https://doi.org/10.1109/ASRU57964.2023.10389756

Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech. / Zhang, YuanYuan ; Herygers, Aaricia ; Patel, Tanvina et al.
Proceedings of the 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech

AU - Zhang, YuanYuan

AU - Herygers, Aaricia

AU - Patel, Tanvina

AU - Yue, Zhengjun

AU - Scharenborg, Odette

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - Automatic speech recognition (ASR) should serve every speaker, not only the majority “standard” speakers of a language. In order to build inclusive ASR, mitigating the bias against speaker groups who speak in a “non-standard” or “diverse” way is crucial. We aim to mitigate the bias against non-native-accented Flemish in a Flemish ASR system. Since this is a low-resource problem, we investigate the optimal type of data augmentation, i.e., speed/pitch perturbation, cross-lingual voice conversion-based methods, and SpecAugment, applied to both native Flemish and non-native-accented Flemish, for bias mitigation. The results showed that specific types of data augmentation applied to both native and non-native-accented speech improve non-native-accented ASR while applying data augmentation to the non-native-accented speech is more conducive to bias reduction. Combining both gave the largest bias reduction for human-machine interaction (HMI) as well as read-type speech.

AB - Automatic speech recognition (ASR) should serve every speaker, not only the majority “standard” speakers of a language. In order to build inclusive ASR, mitigating the bias against speaker groups who speak in a “non-standard” or “diverse” way is crucial. We aim to mitigate the bias against non-native-accented Flemish in a Flemish ASR system. Since this is a low-resource problem, we investigate the optimal type of data augmentation, i.e., speed/pitch perturbation, cross-lingual voice conversion-based methods, and SpecAugment, applied to both native Flemish and non-native-accented Flemish, for bias mitigation. The results showed that specific types of data augmentation applied to both native and non-native-accented speech improve non-native-accented ASR while applying data augmentation to the non-native-accented speech is more conducive to bias reduction. Combining both gave the largest bias reduction for human-machine interaction (HMI) as well as read-type speech.

KW - Speech recognition

KW - bias mitigation

KW - non-native accents

KW - data augmentation

KW - voice conversion

UR - http://www.scopus.com/inward/record.url?scp=85184665695&partnerID=8YFLogxK

U2 - 10.1109/ASRU57964.2023.10389756

DO - 10.1109/ASRU57964.2023.10389756

M3 - Conference contribution

SN - 979-8-3503-0690-3

BT - Proceedings of the 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

PB - IEEE

T2 - 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

Y2 - 16 December 2023 through 20 December 2023

ER -

Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech

Abstract

Workshop

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this