Improving Whispered Speech Recognition Performance Using Pseudo-Whispered Based Data Augmentation

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

Whispering is a distinct form of speech known for its soft, breathy, and hushed characteristics, often used for private communication. The acoustic characteristics of whispered speech differ substantially from normally phonated speech and the scarcity of adequate training data leads to low automatic speech recognition (ASR) performance. To address the data scarcity issue, we use a signal processing-based technique that transforms the spectral characteristics of normal speech to those of pseudo-whispered speech. We augment an End-to-End ASR with pseudo-whispered speech and achieve an 18.2 % relative reduction in word error rate for whispered speech compared to the baseline. Results for the individual speaker groups in the wTIMIT database show the best results for US English. Further investigation showed that the lack of glottal information in whispered speech has the largest impact on whispered speech ASR performance.
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)
PublisherIEEE
Number of pages8
ISBN (Electronic)979-8-3503-0689-7
ISBN (Print)979-8-3503-0690-3
DOIs
Publication statusPublished - 2023
Event2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) - Taipei, Taiwan
Duration: 16 Dec 202320 Dec 2023

Workshop

Workshop2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)
Country/TerritoryTaiwan
CityTaipei
Period16/12/2320/12/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Error analysis
  • Databases
  • Conferences
  • Training data
  • Transforms
  • Data augmentation
  • Acoustics
  • Whispered speech
  • pseudo-whisper
  • end-to-end speech recognition
  • wTIMIT
  • signal processing

Fingerprint

Dive into the research topics of 'Improving Whispered Speech Recognition Performance Using Pseudo-Whispered Based Data Augmentation'. Together they form a unique fingerprint.

Cite this