Cross-modal approach for conversational well-being monitoring with multi-sensory earables

Chulhong Min; Alessandro Montanari; Akhil Mathur; Seungchul Lee; Fahim Kawsar

doi:10.1145/3267305.3267695

Cross-modal approach for conversational well-being monitoring with multi-sensory earables

Chulhong Min, Alessandro Montanari, Akhil Mathur, Seungchul Lee, Fahim Kawsar

Internet of Things

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

8 Citations (Scopus)

Abstract

We propose a cross-modal approach for conversational well-being monitoring with a multi-sensory earable. It consists of motion, audio, and BLE models on earables. Using the IMU sensor, the microphone, and BLE scanning, the models detect speaking activities, stress and emotion, and participants in the conversation, respectively. We discuss the feasibility in qualifying conversations with our purpose-built cross-modal model in an energy-efficient and privacy-preserving way. With the cross-modal model, we develop a mobile application that qualifies on-going conversations and provides personalised feedback on social well-being.

Original language	English
Title of host publication	UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers
Editors	Rajesh K. Balan, Youngki Lee, Kai Kunze
Place of Publication	New York, NY, USA
Publisher	Association for Computing Machinery (ACM)
Pages	706-709
Number of pages	4
ISBN (Electronic)	978-1-4503-5966-5
DOIs	https://doi.org/10.1145/3267305.3267695
Publication status	Published - 2018
Event	2018 Joint ACM International Conference on Pervasive and Ubiquitous Computing, UbiComp 2018 and 2018 ACM International Symposium on Wearable Computers, ISWC 2018 - Singapore, Singapore Duration: 8 Oct 2018 → 12 Oct 2018

Conference

Conference	2018 Joint ACM International Conference on Pervasive and Ubiquitous Computing, UbiComp 2018 and 2018 ACM International Symposium on Wearable Computers, ISWC 2018
Country/Territory	Singapore
City	Singapore
Period	8/10/18 → 12/10/18

Keywords

Earable
Multi-sensory
Well-being

Access to Document

10.1145/3267305.3267695

Cite this

Min, C., Montanari, A., Mathur, A., Lee, S., & Kawsar, F. (2018). Cross-modal approach for conversational well-being monitoring with multi-sensory earables. In R. K. Balan, Y. Lee, & K. Kunze (Eds.), UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers (pp. 706-709). Association for Computing Machinery (ACM). https://doi.org/10.1145/3267305.3267695

Min, Chulhong ; Montanari, Alessandro ; Mathur, Akhil et al. / Cross-modal approach for conversational well-being monitoring with multi-sensory earables. UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers. editor / Rajesh K. Balan ; Youngki Lee ; Kai Kunze. New York, NY, USA : Association for Computing Machinery (ACM), 2018. pp. 706-709

@inproceedings{c470a57e051741b4b5aabc01b24d2735,

title = "Cross-modal approach for conversational well-being monitoring with multi-sensory earables",

abstract = "We propose a cross-modal approach for conversational well-being monitoring with a multi-sensory earable. It consists of motion, audio, and BLE models on earables. Using the IMU sensor, the microphone, and BLE scanning, the models detect speaking activities, stress and emotion, and participants in the conversation, respectively. We discuss the feasibility in qualifying conversations with our purpose-built cross-modal model in an energy-efficient and privacy-preserving way. With the cross-modal model, we develop a mobile application that qualifies on-going conversations and provides personalised feedback on social well-being.",

keywords = "Earable, Multi-sensory, Well-being",

author = "Chulhong Min and Alessandro Montanari and Akhil Mathur and Seungchul Lee and Fahim Kawsar",

year = "2018",

doi = "10.1145/3267305.3267695",

language = "English",

pages = "706--709",

editor = "Balan, {Rajesh K. } and Lee, {Youngki } and Kunze, {Kai }",

booktitle = "UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

note = "2018 Joint ACM International Conference on Pervasive and Ubiquitous Computing, UbiComp 2018 and 2018 ACM International Symposium on Wearable Computers, ISWC 2018 ; Conference date: 08-10-2018 Through 12-10-2018",

}

Min, C, Montanari, A, Mathur, A, Lee, S & Kawsar, F 2018, Cross-modal approach for conversational well-being monitoring with multi-sensory earables. in RK Balan, Y Lee & K Kunze (eds), UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers. Association for Computing Machinery (ACM), New York, NY, USA, pp. 706-709, 2018 Joint ACM International Conference on Pervasive and Ubiquitous Computing, UbiComp 2018 and 2018 ACM International Symposium on Wearable Computers, ISWC 2018, Singapore, Singapore, 8/10/18. https://doi.org/10.1145/3267305.3267695

Cross-modal approach for conversational well-being monitoring with multi-sensory earables. / Min, Chulhong; Montanari, Alessandro; Mathur, Akhil et al.
UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers. ed. / Rajesh K. Balan; Youngki Lee; Kai Kunze. New York, NY, USA: Association for Computing Machinery (ACM), 2018. p. 706-709.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Cross-modal approach for conversational well-being monitoring with multi-sensory earables

AU - Min, Chulhong

AU - Montanari, Alessandro

AU - Mathur, Akhil

AU - Lee, Seungchul

AU - Kawsar, Fahim

PY - 2018

Y1 - 2018

N2 - We propose a cross-modal approach for conversational well-being monitoring with a multi-sensory earable. It consists of motion, audio, and BLE models on earables. Using the IMU sensor, the microphone, and BLE scanning, the models detect speaking activities, stress and emotion, and participants in the conversation, respectively. We discuss the feasibility in qualifying conversations with our purpose-built cross-modal model in an energy-efficient and privacy-preserving way. With the cross-modal model, we develop a mobile application that qualifies on-going conversations and provides personalised feedback on social well-being.

AB - We propose a cross-modal approach for conversational well-being monitoring with a multi-sensory earable. It consists of motion, audio, and BLE models on earables. Using the IMU sensor, the microphone, and BLE scanning, the models detect speaking activities, stress and emotion, and participants in the conversation, respectively. We discuss the feasibility in qualifying conversations with our purpose-built cross-modal model in an energy-efficient and privacy-preserving way. With the cross-modal model, we develop a mobile application that qualifies on-going conversations and provides personalised feedback on social well-being.

KW - Earable

KW - Multi-sensory

KW - Well-being

UR - http://www.scopus.com/inward/record.url?scp=85058307253&partnerID=8YFLogxK

U2 - 10.1145/3267305.3267695

DO - 10.1145/3267305.3267695

M3 - Conference contribution

AN - SCOPUS:85058307253

SP - 706

EP - 709

BT - UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers

A2 - Balan, Rajesh K.

A2 - Lee, Youngki

A2 - Kunze, Kai

PB - Association for Computing Machinery (ACM)

CY - New York, NY, USA

T2 - 2018 Joint ACM International Conference on Pervasive and Ubiquitous Computing, UbiComp 2018 and 2018 ACM International Symposium on Wearable Computers, ISWC 2018

Y2 - 8 October 2018 through 12 October 2018

ER -

Min C, Montanari A, Mathur A, Lee S, Kawsar F. Cross-modal approach for conversational well-being monitoring with multi-sensory earables. In Balan RK, Lee Y, Kunze K, editors, UbiComp/ISWC 2018 - Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers. New York, NY, USA: Association for Computing Machinery (ACM). 2018. p. 706-709 doi: 10.1145/3267305.3267695

Cross-modal approach for conversational well-being monitoring with multi-sensory earables

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this