A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments

Kushal Thirani; David A. Abbink; Luka Peternel

doi:10.1007/978-3-031-22731-8_2

A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments

Kushal Thirani, David A. Abbink, Luka Peternel^*

^*Corresponding author for this work

Human-Robot Interaction

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

Abstract

This paper studies non-physical feedback mechanisms to guide human workers toward ergonomic body postures. Specifically, the focus is to solve the tasks that involve no direct physical interaction between the human and the robotic system, therefore tactile guidance by the robot body is not feasible. We propose a multi-modal ergonomic posture guidance system that comprises visual feedback and speech-based audio feedback. We hypothesise that the proposed multi-modal system leads to better performance compared to uni-modal feedback systems when trying to guide users from one pose to another. To test the hypothesis we conducted an experiment that compared conditions with only audio feedback, only visual feedback and multi-modal feedback. In addition, we examined speech-based audio guidance in joint space and in endpoint space. The results showed that the speech-based feedback in joint space came out as the preferred audio feedback due to its ability to allow users to carry out efficient and coordinated inter-joint movements, especially in cases of high redundancy. Furthermore, the proposed multi-modal feedback system was superior compared to the other feedback modalities both in terms of objective measures and subjective measures.

Original language	English
Title of host publication	Human-Friendly Robotics 2022 - HFR
Subtitle of host publication	Proceedings of the 15th International Workshop on Human-Friendly Robotics
Editors	Pablo Borja, Cosimo Della Santina, Luka Peternel, Elena Torta
Publisher	Springer
Pages	14-29
ISBN (Electronic)	978-3-031-22731-8
ISBN (Print)	978-3-031-22730-1
DOIs	https://doi.org/10.1007/978-3-031-22731-8_2
Publication status	Published - 2023
Event	15th International Workshop on Human-Friendly Robotics, HFR 2022 - Delft, Netherlands Duration: 22 Sep 2022 → 23 Sep 2022

Publication series

Name	Springer Proceedings in Advanced Robotics
Volume	26
ISSN (Print)	2511-1256
ISSN (Electronic)	2511-1264

Conference

Conference	15th International Workshop on Human-Friendly Robotics, HFR 2022
Country/Territory	Netherlands
City	Delft
Period	22/09/22 → 23/09/22

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Audio feedback
Ergonomics
Human factors
Human-machine interaction
Visual feedback

Access to Document

10.1007/978-3-031-22731-8_2

Cite this

Thirani, K., Abbink, D. A., & Peternel, L. (2023). A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments. In P. Borja, C. Della Santina, L. Peternel, & E. Torta (Eds.), Human-Friendly Robotics 2022 - HFR: Proceedings of the 15th International Workshop on Human-Friendly Robotics (pp. 14-29). (Springer Proceedings in Advanced Robotics; Vol. 26). Springer. https://doi.org/10.1007/978-3-031-22731-8_2

Thirani, Kushal ; Abbink, David A. ; Peternel, Luka. / A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments. Human-Friendly Robotics 2022 - HFR: Proceedings of the 15th International Workshop on Human-Friendly Robotics. editor / Pablo Borja ; Cosimo Della Santina ; Luka Peternel ; Elena Torta. Springer, 2023. pp. 14-29 (Springer Proceedings in Advanced Robotics).

@inproceedings{a1c50eeab1244e9f9bd1437f6365415c,

title = "A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments",

abstract = "This paper studies non-physical feedback mechanisms to guide human workers toward ergonomic body postures. Specifically, the focus is to solve the tasks that involve no direct physical interaction between the human and the robotic system, therefore tactile guidance by the robot body is not feasible. We propose a multi-modal ergonomic posture guidance system that comprises visual feedback and speech-based audio feedback. We hypothesise that the proposed multi-modal system leads to better performance compared to uni-modal feedback systems when trying to guide users from one pose to another. To test the hypothesis we conducted an experiment that compared conditions with only audio feedback, only visual feedback and multi-modal feedback. In addition, we examined speech-based audio guidance in joint space and in endpoint space. The results showed that the speech-based feedback in joint space came out as the preferred audio feedback due to its ability to allow users to carry out efficient and coordinated inter-joint movements, especially in cases of high redundancy. Furthermore, the proposed multi-modal feedback system was superior compared to the other feedback modalities both in terms of objective measures and subjective measures.",

keywords = "Audio feedback, Ergonomics, Human factors, Human-machine interaction, Visual feedback",

author = "Kushal Thirani and Abbink, {David A.} and Luka Peternel",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 15th International Workshop on Human-Friendly Robotics, HFR 2022 ; Conference date: 22-09-2022 Through 23-09-2022",

year = "2023",

doi = "10.1007/978-3-031-22731-8_2",

language = "English",

isbn = "978-3-031-22730-1",

series = "Springer Proceedings in Advanced Robotics",

publisher = "Springer",

pages = "14--29",

editor = "Pablo Borja and {Della Santina}, Cosimo and Luka Peternel and Elena Torta",

booktitle = "Human-Friendly Robotics 2022 - HFR",

}

Thirani, K, Abbink, DA & Peternel, L 2023, A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments. in P Borja, C Della Santina, L Peternel & E Torta (eds), Human-Friendly Robotics 2022 - HFR: Proceedings of the 15th International Workshop on Human-Friendly Robotics. Springer Proceedings in Advanced Robotics, vol. 26, Springer, pp. 14-29, 15th International Workshop on Human-Friendly Robotics, HFR 2022, Delft, Netherlands, 22/09/22. https://doi.org/10.1007/978-3-031-22731-8_2

A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments. / Thirani, Kushal; Abbink, David A.; Peternel, Luka.

Human-Friendly Robotics 2022 - HFR: Proceedings of the 15th International Workshop on Human-Friendly Robotics. ed. / Pablo Borja; Cosimo Della Santina; Luka Peternel; Elena Torta. Springer, 2023. p. 14-29 (Springer Proceedings in Advanced Robotics; Vol. 26).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments

AU - Thirani, Kushal

AU - Abbink, David A.

AU - Peternel, Luka

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - This paper studies non-physical feedback mechanisms to guide human workers toward ergonomic body postures. Specifically, the focus is to solve the tasks that involve no direct physical interaction between the human and the robotic system, therefore tactile guidance by the robot body is not feasible. We propose a multi-modal ergonomic posture guidance system that comprises visual feedback and speech-based audio feedback. We hypothesise that the proposed multi-modal system leads to better performance compared to uni-modal feedback systems when trying to guide users from one pose to another. To test the hypothesis we conducted an experiment that compared conditions with only audio feedback, only visual feedback and multi-modal feedback. In addition, we examined speech-based audio guidance in joint space and in endpoint space. The results showed that the speech-based feedback in joint space came out as the preferred audio feedback due to its ability to allow users to carry out efficient and coordinated inter-joint movements, especially in cases of high redundancy. Furthermore, the proposed multi-modal feedback system was superior compared to the other feedback modalities both in terms of objective measures and subjective measures.

AB - This paper studies non-physical feedback mechanisms to guide human workers toward ergonomic body postures. Specifically, the focus is to solve the tasks that involve no direct physical interaction between the human and the robotic system, therefore tactile guidance by the robot body is not feasible. We propose a multi-modal ergonomic posture guidance system that comprises visual feedback and speech-based audio feedback. We hypothesise that the proposed multi-modal system leads to better performance compared to uni-modal feedback systems when trying to guide users from one pose to another. To test the hypothesis we conducted an experiment that compared conditions with only audio feedback, only visual feedback and multi-modal feedback. In addition, we examined speech-based audio guidance in joint space and in endpoint space. The results showed that the speech-based feedback in joint space came out as the preferred audio feedback due to its ability to allow users to carry out efficient and coordinated inter-joint movements, especially in cases of high redundancy. Furthermore, the proposed multi-modal feedback system was superior compared to the other feedback modalities both in terms of objective measures and subjective measures.

KW - Audio feedback

KW - Ergonomics

KW - Human factors

KW - Human-machine interaction

KW - Visual feedback

UR - http://www.scopus.com/inward/record.url?scp=85148045410&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-22731-8_2

DO - 10.1007/978-3-031-22731-8_2

M3 - Conference contribution

AN - SCOPUS:85148045410

SN - 978-3-031-22730-1

T3 - Springer Proceedings in Advanced Robotics

SP - 14

EP - 29

BT - Human-Friendly Robotics 2022 - HFR

A2 - Borja, Pablo

A2 - Della Santina, Cosimo

A2 - Peternel, Luka

A2 - Torta, Elena

PB - Springer

T2 - 15th International Workshop on Human-Friendly Robotics, HFR 2022

Y2 - 22 September 2022 through 23 September 2022

ER -

Thirani K, Abbink DA , Peternel L. A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments. In Borja P, Della Santina C, Peternel L, Torta E, editors, Human-Friendly Robotics 2022 - HFR: Proceedings of the 15th International Workshop on Human-Friendly Robotics. Springer. 2023. p. 14-29. (Springer Proceedings in Advanced Robotics). https://doi.org/10.1007/978-3-031-22731-8_2