EValueAction: a proposal for policy evaluation in simulation to support interactive imitation learning

Fiorella Sibona; Jelle Luijkx; Bas Van Der Heijden; Laura Ferranti; Marina Indri

doi:10.1109/INDIN51400.2023.10218251

EValueAction: a proposal for policy evaluation in simulation to support interactive imitation learning

Fiorella Sibona^*, Jelle Luijkx, Bas Van Der Heijden, Laura Ferranti, Marina Indri

^*Corresponding author for this work

Learning & Autonomous Control

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

12 Downloads (Pure)

Abstract

The up-and-coming concept of Industry 5.0 fore-sees human-centric flexible production lines, where collaborative robots support human workforce. In order to allow a seamless collaboration between intelligent robots and human workers, designing solutions for non-expert users is crucial. Learning from demonstration emerged as the enabling approach to address such a problem. However, more focus should be put on finding safe solutions which optimize the cost associated with the demonstrations collection process. This paper introduces a preliminary outline of a system, namely EValueAction (EVA), designed to assist the human in the process of collecting interactive demonstrations taking advantage of simulation to safely avoid failures. A policy is pre-trained with human-demonstrations and, where needed, new informative data are interactively gathered and aggregated to iteratively improve the initial policy. A trial case study further reinforces the relevance of the work by demonstrating the crucial role of informative demonstrations for generalization.

Original language	English
Title of host publication	Proceedings of the IEEE 21st International Conference on Industrial Informatics, INDIN 2023
Editors	Helene Dorksen, Stefano Scanzio, Jurgen Jasperneite
Publisher	IEEE
Number of pages	6
ISBN (Electronic)	978-1-6654-9313-0
DOIs	https://doi.org/10.1109/INDIN51400.2023.10218251
Publication status	Published - 2023
Event	21st IEEE International Conference on Industrial Informatics, INDIN 2023 - Lemgo, Germany Duration: 17 Jul 2023 → 20 Jul 2023

Conference

Conference	21st IEEE International Conference on Industrial Informatics, INDIN 2023
Country/Territory	Germany
City	Lemgo
Period	17/07/23 → 20/07/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Human-centered manufacturing
Interactive imitation learning
Learning from Demonstration
Simulation

Access to Document

10.1109/INDIN51400.2023.10218251

EValueAction_a_proposal_for_policy_evaluation_in_simulation_to_support_interactive_imitation_learningFinal published version, 2.56 MB

Cite this

Sibona, F., Luijkx, J., Heijden, B. V. D., Ferranti, L., & Indri, M. (2023). EValueAction: a proposal for policy evaluation in simulation to support interactive imitation learning. In H. Dorksen, S. Scanzio, & J. Jasperneite (Eds.), Proceedings of the IEEE 21st International Conference on Industrial Informatics, INDIN 2023 IEEE. https://doi.org/10.1109/INDIN51400.2023.10218251

@inproceedings{2d1e10b9d4354e4d9867de1ccd864d55,

title = "EValueAction: a proposal for policy evaluation in simulation to support interactive imitation learning",

abstract = "The up-and-coming concept of Industry 5.0 fore-sees human-centric flexible production lines, where collaborative robots support human workforce. In order to allow a seamless collaboration between intelligent robots and human workers, designing solutions for non-expert users is crucial. Learning from demonstration emerged as the enabling approach to address such a problem. However, more focus should be put on finding safe solutions which optimize the cost associated with the demonstrations collection process. This paper introduces a preliminary outline of a system, namely EValueAction (EVA), designed to assist the human in the process of collecting interactive demonstrations taking advantage of simulation to safely avoid failures. A policy is pre-trained with human-demonstrations and, where needed, new informative data are interactively gathered and aggregated to iteratively improve the initial policy. A trial case study further reinforces the relevance of the work by demonstrating the crucial role of informative demonstrations for generalization.",

keywords = "Human-centered manufacturing, Interactive imitation learning, Learning from Demonstration, Simulation",

author = "Fiorella Sibona and Jelle Luijkx and Heijden, {Bas Van Der} and Laura Ferranti and Marina Indri",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 21st IEEE International Conference on Industrial Informatics, INDIN 2023 ; Conference date: 17-07-2023 Through 20-07-2023",

year = "2023",

doi = "10.1109/INDIN51400.2023.10218251",

language = "English",

editor = "Helene Dorksen and Stefano Scanzio and Jurgen Jasperneite",

booktitle = "Proceedings of the IEEE 21st International Conference on Industrial Informatics, INDIN 2023",

publisher = "IEEE",

address = "United States",

}

Sibona, F, Luijkx, J , Heijden, BVD , Ferranti, L & Indri, M 2023, EValueAction: a proposal for policy evaluation in simulation to support interactive imitation learning. in H Dorksen, S Scanzio & J Jasperneite (eds), Proceedings of the IEEE 21st International Conference on Industrial Informatics, INDIN 2023. IEEE, 21st IEEE International Conference on Industrial Informatics, INDIN 2023, Lemgo, Germany, 17/07/23. https://doi.org/10.1109/INDIN51400.2023.10218251

EValueAction: a proposal for policy evaluation in simulation to support interactive imitation learning. / Sibona, Fiorella; Luijkx, Jelle ; Heijden, Bas Van Der et al.
Proceedings of the IEEE 21st International Conference on Industrial Informatics, INDIN 2023. ed. / Helene Dorksen; Stefano Scanzio; Jurgen Jasperneite. IEEE, 2023.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - EValueAction

T2 - 21st IEEE International Conference on Industrial Informatics, INDIN 2023

AU - Sibona, Fiorella

AU - Luijkx, Jelle

AU - Heijden, Bas Van Der

AU - Ferranti, Laura

AU - Indri, Marina

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - The up-and-coming concept of Industry 5.0 fore-sees human-centric flexible production lines, where collaborative robots support human workforce. In order to allow a seamless collaboration between intelligent robots and human workers, designing solutions for non-expert users is crucial. Learning from demonstration emerged as the enabling approach to address such a problem. However, more focus should be put on finding safe solutions which optimize the cost associated with the demonstrations collection process. This paper introduces a preliminary outline of a system, namely EValueAction (EVA), designed to assist the human in the process of collecting interactive demonstrations taking advantage of simulation to safely avoid failures. A policy is pre-trained with human-demonstrations and, where needed, new informative data are interactively gathered and aggregated to iteratively improve the initial policy. A trial case study further reinforces the relevance of the work by demonstrating the crucial role of informative demonstrations for generalization.

AB - The up-and-coming concept of Industry 5.0 fore-sees human-centric flexible production lines, where collaborative robots support human workforce. In order to allow a seamless collaboration between intelligent robots and human workers, designing solutions for non-expert users is crucial. Learning from demonstration emerged as the enabling approach to address such a problem. However, more focus should be put on finding safe solutions which optimize the cost associated with the demonstrations collection process. This paper introduces a preliminary outline of a system, namely EValueAction (EVA), designed to assist the human in the process of collecting interactive demonstrations taking advantage of simulation to safely avoid failures. A policy is pre-trained with human-demonstrations and, where needed, new informative data are interactively gathered and aggregated to iteratively improve the initial policy. A trial case study further reinforces the relevance of the work by demonstrating the crucial role of informative demonstrations for generalization.

KW - Human-centered manufacturing

KW - Interactive imitation learning

KW - Learning from Demonstration

KW - Simulation

UR - http://www.scopus.com/inward/record.url?scp=85171129286&partnerID=8YFLogxK

U2 - 10.1109/INDIN51400.2023.10218251

DO - 10.1109/INDIN51400.2023.10218251

M3 - Conference contribution

AN - SCOPUS:85171129286

BT - Proceedings of the IEEE 21st International Conference on Industrial Informatics, INDIN 2023

A2 - Dorksen, Helene

A2 - Scanzio, Stefano

A2 - Jasperneite, Jurgen

PB - IEEE

Y2 - 17 July 2023 through 20 July 2023

ER -

EValueAction: a proposal for policy evaluation in simulation to support interactive imitation learning

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this