Demo Abstract: Catch My Eye: Gaze-Based Activity Recognition in an Augmented Reality Art Gallery

Tim  Scargill; Guohao Lan; Maria  Gorlatova

doi:10.1109/IPSN54338.2022.00052

Demo Abstract: Catch My Eye: Gaze-Based Activity Recognition in an Augmented Reality Art Gallery

Tim Scargill, Guohao Lan, Maria Gorlatova

Embedded Systems

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

1 Citation (Scopus)

10 Downloads (Pure)

Abstract

The personalization of augmented reality (AR) experiences based on environmental and user context is key to unlocking their full potential. The recent addition of eye tracking to AR headsets provides a convenient method for detecting user context, but complex analysis of raw gaze data is required to detect where a user's attention and thoughts truly lie. In this demo we present Catch My Eye, the first system to incorporate deep neural network (DNN)-based activity recognition from user gaze into a realistic mobile AR app. We develop an edge computing-based architecture to offload context computation from resource-constrained AR devices, and present a working example of content adaptation based on user context, for the scenario of a virtual art gallery. It shows that user activities can be accurately recognized and employed with sufficiently low latency for practical AR applications.

Original language	English
Title of host publication	Proceedings of the 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)
Editors	L. O'Conner
Place of Publication	Piscataway
Publisher	IEEE
Pages	503-504
Number of pages	2
ISBN (Electronic)	978-1-6654-9624-7
ISBN (Print)	978-1-6654-9625-4
DOIs	https://doi.org/10.1109/IPSN54338.2022.00052
Publication status	Published - 2022
Event	2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) - Milano, Italy Duration: 4 May 2022 → 6 May 2022 Conference number: 21st

Conference

Conference	2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)
Abbreviated title	IPSN 2022
Country/Territory	Italy
City	Milano
Period	4/05/22 → 6/05/22

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Augmented reality
eye tracking
activity recognition
edge computing
augmented reality art gallery

Access to Document

10.1109/IPSN54338.2022.00052

Demo_Abstract_Catch_My_Eye_Gaze-Based_Activity_Recognition_in_an_Augmented_Reality_Art_GalleryFinal published version, 337 KB

Cite this

@inproceedings{18ab04c67d0f49ca9c01cbc02e9f7f93,

title = "Demo Abstract: Catch My Eye: Gaze-Based Activity Recognition in an Augmented Reality Art Gallery",

abstract = "The personalization of augmented reality (AR) experiences based on environmental and user context is key to unlocking their full potential. The recent addition of eye tracking to AR headsets provides a convenient method for detecting user context, but complex analysis of raw gaze data is required to detect where a user's attention and thoughts truly lie. In this demo we present Catch My Eye, the first system to incorporate deep neural network (DNN)-based activity recognition from user gaze into a realistic mobile AR app. We develop an edge computing-based architecture to offload context computation from resource-constrained AR devices, and present a working example of content adaptation based on user context, for the scenario of a virtual art gallery. It shows that user activities can be accurately recognized and employed with sufficiently low latency for practical AR applications.",

keywords = "Augmented reality, eye tracking, activity recognition, edge computing, augmented reality art gallery",

author = "Tim Scargill and Guohao Lan and Maria Gorlatova",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), IPSN 2022 ; Conference date: 04-05-2022 Through 06-05-2022",

year = "2022",

doi = "10.1109/IPSN54338.2022.00052",

language = "English",

isbn = "978-1-6654-9625-4",

pages = "503--504",

editor = "L. O'Conner",

booktitle = "Proceedings of the 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)",

publisher = "IEEE",

address = "United States",

}

Scargill, T, Lan, G & Gorlatova, M 2022, Demo Abstract: Catch My Eye: Gaze-Based Activity Recognition in an Augmented Reality Art Gallery. in L O'Conner (ed.), Proceedings of the 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)., 9825976, IEEE, Piscataway, pp. 503-504, 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), Milano, Italy, 4/05/22. https://doi.org/10.1109/IPSN54338.2022.00052

Demo Abstract: Catch My Eye: Gaze-Based Activity Recognition in an Augmented Reality Art Gallery. / Scargill, Tim; Lan, Guohao; Gorlatova, Maria .
Proceedings of the 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). ed. / L. O'Conner. Piscataway: IEEE, 2022. p. 503-504 9825976.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Demo Abstract: Catch My Eye

T2 - 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)

AU - Scargill, Tim

AU - Lan, Guohao

AU - Gorlatova, Maria

N1 - Conference code: 21st

PY - 2022

Y1 - 2022

N2 - The personalization of augmented reality (AR) experiences based on environmental and user context is key to unlocking their full potential. The recent addition of eye tracking to AR headsets provides a convenient method for detecting user context, but complex analysis of raw gaze data is required to detect where a user's attention and thoughts truly lie. In this demo we present Catch My Eye, the first system to incorporate deep neural network (DNN)-based activity recognition from user gaze into a realistic mobile AR app. We develop an edge computing-based architecture to offload context computation from resource-constrained AR devices, and present a working example of content adaptation based on user context, for the scenario of a virtual art gallery. It shows that user activities can be accurately recognized and employed with sufficiently low latency for practical AR applications.

AB - The personalization of augmented reality (AR) experiences based on environmental and user context is key to unlocking their full potential. The recent addition of eye tracking to AR headsets provides a convenient method for detecting user context, but complex analysis of raw gaze data is required to detect where a user's attention and thoughts truly lie. In this demo we present Catch My Eye, the first system to incorporate deep neural network (DNN)-based activity recognition from user gaze into a realistic mobile AR app. We develop an edge computing-based architecture to offload context computation from resource-constrained AR devices, and present a working example of content adaptation based on user context, for the scenario of a virtual art gallery. It shows that user activities can be accurately recognized and employed with sufficiently low latency for practical AR applications.

KW - Augmented reality

KW - eye tracking

KW - activity recognition

KW - edge computing

KW - augmented reality art gallery

UR - http://www.scopus.com/inward/record.url?scp=85135874187&partnerID=8YFLogxK

U2 - 10.1109/IPSN54338.2022.00052

DO - 10.1109/IPSN54338.2022.00052

M3 - Conference contribution

SN - 978-1-6654-9625-4

SP - 503

EP - 504

BT - Proceedings of the 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)

A2 - O'Conner, L.

PB - IEEE

CY - Piscataway

Y2 - 4 May 2022 through 6 May 2022

ER -

Demo Abstract: Catch My Eye: Gaze-Based Activity Recognition in an Augmented Reality Art Gallery

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this