3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

Qinyu  Chen; Zuowen  Wang; Shih Chii Liu; Chang Gao

doi:10.1109/BioCAS58349.2023.10389062

3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

Qinyu Chen, Zuowen Wang, Shih Chii Liu, Chang Gao

Electronics

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

23 Citations (SciVal)

71 Downloads (Pure)

Abstract

This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7× without losing accuracy when tested on a v2e-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at https://github.com/qinche106/cb-convlstm-eyetracking.

Original language	English
Title of host publication	Proceedings of the 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)
Place of Publication	Danvers
Publisher	IEEE
Number of pages	5
ISBN (Electronic)	979-8-3503-0026-0
ISBN (Print)	979-8-3503-0027-7
DOIs	https://doi.org/10.1109/BioCAS58349.2023.10389062
Publication status	Published - 2023
Event	2023 IEEE Biomedical Circuits and Systems Conference (BioCAS) - Toronto, Canada Duration: 19 Oct 2023 → 21 Oct 2023

Conference

Conference	2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)
Country/Territory	Canada
City	Toronto
Period	19/10/23 → 21/10/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Pupil tracking
event cameras
sparsity
ConvLSTM
healthcare
AR/VR

Access to Document

10.1109/BioCAS58349.2023.10389062

3ET_Efficient_Event-based_Eye_Tracking_using_a_Change-Based_ConvLSTM_NetworkFinal published version, 2.63 MB

Cite this

@inproceedings{3631b0576e20470aacecceb3476a1137,

title = "3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network",

abstract = "This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7× without losing accuracy when tested on a v2e-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at https://github.com/qinche106/cb-convlstm-eyetracking.",

keywords = "Pupil tracking, event cameras, sparsity, ConvLSTM, healthcare, AR/VR",

author = "Qinyu Chen and Zuowen Wang and Liu, {Shih Chii} and Chang Gao",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS) ; Conference date: 19-10-2023 Through 21-10-2023",

year = "2023",

doi = "10.1109/BioCAS58349.2023.10389062",

language = "English",

isbn = "979-8-3503-0027-7",

booktitle = "Proceedings of the 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)",

publisher = "IEEE",

address = "United States",

}

TY - GEN

T1 - 3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

AU - Chen, Qinyu

AU - Wang, Zuowen

AU - Liu, Shih Chii

AU - Gao, Chang

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7× without losing accuracy when tested on a v2e-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at https://github.com/qinche106/cb-convlstm-eyetracking.

AB - This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7× without losing accuracy when tested on a v2e-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at https://github.com/qinche106/cb-convlstm-eyetracking.

KW - Pupil tracking

KW - event cameras

KW - sparsity

KW - ConvLSTM

KW - healthcare

KW - AR/VR

UR - http://www.scopus.com/inward/record.url?scp=85179172183&partnerID=8YFLogxK

U2 - 10.1109/BioCAS58349.2023.10389062

DO - 10.1109/BioCAS58349.2023.10389062

M3 - Conference contribution

SN - 979-8-3503-0027-7

BT - Proceedings of the 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)

PB - IEEE

CY - Danvers

T2 - 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)

Y2 - 19 October 2023 through 21 October 2023

ER -

3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this