3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

Qinyu Chen, Zuowen Wang, Shih Chii Liu, Chang Gao

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7× without losing accuracy when tested on a v2e-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at https://github.com/qinche106/cb-convlstm-eyetracking.
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)
Place of PublicationDanvers
PublisherIEEE
Number of pages5
ISBN (Electronic)979-8-3503-0026-0
ISBN (Print)979-8-3503-0027-7
DOIs
Publication statusPublished - 2023
Event2023 IEEE Biomedical Circuits and Systems Conference (BioCAS) - Toronto, Canada
Duration: 19 Oct 202321 Oct 2023

Conference

Conference2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)
Country/TerritoryCanada
City Toronto
Period19/10/2321/10/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Pupil tracking
  • event cameras
  • sparsity
  • ConvLSTM
  • healthcare
  • AR/VR

Fingerprint

Dive into the research topics of '3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network'. Together they form a unique fingerprint.

Cite this