Through-Screen Visible Light Sensing Empowered by Embedded Deep Learning

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

2 Citations (Scopus)
48 Downloads (Pure)

Abstract

Motivated by the trend of realizing full screens on devices such as smartphones, in this work we propose through-screen sensing with visible light for the application of fingertip air-writing. The system can recognize handwritten digits with under-screen photodiodes as the receiver. The key idea is to recognize the weak light reflected by the finger when the finger writes the digits on top of a screen. The proposed air-writing system has immunity to scene changes because it has a fixed screen light source. However, the screen is a double-edged sword as both a signal source and a noise source. We propose a data preprocessing method to reduce the interference of the screen as a noise source. We design an embedded deep learning model, a customized model ConvRNN, to model the spatial and temporal patterns in the dynamic and weak reflected signal for air-writing digits recognition. The evaluation results show that our through-screen fingertip air-writing system with visible light can achieve accuracy up to 91%. Results further show that the size of the customized ConvRNN model can be reduced by 94% with less than a 10% drop in performance.

Original languageEnglish
Title of host publicationSenSys 2021 - Proceedings of the 2021 19th ACM Conference on Embedded Networked Sensor Systems
PublisherAssociation for Computing Machinery (ACM)
Pages478-484
Number of pages7
ISBN (Electronic)9781450390972
DOIs
Publication statusPublished - 2021
Event19th ACM Conference on Embedded Networked Sensor Systems, SenSys 2021 - Coimbra, Portugal
Duration: 15 Nov 202117 Nov 2021

Publication series

NameSenSys 2021 - Proceedings of the 2021 19th ACM Conference on Embedded Networked Sensor Systems

Conference

Conference19th ACM Conference on Embedded Networked Sensor Systems, SenSys 2021
Country/TerritoryPortugal
CityCoimbra
Period15/11/2117/11/21

Keywords

  • embedded AI
  • embedded deep learning
  • Through-screen sensing

Fingerprint

Dive into the research topics of 'Through-Screen Visible Light Sensing Empowered by Embedded Deep Learning'. Together they form a unique fingerprint.

Cite this