Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects

Jinchen  Zeng; Rick Butler; John J. van den Dobbelsteen; Benno H.W. Hendriks; Maarten van der Elst; Justin Dauwels

doi:10.1109/ICASSP49357.2023.10095197

Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects

Jinchen Zeng, Rick Butler, John J. van den Dobbelsteen, Benno H.W. Hendriks, Maarten van der Elst, Justin Dauwels

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

40 Downloads (Pure)

Abstract

In this paper, we aim to design an automatic camera pose estimation pipeline for clinical spaces such as catheterization laboratories. Our proposed pipeline exploits Scaled-YOLOv4 to detect fixed objects. We adopt the self-supervised key-point detector SuperPoint in combination with SuperGlue, a keypoint matching technique based on graph neural networks. Thus, we match key-points on input images with annotated reference points. Reference points are chosen on fixed objects in the scene, such as corners of door posts or windows. The point-correspondences between the image coordinates and the 3D coordinates are applied to the Perspective-n-Point algorithm to estimate the pose of each camera. Compared with other camera pose estimation methods, the proposed pipeline does not require the construction of 3D point-cloud model of the scene or placing a polyhedron object in the scene before each required calibration. Using videos from real procedures, we show that the pipeline can estimate the camera pose with high accuracy.

Original language	English
Title of host publication	Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Place of Publication	Piscataway
Publisher	IEEE
Number of pages	5
ISBN (Electronic)	978-1-7281-6327-7
ISBN (Print)	978-1-7281-6328-4
DOIs	https://doi.org/10.1109/ICASSP49357.2023.10095197
Publication status	Published - 2023
Event	48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023 - Rhodes Island, Greece Duration: 4 Jun 2023 → 10 Jun 2023

Publication series

Name	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2023-June
ISSN (Print)	1520-6149

Conference

Conference	48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023
Abbreviated title	ICASSP 2023
Country/Territory	Greece
City	Rhodes Island
Period	4/06/23 → 10/06/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Camera calibration
camera pose estimation
Perspective-n-Point
3D geometry

Access to Document

10.1109/ICASSP49357.2023.10095197

Automatic_Camera_Pose_Estimation_by_Key-Point_Matching_of_Reference_ObjectsFinal published version, 3.69 MB

Cite this

Zeng, J., Butler, R., van den Dobbelsteen, J. J., Hendriks, B. H. W., van der Elst, M., & Dauwels, J. (2023). Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects. In Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2023-June). IEEE. https://doi.org/10.1109/ICASSP49357.2023.10095197

Zeng, Jinchen ; Butler, Rick ; van den Dobbelsteen, John J. et al. / Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects. Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway : IEEE, 2023. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

@inproceedings{166ca190a48546fc8ea7050eb1792438,

title = "Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects",

abstract = "In this paper, we aim to design an automatic camera pose estimation pipeline for clinical spaces such as catheterization laboratories. Our proposed pipeline exploits Scaled-YOLOv4 to detect fixed objects. We adopt the self-supervised key-point detector SuperPoint in combination with SuperGlue, a keypoint matching technique based on graph neural networks. Thus, we match key-points on input images with annotated reference points. Reference points are chosen on fixed objects in the scene, such as corners of door posts or windows. The point-correspondences between the image coordinates and the 3D coordinates are applied to the Perspective-n-Point algorithm to estimate the pose of each camera. Compared with other camera pose estimation methods, the proposed pipeline does not require the construction of 3D point-cloud model of the scene or placing a polyhedron object in the scene before each required calibration. Using videos from real procedures, we show that the pipeline can estimate the camera pose with high accuracy.",

keywords = "Camera calibration, camera pose estimation, Perspective-n-Point, 3D geometry",

author = "Jinchen Zeng and Rick Butler and {van den Dobbelsteen}, {John J.} and Hendriks, {Benno H.W.} and {van der Elst}, Maarten and Justin Dauwels",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023, ICASSP 2023 ; Conference date: 04-06-2023 Through 10-06-2023",

year = "2023",

doi = "10.1109/ICASSP49357.2023.10095197",

language = "English",

isbn = "978-1-7281-6328-4",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

publisher = "IEEE",

booktitle = "Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

}

Zeng, J, Butler, R , van den Dobbelsteen, JJ , Hendriks, BHW , van der Elst, M & Dauwels, J 2023, Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects. in Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2023-June, IEEE, Piscataway, 48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023, Rhodes Island, Greece, 4/06/23. https://doi.org/10.1109/ICASSP49357.2023.10095197

Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects. / Zeng, Jinchen ; Butler, Rick ; van den Dobbelsteen, John J. et al.
Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway: IEEE, 2023. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2023-June).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects

AU - Zeng, Jinchen

AU - Butler, Rick

AU - van den Dobbelsteen, John J.

AU - Hendriks, Benno H.W.

AU - van der Elst, Maarten

AU - Dauwels, Justin

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - In this paper, we aim to design an automatic camera pose estimation pipeline for clinical spaces such as catheterization laboratories. Our proposed pipeline exploits Scaled-YOLOv4 to detect fixed objects. We adopt the self-supervised key-point detector SuperPoint in combination with SuperGlue, a keypoint matching technique based on graph neural networks. Thus, we match key-points on input images with annotated reference points. Reference points are chosen on fixed objects in the scene, such as corners of door posts or windows. The point-correspondences between the image coordinates and the 3D coordinates are applied to the Perspective-n-Point algorithm to estimate the pose of each camera. Compared with other camera pose estimation methods, the proposed pipeline does not require the construction of 3D point-cloud model of the scene or placing a polyhedron object in the scene before each required calibration. Using videos from real procedures, we show that the pipeline can estimate the camera pose with high accuracy.

AB - In this paper, we aim to design an automatic camera pose estimation pipeline for clinical spaces such as catheterization laboratories. Our proposed pipeline exploits Scaled-YOLOv4 to detect fixed objects. We adopt the self-supervised key-point detector SuperPoint in combination with SuperGlue, a keypoint matching technique based on graph neural networks. Thus, we match key-points on input images with annotated reference points. Reference points are chosen on fixed objects in the scene, such as corners of door posts or windows. The point-correspondences between the image coordinates and the 3D coordinates are applied to the Perspective-n-Point algorithm to estimate the pose of each camera. Compared with other camera pose estimation methods, the proposed pipeline does not require the construction of 3D point-cloud model of the scene or placing a polyhedron object in the scene before each required calibration. Using videos from real procedures, we show that the pipeline can estimate the camera pose with high accuracy.

KW - Camera calibration

KW - camera pose estimation

KW - Perspective-n-Point

KW - 3D geometry

UR - http://www.scopus.com/inward/record.url?scp=85177601395&partnerID=8YFLogxK

U2 - 10.1109/ICASSP49357.2023.10095197

DO - 10.1109/ICASSP49357.2023.10095197

M3 - Conference contribution

SN - 978-1-7281-6328-4

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

BT - Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - IEEE

CY - Piscataway

T2 - 48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023

Y2 - 4 June 2023 through 10 June 2023

ER -

Zeng J, Butler R , van den Dobbelsteen JJ , Hendriks BHW , van der Elst M , Dauwels J. Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects. In Proceedings of the ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway: IEEE. 2023. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). doi: 10.1109/ICASSP49357.2023.10095197

Automatic Camera Pose Estimation by Key-Point Matching of Reference Objects

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this