Hallucination In Object Detection: A Study In Visual Part VERIFICATION

Osman Semih Kayhan; Bart  Vredebregt; Jan C. van Gemert

doi:10.1109/ICIP42928.2021.9506670

Hallucination In Object Detection: A Study In Visual Part VERIFICATION

Osman Semih Kayhan, Bart Vredebregt, Jan C. van Gemert

Pattern Recognition and Bioinformatics

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

4 Citations (Scopus)

Abstract

We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes ¹, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.

Original language	English
Title of host publication	2021 IEEE International Conference on Image Processing (ICIP)
Subtitle of host publication	Proceedings
Place of Publication	Piscataway
Publisher	IEEE
Pages	2234-2238
Number of pages	5
ISBN (Electronic)	978-1-6654-4115-5
ISBN (Print)	978-1-6654-3102-6
DOIs	https://doi.org/10.1109/ICIP42928.2021.9506670
Publication status	Published - 2021
Event	2021 IEEE International Conference on Image Processing (ICIP) - Virtual at Anchorage, United States Duration: 19 Sept 2021 → 22 Sept 2021

Conference

Conference	2021 IEEE International Conference on Image Processing (ICIP)
Country/Territory	United States
City	Virtual at Anchorage
Period	19/09/21 → 22/09/21

Keywords

Visual part verification
object detection

Access to Document

10.1109/ICIP42928.2021.9506670

Cite this

@inproceedings{97e6a49325944e668981c1f0bcee701d,

title = "Hallucination In Object Detection: A Study In Visual Part VERIFICATION",

abstract = "We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes 1, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes. ",

keywords = "Visual part verification, object detection",

author = "Kayhan, {Osman Semih} and Bart Vredebregt and {van Gemert}, {Jan C.}",

year = "2021",

doi = "10.1109/ICIP42928.2021.9506670",

language = "English",

isbn = "978-1-6654-3102-6",

pages = "2234--2238",

booktitle = "2021 IEEE International Conference on Image Processing (ICIP)",

publisher = "IEEE",

address = "United States",

note = "2021 IEEE International Conference on Image Processing (ICIP) ; Conference date: 19-09-2021 Through 22-09-2021",

}

Kayhan, OS, Vredebregt, B & van Gemert, JC 2021, Hallucination In Object Detection: A Study In Visual Part VERIFICATION. in 2021 IEEE International Conference on Image Processing (ICIP): Proceedings., 9506670, IEEE, Piscataway, pp. 2234-2238, 2021 IEEE International Conference on Image Processing (ICIP), Virtual at Anchorage, United States, 19/09/21. https://doi.org/10.1109/ICIP42928.2021.9506670

Hallucination In Object Detection: A Study In Visual Part VERIFICATION. / Kayhan, Osman Semih; Vredebregt, Bart ; van Gemert, Jan C.
2021 IEEE International Conference on Image Processing (ICIP): Proceedings. Piscataway: IEEE, 2021. p. 2234-2238 9506670.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Hallucination In Object Detection

T2 - 2021 IEEE International Conference on Image Processing (ICIP)

AU - Kayhan, Osman Semih

AU - Vredebregt, Bart

AU - van Gemert, Jan C.

PY - 2021

Y1 - 2021

N2 - We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes 1, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.

AB - We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes 1, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.

KW - Visual part verification

KW - object detection

UR - http://www.scopus.com/inward/record.url?scp=85125582972&partnerID=8YFLogxK

U2 - 10.1109/ICIP42928.2021.9506670

DO - 10.1109/ICIP42928.2021.9506670

M3 - Conference contribution

SN - 978-1-6654-3102-6

SP - 2234

EP - 2238

BT - 2021 IEEE International Conference on Image Processing (ICIP)

PB - IEEE

CY - Piscataway

Y2 - 19 September 2021 through 22 September 2021

ER -

Hallucination In Object Detection: A Study In Visual Part VERIFICATION

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

DelftBikes, data underlying the publication: Hallucination In Object Detection-A Study In Visual Part Verification

Cite this

Hallucination In Object Detection: A Study In Visual Part VERIFICATION

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Datasets

DelftBikes, data underlying the publication: Hallucination In Object Detection-A Study In Visual Part Verification

Cite this