Effects of sampling skewness of the importance-weighted risk estimator on model selection

Wouter Kouw, Marco Loog

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

2 Citations (Scopus)

Abstract

Importance-weighting is a popular and well-researched technique for dealing with sample selection bias and covariate shift. It has desirable characteristics such as unbiasedness, consistency and low computational complexity. However, weighting can have a detrimental effect on an estimator as well. In this work, we empirically show that the sampling distribution of an importance-weighted estimator can be skewed. For sample selection bias settings, and for small sample sizes, the importance-weighted risk estimator produces overestimates for data sets in the body of the sampling distribution, i.e. the majority of cases, and large underestimates for data sets in the tail of the sampling distribution. These over- and underestimates of the risk lead to sub-optimal regularization parameters when used for importance-weighted validation.
Original languageEnglish
Title of host publication2018 24th International Conference on Pattern Recognition (ICPR)
PublisherIEEE
Pages1468-1473
Number of pages6
ISBN (Electronic)978-1-5386-3788-3
ISBN (Print)978-1-5386-3789-0
DOIs
Publication statusPublished - 2018
Event2018 24th International Conference on Pattern Recognition, ICPR 2018 - Beijing, China
Duration: 20 Aug 201824 Aug 2018

Conference

Conference2018 24th International Conference on Pattern Recognition, ICPR 2018
CountryChina
CityBeijing
Period20/08/1824/08/18

Fingerprint Dive into the research topics of 'Effects of sampling skewness of the importance-weighted risk estimator on model selection'. Together they form a unique fingerprint.

Cite this