Document performance prediction for automatic text classification

Gustavo Penha; Raphael Campos; Sérgio Canuto; Marcos André Gonçalves; Rodrygo L.T. Santos

doi:10.1007/978-3-030-15719-7_17

Document performance prediction for automatic text classification

Gustavo Penha, Raphael Campos, Sérgio Canuto, Marcos André Gonçalves, Rodrygo L.T. Santos^*

^*Corresponding author for this work

Web Information Systems

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

1 Citation (Scopus)

4 Downloads (Pure)

Abstract

Query performance prediction (QPP) is a fundamental task in information retrieval, which concerns predicting the effectiveness of a ranking model for a given query in the absence of relevance information. Despite being an active research area, this task has not yet been explored in the context of automatic text classification. In this paper, we study the task of predicting the effectiveness of a classifier for a given document, which we refer to as document performance prediction (DPP). Our experiments on several text classification datasets for both categorization and sentiment analysis attest the effectiveness and complementarity of several DPP inspired by related QPP approaches. Finally, we also explore the usefulness of DPP for improving the classification itself, by using them as additional features in a classification ensemble.

Original language	English
Title of host publication	Advances in Information Retrieval
Subtitle of host publication	41st European Conference on IR Research, ECIR 2019
Editors	Leif Azzopardi, Benno Stein, Norbert Fuhr, Claudia Hauff, Philipp Mayr, Djoerd Hiemstra
Place of Publication	Cham
Publisher	Springer
Pages	132-139
Number of pages	8
ISBN (Electronic)	978-3-030-15719-7
ISBN (Print)	978-3-030-15718-0
DOIs	https://doi.org/10.1007/978-3-030-15719-7_17
Publication status	Published - 2019
Event	41st European Conference on Information Retrieval, ECIR 2019 - Cologne, Germany Duration: 14 Apr 2019 → 18 Apr 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11438 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	41st European Conference on Information Retrieval, ECIR 2019
Country/Territory	Germany
City	Cologne
Period	14/04/19 → 18/04/19

Keywords

Automatic text classification
Performance prediction

Access to Document

10.1007/978-3-030-15719-7_17

Cite this

Penha, G., Campos, R., Canuto, S., Gonçalves, M. A., & Santos, R. L. T. (2019). Document performance prediction for automatic text classification. In L. Azzopardi, B. Stein, N. Fuhr, C. Hauff, P. Mayr, & D. Hiemstra (Eds.), Advances in Information Retrieval : 41st European Conference on IR Research, ECIR 2019 (pp. 132-139). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11438 LNCS). Springer. https://doi.org/10.1007/978-3-030-15719-7_17

Penha, Gustavo ; Campos, Raphael ; Canuto, Sérgio et al. / Document performance prediction for automatic text classification. Advances in Information Retrieval : 41st European Conference on IR Research, ECIR 2019. editor / Leif Azzopardi ; Benno Stein ; Norbert Fuhr ; Claudia Hauff ; Philipp Mayr ; Djoerd Hiemstra. Cham : Springer, 2019. pp. 132-139 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{5592ec0dcb11488a91d47f60e98d9a3c,

title = "Document performance prediction for automatic text classification",

abstract = "Query performance prediction (QPP) is a fundamental task in information retrieval, which concerns predicting the effectiveness of a ranking model for a given query in the absence of relevance information. Despite being an active research area, this task has not yet been explored in the context of automatic text classification. In this paper, we study the task of predicting the effectiveness of a classifier for a given document, which we refer to as document performance prediction (DPP). Our experiments on several text classification datasets for both categorization and sentiment analysis attest the effectiveness and complementarity of several DPP inspired by related QPP approaches. Finally, we also explore the usefulness of DPP for improving the classification itself, by using them as additional features in a classification ensemble.",

keywords = "Automatic text classification, Performance prediction",

author = "Gustavo Penha and Raphael Campos and S{\'e}rgio Canuto and Gon{\c c}alves, {Marcos Andr{\'e}} and Santos, {Rodrygo L.T.}",

year = "2019",

doi = "10.1007/978-3-030-15719-7_17",

language = "English",

isbn = "978-3-030-15718-0",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "132--139",

editor = "Leif Azzopardi and Benno Stein and Norbert Fuhr and Claudia Hauff and Philipp Mayr and Djoerd Hiemstra",

booktitle = "Advances in Information Retrieval",

note = "41st European Conference on Information Retrieval, ECIR 2019 ; Conference date: 14-04-2019 Through 18-04-2019",

}

Penha, G, Campos, R, Canuto, S, Gonçalves, MA & Santos, RLT 2019, Document performance prediction for automatic text classification. in L Azzopardi, B Stein, N Fuhr, C Hauff, P Mayr & D Hiemstra (eds), Advances in Information Retrieval : 41st European Conference on IR Research, ECIR 2019. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11438 LNCS, Springer, Cham, pp. 132-139, 41st European Conference on Information Retrieval, ECIR 2019, Cologne, Germany, 14/04/19. https://doi.org/10.1007/978-3-030-15719-7_17

Document performance prediction for automatic text classification. / Penha, Gustavo; Campos, Raphael; Canuto, Sérgio et al.
Advances in Information Retrieval : 41st European Conference on IR Research, ECIR 2019. ed. / Leif Azzopardi; Benno Stein; Norbert Fuhr; Claudia Hauff; Philipp Mayr; Djoerd Hiemstra. Cham: Springer, 2019. p. 132-139 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11438 LNCS).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Document performance prediction for automatic text classification

AU - Penha, Gustavo

AU - Campos, Raphael

AU - Canuto, Sérgio

AU - Gonçalves, Marcos André

AU - Santos, Rodrygo L.T.

PY - 2019

Y1 - 2019

N2 - Query performance prediction (QPP) is a fundamental task in information retrieval, which concerns predicting the effectiveness of a ranking model for a given query in the absence of relevance information. Despite being an active research area, this task has not yet been explored in the context of automatic text classification. In this paper, we study the task of predicting the effectiveness of a classifier for a given document, which we refer to as document performance prediction (DPP). Our experiments on several text classification datasets for both categorization and sentiment analysis attest the effectiveness and complementarity of several DPP inspired by related QPP approaches. Finally, we also explore the usefulness of DPP for improving the classification itself, by using them as additional features in a classification ensemble.

AB - Query performance prediction (QPP) is a fundamental task in information retrieval, which concerns predicting the effectiveness of a ranking model for a given query in the absence of relevance information. Despite being an active research area, this task has not yet been explored in the context of automatic text classification. In this paper, we study the task of predicting the effectiveness of a classifier for a given document, which we refer to as document performance prediction (DPP). Our experiments on several text classification datasets for both categorization and sentiment analysis attest the effectiveness and complementarity of several DPP inspired by related QPP approaches. Finally, we also explore the usefulness of DPP for improving the classification itself, by using them as additional features in a classification ensemble.

KW - Automatic text classification

KW - Performance prediction

UR - http://www.scopus.com/inward/record.url?scp=85064858139&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-15719-7_17

DO - 10.1007/978-3-030-15719-7_17

M3 - Conference contribution

AN - SCOPUS:85064858139

SN - 978-3-030-15718-0

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 132

EP - 139

BT - Advances in Information Retrieval

A2 - Azzopardi, Leif

A2 - Stein, Benno

A2 - Fuhr, Norbert

A2 - Hauff, Claudia

A2 - Mayr, Philipp

A2 - Hiemstra, Djoerd

PB - Springer

CY - Cham

T2 - 41st European Conference on Information Retrieval, ECIR 2019

Y2 - 14 April 2019 through 18 April 2019

ER -

Penha G, Campos R, Canuto S, Gonçalves MA, Santos RLT. Document performance prediction for automatic text classification. In Azzopardi L, Stein B, Fuhr N, Hauff C, Mayr P, Hiemstra D, editors, Advances in Information Retrieval : 41st European Conference on IR Research, ECIR 2019. Cham: Springer. 2019. p. 132-139. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-15719-7_17

Document performance prediction for automatic text classification

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this