SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals

Zijian Zhang; Vinay Setty; Avishek Anand

doi:10.1145/3477495.3531677

SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals

Zijian Zhang, Vinay Setty, Avishek Anand

Web Information Systems

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

3 Citations (Scopus)

23 Downloads (Pure)

Abstract

We introduce SparCAssist, a general-purpose risk assessment tool for the machine learning models trained for language tasks. It evaluates models' risk by inspecting their behavior on counterfactuals, namely out-of-distribution instances generated based on the given data instance. The counterfactuals are generated by replacing tokens in rational subsequences identified by ExPred, while the replacements are retrieved using HotFlip or the Masked-Language-Model-based algorithms. The main purpose of our system is to help the human annotators to assess the model's risk on deployment. The counterfactual instances generated during the assessment are the by-product and can be used to train more robust NLP models in the future.

Original language	English
Title of host publication	SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
Publisher	Association for Computing Machinery (ACM)
Pages	3219-3223
Number of pages	5
ISBN (Electronic)	978-1-4503-8732-3
DOIs	https://doi.org/10.1145/3477495.3531677
Publication status	Published - 2022
Event	45th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022 - Madrid, Spain Duration: 11 Jul 2022 → 15 Jul 2022

Publication series

Name	SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

Conference	45th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022
Country/Territory	Spain
City	Madrid
Period	11/07/22 → 15/07/22

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

counterfactual interpretation
data-annotation tools
human-in-the-loop machine learning
interpretable machine learning

Access to Document

10.1145/3477495.3531677

3477495.3531677Final published version, 1.2 MB

Cite this

Zhang, Z., Setty, V., & Anand, A. (2022). SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals. In SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 3219-3223). (SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval). Association for Computing Machinery (ACM). https://doi.org/10.1145/3477495.3531677

Zhang, Zijian ; Setty, Vinay ; Anand, Avishek. / SparCAssist : A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals. SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery (ACM), 2022. pp. 3219-3223 (SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval).

@inproceedings{553f3377f4bd4b19a0ee974d9947f62b,

title = "SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals",

abstract = "We introduce SparCAssist, a general-purpose risk assessment tool for the machine learning models trained for language tasks. It evaluates models' risk by inspecting their behavior on counterfactuals, namely out-of-distribution instances generated based on the given data instance. The counterfactuals are generated by replacing tokens in rational subsequences identified by ExPred, while the replacements are retrieved using HotFlip or the Masked-Language-Model-based algorithms. The main purpose of our system is to help the human annotators to assess the model's risk on deployment. The counterfactual instances generated during the assessment are the by-product and can be used to train more robust NLP models in the future.",

keywords = "counterfactual interpretation, data-annotation tools, human-in-the-loop machine learning, interpretable machine learning",

author = "Zijian Zhang and Vinay Setty and Avishek Anand",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 45th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022 ; Conference date: 11-07-2022 Through 15-07-2022",

year = "2022",

doi = "10.1145/3477495.3531677",

language = "English",

series = "SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval",

publisher = "Association for Computing Machinery (ACM)",

pages = "3219--3223",

booktitle = "SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval",

address = "United States",

}

Zhang, Z, Setty, V & Anand, A 2022, SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals. in SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery (ACM), pp. 3219-3223, 45th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022, Madrid, Spain, 11/07/22. https://doi.org/10.1145/3477495.3531677

SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals. / Zhang, Zijian; Setty, Vinay; Anand, Avishek.
SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery (ACM), 2022. p. 3219-3223 (SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - SparCAssist

T2 - 45th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022

AU - Zhang, Zijian

AU - Setty, Vinay

AU - Anand, Avishek

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2022

Y1 - 2022

N2 - We introduce SparCAssist, a general-purpose risk assessment tool for the machine learning models trained for language tasks. It evaluates models' risk by inspecting their behavior on counterfactuals, namely out-of-distribution instances generated based on the given data instance. The counterfactuals are generated by replacing tokens in rational subsequences identified by ExPred, while the replacements are retrieved using HotFlip or the Masked-Language-Model-based algorithms. The main purpose of our system is to help the human annotators to assess the model's risk on deployment. The counterfactual instances generated during the assessment are the by-product and can be used to train more robust NLP models in the future.

AB - We introduce SparCAssist, a general-purpose risk assessment tool for the machine learning models trained for language tasks. It evaluates models' risk by inspecting their behavior on counterfactuals, namely out-of-distribution instances generated based on the given data instance. The counterfactuals are generated by replacing tokens in rational subsequences identified by ExPred, while the replacements are retrieved using HotFlip or the Masked-Language-Model-based algorithms. The main purpose of our system is to help the human annotators to assess the model's risk on deployment. The counterfactual instances generated during the assessment are the by-product and can be used to train more robust NLP models in the future.

KW - counterfactual interpretation

KW - data-annotation tools

KW - human-in-the-loop machine learning

KW - interpretable machine learning

UR - http://www.scopus.com/inward/record.url?scp=85135007122&partnerID=8YFLogxK

U2 - 10.1145/3477495.3531677

DO - 10.1145/3477495.3531677

M3 - Conference contribution

AN - SCOPUS:85135007122

T3 - SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

SP - 3219

EP - 3223

BT - SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

PB - Association for Computing Machinery (ACM)

Y2 - 11 July 2022 through 15 July 2022

ER -

Zhang Z, Setty V, Anand A. SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals. In SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery (ACM). 2022. p. 3219-3223. (SIGIR 2022 - Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval). doi: 10.1145/3477495.3531677

SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this