ManyTypes4Py: A benchmark python dataset for machine learning-based type inference

Amir M. Mir; Evaldas Latoskinas; Georgios Gousios

doi:10.1109/MSR52588.2021.00079

ManyTypes4Py: A benchmark python dataset for machine learning-based type inference

Amir M. Mir, Evaldas Latoskinas, Georgios Gousios

Software Engineering

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

9 Citations (Scopus)

77 Downloads (Pure)

Abstract

In this paper, we present ManyTypes4Py, a large Python dataset for machine learning (ML)-based type inference. The dataset contains a total of 5, 382 Python projects with more than 869K type annotations. Duplicate source code files were removed to eliminate the negative effect of the duplication bias. To facilitate training and evaluation of ML models, the dataset was split into training, validation and test sets by files. To extract type information from abstract syntax trees (ASTs), a light-weight static analyzer pipeline is developed and accompanied with the dataset. Using this pipeline, the collected Python projects were analyzed and the results of the AST analysis were stored in JSON-formatted files. The ManyTypes4Py dataset is shared on zenodo and its tools are publicly available on GitHub.

Original language	English
Title of host publication	2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021
Subtitle of host publication	Proceedings
Editors	L. O'Conner
Place of Publication	Piscataway
Publisher	IEEE
Pages	585-589
Number of pages	5
ISBN (Electronic)	978-1-7281-8710-5
ISBN (Print)	978-1-6654-2985-6
DOIs	https://doi.org/10.1109/MSR52588.2021.00079
Publication status	Published - 2021
Event	2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) - Virtual at Madrid, Spain Duration: 17 May 2021 → 19 May 2021 Conference number: 18th

Publication series

Name	2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021)
ISSN (Print)	2160-1852

Conference

Conference	2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)
Abbreviated title	MSR21
Country/Territory	Spain
City	Virtual at Madrid
Period	17/05/21 → 19/05/21

Bibliographical note

Accepted author manuscript

Keywords

Machine Learning
Python
Static Analysis
Type Annotations
Type Inference

Access to Document

10.1109/MSR52588.2021.00079

mt4py_paperAccepted author manuscript, 543 KB

Cite this

Mir, A. M., Latoskinas, E., & Gousios, G. (2021). ManyTypes4Py: A benchmark python dataset for machine learning-based type inference. In L. O'Conner (Ed.), 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021: Proceedings (pp. 585-589). Article 9463150 (2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021)). IEEE. https://doi.org/10.1109/MSR52588.2021.00079

Mir, Amir M. ; Latoskinas, Evaldas ; Gousios, Georgios. / ManyTypes4Py : A benchmark python dataset for machine learning-based type inference. 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021: Proceedings. editor / L. O'Conner. Piscataway : IEEE, 2021. pp. 585-589 (2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021)).

@inproceedings{899fabe315fc40b5afb459444d6f05ea,

title = "ManyTypes4Py: A benchmark python dataset for machine learning-based type inference",

abstract = "In this paper, we present ManyTypes4Py, a large Python dataset for machine learning (ML)-based type inference. The dataset contains a total of 5, 382 Python projects with more than 869K type annotations. Duplicate source code files were removed to eliminate the negative effect of the duplication bias. To facilitate training and evaluation of ML models, the dataset was split into training, validation and test sets by files. To extract type information from abstract syntax trees (ASTs), a light-weight static analyzer pipeline is developed and accompanied with the dataset. Using this pipeline, the collected Python projects were analyzed and the results of the AST analysis were stored in JSON-formatted files. The ManyTypes4Py dataset is shared on zenodo and its tools are publicly available on GitHub. ",

keywords = "Machine Learning, Python, Static Analysis, Type Annotations, Type Inference",

author = "Mir, {Amir M.} and Evaldas Latoskinas and Georgios Gousios",

note = "Accepted author manuscript; 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), MSR21 ; Conference date: 17-05-2021 Through 19-05-2021",

year = "2021",

doi = "10.1109/MSR52588.2021.00079",

language = "English",

isbn = "978-1-6654-2985-6",

series = "2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021)",

publisher = "IEEE",

pages = "585--589",

editor = "L. O'Conner",

booktitle = "2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021",

address = "United States",

}

Mir, AM, Latoskinas, E & Gousios, G 2021, ManyTypes4Py: A benchmark python dataset for machine learning-based type inference. in L O'Conner (ed.), 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021: Proceedings., 9463150, 2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021), IEEE, Piscataway, pp. 585-589, 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), Virtual at Madrid, Spain, 17/05/21. https://doi.org/10.1109/MSR52588.2021.00079

ManyTypes4Py: A benchmark python dataset for machine learning-based type inference. / Mir, Amir M.; Latoskinas, Evaldas; Gousios, Georgios.
2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021: Proceedings. ed. / L. O'Conner. Piscataway: IEEE, 2021. p. 585-589 9463150 (2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021)).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - ManyTypes4Py

T2 - 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

AU - Mir, Amir M.

AU - Latoskinas, Evaldas

AU - Gousios, Georgios

N1 - Conference code: 18th

PY - 2021

Y1 - 2021

N2 - In this paper, we present ManyTypes4Py, a large Python dataset for machine learning (ML)-based type inference. The dataset contains a total of 5, 382 Python projects with more than 869K type annotations. Duplicate source code files were removed to eliminate the negative effect of the duplication bias. To facilitate training and evaluation of ML models, the dataset was split into training, validation and test sets by files. To extract type information from abstract syntax trees (ASTs), a light-weight static analyzer pipeline is developed and accompanied with the dataset. Using this pipeline, the collected Python projects were analyzed and the results of the AST analysis were stored in JSON-formatted files. The ManyTypes4Py dataset is shared on zenodo and its tools are publicly available on GitHub.

AB - In this paper, we present ManyTypes4Py, a large Python dataset for machine learning (ML)-based type inference. The dataset contains a total of 5, 382 Python projects with more than 869K type annotations. Duplicate source code files were removed to eliminate the negative effect of the duplication bias. To facilitate training and evaluation of ML models, the dataset was split into training, validation and test sets by files. To extract type information from abstract syntax trees (ASTs), a light-weight static analyzer pipeline is developed and accompanied with the dataset. Using this pipeline, the collected Python projects were analyzed and the results of the AST analysis were stored in JSON-formatted files. The ManyTypes4Py dataset is shared on zenodo and its tools are publicly available on GitHub.

KW - Machine Learning

KW - Python

KW - Static Analysis

KW - Type Annotations

KW - Type Inference

UR - http://www.scopus.com/inward/record.url?scp=85113652650&partnerID=8YFLogxK

U2 - 10.1109/MSR52588.2021.00079

DO - 10.1109/MSR52588.2021.00079

M3 - Conference contribution

AN - SCOPUS:85113652650

SN - 978-1-6654-2985-6

T3 - 2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021)

SP - 585

EP - 589

BT - 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021

A2 - O'Conner, L.

PB - IEEE

CY - Piscataway

Y2 - 17 May 2021 through 19 May 2021

ER -

Mir AM, Latoskinas E, Gousios G. ManyTypes4Py: A benchmark python dataset for machine learning-based type inference. In O'Conner L, editor, 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021: Proceedings. Piscataway: IEEE. 2021. p. 585-589. 9463150. (2021 IEEE/ACM 18TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2021)). doi: 10.1109/MSR52588.2021.00079

ManyTypes4Py: A benchmark python dataset for machine learning-based type inference

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this