The Power of Universal Contextualized Protein Embeddings in Cross-species Protein Function Prediction

Irene van den Bent, Stavros Makrodimitris, Marcel Reinders*

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

2 Citations (Scopus)
11 Downloads (Pure)


Computationally annotating proteins with a molecular function is a difficult problem that is made even harder due to the limited amount of available labeled protein training data. Unsupervised protein embeddings partly circumvent this limitation by learning a universal protein representation from many unlabeled sequences. Such embeddings incorporate contextual information of amino acids, thereby modeling the underlying principles of protein sequences insensitive to the context of species. We used an existing pre-trained protein embedding method and subjected its molecular function prediction performance to detailed characterization, first to advance the understanding of protein language models, and second to determine areas of improvement. Then, we applied the model in a transfer learning task by training a function predictor based on the embeddings of annotated protein sequences of one training species and making predictions on the proteins of several test species with varying evolutionary distance. We show that this approach successfully generalizes knowledge about protein function from one eukaryotic species to various other species, outperforming both an alignment-based and a supervised-learning-based baseline. This implies that such a method could be effective for molecular function prediction in inadequately annotated species from understudied taxonomic kingdoms.

Original languageEnglish
Number of pages15
JournalEvolutionary Bioinformatics
Publication statusPublished - 2021


  • annotating evolutionary distant proteins
  • protein embedding
  • Protein function prediction
  • protein language models
  • transfer learning


Dive into the research topics of 'The Power of Universal Contextualized Protein Embeddings in Cross-species Protein Function Prediction'. Together they form a unique fingerprint.

Cite this