Benchmarks · protein-ml/esm2-150m-protein-molecular-function

ProteinGym protein-ml/proteingym

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.466 [0.294, 0.624]	100
CASP3_HUMAN_Roychowdhury_2020	0.222 [0.019, 0.405]	100
CCR5_HUMAN_Gill_2023	0.177 [−0.020, 0.381]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.442 [0.228, 0.609]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.583 [0.403, 0.724]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.505 [0.336, 0.649]	100
MLAC_ECOLI_MacRae_2023	0.374 [0.166, 0.557]	100
MTH3_HAEAE_RockahShmuel_2015	0.491 [0.305, 0.648]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.860 [0.789, 0.905]	100
SPG1_STRSG_Olson_2014	0.701 [0.559, 0.814]	100

0.482

[0.426, 0.528]

1000

15 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.696 [0.594, 0.798]	100
CASP3_HUMAN_Roychowdhury_2020	0.624 [0.515, 0.731]	100
CCR5_HUMAN_Gill_2023	0.591 [0.478, 0.702]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.727 [0.616, 0.818]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.838 [0.734, 0.921]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.766 [0.664, 0.859]	100
MLAC_ECOLI_MacRae_2023	0.685 [0.580, 0.788]	100
MTH3_HAEAE_RockahShmuel_2015	0.748 [0.627, 0.854]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.967 [0.927, 0.994]	100
SPG1_STRSG_Olson_2014	0.970 [0.930, 0.994]	100

0.761

[0.731, 0.790]

1000

15 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.300 [0.102, 0.501]	100
CASP3_HUMAN_Roychowdhury_2020	0.140 [−0.098, 0.321]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.099, 0.320]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.281 [0.082, 0.490]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.436 [0.231, 0.577]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.401 [0.208, 0.535]	100
MLAC_ECOLI_MacRae_2023	0.240 [0.061, 0.461]	100
MTH3_HAEAE_RockahShmuel_2015	0.328 [0.134, 0.494]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.776 [0.625, 0.869]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.531]	100

0.347

[0.282, 0.392]

1000

15 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.734 [0.624, 0.897]	100
CASP3_HUMAN_Roychowdhury_2020	0.709 [0.613, 0.813]	100
CCR5_HUMAN_Gill_2023	0.768 [0.536, 0.859]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.722 [0.547, 0.882]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.932 [0.859, 0.969]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.702 [0.552, 0.859]	100
MLAC_ECOLI_MacRae_2023	0.818 [0.694, 0.937]	100
MTH3_HAEAE_RockahShmuel_2015	0.591 [0.377, 0.796]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.907 [0.842, 0.963]	100
SPG1_STRSG_Olson_2014	0.845 [0.771, 0.943]	100

0.773

[0.732, 0.819]

1000

16 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.200 [0.000, 0.500]	100
CASP3_HUMAN_Roychowdhury_2020	0.000 [0.000, 0.308]	100
CCR5_HUMAN_Gill_2023	0.100 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.500]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.400 [0.091, 0.700]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.300 [0.083, 0.600]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.000, 0.545]	100
MTH3_HAEAE_RockahShmuel_2015	0.091 [0.000, 0.364]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.400 [0.167, 0.727]	100
SPG1_STRSG_Olson_2014	0.300 [0.091, 0.636]	100

0.219

[0.165, 0.343]

1000

12 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

ProteinGym_v1 protein-ml/proteingym-v1

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.466 [0.294, 0.624]	100
CASP3_HUMAN_Roychowdhury_2020	0.222 [0.019, 0.405]	100
CCR5_HUMAN_Gill_2023	0.177 [−0.020, 0.381]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.442 [0.228, 0.609]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.583 [0.403, 0.724]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.505 [0.336, 0.649]	100
MLAC_ECOLI_MacRae_2023	0.374 [0.166, 0.557]	100
MTH3_HAEAE_RockahShmuel_2015	0.491 [0.305, 0.648]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.860 [0.789, 0.905]	100
SPG1_STRSG_Olson_2014	0.701 [0.559, 0.814]	100

0.482

[0.426, 0.528]

1000

15 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.696 [0.594, 0.798]	100
CASP3_HUMAN_Roychowdhury_2020	0.624 [0.515, 0.731]	100
CCR5_HUMAN_Gill_2023	0.591 [0.478, 0.702]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.727 [0.616, 0.818]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.838 [0.734, 0.921]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.766 [0.664, 0.859]	100
MLAC_ECOLI_MacRae_2023	0.685 [0.580, 0.788]	100
MTH3_HAEAE_RockahShmuel_2015	0.748 [0.627, 0.854]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.967 [0.927, 0.994]	100
SPG1_STRSG_Olson_2014	0.970 [0.930, 0.994]	100

0.761

[0.731, 0.790]

1000

15 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.300 [0.102, 0.501]	100
CASP3_HUMAN_Roychowdhury_2020	0.140 [−0.098, 0.321]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.099, 0.320]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.281 [0.082, 0.490]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.436 [0.231, 0.577]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.401 [0.208, 0.535]	100
MLAC_ECOLI_MacRae_2023	0.240 [0.061, 0.461]	100
MTH3_HAEAE_RockahShmuel_2015	0.328 [0.134, 0.494]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.776 [0.625, 0.869]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.531]	100

0.347

[0.282, 0.392]

1000

15 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.734 [0.624, 0.897]	100
CASP3_HUMAN_Roychowdhury_2020	0.709 [0.613, 0.813]	100
CCR5_HUMAN_Gill_2023	0.768 [0.536, 0.859]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.722 [0.547, 0.882]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.932 [0.859, 0.969]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.702 [0.552, 0.859]	100
MLAC_ECOLI_MacRae_2023	0.818 [0.694, 0.937]	100
MTH3_HAEAE_RockahShmuel_2015	0.591 [0.377, 0.796]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.907 [0.842, 0.963]	100
SPG1_STRSG_Olson_2014	0.845 [0.771, 0.943]	100

0.773

[0.732, 0.819]

1000

16 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.200 [0.000, 0.500]	100
CASP3_HUMAN_Roychowdhury_2020	0.000 [0.000, 0.308]	100
CCR5_HUMAN_Gill_2023	0.100 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.500]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.400 [0.091, 0.700]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.300 [0.083, 0.600]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.000, 0.545]	100
MTH3_HAEAE_RockahShmuel_2015	0.091 [0.000, 0.364]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.400 [0.167, 0.727]	100
SPG1_STRSG_Olson_2014	0.300 [0.091, 0.636]	100

0.219

[0.165, 0.343]

1000

12 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

tape protein-ml/tape

Secondary structure (Q3) protein-ml/secondary-structure-q3

Drawn from: tape

atlas-n100-v2

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q3 accuracy (per-residue probe) ↑ higher is better · probe	0.749	[0.735, 0.763]	100	9 / 17	0.813	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
Q8 accuracy (per-residue probe) ↑ higher is better · probe	0.586	[0.565, 0.605]	100	9 / 17	0.663	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board

UniProt Knowledgebase protein-ml/uniprot-knowledgebase

GO function prediction (Swiss-Prot 2025H2) protein-ml/protein-function-go-prediction

Drawn from: UniProt Knowledgebase

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Fmax (molecular function, native head) ↑ higher is better · zero-shot	0.868 best	[0.804, 0.928]	100	1 / 1	0.868	reproduced	unknown	Board
Fmax (molecular function, probe) ↑ higher is better · probe	0.844 best	[0.786, 0.894]	100	1 / 17	0.844	reproduced	unknown	Board
Fmax (biological process, probe) ↑ higher is better · probe	0.527	[0.426, 0.626]	100	2 / 17	0.538	reproduced	unknown	Board
Fmax (cellular component, probe) ↑ higher is better · probe	0.727	[0.672, 0.784]	100	11 / 17	0.756	reproduced	unknown	Board
Micro AUPR (molecular function, native head) ↑ higher is better · zero-shot · secondary	0.909 best	[0.847, 0.954]	100	—	0.909	reproduced	unknown	Board
Smin (molecular function, native head) ↓ lower is better · bits · zero-shot · secondary	1.416 best	[0.778, 2.072]	100	—	1.416	reproduced	unknown	Board
Micro AUPR (molecular function, probe) ↑ higher is better · probe · secondary	0.875 best	[0.813, 0.934]	100	—	0.875	reproduced	unknown	Board
Smin (molecular function, probe) ↓ lower is better · bits · probe · secondary	1.727 best	[1.080, 2.467]	100	—	1.727	reproduced	unknown	Board
Micro AUPR (biological process, probe) ↑ higher is better · probe · secondary	0.500	[0.361, 0.652]	100	—	0.595	reproduced	unknown	Board
Smin (biological process, probe) ↓ lower is better · bits · probe · secondary	5.626	[4.245, 6.779]	100	—	4.842	reproduced	unknown	Board
Micro AUPR (cellular component, probe) ↑ higher is better · probe · secondary	0.693	[0.623, 0.763]	100	—	0.758	reproduced	unknown	Board
Smin (cellular component, probe) ↓ lower is better · bits · probe · secondary	3.365	[2.553, 4.126]	100	—	3.092	reproduced	unknown	Board

Other Benchmarks drawn from no hub dataset you can see

Protein membrane association (DeepLoc) protein-ml/protein-membrane-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q2 accuracy (logistic probe) ↑ higher is better · probe	0.849	[0.833, 0.866]	1728	6 / 17	0.888	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.690	[0.657, 0.726]	1728	—	0.771	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board

Protein stability ΔΔG (Mega-scale) protein-ml/protein-stability-ddg

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Spearman ρ (ridge probe) ↑ higher is better · probe	0.116	[−0.121, 0.315]	100	10 / 17	0.297	reproduced	unknown	Board
RMSE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe	1.102	[0.876, 1.302]	100	4 / 17	1.095	reproduced	unknown	Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary	0.143	[−0.081, 0.297]	100	—	0.349	reproduced	unknown	Board
MAE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe · secondary	0.844	[0.702, 1.001]	100	—	0.819	reproduced	unknown	Board

Protein subcellular localization (DeepLoc) protein-ml/protein-subcellular-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.677	[0.662, 0.693]	2742	8 / 17	0.801	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.480	[0.461, 0.500]	2742	10 / 17	0.640	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.607	[0.589, 0.626]	2742	—	0.758	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board