Benchmarks · protein-ml/esm2-35m-protein-biological-process

ProteinGym protein-ml/proteingym

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.587 [0.410, 0.721]	100
CASP3_HUMAN_Roychowdhury_2020	0.212 [0.013, 0.399]	100
CCR5_HUMAN_Gill_2023	0.232 [0.034, 0.412]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.440 [0.260, 0.579]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.730 [0.585, 0.839]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.491 [0.323, 0.639]	100
MLAC_ECOLI_MacRae_2023	0.317 [0.121, 0.505]	100
MTH3_HAEAE_RockahShmuel_2015	0.409 [0.212, 0.579]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.845 [0.767, 0.892]	100
SPG1_STRSG_Olson_2014	0.705 [0.577, 0.799]	100

0.497

[0.438, 0.541]

1000

14 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.821 [0.717, 0.909]	100
CASP3_HUMAN_Roychowdhury_2020	0.605 [0.494, 0.711]	100
CCR5_HUMAN_Gill_2023	0.597 [0.485, 0.702]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.752 [0.646, 0.847]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.929 [0.845, 0.983]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.749 [0.644, 0.843]	100
MLAC_ECOLI_MacRae_2023	0.668 [0.554, 0.780]	100
MTH3_HAEAE_RockahShmuel_2015	0.667 [0.535, 0.796]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.937 [0.884, 0.977]	100
SPG1_STRSG_Olson_2014	0.904 [0.821, 0.967]	100

0.763

[0.731, 0.795]

1000

14 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.500 [0.279, 0.740]	100
CASP3_HUMAN_Roychowdhury_2020	0.180 [−0.081, 0.341]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.100, 0.321]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.401 [0.201, 0.602]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.611 [0.439, 0.735]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.356 [0.187, 0.535]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.040, 0.443]	100
MTH3_HAEAE_RockahShmuel_2015	0.281 [0.074, 0.462]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.735 [0.542, 0.841]	100
SPG1_STRSG_Olson_2014	0.403 [0.280, 0.516]	100

0.383

[0.309, 0.435]

1000

13 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.728 [0.619, 0.895]	100
CASP3_HUMAN_Roychowdhury_2020	0.712 [0.583, 0.814]	100
CCR5_HUMAN_Gill_2023	0.837 [0.650, 0.904]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.646 [0.486, 0.794]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.875 [0.781, 0.958]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.706 [0.576, 0.833]	100
MLAC_ECOLI_MacRae_2023	0.767 [0.644, 0.874]	100
MTH3_HAEAE_RockahShmuel_2015	0.542 [0.310, 0.783]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.880 [0.795, 0.954]	100
SPG1_STRSG_Olson_2014	0.880 [0.835, 0.939]	100

0.757

[0.714, 0.800]

1000

17 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.100 [0.000, 0.400]	100
CASP3_HUMAN_Roychowdhury_2020	0.200 [0.000, 0.455]	100
CCR5_HUMAN_Gill_2023	0.200 [0.000, 0.500]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.000 [0.000, 0.333]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.300 [0.083, 0.667]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.200 [0.000, 0.500]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.000, 0.455]	100
MTH3_HAEAE_RockahShmuel_2015	0.000 [0.000, 0.251]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.300 [0.000, 0.700]	100
SPG1_STRSG_Olson_2014	0.000 [0.000, 0.400]	100

0.150

[0.100, 0.255]

1000

17 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

ProteinGym_v1 protein-ml/proteingym-v1

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.587 [0.410, 0.721]	100
CASP3_HUMAN_Roychowdhury_2020	0.212 [0.013, 0.399]	100
CCR5_HUMAN_Gill_2023	0.232 [0.034, 0.412]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.440 [0.260, 0.579]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.730 [0.585, 0.839]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.491 [0.323, 0.639]	100
MLAC_ECOLI_MacRae_2023	0.317 [0.121, 0.505]	100
MTH3_HAEAE_RockahShmuel_2015	0.409 [0.212, 0.579]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.845 [0.767, 0.892]	100
SPG1_STRSG_Olson_2014	0.705 [0.577, 0.799]	100

0.497

[0.438, 0.541]

1000

14 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.821 [0.717, 0.909]	100
CASP3_HUMAN_Roychowdhury_2020	0.605 [0.494, 0.711]	100
CCR5_HUMAN_Gill_2023	0.597 [0.485, 0.702]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.752 [0.646, 0.847]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.929 [0.845, 0.983]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.749 [0.644, 0.843]	100
MLAC_ECOLI_MacRae_2023	0.668 [0.554, 0.780]	100
MTH3_HAEAE_RockahShmuel_2015	0.667 [0.535, 0.796]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.937 [0.884, 0.977]	100
SPG1_STRSG_Olson_2014	0.904 [0.821, 0.967]	100

0.763

[0.731, 0.795]

1000

14 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.500 [0.279, 0.740]	100
CASP3_HUMAN_Roychowdhury_2020	0.180 [−0.081, 0.341]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.100, 0.321]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.401 [0.201, 0.602]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.611 [0.439, 0.735]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.356 [0.187, 0.535]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.040, 0.443]	100
MTH3_HAEAE_RockahShmuel_2015	0.281 [0.074, 0.462]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.735 [0.542, 0.841]	100
SPG1_STRSG_Olson_2014	0.403 [0.280, 0.516]	100

0.383

[0.309, 0.435]

1000

13 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.728 [0.619, 0.895]	100
CASP3_HUMAN_Roychowdhury_2020	0.712 [0.583, 0.814]	100
CCR5_HUMAN_Gill_2023	0.837 [0.650, 0.904]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.646 [0.486, 0.794]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.875 [0.781, 0.958]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.706 [0.576, 0.833]	100
MLAC_ECOLI_MacRae_2023	0.767 [0.644, 0.874]	100
MTH3_HAEAE_RockahShmuel_2015	0.542 [0.310, 0.783]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.880 [0.795, 0.954]	100
SPG1_STRSG_Olson_2014	0.880 [0.835, 0.939]	100

0.757

[0.714, 0.800]

1000

17 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.100 [0.000, 0.400]	100
CASP3_HUMAN_Roychowdhury_2020	0.200 [0.000, 0.455]	100
CCR5_HUMAN_Gill_2023	0.200 [0.000, 0.500]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.000 [0.000, 0.333]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.300 [0.083, 0.667]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.200 [0.000, 0.500]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.000, 0.455]	100
MTH3_HAEAE_RockahShmuel_2015	0.000 [0.000, 0.251]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.300 [0.000, 0.700]	100
SPG1_STRSG_Olson_2014	0.000 [0.000, 0.400]	100

0.150

[0.100, 0.255]

1000

17 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

tape protein-ml/tape

Secondary structure (Q3) protein-ml/secondary-structure-q3

Drawn from: tape

atlas-n100-v2

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q3 accuracy (per-residue probe) ↑ higher is better · probe	0.674	[0.658, 0.689]	100	14 / 17	0.813	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
Q8 accuracy (per-residue probe) ↑ higher is better · probe	0.519	[0.501, 0.539]	100	14 / 17	0.663	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board

UniProt Knowledgebase protein-ml/uniprot-knowledgebase

GO function prediction (Swiss-Prot 2025H2) protein-ml/protein-function-go-prediction

Drawn from: UniProt Knowledgebase

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Fmax (biological process, native head) ↑ higher is better · zero-shot	0.516 best	[0.387, 0.662]	100	1 / 1	0.516	reproduced	unknown	Board
Fmax (molecular function, probe) ↑ higher is better · probe	0.715	[0.655, 0.804]	100	2 / 17	0.844	reproduced	unknown	Board
Fmax (biological process, probe) ↑ higher is better · probe	0.538 best	[0.444, 0.638]	100	1 / 17	0.538	reproduced	unknown	Board
Fmax (cellular component, probe) ↑ higher is better · probe	0.728	[0.675, 0.790]	100	10 / 17	0.756	reproduced	unknown	Board
Micro AUPR (biological process, native head) ↑ higher is better · zero-shot · secondary	0.547 best	[0.365, 0.713]	100	—	0.547	reproduced	unknown	Board
Smin (biological process, native head) ↓ lower is better · bits · zero-shot · secondary	4.465 best	[3.283, 5.699]	100	—	4.465	reproduced	unknown	Board
Micro AUPR (molecular function, probe) ↑ higher is better · probe · secondary	0.756	[0.688, 0.830]	100	—	0.875	reproduced	unknown	Board
Smin (molecular function, probe) ↓ lower is better · bits · probe · secondary	2.565	[1.822, 3.195]	100	—	1.727	reproduced	unknown	Board
Micro AUPR (biological process, probe) ↑ higher is better · probe · secondary	0.595 best	[0.409, 0.743]	100	—	0.595	reproduced	unknown	Board
Smin (biological process, probe) ↓ lower is better · bits · probe · secondary	4.842 best	[3.520, 5.862]	100	—	4.842	reproduced	unknown	Board
Micro AUPR (cellular component, probe) ↑ higher is better · probe · secondary	0.716	[0.652, 0.784]	100	—	0.758	reproduced	unknown	Board
Smin (cellular component, probe) ↓ lower is better · bits · probe · secondary	3.242	[2.547, 3.897]	100	—	3.092	reproduced	unknown	Board

Other Benchmarks drawn from no hub dataset you can see

Protein membrane association (DeepLoc) protein-ml/protein-membrane-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q2 accuracy (logistic probe) ↑ higher is better · probe	0.837	[0.821, 0.854]	1728	10 / 17	0.888	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.667	[0.632, 0.701]	1728	—	0.771	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board

Protein stability ΔΔG (Mega-scale) protein-ml/protein-stability-ddg

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Spearman ρ (ridge probe) ↑ higher is better · probe	0.102	[−0.114, 0.268]	100	12 / 17	0.297	reproduced	unknown	Board
RMSE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe	1.107	[0.864, 1.316]	100	7 / 17	1.095	reproduced	unknown	Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary	0.136	[−0.090, 0.282]	100	—	0.349	reproduced	unknown	Board
MAE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe · secondary	0.836	[0.686, 0.992]	100	—	0.819	reproduced	unknown	Board

Protein subcellular localization (DeepLoc) protein-ml/protein-subcellular-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.675	[0.659, 0.690]	2742	10 / 17	0.801	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.479	[0.458, 0.502]	2742	11 / 17	0.640	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.603	[0.584, 0.622]	2742	—	0.758	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board