Benchmarks · protein-ml/esm2-35m-protein-cellular-component

ProteinGym protein-ml/proteingym

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.476 [0.296, 0.621]	100
CASP3_HUMAN_Roychowdhury_2020	0.135 [−0.065, 0.339]	100
CCR5_HUMAN_Gill_2023	0.124 [−0.076, 0.307]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.487 [0.306, 0.647]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.674 [0.532, 0.784]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.336 [0.146, 0.538]	100
MLAC_ECOLI_MacRae_2023	0.330 [0.112, 0.510]	100
MTH3_HAEAE_RockahShmuel_2015	0.410 [0.208, 0.567]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.862 [0.808, 0.895]	100
SPG1_STRSG_Olson_2014	0.679 [0.538, 0.783]	100

0.451

[0.393, 0.499]

1000

17 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.737 [0.630, 0.830]	100
CASP3_HUMAN_Roychowdhury_2020	0.564 [0.452, 0.674]	100
CCR5_HUMAN_Gill_2023	0.551 [0.443, 0.655]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.724 [0.620, 0.822]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.911 [0.837, 0.969]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.678 [0.561, 0.799]	100
MLAC_ECOLI_MacRae_2023	0.655 [0.543, 0.756]	100
MTH3_HAEAE_RockahShmuel_2015	0.725 [0.597, 0.835]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.951 [0.907, 0.986]	100
SPG1_STRSG_Olson_2014	0.951 [0.900, 0.986]	100

0.745

[0.713, 0.774]

1000

17 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.300 [0.124, 0.508]	100
CASP3_HUMAN_Roychowdhury_2020	0.060 [−0.122, 0.267]	100
CCR5_HUMAN_Gill_2023	0.080 [−0.186, 0.280]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.281 [0.138, 0.520]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.524 [0.344, 0.672]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.223 [0.038, 0.428]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.020, 0.420]	100
MTH3_HAEAE_RockahShmuel_2015	0.281 [0.082, 0.467]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.735 [0.604, 0.847]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.521]	100

0.309

[0.257, 0.372]

1000

17 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.696 [0.539, 0.876]	100
CASP3_HUMAN_Roychowdhury_2020	0.683 [0.515, 0.801]	100
CCR5_HUMAN_Gill_2023	0.836 [0.609, 0.886]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.744 [0.586, 0.887]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.872 [0.807, 0.935]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.707 [0.539, 0.844]	100
MLAC_ECOLI_MacRae_2023	0.870 [0.789, 0.934]	100
MTH3_HAEAE_RockahShmuel_2015	0.762 [0.637, 0.883]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.884 [0.816, 0.949]	100
SPG1_STRSG_Olson_2014	0.902 [0.850, 0.950]	100

0.796

[0.749, 0.831]

1000

12 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.000 [0.000, 0.300]	100
CASP3_HUMAN_Roychowdhury_2020	0.000 [0.000, 0.333]	100
CCR5_HUMAN_Gill_2023	0.200 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.500]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.200 [0.000, 0.545]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.300 [0.000, 0.600]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.400]	100
MTH3_HAEAE_RockahShmuel_2015	0.273 [0.000, 0.545]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.500 [0.000, 0.700]	100
SPG1_STRSG_Olson_2014	0.400 [0.083, 0.636]	100

0.217

[0.135, 0.297]

1000

15 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

ProteinGym_v1 protein-ml/proteingym-v1

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.476 [0.296, 0.621]	100
CASP3_HUMAN_Roychowdhury_2020	0.135 [−0.065, 0.339]	100
CCR5_HUMAN_Gill_2023	0.124 [−0.076, 0.307]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.487 [0.306, 0.647]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.674 [0.532, 0.784]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.336 [0.146, 0.538]	100
MLAC_ECOLI_MacRae_2023	0.330 [0.112, 0.510]	100
MTH3_HAEAE_RockahShmuel_2015	0.410 [0.208, 0.567]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.862 [0.808, 0.895]	100
SPG1_STRSG_Olson_2014	0.679 [0.538, 0.783]	100

0.451

[0.393, 0.499]

1000

17 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.737 [0.630, 0.830]	100
CASP3_HUMAN_Roychowdhury_2020	0.564 [0.452, 0.674]	100
CCR5_HUMAN_Gill_2023	0.551 [0.443, 0.655]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.724 [0.620, 0.822]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.911 [0.837, 0.969]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.678 [0.561, 0.799]	100
MLAC_ECOLI_MacRae_2023	0.655 [0.543, 0.756]	100
MTH3_HAEAE_RockahShmuel_2015	0.725 [0.597, 0.835]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.951 [0.907, 0.986]	100
SPG1_STRSG_Olson_2014	0.951 [0.900, 0.986]	100

0.745

[0.713, 0.774]

1000

17 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.300 [0.124, 0.508]	100
CASP3_HUMAN_Roychowdhury_2020	0.060 [−0.122, 0.267]	100
CCR5_HUMAN_Gill_2023	0.080 [−0.186, 0.280]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.281 [0.138, 0.520]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.524 [0.344, 0.672]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.223 [0.038, 0.428]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.020, 0.420]	100
MTH3_HAEAE_RockahShmuel_2015	0.281 [0.082, 0.467]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.735 [0.604, 0.847]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.521]	100

0.309

[0.257, 0.372]

1000

17 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.696 [0.539, 0.876]	100
CASP3_HUMAN_Roychowdhury_2020	0.683 [0.515, 0.801]	100
CCR5_HUMAN_Gill_2023	0.836 [0.609, 0.886]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.744 [0.586, 0.887]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.872 [0.807, 0.935]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.707 [0.539, 0.844]	100
MLAC_ECOLI_MacRae_2023	0.870 [0.789, 0.934]	100
MTH3_HAEAE_RockahShmuel_2015	0.762 [0.637, 0.883]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.884 [0.816, 0.949]	100
SPG1_STRSG_Olson_2014	0.902 [0.850, 0.950]	100

0.796

[0.749, 0.831]

1000

12 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.000 [0.000, 0.300]	100
CASP3_HUMAN_Roychowdhury_2020	0.000 [0.000, 0.333]	100
CCR5_HUMAN_Gill_2023	0.200 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.500]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.200 [0.000, 0.545]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.300 [0.000, 0.600]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.400]	100
MTH3_HAEAE_RockahShmuel_2015	0.273 [0.000, 0.545]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.500 [0.000, 0.700]	100
SPG1_STRSG_Olson_2014	0.400 [0.083, 0.636]	100

0.217

[0.135, 0.297]

1000

15 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: protein-ml/uniref50

Board

tape protein-ml/tape

Secondary structure (Q3) protein-ml/secondary-structure-q3

Drawn from: tape

atlas-n100-v2

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q3 accuracy (per-residue probe) ↑ higher is better · probe	0.678	[0.662, 0.695]	100	13 / 17	0.813	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
Q8 accuracy (per-residue probe) ↑ higher is better · probe	0.525	[0.503, 0.545]	100	13 / 17	0.663	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board

UniProt Knowledgebase protein-ml/uniprot-knowledgebase

GO function prediction (Swiss-Prot 2025H2) protein-ml/protein-function-go-prediction

Drawn from: UniProt Knowledgebase

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Fmax (cellular component, native head) ↑ higher is better · zero-shot	0.718 best	[0.673, 0.776]	100	1 / 1	0.718	reproduced	unknown	Board
Fmax (molecular function, probe) ↑ higher is better · probe	0.683	[0.631, 0.749]	100	8 / 17	0.844	reproduced	unknown	Board
Fmax (biological process, probe) ↑ higher is better · probe	0.477	[0.396, 0.577]	100	7 / 17	0.538	reproduced	unknown	Board
Fmax (cellular component, probe) ↑ higher is better · probe	0.755	[0.709, 0.813]	100	2 / 17	0.756	reproduced	unknown	Board
Micro AUPR (cellular component, native head) ↑ higher is better · zero-shot · secondary	0.695 best	[0.606, 0.783]	100	—	0.695	reproduced	unknown	Board
Smin (cellular component, native head) ↓ lower is better · bits · zero-shot · secondary	3.466 best	[2.746, 3.975]	100	—	3.466	reproduced	unknown	Board
Micro AUPR (molecular function, probe) ↑ higher is better · probe · secondary	0.695	[0.618, 0.771]	100	—	0.875	reproduced	unknown	Board
Smin (molecular function, probe) ↓ lower is better · bits · probe · secondary	3.121	[2.347, 3.973]	100	—	1.727	reproduced	unknown	Board
Micro AUPR (biological process, probe) ↑ higher is better · probe · secondary	0.470	[0.328, 0.603]	100	—	0.595	reproduced	unknown	Board
Smin (biological process, probe) ↓ lower is better · bits · probe · secondary	5.979	[4.317, 7.450]	100	—	4.842	reproduced	unknown	Board
Micro AUPR (cellular component, probe) ↑ higher is better · probe · secondary	0.758 best	[0.695, 0.821]	100	—	0.758	reproduced	unknown	Board
Smin (cellular component, probe) ↓ lower is better · bits · probe · secondary	3.116	[2.456, 3.770]	100	—	3.092	reproduced	unknown	Board

Other Benchmarks drawn from no hub dataset you can see

Protein membrane association (DeepLoc) protein-ml/protein-membrane-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q2 accuracy (logistic probe) ↑ higher is better · probe	0.888 best	[0.871, 0.903]	1728	1 / 17	0.888	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.771 best	[0.736, 0.802]	1728	—	0.771	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board

Protein stability ΔΔG (Mega-scale) protein-ml/protein-stability-ddg

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Spearman ρ (ridge probe) ↑ higher is better · probe	0.166	[−0.071, 0.380]	100	6 / 17	0.297	reproduced	unknown	Board
RMSE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe	1.095 best	[0.864, 1.295]	100	1 / 17	1.095	reproduced	unknown	Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary	0.172	[−0.065, 0.369]	100	—	0.349	reproduced	unknown	Board
MAE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe · secondary	0.836	[0.688, 0.987]	100	—	0.819	reproduced	unknown	Board

Protein subcellular localization (DeepLoc) protein-ml/protein-subcellular-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.801 best	[0.788, 0.814]	2742	1 / 17	0.801	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.640 best	[0.618, 0.664]	2742	1 / 17	0.640	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.758 best	[0.743, 0.775]	2742	—	0.758	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50	Board