Benchmarks · protein-ml/progen2-base

ProteinGym protein-ml/proteingym

Domain variant fitness, human PLM 0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.292 [−0.139, 0.689]	25
MET_HUMAN_Estevam_2023	0.381 [0.066, 0.669]	25
S22A1_HUMAN_Yee_2023_activity	0.693 [0.462, 0.828]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.538 [0.223, 0.705]	25

0.476 best

[0.299, 0.608]

100

1 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.596 [0.360, 0.818]	25
MET_HUMAN_Estevam_2023	0.747 [0.551, 0.922]	25
S22A1_HUMAN_Yee_2023_activity	0.893 [0.719, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.799 [0.600, 0.948]	25

0.759 best

[0.664, 0.853]

100

1 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.359 [−0.230, 0.621]	25
MET_HUMAN_Estevam_2023	0.277 [−0.099, 0.690]	25
S22A1_HUMAN_Yee_2023_activity	0.621 [0.206, 0.923]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.447 [−0.081, 0.761]	25

0.426 best

[0.159, 0.572]

100

1 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.248 [0.000, 0.758]	25
MET_HUMAN_Estevam_2023	0.951 [0.805, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.813 [0.687, 0.931]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.207, 0.762]	25

0.596

[0.489, 0.773]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.333]	25
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.667]	25

0.083

[0.000, 0.375]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.292 [−0.139, 0.689]	25
MET_HUMAN_Estevam_2023	0.381 [0.066, 0.669]	25
S22A1_HUMAN_Yee_2023_activity	0.693 [0.462, 0.828]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.538 [0.223, 0.705]	25

0.476 best

[0.299, 0.608]

100

1 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.596 [0.360, 0.818]	25
MET_HUMAN_Estevam_2023	0.747 [0.551, 0.922]	25
S22A1_HUMAN_Yee_2023_activity	0.893 [0.719, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.799 [0.600, 0.948]	25

0.759 best

[0.664, 0.853]

100

1 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.359 [−0.230, 0.621]	25
MET_HUMAN_Estevam_2023	0.277 [−0.099, 0.690]	25
S22A1_HUMAN_Yee_2023_activity	0.621 [0.206, 0.923]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.447 [−0.081, 0.761]	25

0.426 best

[0.159, 0.572]

100

1 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.248 [0.000, 0.758]	25
MET_HUMAN_Estevam_2023	0.951 [0.805, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.813 [0.687, 0.931]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.207, 0.762]	25

0.596

[0.489, 0.773]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.333]	25
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.667]	25

0.083

[0.000, 0.375]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Domain variant fitness, human PLM Clans-0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.403 [0.042, 0.665]	25
CCR5_HUMAN_Gill_2023	0.292 [−0.139, 0.689]	25
OPSD_HUMAN_Wan_2019	0.463 [0.032, 0.783]	25
S22A1_HUMAN_Yee_2023_activity	0.693 [0.462, 0.828]	25

0.463 best

[0.271, 0.612]

100

1 / 13

0.463

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.701 [0.462, 0.897]	25
CCR5_HUMAN_Gill_2023	0.596 [0.360, 0.818]	25
OPSD_HUMAN_Wan_2019	0.813 [0.603, 0.968]	25
S22A1_HUMAN_Yee_2023_activity	0.893 [0.719, 1.000]	25

0.751 best

[0.645, 0.847]

100

1 / 13

0.751

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.277 [−0.199, 0.678]	25
CCR5_HUMAN_Gill_2023	0.359 [−0.230, 0.621]	25
OPSD_HUMAN_Wan_2019	0.447 [−0.014, 0.774]	25
S22A1_HUMAN_Yee_2023_activity	0.621 [0.206, 0.923]	25

0.426 best

[0.155, 0.559]

100

1 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.653 [0.354, 0.903]	25
CCR5_HUMAN_Gill_2023	0.248 [0.000, 0.758]	25
OPSD_HUMAN_Wan_2019	0.833 [0.352, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.813 [0.687, 0.931]	25

0.637

[0.477, 0.795]

100

—

0.709

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.000 [0.000, 1.000]	25
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.333]	25
OPSD_HUMAN_Wan_2019	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25

0.083

[0.000, 0.438]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Domain variant fitness, human PLM Clans-0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.381 [0.066, 0.669]	25
MK01_HUMAN_Brenan_2016	0.430 [0.016, 0.758]	25
SCN5A_HUMAN_Glazer_2019	0.104 [−0.390, 0.549]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.532 [0.210, 0.695]	25

0.362 best

[0.167, 0.522]

100

1 / 13

0.362

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.747 [0.551, 0.922]	25
MK01_HUMAN_Brenan_2016	0.816 [0.545, 1.000]	25
SCN5A_HUMAN_Glazer_2019	0.533 [0.279, 0.762]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.792 [0.587, 0.942]	25

0.722 best

[0.616, 0.821]

100

1 / 13

0.722

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.277 [−0.099, 0.690]	25
MK01_HUMAN_Brenan_2016	0.316 [−0.068, 0.721]	25
SCN5A_HUMAN_Glazer_2019	0.033 [−0.442, 0.428]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.447 [−0.136, 0.761]	25

0.268

[0.024, 0.458]

100

3 / 13

0.310

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.951 [0.805, 1.000]	25
MK01_HUMAN_Brenan_2016	0.589 [0.000, 0.948]	25
SCN5A_HUMAN_Glazer_2019	0.986 [0.213, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.207, 0.762]	25

0.725 best

[0.484, 0.865]

100

—

0.725

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
MK01_HUMAN_Brenan_2016	0.000 [0.000, 0.667]	25
SCN5A_HUMAN_Glazer_2019	0.667 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.667]	25

0.250 best

[0.000, 0.504]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.570 [0.399, 0.717]	100
CASP3_HUMAN_Roychowdhury_2020	0.528 [0.378, 0.663]	100
CCR5_HUMAN_Gill_2023	0.312 [0.133, 0.497]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.596 [0.447, 0.706]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.345 [0.152, 0.509]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.059 [−0.139, 0.261]	100
MLAC_ECOLI_MacRae_2023	0.270 [0.089, 0.433]	100
MTH3_HAEAE_RockahShmuel_2015	0.770 [0.674, 0.837]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.704 [0.583, 0.789]	100
SPG1_STRSG_Olson_2014	0.083 [−0.135, 0.285]	100

0.424

[0.366, 0.471]

1000

3 / 11

0.434

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.766 [0.668, 0.857]	100
CASP3_HUMAN_Roychowdhury_2020	0.788 [0.697, 0.874]	100
CCR5_HUMAN_Gill_2023	0.637 [0.528, 0.743]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.808 [0.711, 0.883]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.690 [0.571, 0.802]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.680 [0.569, 0.789]	100
MLAC_ECOLI_MacRae_2023	0.707 [0.602, 0.799]	100
MTH3_HAEAE_RockahShmuel_2015	0.859 [0.772, 0.932]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.935 [0.881, 0.976]	100
SPG1_STRSG_Olson_2014	0.675 [0.530, 0.802]	100

0.754

[0.721, 0.786]

1000

5 / 11

0.775

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.420 [0.220, 0.620]	100
CASP3_HUMAN_Roychowdhury_2020	0.381 [0.185, 0.583]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.038, 0.363]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.482 [0.267, 0.641]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.306 [0.075, 0.480]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.312 [0.094, 0.472]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.041, 0.431]	100
MTH3_HAEAE_RockahShmuel_2015	0.468 [0.319, 0.626]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.612 [0.430, 0.803]	100
SPG1_STRSG_Olson_2014	0.173 [−0.032, 0.364]	100

0.351

[0.291, 0.412]

1000

5 / 11

0.370

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.809 [0.694, 0.943]	100
CASP3_HUMAN_Roychowdhury_2020	0.793 [0.691, 0.888]	100
CCR5_HUMAN_Gill_2023	0.834 [0.617, 0.897]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.856 [0.770, 0.921]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.824 [0.745, 0.903]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.550 [0.430, 0.662]	100
MLAC_ECOLI_MacRae_2023	0.835 [0.717, 0.911]	100
MTH3_HAEAE_RockahShmuel_2015	0.894 [0.823, 0.945]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.883 [0.804, 0.958]	100
SPG1_STRSG_Olson_2014	0.857 [0.733, 0.943]	100

0.814

[0.773, 0.841]

1000

2 / 11

0.814

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.100 [0.000, 0.334]	100
CASP3_HUMAN_Roychowdhury_2020	0.100 [0.000, 0.364]	100
CCR5_HUMAN_Gill_2023	0.200 [0.000, 0.455]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.300 [0.000, 0.600]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.100 [0.000, 0.400]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.000 [0.000, 0.000]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.300]	100
MTH3_HAEAE_RockahShmuel_2015	0.364 [0.091, 0.616]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.400 [0.083, 0.636]	100
SPG1_STRSG_Olson_2014	0.300 [0.000, 0.545]	100

0.196 best

[0.106, 0.262]

1000

1 / 11

0.196

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.698 [0.558, 0.802]	100
CASP3_HUMAN_Roychowdhury_2020	0.602 [0.464, 0.710]	100
CCR5_HUMAN_Gill_2023	0.311 [0.127, 0.483]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.740 [0.620, 0.824]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.757 [0.616, 0.847]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.654 [0.502, 0.777]	100
MLAC_ECOLI_MacRae_2023	0.469 [0.292, 0.622]	100
MTH3_HAEAE_RockahShmuel_2015	0.714 [0.598, 0.805]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.895 [0.832, 0.929]	100
SPG1_STRSG_Olson_2014	0.765 [0.644, 0.855]	100

0.661 best

[0.617, 0.693]

1000

1 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.848 [0.773, 0.913]	100
CASP3_HUMAN_Roychowdhury_2020	0.807 [0.722, 0.892]	100
CCR5_HUMAN_Gill_2023	0.643 [0.541, 0.747]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.867 [0.786, 0.936]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.929 [0.873, 0.972]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.855 [0.751, 0.940]	100
MLAC_ECOLI_MacRae_2023	0.784 [0.688, 0.875]	100
MTH3_HAEAE_RockahShmuel_2015	0.826 [0.730, 0.911]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.978 [0.947, 0.999]	100
SPG1_STRSG_Olson_2014	0.949 [0.899, 0.984]	100

0.848 best

[0.825, 0.873]

1000

1 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.460 [0.260, 0.620]	100
CASP3_HUMAN_Roychowdhury_2020	0.501 [0.253, 0.667]	100
CCR5_HUMAN_Gill_2023	0.120 [−0.080, 0.363]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.562 [0.380, 0.721]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.567 [0.407, 0.692]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.401 [0.200, 0.574]	100
MLAC_ECOLI_MacRae_2023	0.480 [0.276, 0.640]	100
MTH3_HAEAE_RockahShmuel_2015	0.421 [0.240, 0.574]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.776 [0.600, 0.905]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.521]	100

0.469

[0.400, 0.517]

1000

4 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.825 [0.719, 0.949]	100
CASP3_HUMAN_Roychowdhury_2020	0.792 [0.701, 0.903]	100
CCR5_HUMAN_Gill_2023	0.815 [0.600, 0.887]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.845 [0.758, 0.923]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.952 [0.896, 0.979]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.761 [0.624, 0.880]	100
MLAC_ECOLI_MacRae_2023	0.885 [0.828, 0.944]	100
MTH3_HAEAE_RockahShmuel_2015	0.888 [0.780, 0.958]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.921 [0.857, 0.964]	100
SPG1_STRSG_Olson_2014	0.912 [0.862, 0.966]	100

0.860 best

[0.826, 0.886]

1000

1 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.300 [0.000, 0.600]	100
CASP3_HUMAN_Roychowdhury_2020	0.300 [0.000, 0.600]	100
CCR5_HUMAN_Gill_2023	0.100 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.600]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.300 [0.077, 0.600]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.400 [0.100, 0.636]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.500]	100
MTH3_HAEAE_RockahShmuel_2015	0.364 [0.091, 0.615]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.500 [0.182, 0.728]	100
SPG1_STRSG_Olson_2014	0.300 [0.000, 0.601]	100

0.286

[0.209, 0.382]

1000

7 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

ProteinGym_v1 protein-ml/proteingym-v1

Domain variant fitness, human PLM 0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.292 [−0.139, 0.689]	25
MET_HUMAN_Estevam_2023	0.381 [0.066, 0.669]	25
S22A1_HUMAN_Yee_2023_activity	0.693 [0.462, 0.828]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.538 [0.223, 0.705]	25

0.476 best

[0.299, 0.608]

100

1 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.596 [0.360, 0.818]	25
MET_HUMAN_Estevam_2023	0.747 [0.551, 0.922]	25
S22A1_HUMAN_Yee_2023_activity	0.893 [0.719, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.799 [0.600, 0.948]	25

0.759 best

[0.664, 0.853]

100

1 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.359 [−0.230, 0.621]	25
MET_HUMAN_Estevam_2023	0.277 [−0.099, 0.690]	25
S22A1_HUMAN_Yee_2023_activity	0.621 [0.206, 0.923]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.447 [−0.081, 0.761]	25

0.426 best

[0.159, 0.572]

100

1 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.248 [0.000, 0.758]	25
MET_HUMAN_Estevam_2023	0.951 [0.805, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.813 [0.687, 0.931]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.207, 0.762]	25

0.596

[0.489, 0.773]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.333]	25
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.667]	25

0.083

[0.000, 0.375]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.292 [−0.139, 0.689]	25
MET_HUMAN_Estevam_2023	0.381 [0.066, 0.669]	25
S22A1_HUMAN_Yee_2023_activity	0.693 [0.462, 0.828]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.538 [0.223, 0.705]	25

0.476 best

[0.299, 0.608]

100

1 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.596 [0.360, 0.818]	25
MET_HUMAN_Estevam_2023	0.747 [0.551, 0.922]	25
S22A1_HUMAN_Yee_2023_activity	0.893 [0.719, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.799 [0.600, 0.948]	25

0.759 best

[0.664, 0.853]

100

1 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.359 [−0.230, 0.621]	25
MET_HUMAN_Estevam_2023	0.277 [−0.099, 0.690]	25
S22A1_HUMAN_Yee_2023_activity	0.621 [0.206, 0.923]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.447 [−0.081, 0.761]	25

0.426 best

[0.159, 0.572]

100

1 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.248 [0.000, 0.758]	25
MET_HUMAN_Estevam_2023	0.951 [0.805, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.813 [0.687, 0.931]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.207, 0.762]	25

0.596

[0.489, 0.773]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.333]	25
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.667]	25

0.083

[0.000, 0.375]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Domain variant fitness, human PLM Clans-0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.403 [0.042, 0.665]	25
CCR5_HUMAN_Gill_2023	0.292 [−0.139, 0.689]	25
OPSD_HUMAN_Wan_2019	0.463 [0.032, 0.783]	25
S22A1_HUMAN_Yee_2023_activity	0.693 [0.462, 0.828]	25

0.463 best

[0.271, 0.612]

100

1 / 13

0.463

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.701 [0.462, 0.897]	25
CCR5_HUMAN_Gill_2023	0.596 [0.360, 0.818]	25
OPSD_HUMAN_Wan_2019	0.813 [0.603, 0.968]	25
S22A1_HUMAN_Yee_2023_activity	0.893 [0.719, 1.000]	25

0.751 best

[0.645, 0.847]

100

1 / 13

0.751

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.277 [−0.199, 0.678]	25
CCR5_HUMAN_Gill_2023	0.359 [−0.230, 0.621]	25
OPSD_HUMAN_Wan_2019	0.447 [−0.014, 0.774]	25
S22A1_HUMAN_Yee_2023_activity	0.621 [0.206, 0.923]	25

0.426 best

[0.155, 0.559]

100

1 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.653 [0.354, 0.903]	25
CCR5_HUMAN_Gill_2023	0.248 [0.000, 0.758]	25
OPSD_HUMAN_Wan_2019	0.833 [0.352, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.813 [0.687, 0.931]	25

0.637

[0.477, 0.795]

100

—

0.709

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.000 [0.000, 1.000]	25
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.333]	25
OPSD_HUMAN_Wan_2019	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25

0.083

[0.000, 0.438]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Domain variant fitness, human PLM Clans-0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.381 [0.066, 0.669]	25
MK01_HUMAN_Brenan_2016	0.430 [0.016, 0.758]	25
SCN5A_HUMAN_Glazer_2019	0.104 [−0.390, 0.549]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.532 [0.210, 0.695]	25

0.362 best

[0.167, 0.522]

100

1 / 13

0.362

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.747 [0.551, 0.922]	25
MK01_HUMAN_Brenan_2016	0.816 [0.545, 1.000]	25
SCN5A_HUMAN_Glazer_2019	0.533 [0.279, 0.762]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.792 [0.587, 0.942]	25

0.722 best

[0.616, 0.821]

100

1 / 13

0.722

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.277 [−0.099, 0.690]	25
MK01_HUMAN_Brenan_2016	0.316 [−0.068, 0.721]	25
SCN5A_HUMAN_Glazer_2019	0.033 [−0.442, 0.428]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.447 [−0.136, 0.761]	25

0.268

[0.024, 0.458]

100

3 / 13

0.310

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.951 [0.805, 1.000]	25
MK01_HUMAN_Brenan_2016	0.589 [0.000, 0.948]	25
SCN5A_HUMAN_Glazer_2019	0.986 [0.213, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.207, 0.762]	25

0.725 best

[0.484, 0.865]

100

—

0.725

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
MK01_HUMAN_Brenan_2016	0.000 [0.000, 0.667]	25
SCN5A_HUMAN_Glazer_2019	0.667 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.667]	25

0.250 best

[0.000, 0.504]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.570 [0.399, 0.717]	100
CASP3_HUMAN_Roychowdhury_2020	0.528 [0.378, 0.663]	100
CCR5_HUMAN_Gill_2023	0.312 [0.133, 0.497]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.596 [0.447, 0.706]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.345 [0.152, 0.509]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.059 [−0.139, 0.261]	100
MLAC_ECOLI_MacRae_2023	0.270 [0.089, 0.433]	100
MTH3_HAEAE_RockahShmuel_2015	0.770 [0.674, 0.837]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.704 [0.583, 0.789]	100
SPG1_STRSG_Olson_2014	0.083 [−0.135, 0.285]	100

0.424

[0.366, 0.471]

1000

3 / 11

0.434

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.766 [0.668, 0.857]	100
CASP3_HUMAN_Roychowdhury_2020	0.788 [0.697, 0.874]	100
CCR5_HUMAN_Gill_2023	0.637 [0.528, 0.743]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.808 [0.711, 0.883]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.690 [0.571, 0.802]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.680 [0.569, 0.789]	100
MLAC_ECOLI_MacRae_2023	0.707 [0.602, 0.799]	100
MTH3_HAEAE_RockahShmuel_2015	0.859 [0.772, 0.932]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.935 [0.881, 0.976]	100
SPG1_STRSG_Olson_2014	0.675 [0.530, 0.802]	100

0.754

[0.721, 0.786]

1000

5 / 11

0.775

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.420 [0.220, 0.620]	100
CASP3_HUMAN_Roychowdhury_2020	0.381 [0.185, 0.583]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.038, 0.363]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.482 [0.267, 0.641]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.306 [0.075, 0.480]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.312 [0.094, 0.472]	100
MLAC_ECOLI_MacRae_2023	0.200 [0.041, 0.431]	100
MTH3_HAEAE_RockahShmuel_2015	0.468 [0.319, 0.626]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.612 [0.430, 0.803]	100
SPG1_STRSG_Olson_2014	0.173 [−0.032, 0.364]	100

0.351

[0.291, 0.412]

1000

5 / 11

0.370

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.809 [0.694, 0.943]	100
CASP3_HUMAN_Roychowdhury_2020	0.793 [0.691, 0.888]	100
CCR5_HUMAN_Gill_2023	0.834 [0.617, 0.897]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.856 [0.770, 0.921]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.824 [0.745, 0.903]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.550 [0.430, 0.662]	100
MLAC_ECOLI_MacRae_2023	0.835 [0.717, 0.911]	100
MTH3_HAEAE_RockahShmuel_2015	0.894 [0.823, 0.945]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.883 [0.804, 0.958]	100
SPG1_STRSG_Olson_2014	0.857 [0.733, 0.943]	100

0.814

[0.773, 0.841]

1000

2 / 11

0.814

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.100 [0.000, 0.334]	100
CASP3_HUMAN_Roychowdhury_2020	0.100 [0.000, 0.364]	100
CCR5_HUMAN_Gill_2023	0.200 [0.000, 0.455]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.300 [0.000, 0.600]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.100 [0.000, 0.400]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.000 [0.000, 0.000]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.300]	100
MTH3_HAEAE_RockahShmuel_2015	0.364 [0.091, 0.616]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.400 [0.083, 0.636]	100
SPG1_STRSG_Olson_2014	0.300 [0.000, 0.545]	100

0.196 best

[0.106, 0.262]

1000

1 / 11

0.196

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.698 [0.558, 0.802]	100
CASP3_HUMAN_Roychowdhury_2020	0.602 [0.464, 0.710]	100
CCR5_HUMAN_Gill_2023	0.311 [0.127, 0.483]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.740 [0.620, 0.824]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.757 [0.616, 0.847]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.654 [0.502, 0.777]	100
MLAC_ECOLI_MacRae_2023	0.469 [0.292, 0.622]	100
MTH3_HAEAE_RockahShmuel_2015	0.714 [0.598, 0.805]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.895 [0.832, 0.929]	100
SPG1_STRSG_Olson_2014	0.765 [0.644, 0.855]	100

0.661 best

[0.617, 0.693]

1000

1 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.848 [0.773, 0.913]	100
CASP3_HUMAN_Roychowdhury_2020	0.807 [0.722, 0.892]	100
CCR5_HUMAN_Gill_2023	0.643 [0.541, 0.747]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.867 [0.786, 0.936]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.929 [0.873, 0.972]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.855 [0.751, 0.940]	100
MLAC_ECOLI_MacRae_2023	0.784 [0.688, 0.875]	100
MTH3_HAEAE_RockahShmuel_2015	0.826 [0.730, 0.911]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.978 [0.947, 0.999]	100
SPG1_STRSG_Olson_2014	0.949 [0.899, 0.984]	100

0.848 best

[0.825, 0.873]

1000

1 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.460 [0.260, 0.620]	100
CASP3_HUMAN_Roychowdhury_2020	0.501 [0.253, 0.667]	100
CCR5_HUMAN_Gill_2023	0.120 [−0.080, 0.363]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.562 [0.380, 0.721]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.567 [0.407, 0.692]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.401 [0.200, 0.574]	100
MLAC_ECOLI_MacRae_2023	0.480 [0.276, 0.640]	100
MTH3_HAEAE_RockahShmuel_2015	0.421 [0.240, 0.574]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.776 [0.600, 0.905]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.521]	100

0.469

[0.400, 0.517]

1000

4 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.825 [0.719, 0.949]	100
CASP3_HUMAN_Roychowdhury_2020	0.792 [0.701, 0.903]	100
CCR5_HUMAN_Gill_2023	0.815 [0.600, 0.887]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.845 [0.758, 0.923]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.952 [0.896, 0.979]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.761 [0.624, 0.880]	100
MLAC_ECOLI_MacRae_2023	0.885 [0.828, 0.944]	100
MTH3_HAEAE_RockahShmuel_2015	0.888 [0.780, 0.958]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.921 [0.857, 0.964]	100
SPG1_STRSG_Olson_2014	0.912 [0.862, 0.966]	100

0.860 best

[0.826, 0.886]

1000

1 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.300 [0.000, 0.600]	100
CASP3_HUMAN_Roychowdhury_2020	0.300 [0.000, 0.600]	100
CCR5_HUMAN_Gill_2023	0.100 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.600]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.300 [0.077, 0.600]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.400 [0.100, 0.636]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.500]	100
MTH3_HAEAE_RockahShmuel_2015	0.364 [0.091, 0.615]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.500 [0.182, 0.728]	100
SPG1_STRSG_Olson_2014	0.300 [0.000, 0.601]	100

0.286

[0.209, 0.382]

1000

7 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

Board

tape protein-ml/tape

Secondary structure (Q3) protein-ml/secondary-structure-q3

Drawn from: tape

atlas-n100-v2

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q3 accuracy (per-residue probe) ↑ higher is better · probe	0.659	[0.643, 0.673]	100	15 / 17	0.813	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90	Board
Q8 accuracy (per-residue probe) ↑ higher is better · probe	0.511	[0.492, 0.529]	100	15 / 17	0.663	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90	Board

UniProt Knowledgebase protein-ml/uniprot-knowledgebase

GO function prediction (Swiss-Prot 2025H2) protein-ml/protein-function-go-prediction

Drawn from: UniProt Knowledgebase

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Fmax (molecular function, probe) ↑ higher is better · probe	0.705	[0.649, 0.767]	100	5 / 17	0.844	reproduced	unknown	Board
Fmax (biological process, probe) ↑ higher is better · probe	0.453	[0.357, 0.569]	100	10 / 17	0.538	reproduced	unknown	Board
Fmax (cellular component, probe) ↑ higher is better · probe	0.732	[0.682, 0.791]	100	8 / 17	0.756	reproduced	unknown	Board
Micro AUPR (molecular function, probe) ↑ higher is better · probe · secondary	0.674	[0.588, 0.769]	100	—	0.875	reproduced	unknown	Board
Smin (molecular function, probe) ↓ lower is better · bits · probe · secondary	2.992	[2.391, 3.670]	100	—	1.727	reproduced	unknown	Board
Micro AUPR (biological process, probe) ↑ higher is better · probe · secondary	0.409	[0.281, 0.551]	100	—	0.595	reproduced	unknown	Board
Smin (biological process, probe) ↓ lower is better · bits · probe · secondary	6.127	[4.393, 7.920]	100	—	4.842	reproduced	unknown	Board
Micro AUPR (cellular component, probe) ↑ higher is better · probe · secondary	0.715	[0.648, 0.781]	100	—	0.758	reproduced	unknown	Board
Smin (cellular component, probe) ↓ lower is better · bits · probe · secondary	3.281	[2.557, 3.922]	100	—	3.092	reproduced	unknown	Board

Other Benchmarks drawn from no hub dataset you can see

Protein membrane association (DeepLoc) protein-ml/protein-membrane-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q2 accuracy (logistic probe) ↑ higher is better · probe	0.824	[0.807, 0.843]	1728	14 / 17	0.888	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.641	[0.605, 0.679]	1728	—	0.771	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90	Board

Protein stability ΔΔG (Mega-scale) protein-ml/protein-stability-ddg

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Spearman ρ (zero-shot) ↑ higher is better · zero-shot	0.237	[0.014, 0.458]	100	11 / 12	0.410	reproduced	unknown	Board
Spearman ρ (ridge probe) ↑ higher is better · probe	0.111	[−0.092, 0.314]	100	11 / 17	0.297	reproduced	unknown	Board
RMSE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe	1.101	[0.879, 1.294]	100	3 / 17	1.095	reproduced	unknown	Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary	0.142	[−0.079, 0.330]	100	—	0.349	reproduced	unknown	Board
MAE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe · secondary	0.840	[0.695, 0.991]	100	—	0.819	reproduced	unknown	Board

Protein subcellular localization (DeepLoc) protein-ml/protein-subcellular-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.614	[0.598, 0.629]	2742	15 / 17	0.801	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.438	[0.415, 0.462]	2742	15 / 17	0.640	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.528	[0.508, 0.546]	2742	—	0.758	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90	Board