Benchmarks · protein-ml/prot-bert

ProteinGym protein-ml/proteingym

Domain variant fitness, human PLM 0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.291 [−0.179, 0.645]	25
MET_HUMAN_Estevam_2023	0.289 [−0.081, 0.584]	25
S22A1_HUMAN_Yee_2023_activity	0.398 [0.007, 0.658]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.066 [−0.450, 0.345]	25

0.228

[0.034, 0.404]

100

11 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.647 [0.393, 0.853]	25
MET_HUMAN_Estevam_2023	0.695 [0.467, 0.903]	25
S22A1_HUMAN_Yee_2023_activity	0.673 [0.441, 0.877]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.500 [0.235, 0.764]	25

0.629

[0.509, 0.754]

100

10 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.038 [−0.342, 0.529]	25
MET_HUMAN_Estevam_2023	0.439 [−0.131, 0.761]	25
S22A1_HUMAN_Yee_2023_activity	0.294 [−0.175, 0.621]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.421, 0.447]	25

0.179

[−0.056, 0.393]

100

11 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.448 [0.000, 0.959]	25
MET_HUMAN_Estevam_2023	0.597 [0.394, 0.898]	25
S22A1_HUMAN_Yee_2023_activity	0.793 [0.604, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.068, 0.709]	25

0.553

[0.409, 0.731]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
MET_HUMAN_Estevam_2023	0.000 [0.000, 0.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.333]	25

0.000

[0.000, 0.292]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.291 [−0.179, 0.645]	25
MET_HUMAN_Estevam_2023	0.289 [−0.081, 0.584]	25
S22A1_HUMAN_Yee_2023_activity	0.398 [0.007, 0.658]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.066 [−0.450, 0.345]	25

0.228

[0.034, 0.404]

100

11 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.647 [0.393, 0.853]	25
MET_HUMAN_Estevam_2023	0.695 [0.467, 0.903]	25
S22A1_HUMAN_Yee_2023_activity	0.673 [0.441, 0.877]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.500 [0.235, 0.764]	25

0.629

[0.509, 0.754]

100

10 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.038 [−0.342, 0.529]	25
MET_HUMAN_Estevam_2023	0.439 [−0.131, 0.761]	25
S22A1_HUMAN_Yee_2023_activity	0.294 [−0.175, 0.621]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.421, 0.447]	25

0.179

[−0.056, 0.393]

100

10 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.448 [0.000, 0.959]	25
MET_HUMAN_Estevam_2023	0.597 [0.394, 0.898]	25
S22A1_HUMAN_Yee_2023_activity	0.793 [0.604, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.068, 0.709]	25

0.553

[0.409, 0.731]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
MET_HUMAN_Estevam_2023	0.000 [0.000, 0.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.333]	25

0.000

[0.000, 0.292]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Domain variant fitness, human PLM Clans-0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.289 [−0.066, 0.609]	25
CCR5_HUMAN_Gill_2023	0.291 [−0.179, 0.645]	25
OPSD_HUMAN_Wan_2019	0.548 [0.129, 0.816]	25
S22A1_HUMAN_Yee_2023_activity	0.398 [0.007, 0.658]	25

0.382

[0.184, 0.548]

100

2 / 13

0.463

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.662 [0.417, 0.887]	25
CCR5_HUMAN_Gill_2023	0.647 [0.393, 0.853]	25
OPSD_HUMAN_Wan_2019	0.819 [0.607, 0.958]	25
S22A1_HUMAN_Yee_2023_activity	0.673 [0.441, 0.877]	25

0.701

[0.588, 0.811]

100

3 / 13

0.751

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.277 [−0.116, 0.704]	25
CCR5_HUMAN_Gill_2023	0.038 [−0.342, 0.529]	25
OPSD_HUMAN_Wan_2019	0.447 [−0.015, 0.782]	25
S22A1_HUMAN_Yee_2023_activity	0.294 [−0.175, 0.621]	25

0.264

[0.071, 0.473]

100

2 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.457 [0.322, 0.750]	25
CCR5_HUMAN_Gill_2023	0.448 [0.000, 0.959]	25
OPSD_HUMAN_Wan_2019	0.620 [0.404, 0.921]	25
S22A1_HUMAN_Yee_2023_activity	0.793 [0.604, 1.000]	25

0.580

[0.448, 0.767]

100

—

0.709

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.000 [0.000, 0.667]	25
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
OPSD_HUMAN_Wan_2019	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25

0.083

[0.000, 0.438]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Domain variant fitness, human PLM Clans-0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.289 [−0.081, 0.584]	25
MK01_HUMAN_Brenan_2016	−0.054 [−0.464, 0.374]	25
SCN5A_HUMAN_Glazer_2019	0.409 [−0.013, 0.732]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.066 [−0.450, 0.345]	25

0.145

[−0.043, 0.329]

100

12 / 13

0.362

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.695 [0.467, 0.903]	25
MK01_HUMAN_Brenan_2016	0.515 [0.240, 0.772]	25
SCN5A_HUMAN_Glazer_2019	0.693 [0.468, 0.875]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.500 [0.235, 0.764]	25

0.601

[0.485, 0.723]

100

12 / 13

0.722

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.439 [−0.131, 0.761]	25
MK01_HUMAN_Brenan_2016	0.144 [−0.371, 0.435]	25
SCN5A_HUMAN_Glazer_2019	0.196 [−0.263, 0.600]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.421, 0.447]	25

0.181

[−0.076, 0.357]

100

10 / 13

0.310

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.597 [0.394, 0.898]	25
MK01_HUMAN_Brenan_2016	0.409 [0.000, 0.911]	25
SCN5A_HUMAN_Glazer_2019	0.986 [0.385, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.068, 0.709]	25

0.591

[0.383, 0.765]

100

—

0.725

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.000 [0.000, 0.000]	25
MK01_HUMAN_Brenan_2016	0.000 [0.000, 1.000]	25
SCN5A_HUMAN_Glazer_2019	0.667 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.333]	25

0.167

[0.000, 0.417]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.345 [0.135, 0.541]	100
CASP3_HUMAN_Roychowdhury_2020	0.585 [0.453, 0.699]	100
CCR5_HUMAN_Gill_2023	0.234 [0.044, 0.434]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.470 [0.301, 0.605]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.419 [0.242, 0.576]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.109 [−0.092, 0.290]	100
MLAC_ECOLI_MacRae_2023	0.298 [0.129, 0.450]	100
MTH3_HAEAE_RockahShmuel_2015	0.738 [0.644, 0.806]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.698 [0.551, 0.812]	100
SPG1_STRSG_Olson_2014	0.125 [−0.086, 0.337]	100

0.402

[0.341, 0.452]

1000

6 / 11

0.434

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.624 [0.508, 0.746]	100
CASP3_HUMAN_Roychowdhury_2020	0.799 [0.711, 0.880]	100
CCR5_HUMAN_Gill_2023	0.584 [0.479, 0.700]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.710 [0.591, 0.806]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.762 [0.644, 0.861]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.695 [0.578, 0.806]	100
MLAC_ECOLI_MacRae_2023	0.694 [0.588, 0.791]	100
MTH3_HAEAE_RockahShmuel_2015	0.848 [0.764, 0.922]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.925 [0.867, 0.968]	100
SPG1_STRSG_Olson_2014	0.772 [0.636, 0.888]	100

0.741

[0.708, 0.773]

1000

9 / 11

0.775

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.180 [−0.019, 0.420]	100
CASP3_HUMAN_Roychowdhury_2020	0.461 [0.280, 0.640]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.078, 0.341]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.241 [0.040, 0.440]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.349 [0.155, 0.524]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.178 [−0.022, 0.401]	100
MLAC_ECOLI_MacRae_2023	0.240 [0.080, 0.460]	100
MTH3_HAEAE_RockahShmuel_2015	0.468 [0.300, 0.614]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.572 [0.430, 0.766]	100
SPG1_STRSG_Olson_2014	0.288 [0.123, 0.450]	100

0.314

[0.261, 0.380]

1000

9 / 11

0.370

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.694 [0.527, 0.889]	100
CASP3_HUMAN_Roychowdhury_2020	0.780 [0.692, 0.863]	100
CCR5_HUMAN_Gill_2023	0.839 [0.624, 0.911]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.822 [0.760, 0.881]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.790 [0.707, 0.874]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.612 [0.486, 0.732]	100
MLAC_ECOLI_MacRae_2023	0.864 [0.779, 0.919]	100
MTH3_HAEAE_RockahShmuel_2015	0.890 [0.776, 0.958]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.864 [0.778, 0.934]	100
SPG1_STRSG_Olson_2014	0.805 [0.615, 0.928]	100

0.796

[0.753, 0.827]

1000

4 / 11

0.814

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.100 [0.000, 0.334]	100
CASP3_HUMAN_Roychowdhury_2020	0.200 [0.000, 0.400]	100
CCR5_HUMAN_Gill_2023	0.100 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.545]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.000 [0.000, 0.300]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.100 [0.000, 0.300]	100
MLAC_ECOLI_MacRae_2023	0.000 [0.000, 0.273]	100
MTH3_HAEAE_RockahShmuel_2015	0.364 [0.091, 0.616]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.400 [0.091, 0.636]	100
SPG1_STRSG_Olson_2014	0.200 [0.000, 0.500]	100

0.166

[0.095, 0.246]

1000

7 / 11

0.196

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.567 [0.387, 0.688]	100
CASP3_HUMAN_Roychowdhury_2020	0.456 [0.282, 0.607]	100
CCR5_HUMAN_Gill_2023	0.118 [−0.078, 0.303]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.481 [0.273, 0.623]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.681 [0.529, 0.793]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.570 [0.394, 0.720]	100
MLAC_ECOLI_MacRae_2023	0.452 [0.263, 0.605]	100
MTH3_HAEAE_RockahShmuel_2015	0.422 [0.253, 0.577]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.881 [0.817, 0.920]	100
SPG1_STRSG_Olson_2014	0.783 [0.692, 0.846]	100

0.541

[0.488, 0.585]

1000

12 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.790 [0.690, 0.872]	100
CASP3_HUMAN_Roychowdhury_2020	0.738 [0.632, 0.837]	100
CCR5_HUMAN_Gill_2023	0.567 [0.449, 0.680]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.740 [0.621, 0.829]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.892 [0.807, 0.956]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.825 [0.734, 0.906]	100
MLAC_ECOLI_MacRae_2023	0.760 [0.647, 0.853]	100
MTH3_HAEAE_RockahShmuel_2015	0.683 [0.562, 0.803]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.965 [0.925, 0.994]	100
SPG1_STRSG_Olson_2014	0.927 [0.861, 0.975]	100

0.789

[0.759, 0.815]

1000

13 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.340 [0.180, 0.581]	100
CASP3_HUMAN_Roychowdhury_2020	0.341 [0.153, 0.543]	100
CCR5_HUMAN_Gill_2023	0.120 [−0.121, 0.320]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.321 [0.102, 0.522]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.480 [0.318, 0.643]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.356 [0.186, 0.541]	100
MLAC_ECOLI_MacRae_2023	0.440 [0.241, 0.620]	100
MTH3_HAEAE_RockahShmuel_2015	0.187 [0.013, 0.401]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.735 [0.571, 0.864]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.521]	100

0.372

[0.320, 0.433]

1000

14 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.787 [0.685, 0.918]	100
CASP3_HUMAN_Roychowdhury_2020	0.816 [0.714, 0.892]	100
CCR5_HUMAN_Gill_2023	0.762 [0.566, 0.813]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.731 [0.492, 0.902]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.864 [0.803, 0.932]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.731 [0.604, 0.882]	100
MLAC_ECOLI_MacRae_2023	0.773 [0.596, 0.888]	100
MTH3_HAEAE_RockahShmuel_2015	0.780 [0.560, 0.921]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.896 [0.854, 0.965]	100
SPG1_STRSG_Olson_2014	0.914 [0.887, 0.956]	100

0.805

[0.755, 0.841]

1000

9 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.200 [0.000, 0.500]	100
CASP3_HUMAN_Roychowdhury_2020	0.200 [0.000, 0.600]	100
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.273]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.300 [0.000, 0.636]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.200 [0.000, 0.500]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.400 [0.000, 0.667]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.333]	100
MTH3_HAEAE_RockahShmuel_2015	0.273 [0.000, 0.500]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.500 [0.166, 0.800]	100
SPG1_STRSG_Olson_2014	0.200 [0.000, 0.500]	100

0.237

[0.155, 0.336]

1000

11 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

ProteinGym_v1 protein-ml/proteingym-v1

Domain variant fitness, human PLM 0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.291 [−0.179, 0.645]	25
MET_HUMAN_Estevam_2023	0.289 [−0.081, 0.584]	25
S22A1_HUMAN_Yee_2023_activity	0.398 [0.007, 0.658]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.066 [−0.450, 0.345]	25

0.228

[0.034, 0.404]

100

11 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.647 [0.393, 0.853]	25
MET_HUMAN_Estevam_2023	0.695 [0.467, 0.903]	25
S22A1_HUMAN_Yee_2023_activity	0.673 [0.441, 0.877]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.500 [0.235, 0.764]	25

0.629

[0.509, 0.754]

100

10 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.038 [−0.342, 0.529]	25
MET_HUMAN_Estevam_2023	0.439 [−0.131, 0.761]	25
S22A1_HUMAN_Yee_2023_activity	0.294 [−0.175, 0.621]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.421, 0.447]	25

0.179

[−0.056, 0.393]

100

11 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.448 [0.000, 0.959]	25
MET_HUMAN_Estevam_2023	0.597 [0.394, 0.898]	25
S22A1_HUMAN_Yee_2023_activity	0.793 [0.604, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.068, 0.709]	25

0.553

[0.409, 0.731]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
MET_HUMAN_Estevam_2023	0.000 [0.000, 0.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.333]	25

0.000

[0.000, 0.292]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.291 [−0.179, 0.645]	25
MET_HUMAN_Estevam_2023	0.289 [−0.081, 0.584]	25
S22A1_HUMAN_Yee_2023_activity	0.398 [0.007, 0.658]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.066 [−0.450, 0.345]	25

0.228

[0.034, 0.404]

100

11 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.647 [0.393, 0.853]	25
MET_HUMAN_Estevam_2023	0.695 [0.467, 0.903]	25
S22A1_HUMAN_Yee_2023_activity	0.673 [0.441, 0.877]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.500 [0.235, 0.764]	25

0.629

[0.509, 0.754]

100

10 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.038 [−0.342, 0.529]	25
MET_HUMAN_Estevam_2023	0.439 [−0.131, 0.761]	25
S22A1_HUMAN_Yee_2023_activity	0.294 [−0.175, 0.621]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.421, 0.447]	25

0.179

[−0.056, 0.393]

100

10 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.448 [0.000, 0.959]	25
MET_HUMAN_Estevam_2023	0.597 [0.394, 0.898]	25
S22A1_HUMAN_Yee_2023_activity	0.793 [0.604, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.068, 0.709]	25

0.553

[0.409, 0.731]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
MET_HUMAN_Estevam_2023	0.000 [0.000, 0.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.333]	25

0.000

[0.000, 0.292]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Domain variant fitness, human PLM Clans-0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.289 [−0.066, 0.609]	25
CCR5_HUMAN_Gill_2023	0.291 [−0.179, 0.645]	25
OPSD_HUMAN_Wan_2019	0.548 [0.129, 0.816]	25
S22A1_HUMAN_Yee_2023_activity	0.398 [0.007, 0.658]	25

0.382

[0.184, 0.548]

100

2 / 13

0.463

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.662 [0.417, 0.887]	25
CCR5_HUMAN_Gill_2023	0.647 [0.393, 0.853]	25
OPSD_HUMAN_Wan_2019	0.819 [0.607, 0.958]	25
S22A1_HUMAN_Yee_2023_activity	0.673 [0.441, 0.877]	25

0.701

[0.588, 0.811]

100

3 / 13

0.751

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.277 [−0.116, 0.704]	25
CCR5_HUMAN_Gill_2023	0.038 [−0.342, 0.529]	25
OPSD_HUMAN_Wan_2019	0.447 [−0.015, 0.782]	25
S22A1_HUMAN_Yee_2023_activity	0.294 [−0.175, 0.621]	25

0.264

[0.071, 0.473]

100

2 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.457 [0.322, 0.750]	25
CCR5_HUMAN_Gill_2023	0.448 [0.000, 0.959]	25
OPSD_HUMAN_Wan_2019	0.620 [0.404, 0.921]	25
S22A1_HUMAN_Yee_2023_activity	0.793 [0.604, 1.000]	25

0.580

[0.448, 0.767]

100

—

0.709

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
ADRB2_HUMAN_Jones_2020	0.000 [0.000, 0.667]	25
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
OPSD_HUMAN_Wan_2019	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25

0.083

[0.000, 0.438]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Domain variant fitness, human PLM Clans-0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.289 [−0.081, 0.584]	25
MK01_HUMAN_Brenan_2016	−0.054 [−0.464, 0.374]	25
SCN5A_HUMAN_Glazer_2019	0.409 [−0.013, 0.732]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.066 [−0.450, 0.345]	25

0.145

[−0.043, 0.329]

100

12 / 13

0.362

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.695 [0.467, 0.903]	25
MK01_HUMAN_Brenan_2016	0.515 [0.240, 0.772]	25
SCN5A_HUMAN_Glazer_2019	0.693 [0.468, 0.875]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.500 [0.235, 0.764]	25

0.601

[0.485, 0.723]

100

12 / 13

0.722

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.439 [−0.131, 0.761]	25
MK01_HUMAN_Brenan_2016	0.144 [−0.371, 0.435]	25
SCN5A_HUMAN_Glazer_2019	0.196 [−0.263, 0.600]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.421, 0.447]	25

0.181

[−0.076, 0.357]

100

10 / 13

0.310

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.597 [0.394, 0.898]	25
MK01_HUMAN_Brenan_2016	0.409 [0.000, 0.911]	25
SCN5A_HUMAN_Glazer_2019	0.986 [0.385, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.372 [0.068, 0.709]	25

0.591

[0.383, 0.765]

100

—

0.725

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
MET_HUMAN_Estevam_2023	0.000 [0.000, 0.000]	25
MK01_HUMAN_Brenan_2016	0.000 [0.000, 1.000]	25
SCN5A_HUMAN_Glazer_2019	0.667 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.000 [0.000, 0.333]	25

0.167

[0.000, 0.417]

100

—

0.250

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric

Value

95% CI

n

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.345 [0.135, 0.541]	100
CASP3_HUMAN_Roychowdhury_2020	0.585 [0.453, 0.699]	100
CCR5_HUMAN_Gill_2023	0.234 [0.044, 0.434]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.470 [0.301, 0.605]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.419 [0.242, 0.576]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.109 [−0.092, 0.290]	100
MLAC_ECOLI_MacRae_2023	0.298 [0.129, 0.450]	100
MTH3_HAEAE_RockahShmuel_2015	0.738 [0.644, 0.806]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.698 [0.551, 0.812]	100
SPG1_STRSG_Olson_2014	0.125 [−0.086, 0.337]	100

0.402

[0.341, 0.452]

1000

6 / 11

0.434

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.624 [0.508, 0.746]	100
CASP3_HUMAN_Roychowdhury_2020	0.799 [0.711, 0.880]	100
CCR5_HUMAN_Gill_2023	0.584 [0.479, 0.700]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.710 [0.591, 0.806]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.762 [0.644, 0.861]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.695 [0.578, 0.806]	100
MLAC_ECOLI_MacRae_2023	0.694 [0.588, 0.791]	100
MTH3_HAEAE_RockahShmuel_2015	0.848 [0.764, 0.922]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.925 [0.867, 0.968]	100
SPG1_STRSG_Olson_2014	0.772 [0.636, 0.888]	100

0.741

[0.708, 0.773]

1000

9 / 11

0.775

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.180 [−0.019, 0.420]	100
CASP3_HUMAN_Roychowdhury_2020	0.461 [0.280, 0.640]	100
CCR5_HUMAN_Gill_2023	0.160 [−0.078, 0.341]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.241 [0.040, 0.440]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.349 [0.155, 0.524]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.178 [−0.022, 0.401]	100
MLAC_ECOLI_MacRae_2023	0.240 [0.080, 0.460]	100
MTH3_HAEAE_RockahShmuel_2015	0.468 [0.300, 0.614]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.572 [0.430, 0.766]	100
SPG1_STRSG_Olson_2014	0.288 [0.123, 0.450]	100

0.314

[0.261, 0.380]

1000

9 / 11

0.370

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.694 [0.527, 0.889]	100
CASP3_HUMAN_Roychowdhury_2020	0.780 [0.692, 0.863]	100
CCR5_HUMAN_Gill_2023	0.839 [0.624, 0.911]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.822 [0.760, 0.881]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.790 [0.707, 0.874]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.612 [0.486, 0.732]	100
MLAC_ECOLI_MacRae_2023	0.864 [0.779, 0.919]	100
MTH3_HAEAE_RockahShmuel_2015	0.890 [0.776, 0.958]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.864 [0.778, 0.934]	100
SPG1_STRSG_Olson_2014	0.805 [0.615, 0.928]	100

0.796

[0.753, 0.827]

1000

4 / 11

0.814

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (zero-shot) ↑ higher is better · zero-shot

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.100 [0.000, 0.334]	100
CASP3_HUMAN_Roychowdhury_2020	0.200 [0.000, 0.400]	100
CCR5_HUMAN_Gill_2023	0.100 [0.000, 0.400]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.200 [0.000, 0.545]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.000 [0.000, 0.300]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.100 [0.000, 0.300]	100
MLAC_ECOLI_MacRae_2023	0.000 [0.000, 0.273]	100
MTH3_HAEAE_RockahShmuel_2015	0.364 [0.091, 0.616]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.400 [0.091, 0.636]	100
SPG1_STRSG_Olson_2014	0.200 [0.000, 0.500]	100

0.166

[0.095, 0.246]

1000

7 / 11

0.196

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Spearman ρ (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.567 [0.387, 0.688]	100
CASP3_HUMAN_Roychowdhury_2020	0.456 [0.282, 0.607]	100
CCR5_HUMAN_Gill_2023	0.118 [−0.078, 0.303]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.481 [0.273, 0.623]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.681 [0.529, 0.793]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.570 [0.394, 0.720]	100
MLAC_ECOLI_MacRae_2023	0.452 [0.263, 0.605]	100
MTH3_HAEAE_RockahShmuel_2015	0.422 [0.253, 0.577]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.881 [0.817, 0.920]	100
SPG1_STRSG_Olson_2014	0.783 [0.692, 0.846]	100

0.541

[0.488, 0.585]

1000

12 / 17

0.661

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

AUC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.790 [0.690, 0.872]	100
CASP3_HUMAN_Roychowdhury_2020	0.738 [0.632, 0.837]	100
CCR5_HUMAN_Gill_2023	0.567 [0.449, 0.680]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.740 [0.621, 0.829]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.892 [0.807, 0.956]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.825 [0.734, 0.906]	100
MLAC_ECOLI_MacRae_2023	0.760 [0.647, 0.853]	100
MTH3_HAEAE_RockahShmuel_2015	0.683 [0.562, 0.803]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.965 [0.925, 0.994]	100
SPG1_STRSG_Olson_2014	0.927 [0.861, 0.975]	100

0.789

[0.759, 0.815]

1000

13 / 17

0.848

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

MCC (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.340 [0.180, 0.581]	100
CASP3_HUMAN_Roychowdhury_2020	0.341 [0.153, 0.543]	100
CCR5_HUMAN_Gill_2023	0.120 [−0.121, 0.320]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.321 [0.102, 0.522]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.480 [0.318, 0.643]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.356 [0.186, 0.541]	100
MLAC_ECOLI_MacRae_2023	0.440 [0.241, 0.620]	100
MTH3_HAEAE_RockahShmuel_2015	0.187 [0.013, 0.401]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.735 [0.571, 0.864]	100
SPG1_STRSG_Olson_2014	0.403 [0.289, 0.521]	100

0.372

[0.320, 0.433]

1000

14 / 17

0.472

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

NDCG, top 10 % (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.787 [0.685, 0.918]	100
CASP3_HUMAN_Roychowdhury_2020	0.816 [0.714, 0.892]	100
CCR5_HUMAN_Gill_2023	0.762 [0.566, 0.813]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.731 [0.492, 0.902]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.864 [0.803, 0.932]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.731 [0.604, 0.882]	100
MLAC_ECOLI_MacRae_2023	0.773 [0.596, 0.888]	100
MTH3_HAEAE_RockahShmuel_2015	0.780 [0.560, 0.921]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.896 [0.854, 0.965]	100
SPG1_STRSG_Olson_2014	0.914 [0.887, 0.956]	100

0.805

[0.755, 0.841]

1000

9 / 17

0.860

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

Top-10 % recall (ridge probe) ↑ higher is better · probe

By assay: 10 levels

assay	Value [95% CI]	n
ANCSZ_Hobbs_2022	0.200 [0.000, 0.500]	100
CASP3_HUMAN_Roychowdhury_2020	0.200 [0.000, 0.600]	100
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.273]	100
CP2C9_HUMAN_Amorosi_2021_abundance	0.300 [0.000, 0.636]	100
ILF3_HUMAN_Tsuboyama_2023_2L33	0.200 [0.000, 0.500]	100
KCNE1_HUMAN_Muhammad_2023_expression	0.400 [0.000, 0.667]	100
MLAC_ECOLI_MacRae_2023	0.100 [0.000, 0.333]	100
MTH3_HAEAE_RockahShmuel_2015	0.273 [0.000, 0.500]	100
RPC1_BP434_Tsuboyama_2023_1R69	0.500 [0.166, 0.800]	100
SPG1_STRSG_Olson_2014	0.200 [0.000, 0.500]	100

0.237

[0.155, 0.336]

1000

11 / 17

0.360

reproduced

overlap

Train/test overlap declared by the evaluator: UniRef100

Board

tape protein-ml/tape

Secondary structure (Q3) protein-ml/secondary-structure-q3

Drawn from: tape

atlas-n100-v2

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q3 accuracy (per-residue probe) ↑ higher is better · probe	0.722	[0.705, 0.740]	100	12 / 17	0.813	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board
Q8 accuracy (per-residue probe) ↑ higher is better · probe	0.558	[0.536, 0.580]	100	12 / 17	0.663	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board

UniProt Knowledgebase protein-ml/uniprot-knowledgebase

GO function prediction (Swiss-Prot 2025H2) protein-ml/protein-function-go-prediction

Drawn from: UniProt Knowledgebase

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Fmax (molecular function, probe) ↑ higher is better · probe	0.566	[0.505, 0.661]	100	17 / 17	0.844	reproduced	unknown	Board
Fmax (biological process, probe) ↑ higher is better · probe	0.484	[0.394, 0.574]	100	6 / 17	0.538	reproduced	unknown	Board
Fmax (cellular component, probe) ↑ higher is better · probe	0.738	[0.683, 0.795]	100	6 / 17	0.756	reproduced	unknown	Board
Micro AUPR (molecular function, probe) ↑ higher is better · probe · secondary	0.556	[0.474, 0.645]	100	—	0.875	reproduced	unknown	Board
Smin (molecular function, probe) ↓ lower is better · bits · probe · secondary	4.243	[3.366, 5.075]	100	—	1.727	reproduced	unknown	Board
Micro AUPR (biological process, probe) ↑ higher is better · probe · secondary	0.377	[0.250, 0.512]	100	—	0.595	reproduced	unknown	Board
Smin (biological process, probe) ↓ lower is better · bits · probe · secondary	5.918	[4.217, 7.829]	100	—	4.842	reproduced	unknown	Board
Micro AUPR (cellular component, probe) ↑ higher is better · probe · secondary	0.719	[0.651, 0.782]	100	—	0.758	reproduced	unknown	Board
Smin (cellular component, probe) ↓ lower is better · bits · probe · secondary	3.222	[2.505, 3.914]	100	—	3.092	reproduced	unknown	Board

Other Benchmarks drawn from no hub dataset you can see

Protein membrane association (DeepLoc) protein-ml/protein-membrane-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Q2 accuracy (logistic probe) ↑ higher is better · probe	0.830	[0.813, 0.848]	1728	11 / 17	0.888	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.652	[0.617, 0.689]	1728	—	0.771	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board

Protein stability ΔΔG (Mega-scale) protein-ml/protein-stability-ddg

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Spearman ρ (zero-shot) ↑ higher is better · zero-shot	0.308	[0.141, 0.469]	100	6 / 12	0.410	reproduced	unknown	Board
Spearman ρ (ridge probe) ↑ higher is better · probe	0.058	[−0.135, 0.238]	100	15 / 17	0.297	reproduced	unknown	Board
RMSE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe	1.117	[0.896, 1.319]	100	11 / 17	1.095	reproduced	unknown	Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary	0.096	[−0.074, 0.237]	100	—	0.349	reproduced	unknown	Board
MAE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe · secondary	0.855	[0.714, 1.007]	100	—	0.819	reproduced	unknown	Board

Protein subcellular localization (DeepLoc) protein-ml/protein-subcellular-localization

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.651	[0.635, 0.666]	2742	13 / 17	0.801	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.481	[0.461, 0.497]	2742	9 / 17	0.640	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.575	[0.555, 0.593]	2742	—	0.758	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.600	[0.530, 0.670]	100	2 / 2	0.700	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.470	[0.389, 0.543]	100	2 / 2	0.563	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board
MCC (logistic probe) ↑ higher is better · probe · secondary	0.533	[0.448, 0.623]	100	—	0.652	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100	Board