Models / protein-ml / human-protein-language-model-0-9 / benchmarks

protein-ml/human-protein-language-model-0-9

Visibility: PUBLIC

Model card FilesBenchmarks

Group by

ProteinGym protein-ml/proteingym

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.205 [−0.237, 0.604]	25
MET_HUMAN_Estevam_2023	0.361 [−0.077, 0.697]	25
S22A1_HUMAN_Yee_2023_activity	0.073 [−0.334, 0.441]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.104 [−0.489, 0.308]	25

0.134

[−0.075, 0.344]

100

13 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.625 [0.372, 0.857]	25
MET_HUMAN_Estevam_2023	0.825 [0.617, 0.986]	25
S22A1_HUMAN_Yee_2023_activity	0.513 [0.262, 0.760]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.521 [0.254, 0.771]	25

0.621

[0.506, 0.746]

100

12 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.199 [−0.294, 0.554]	25
MET_HUMAN_Estevam_2023	0.600 [0.116, 0.850]	25
S22A1_HUMAN_Yee_2023_activity	−0.033 [−0.447, 0.360]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.487, 0.408]	25

0.178

[−0.082, 0.352]

100

11 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.555 [0.000, 0.810]	25
MET_HUMAN_Estevam_2023	0.925 [0.847, 0.990]	25
S22A1_HUMAN_Yee_2023_activity	0.593 [0.192, 0.928]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.468 [0.059, 1.000]	25

0.635

[0.454, 0.827]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.333 [0.000, 0.800]	25

0.167 best

[0.000, 0.500]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

ProteinGym_v1 protein-ml/proteingym-v1

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric

Value

95% CI

Rank

Best in catalogue

Source

Overlap

Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.205 [−0.237, 0.604]	25
MET_HUMAN_Estevam_2023	0.361 [−0.077, 0.697]	25
S22A1_HUMAN_Yee_2023_activity	0.073 [−0.334, 0.441]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.104 [−0.489, 0.308]	25

0.134

[−0.075, 0.344]

100

13 / 13

0.476

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

AUC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.625 [0.372, 0.857]	25
MET_HUMAN_Estevam_2023	0.825 [0.617, 0.986]	25
S22A1_HUMAN_Yee_2023_activity	0.513 [0.262, 0.760]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.521 [0.254, 0.771]	25

0.621

[0.506, 0.746]

100

12 / 13

0.759

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

MCC (zero-shot, domain input) ↑ higher is better · zero-shot

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.199 [−0.294, 0.554]	25
MET_HUMAN_Estevam_2023	0.600 [0.116, 0.850]	25
S22A1_HUMAN_Yee_2023_activity	−0.033 [−0.447, 0.360]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	−0.053 [−0.487, 0.408]	25

0.178

[−0.082, 0.352]

100

11 / 13

0.426

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.555 [0.000, 0.810]	25
MET_HUMAN_Estevam_2023	0.925 [0.847, 0.990]	25
S22A1_HUMAN_Yee_2023_activity	0.593 [0.192, 0.928]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.468 [0.059, 1.000]	25

0.635

[0.454, 0.827]

100

—

0.635

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board

Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary

By assay: 4 levels

assay	Value [95% CI]	n
CCR5_HUMAN_Gill_2023	0.000 [0.000, 0.600]	25
MET_HUMAN_Estevam_2023	0.333 [0.000, 1.000]	25
S22A1_HUMAN_Yee_2023_activity	0.000 [0.000, 1.000]	25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM	0.333 [0.000, 0.800]	25

0.167 best

[0.000, 0.500]

100

—

0.167

reproduced

overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

Board