Atlas
Sign in
Models / protein-ml / human-protein-language-model-0-9 / benchmarks

protein-ml/human-protein-language-model-0-9

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.205 [−0.237, 0.604]25
MET_HUMAN_Estevam_20230.361 [−0.077, 0.697]25
S22A1_HUMAN_Yee_2023_activity0.073 [−0.334, 0.441]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM−0.104 [−0.489, 0.308]25
 0.134 [−0.075, 0.344] 100 13 / 13 0.476 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
AUC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.625 [0.372, 0.857]25
MET_HUMAN_Estevam_20230.825 [0.617, 0.986]25
S22A1_HUMAN_Yee_2023_activity0.513 [0.262, 0.760]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.521 [0.254, 0.771]25
 0.621 [0.506, 0.746] 100 12 / 13 0.759 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
MCC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.199 [−0.294, 0.554]25
MET_HUMAN_Estevam_20230.600 [0.116, 0.850]25
S22A1_HUMAN_Yee_2023_activity−0.033 [−0.447, 0.360]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM−0.053 [−0.487, 0.408]25
 0.178 [−0.082, 0.352] 100 11 / 13 0.426 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.555 [0.000, 0.810]25
MET_HUMAN_Estevam_20230.925 [0.847, 0.990]25
S22A1_HUMAN_Yee_2023_activity0.593 [0.192, 0.928]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.468 [0.059, 1.000]25
 0.635 [0.454, 0.827] 100 — 0.635 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.000 [0.000, 0.600]25
MET_HUMAN_Estevam_20230.333 [0.000, 1.000]25
S22A1_HUMAN_Yee_2023_activity0.000 [0.000, 1.000]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.333 [0.000, 0.800]25
 0.167 best [0.000, 0.500] 100 — 0.167 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board