Atlas
Sign in
Models / protein-ml / human-protein-language-model-0-5 / benchmarks

protein-ml/human-protein-language-model-0-5

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

Domain variant fitness, human PLM 0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.053 [−0.353, 0.455]25
MET_HUMAN_Estevam_20230.246 [−0.188, 0.644]25
S22A1_HUMAN_Yee_2023_activity0.147 [−0.238, 0.524]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.192 [−0.215, 0.553]25
 0.160 [−0.051, 0.372] 100 13 / 13 0.476 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
AUC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.519 [0.273, 0.760]25
MET_HUMAN_Estevam_20230.708 [0.479, 0.904]25
S22A1_HUMAN_Yee_2023_activity0.487 [0.214, 0.738]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.514 [0.267, 0.770]25
 0.557 [0.436, 0.687] 100 13 / 13 0.759 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
MCC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.199 [−0.315, 0.554]25
MET_HUMAN_Estevam_20230.439 [−0.053, 0.763]25
S22A1_HUMAN_Yee_2023_activity−0.033 [−0.523, 0.360]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.113 [−0.368, 0.510]25
 0.180 [−0.086, 0.359] 100 10 / 13 0.426 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.486 [0.331, 0.770]25
MET_HUMAN_Estevam_20230.940 [0.419, 0.989]25
S22A1_HUMAN_Yee_2023_activity0.594 [0.107, 0.891]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.103 [0.061, 0.724]25
 0.530 [0.385, 0.720] 100 — 0.635 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
CCR5_HUMAN_Gill_20230.000 [0.000, 0.333]25
MET_HUMAN_Estevam_20230.000 [0.000, 0.750]25
S22A1_HUMAN_Yee_2023_activity0.000 [0.000, 0.333]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.333 [0.000, 0.800]25
 0.083 [0.000, 0.333] 100 — 0.167 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board