Atlas
Sign in
Models / protein-ml / human-protein-language-model-clans-0-9 / benchmarks

protein-ml/human-protein-language-model-clans-0-9

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

Domain variant fitness, human PLM Clans-0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-09

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
MET_HUMAN_Estevam_20230.113 [−0.316, 0.507]25
MK01_HUMAN_Brenan_20160.388 [−0.079, 0.722]25
SCN5A_HUMAN_Glazer_20190.398 [0.015, 0.676]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM−0.008 [−0.445, 0.402]25
 0.223 [0.021, 0.412] 100 9 / 13 0.362 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
AUC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
MET_HUMAN_Estevam_20230.604 [0.365, 0.821]25
MK01_HUMAN_Brenan_20160.846 [0.603, 1.000]25
SCN5A_HUMAN_Glazer_20190.807 [0.596, 0.968]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.576 [0.333, 0.790]25
 0.708 [0.596, 0.818] 100 2 / 13 0.722 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
MCC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
MET_HUMAN_Estevam_20230.116 [−0.447, 0.529]25
MK01_HUMAN_Brenan_20160.487 [0.045, 0.763]25
SCN5A_HUMAN_Glazer_20190.523 [0.033, 0.840]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM−0.053 [−0.458, 0.445]25
 0.268 [0.018, 0.459] 100 2 / 13 0.310 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
MET_HUMAN_Estevam_20230.637 [0.083, 0.976]25
MK01_HUMAN_Brenan_20160.819 [0.394, 0.938]25
SCN5A_HUMAN_Glazer_20190.774 [0.297, 1.000]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.266 [0.078, 1.000]25
 0.624 [0.406, 0.872] 100 — 0.725 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
MET_HUMAN_Estevam_20230.000 [0.000, 0.600]25
MK01_HUMAN_Brenan_20160.000 [0.000, 0.667]25
SCN5A_HUMAN_Glazer_20190.333 [0.000, 1.000]25
SRC_HUMAN_Chakraborty_2023_binding-DAS_25uM0.333 [0.000, 1.000]25
 0.167 [0.000, 0.475] 100 — 0.250 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board