Atlas
Sign in
Models / protein-ml / human-protein-language-model-clans-0-5 / benchmarks

protein-ml/human-protein-language-model-clans-0-5

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

Domain variant fitness, human PLM Clans-0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-05

Drawn from: ProteinGym, ProteinGym_v1

atlas-n25-v2

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
ADRB2_HUMAN_Jones_20200.121 [−0.235, 0.406]25
CCR5_HUMAN_Gill_20230.154 [−0.302, 0.595]25
OPSD_HUMAN_Wan_20190.105 [−0.355, 0.509]25
S22A1_HUMAN_Yee_2023_activity0.196 [−0.236, 0.593]25
 0.144 [−0.063, 0.361] 100 12 / 13 0.463 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
AUC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
ADRB2_HUMAN_Jones_20200.552 [0.292, 0.776]25
CCR5_HUMAN_Gill_20230.596 [0.357, 0.814]25
OPSD_HUMAN_Wan_20190.521 [0.250, 0.785]25
S22A1_HUMAN_Yee_2023_activity0.487 [0.220, 0.744]25
 0.539 [0.408, 0.671] 100 12 / 13 0.751 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
MCC (zero-shot, domain input) ↑ higher is better · zero-shot
By assay: 4 levels
assayValue [95% CI]n
ADRB2_HUMAN_Jones_20200.116 [−0.439, 0.519]25
CCR5_HUMAN_Gill_20230.199 [−0.277, 0.600]25
OPSD_HUMAN_Wan_20190.113 [−0.387, 0.471]25
S22A1_HUMAN_Yee_2023_activity−0.033 [−0.510, 0.421]25
 0.099 [−0.150, 0.293] 100 11 / 13 0.426 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
NDCG, top 10 % (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
ADRB2_HUMAN_Jones_20200.475 [0.347, 0.701]25
CCR5_HUMAN_Gill_20230.548 [0.000, 0.767]25
OPSD_HUMAN_Wan_20190.275 [0.000, 0.846]25
S22A1_HUMAN_Yee_2023_activity0.565 [0.000, 0.841]25
 0.466 [0.252, 0.653] 100 — 0.709 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board
Top-10 % recall (zero-shot, domain input) ↑ higher is better · zero-shot · secondary
By assay: 4 levels
assayValue [95% CI]n
ADRB2_HUMAN_Jones_20200.000 [0.000, 0.333]25
CCR5_HUMAN_Gill_20230.000 [0.000, 0.000]25
OPSD_HUMAN_Wan_20190.000 [0.000, 1.000]25
S22A1_HUMAN_Yee_2023_activity0.000 [0.000, 0.333]25
 0.000 [0.000, 0.250] 100 — 0.250 reproduced
overlap

Train/test overlap declared by the evaluator: human-proteome Pfam/clan domain sequences (DS-UFT), protein-ml/uniref50

 Board