Atlas
Sign in
Models / protein-ml / prot-t5-xl-uniref50 / benchmarks

protein-ml/prot-t5-xl-uniref50

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

ProteinGym protein-ml/proteingym

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.770 [0.661, 0.855]100
CASP3_HUMAN_Roychowdhury_20200.572 [0.410, 0.712]100
CCR5_HUMAN_Gill_20230.279 [0.091, 0.465]100
CP2C9_HUMAN_Amorosi_2021_abundance0.784 [0.673, 0.855]100
ILF3_HUMAN_Tsuboyama_2023_2L330.754 [0.615, 0.862]100
KCNE1_HUMAN_Muhammad_2023_expression0.505 [0.319, 0.658]100
MLAC_ECOLI_MacRae_20230.541 [0.355, 0.685]100
MTH3_HAEAE_RockahShmuel_20150.509 [0.330, 0.654]100
RPC1_BP434_Tsuboyama_2023_1R690.917 [0.866, 0.945]100
SPG1_STRSG_Olson_20140.847 [0.768, 0.908]100
 0.648 [0.599, 0.688] 1000 2 / 17 0.661 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
AUC (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.852 [0.773, 0.929]100
CASP3_HUMAN_Roychowdhury_20200.777 [0.684, 0.868]100
CCR5_HUMAN_Gill_20230.670 [0.564, 0.772]100
CP2C9_HUMAN_Amorosi_2021_abundance0.897 [0.826, 0.948]100
ILF3_HUMAN_Tsuboyama_2023_2L330.922 [0.849, 0.977]100
KCNE1_HUMAN_Muhammad_2023_expression0.803 [0.689, 0.906]100
MLAC_ECOLI_MacRae_20230.784 [0.682, 0.871]100
MTH3_HAEAE_RockahShmuel_20150.713 [0.576, 0.834]100
RPC1_BP434_Tsuboyama_2023_1R690.974 [0.937, 1.000]100
SPG1_STRSG_Olson_20140.974 [0.935, 0.999]100
 0.837 [0.810, 0.865] 1000 2 / 17 0.848 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
MCC (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.540 [0.360, 0.740]100
CASP3_HUMAN_Roychowdhury_20200.341 [0.160, 0.600]100
CCR5_HUMAN_Gill_20230.200 [0.020, 0.441]100
CP2C9_HUMAN_Amorosi_2021_abundance0.682 [0.485, 0.787]100
ILF3_HUMAN_Tsuboyama_2023_2L330.567 [0.388, 0.708]100
KCNE1_HUMAN_Muhammad_2023_expression0.445 [0.234, 0.614]100
MLAC_ECOLI_MacRae_20230.520 [0.307, 0.666]100
MTH3_HAEAE_RockahShmuel_20150.281 [0.048, 0.436]100
RPC1_BP434_Tsuboyama_2023_1R690.735 [0.595, 0.905]100
SPG1_STRSG_Olson_20140.403 [0.289, 0.531]100
 0.472 [0.415, 0.527] 1000 2 / 17 0.472 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
NDCG, top 10 % (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.822 [0.713, 0.951]100
CASP3_HUMAN_Roychowdhury_20200.864 [0.708, 0.932]100
CCR5_HUMAN_Gill_20230.821 [0.583, 0.883]100
CP2C9_HUMAN_Amorosi_2021_abundance0.922 [0.862, 0.971]100
ILF3_HUMAN_Tsuboyama_2023_2L330.957 [0.891, 0.982]100
KCNE1_HUMAN_Muhammad_2023_expression0.648 [0.472, 0.831]100
MLAC_ECOLI_MacRae_20230.888 [0.802, 0.967]100
MTH3_HAEAE_RockahShmuel_20150.636 [0.383, 0.813]100
RPC1_BP434_Tsuboyama_2023_1R690.946 [0.897, 0.987]100
SPG1_STRSG_Olson_20140.895 [0.844, 0.962]100
 0.840 [0.791, 0.872] 1000 4 / 17 0.860 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
Top-10 % recall (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.400 [0.100, 0.727]100
CASP3_HUMAN_Roychowdhury_20200.300 [0.091, 0.600]100
CCR5_HUMAN_Gill_20230.200 [0.000, 0.400]100
CP2C9_HUMAN_Amorosi_2021_abundance0.700 [0.300, 0.833]100
ILF3_HUMAN_Tsuboyama_2023_2L330.500 [0.200, 0.800]100
KCNE1_HUMAN_Muhammad_2023_expression0.300 [0.091, 0.600]100
MLAC_ECOLI_MacRae_20230.400 [0.091, 0.636]100
MTH3_HAEAE_RockahShmuel_20150.000 [0.000, 0.333]100
RPC1_BP434_Tsuboyama_2023_1R690.700 [0.333, 0.900]100
SPG1_STRSG_Olson_20140.100 [0.000, 0.500]100
 0.360 best [0.259, 0.436] 1000 1 / 17 0.360 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board

ProteinGym_v1 protein-ml/proteingym-v1

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Drawn from: ProteinGym, ProteinGym_v1

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.770 [0.661, 0.855]100
CASP3_HUMAN_Roychowdhury_20200.572 [0.410, 0.712]100
CCR5_HUMAN_Gill_20230.279 [0.091, 0.465]100
CP2C9_HUMAN_Amorosi_2021_abundance0.784 [0.673, 0.855]100
ILF3_HUMAN_Tsuboyama_2023_2L330.754 [0.615, 0.862]100
KCNE1_HUMAN_Muhammad_2023_expression0.505 [0.319, 0.658]100
MLAC_ECOLI_MacRae_20230.541 [0.355, 0.685]100
MTH3_HAEAE_RockahShmuel_20150.509 [0.330, 0.654]100
RPC1_BP434_Tsuboyama_2023_1R690.917 [0.866, 0.945]100
SPG1_STRSG_Olson_20140.847 [0.768, 0.908]100
 0.648 [0.599, 0.688] 1000 2 / 17 0.661 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
AUC (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.852 [0.773, 0.929]100
CASP3_HUMAN_Roychowdhury_20200.777 [0.684, 0.868]100
CCR5_HUMAN_Gill_20230.670 [0.564, 0.772]100
CP2C9_HUMAN_Amorosi_2021_abundance0.897 [0.826, 0.948]100
ILF3_HUMAN_Tsuboyama_2023_2L330.922 [0.849, 0.977]100
KCNE1_HUMAN_Muhammad_2023_expression0.803 [0.689, 0.906]100
MLAC_ECOLI_MacRae_20230.784 [0.682, 0.871]100
MTH3_HAEAE_RockahShmuel_20150.713 [0.576, 0.834]100
RPC1_BP434_Tsuboyama_2023_1R690.974 [0.937, 1.000]100
SPG1_STRSG_Olson_20140.974 [0.935, 0.999]100
 0.837 [0.810, 0.865] 1000 2 / 17 0.848 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
MCC (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.540 [0.360, 0.740]100
CASP3_HUMAN_Roychowdhury_20200.341 [0.160, 0.600]100
CCR5_HUMAN_Gill_20230.200 [0.020, 0.441]100
CP2C9_HUMAN_Amorosi_2021_abundance0.682 [0.485, 0.787]100
ILF3_HUMAN_Tsuboyama_2023_2L330.567 [0.388, 0.708]100
KCNE1_HUMAN_Muhammad_2023_expression0.445 [0.234, 0.614]100
MLAC_ECOLI_MacRae_20230.520 [0.307, 0.666]100
MTH3_HAEAE_RockahShmuel_20150.281 [0.048, 0.436]100
RPC1_BP434_Tsuboyama_2023_1R690.735 [0.595, 0.905]100
SPG1_STRSG_Olson_20140.403 [0.289, 0.531]100
 0.472 [0.415, 0.527] 1000 2 / 17 0.472 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
NDCG, top 10 % (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.822 [0.713, 0.951]100
CASP3_HUMAN_Roychowdhury_20200.864 [0.708, 0.932]100
CCR5_HUMAN_Gill_20230.821 [0.583, 0.883]100
CP2C9_HUMAN_Amorosi_2021_abundance0.922 [0.862, 0.971]100
ILF3_HUMAN_Tsuboyama_2023_2L330.957 [0.891, 0.982]100
KCNE1_HUMAN_Muhammad_2023_expression0.648 [0.472, 0.831]100
MLAC_ECOLI_MacRae_20230.888 [0.802, 0.967]100
MTH3_HAEAE_RockahShmuel_20150.636 [0.383, 0.813]100
RPC1_BP434_Tsuboyama_2023_1R690.946 [0.897, 0.987]100
SPG1_STRSG_Olson_20140.895 [0.844, 0.962]100
 0.840 [0.791, 0.872] 1000 4 / 17 0.860 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
Top-10 % recall (ridge probe) ↑ higher is better · probe
By assay: 10 levels
assayValue [95% CI]n
ANCSZ_Hobbs_20220.400 [0.100, 0.727]100
CASP3_HUMAN_Roychowdhury_20200.300 [0.091, 0.600]100
CCR5_HUMAN_Gill_20230.200 [0.000, 0.400]100
CP2C9_HUMAN_Amorosi_2021_abundance0.700 [0.300, 0.833]100
ILF3_HUMAN_Tsuboyama_2023_2L330.500 [0.200, 0.800]100
KCNE1_HUMAN_Muhammad_2023_expression0.300 [0.091, 0.600]100
MLAC_ECOLI_MacRae_20230.400 [0.091, 0.636]100
MTH3_HAEAE_RockahShmuel_20150.000 [0.000, 0.333]100
RPC1_BP434_Tsuboyama_2023_1R690.700 [0.333, 0.900]100
SPG1_STRSG_Olson_20140.100 [0.000, 0.500]100
 0.360 best [0.259, 0.436] 1000 1 / 17 0.360 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board

tape protein-ml/tape

Secondary structure (Q3) protein-ml/secondary-structure-q3

Drawn from: tape

atlas-n100-v2

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Q3 accuracy (per-residue probe) ↑ higher is better · probe 0.813 best [0.799, 0.828] 100 1 / 17 0.813 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
Q8 accuracy (per-residue probe) ↑ higher is better · probe 0.663 best [0.644, 0.683] 100 1 / 17 0.663 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board

UniProt Knowledgebase protein-ml/uniprot-knowledgebase

GO function prediction (Swiss-Prot 2025H2) protein-ml/protein-function-go-prediction

Drawn from: UniProt Knowledgebase

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Fmax (molecular function, probe) ↑ higher is better · probe 0.654 [0.601, 0.715] 100 10 / 17 0.844 reproduced unknown Board
Fmax (biological process, probe) ↑ higher is better · probe 0.423 [0.331, 0.527] 100 14 / 17 0.538 reproduced unknown Board
Fmax (cellular component, probe) ↑ higher is better · probe 0.742 [0.694, 0.803] 100 4 / 17 0.756 reproduced unknown Board
Micro AUPR (molecular function, probe) ↑ higher is better · probe · secondary 0.625 [0.554, 0.704] 100 — 0.875 reproduced unknown Board
Smin (molecular function, probe) ↓ lower is better · bits · probe · secondary 3.804 [2.904, 4.454] 100 — 1.727 reproduced unknown Board
Micro AUPR (biological process, probe) ↑ higher is better · probe · secondary 0.371 [0.251, 0.509] 100 — 0.595 reproduced unknown Board
Smin (biological process, probe) ↓ lower is better · bits · probe · secondary 6.816 [4.728, 8.377] 100 — 4.842 reproduced unknown Board
Micro AUPR (cellular component, probe) ↑ higher is better · probe · secondary 0.751 [0.694, 0.812] 100 — 0.758 reproduced unknown Board
Smin (cellular component, probe) ↓ lower is better · bits · probe · secondary 3.179 [2.446, 3.843] 100 — 3.092 reproduced unknown Board

Other Benchmarks drawn from no hub dataset you can see

Protein membrane association (DeepLoc) protein-ml/protein-membrane-localization

atlas-full-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Q2 accuracy (logistic probe) ↑ higher is better · probe 0.861 [0.845, 0.878] 1728 2 / 17 0.888 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
MCC (logistic probe) ↑ higher is better · probe · secondary 0.715 [0.681, 0.751] 1728 — 0.771 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board

Protein stability ΔΔG (Mega-scale) protein-ml/protein-stability-ddg

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (ridge probe) ↑ higher is better · probe 0.297 best [0.114, 0.481] 100 1 / 17 0.297 reproduced unknown Board
RMSE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe 1.134 [0.913, 1.346] 100 13 / 17 1.095 reproduced unknown Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary 0.349 best [0.124, 0.553] 100 — 0.349 reproduced unknown Board
MAE, kcal/mol (ridge probe) ↓ lower is better · kcal/mol · probe · secondary 0.863 [0.712, 1.002] 100 — 0.819 reproduced unknown Board

Protein subcellular localization (DeepLoc) protein-ml/protein-subcellular-localization

atlas-full-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe) ↑ higher is better · probe 0.754 [0.740, 0.768] 2742 3 / 17 0.801 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
Macro F1 (logistic probe) ↑ higher is better · probe 0.568 [0.547, 0.591] 2742 2 / 17 0.640 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
MCC (logistic probe) ↑ higher is better · probe · secondary 0.701 [0.684, 0.718] 2742 — 0.758 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe) ↑ higher is better · probe 0.700 best [0.630, 0.770] 100 1 / 2 0.700 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
Macro F1 (logistic probe) ↑ higher is better · probe 0.563 best [0.481, 0.636] 100 1 / 2 0.563 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board
MCC (logistic probe) ↑ higher is better · probe · secondary 0.652 best [0.570, 0.735] 100 — 0.652 reproduced
overlap

Train/test overlap declared by the evaluator: BFD, protein-ml/uniref50

 Board