Atlas
Sign in
Models / genomics / caduceus-ph-seqlen-131k-d-model-256-n-layer-16 / benchmarks

genomics/caduceus-ph-seqlen-131k-d-model-256-n-layer-16

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

Genomic Benchmarks accuracy genomics/genomic-benchmarks-accuracy

Drawn from: Genomic_Benchmarks_demo_coding_vs_intergenomic_seqs, Genomic_Benchmarks_human_enhancers_cohn, Genomic_Benchmarks_human_nontata_promoters

atlas-full-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe, mean of 3 sets) ↑ higher is better · probe
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.899 [0.895, 0.902]25000
Human enhancers (Cohn)0.723 [0.711, 0.733]6948
Human non-TATA promoters0.828 [0.821, 0.836]9034
 0.817 [0.812, 0.821] 40982 2 / 5 0.821 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board
Macro F1 (logistic probe, mean of 3 sets) ↑ higher is better · probe · secondary
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.899 [0.895, 0.902]25000
Human enhancers (Cohn)0.722 [0.711, 0.733]6948
Human non-TATA promoters0.828 [0.820, 0.836]9034
 0.816 [0.812, 0.821] 40982 — 0.821 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe, mean of 3 sets) ↑ higher is better · probe
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.940 [0.890, 0.980]100
Human enhancers (Cohn)0.790 [0.700, 0.860]100
Human non-TATA promoters0.830 [0.750, 0.900]100
 0.853 best [0.810, 0.887] 300 1 / 2 0.853 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board
Macro F1 (logistic probe, mean of 3 sets) ↑ higher is better · probe · secondary
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.940 [0.889, 0.980]100
Human enhancers (Cohn)0.789 [0.694, 0.860]100
Human non-TATA promoters0.830 [0.750, 0.900]100
 0.853 best [0.809, 0.886] 300 — 0.853 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board

Protein abundance from coding sequence (E. coli) rna-ml/coding-sequence-protein-abundance

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (zero-shot pseudo-log-likelihood) ↑ higher is better · zero-shot 0.304 [0.089, 0.499] 100 5 / 11 0.552 reproduced none declared Board
Spearman ρ (ridge probe) ↑ higher is better · probe 0.495 [0.325, 0.638] 100 9 / 11 0.688 reproduced none declared Board
RMSE (ridge probe) ↓ lower is better · log10 ppm · probe 1.119 [0.977, 1.263] 100 9 / 11 0.973 reproduced none declared Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary 0.513 [0.375, 0.631] 100 — 0.665 reproduced none declared Board
R² (ridge probe) ↑ higher is better · probe · secondary 0.220 [0.011, 0.376] 100 — 0.411 reproduced none declared Board