Atlas
Sign in
Models / genomics / caduceus-ps-seqlen-131k-d-model-256-n-layer-16 / benchmarks

genomics/caduceus-ps-seqlen-131k-d-model-256-n-layer-16

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

Genomic Benchmarks accuracy genomics/genomic-benchmarks-accuracy

Drawn from: Genomic_Benchmarks_demo_coding_vs_intergenomic_seqs, Genomic_Benchmarks_human_enhancers_cohn, Genomic_Benchmarks_human_nontata_promoters

atlas-full-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe, mean of 3 sets) ↑ higher is better · probe
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.920 [0.916, 0.923]25000
Human enhancers (Cohn)0.722 [0.711, 0.733]6948
Human non-TATA promoters0.822 [0.814, 0.830]9034
 0.821 best [0.817, 0.826] 40982 1 / 5 0.821 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board
Macro F1 (logistic probe, mean of 3 sets) ↑ higher is better · probe · secondary
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.920 [0.916, 0.923]25000
Human enhancers (Cohn)0.722 [0.711, 0.732]6948
Human non-TATA promoters0.822 [0.814, 0.829]9034
 0.821 best [0.816, 0.826] 40982 — 0.821 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe, mean of 3 sets) ↑ higher is better · probe
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.910 [0.850, 0.960]100
Human enhancers (Cohn)0.800 [0.710, 0.870]100
Human non-TATA promoters0.800 [0.720, 0.880]100
 0.837 [0.793, 0.877] 300 2 / 2 0.853 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board
Macro F1 (logistic probe, mean of 3 sets) ↑ higher is better · probe · secondary
By dataset: 3 levels
datasetValue [95% CI]n
Coding vs intergenomic sequences (demo)0.910 [0.848, 0.960]100
Human enhancers (Cohn)0.799 [0.708, 0.870]100
Human non-TATA promoters0.799 [0.713, 0.878]100
 0.836 [0.791, 0.875] 300 — 0.853 reproduced
overlap

Train/test overlap declared by the evaluator: genomics/ucsc-hg38

 Board

Protein abundance from coding sequence (E. coli) rna-ml/coding-sequence-protein-abundance

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Spearman ρ (zero-shot pseudo-log-likelihood) ↑ higher is better · zero-shot 0.349 [0.139, 0.537] 100 3 / 11 0.552 reproduced none declared Board
Spearman ρ (ridge probe) ↑ higher is better · probe 0.600 [0.443, 0.716] 100 4 / 11 0.688 reproduced none declared Board
RMSE (ridge probe) ↓ lower is better · log10 ppm · probe 1.064 [0.941, 1.185] 100 5 / 11 0.973 reproduced none declared Board
Pearson r (ridge probe) ↑ higher is better · probe · secondary 0.570 [0.458, 0.665] 100 — 0.665 reproduced none declared Board
R² (ridge probe) ↑ higher is better · probe · secondary 0.296 [0.138, 0.411] 100 — 0.411 reproduced none declared Board