Atlas
Sign in
Models / single-cell / geneformer / benchmarks

single-cell/geneformer

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

cellxgene-census single-cell/cellxgene-census

Cell type annotation (macro F1) single-cell/cell-type-annotation-macro-f1

Drawn from: cellxgene-census

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Macro F1 (zero-shot) ↑ higher is better · zero-shot 0.562 [0.499, 0.618] 100 3 / 3 0.688 reproduced unknown Board
ARI (zero-shot clustering) ↑ higher is better · zero-shot 0.427 [0.410, 0.510] 100 3 / 3 0.557 reproduced unknown Board
Macro F1 (logistic probe) ↑ higher is better · probe 0.663 best [0.608, 0.714] 100 1 / 3 0.663 reproduced unknown Board
Accuracy (zero-shot) ↑ higher is better · zero-shot · secondary 0.590 [0.520, 0.660] 100 — 0.710 reproduced unknown Board
NMI (zero-shot clustering) ↑ higher is better · zero-shot · secondary 0.692 [0.687, 0.753] 100 — 0.758 reproduced unknown Board
Cell-type ASW (zero-shot) ↑ higher is better · zero-shot · secondary 0.550 [0.542, 0.558] 100 — 0.560 reproduced unknown Board
Accuracy (logistic probe) ↑ higher is better · probe · secondary 0.700 best [0.650, 0.760] 100 — 0.700 reproduced unknown Board

Single-cell batch integration (scIB) single-cell/single-cell-batch-integration

Drawn from: cellxgene-census

atlas-n500-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Cell-type ASW (zero-shot) ↑ higher is better · zero-shot 0.549 [0.544, 0.553] 500 4 / 4 0.624 reproduced unknown Board
Batch ASW (zero-shot) ↑ higher is better · zero-shot 0.811 [0.802, 0.819] 500 2 / 4 0.875 reproduced unknown Board
Graph connectivity (zero-shot) ↑ higher is better · zero-shot 0.990 [0.980, 0.998] 500 4 / 4 1.000 reproduced unknown Board
scIB bio conservation, unscaled (zero-shot) ↑ higher is better · zero-shot 0.634 [0.623, 0.652] 500 4 / 4 0.711 reproduced unknown Board
scIB batch correction, unscaled (zero-shot) ↑ higher is better · zero-shot 0.510 [0.495, 0.523] 500 3 / 4 0.644 reproduced unknown Board
scIB total, unscaled (zero-shot) ↑ higher is better · zero-shot 0.584 [0.575, 0.597] 500 4 / 4 0.684 reproduced unknown Board
NMI, k-means (zero-shot) ↑ higher is better · zero-shot · secondary 0.685 [0.671, 0.721] 500 — 0.764 reproduced unknown Board
ARI, k-means (zero-shot) ↑ higher is better · zero-shot · secondary 0.438 [0.413, 0.477] 500 — 0.534 reproduced unknown Board
Isolated-label ASW (zero-shot) ↑ higher is better · zero-shot · secondary 0.604 [0.593, 0.614] 500 — 0.681 reproduced unknown Board
cLISI (zero-shot) ↑ higher is better · zero-shot · secondary 0.896 [0.886, 0.905] 500 — 0.952 reproduced unknown Board
iLISI (zero-shot) ↑ higher is better · zero-shot · secondary 0.120 [0.103, 0.135] 500 — 0.245 reproduced unknown Board
kBET acceptance (zero-shot) ↑ higher is better · zero-shot · secondary 0.505 [0.470, 0.541] 500 — 0.632 reproduced unknown Board
scIB batch correction, unscaled, without PCR (zero-shot) ↑ higher is better · zero-shot · secondary 0.606 [0.596, 0.617] 500 — 0.673 reproduced unknown Board
scIB total, unscaled, without PCR (zero-shot) ↑ higher is better · zero-shot · secondary 0.623 [0.615, 0.636] 500 — 0.692 reproduced unknown Board
PCR comparison (zero-shot) ↑ higher is better · zero-shot · secondary 0.122 [0.062, 0.174] 500 — 0.573 reproduced unknown Board