Models / single-cell / scgpt / benchmarks

single-cell/scgpt

Visibility: PUBLIC

Model card FilesBenchmarks

Group by

cellxgene-census single-cell/cellxgene-census

Cell type annotation (macro F1) single-cell/cell-type-annotation-macro-f1

Drawn from: cellxgene-census

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Macro F1 (zero-shot) ↑ higher is better · zero-shot	0.688 best	[0.623, 0.751]	100	1 / 3	0.688	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
ARI (zero-shot clustering) ↑ higher is better · zero-shot	0.557 best	[0.510, 0.657]	100	1 / 3	0.557	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.634	[0.578, 0.683]	100	2 / 3	0.663	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
Accuracy (zero-shot) ↑ higher is better · zero-shot · secondary	0.710 best	[0.640, 0.780]	100	—	0.710	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
NMI (zero-shot clustering) ↑ higher is better · zero-shot · secondary	0.758 best	[0.742, 0.821]	100	—	0.758	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
Cell-type ASW (zero-shot) ↑ higher is better · zero-shot · secondary	0.560 best	[0.552, 0.569]	100	—	0.560	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
Accuracy (logistic probe) ↑ higher is better · probe · secondary	0.690	[0.640, 0.750]	100	—	0.700	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board

Single-cell batch integration (scIB) single-cell/single-cell-batch-integration

Drawn from: cellxgene-census

atlas-n500-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Cell-type ASW (zero-shot) ↑ higher is better · zero-shot	0.553	[0.548, 0.557]	500	3 / 4	0.624	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
Batch ASW (zero-shot) ↑ higher is better · zero-shot	0.794	[0.785, 0.803]	500	3 / 4	0.875	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
Graph connectivity (zero-shot) ↑ higher is better · zero-shot	0.996	[0.991, 1.000]	500	3 / 4	1.000	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
scIB bio conservation, unscaled (zero-shot) ↑ higher is better · zero-shot	0.642	[0.631, 0.660]	500	3 / 4	0.711	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
scIB batch correction, unscaled (zero-shot) ↑ higher is better · zero-shot	0.508	[0.492, 0.521]	500	4 / 4	0.644	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
scIB total, unscaled (zero-shot) ↑ higher is better · zero-shot	0.588	[0.579, 0.600]	500	3 / 4	0.684	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
NMI, k-means (zero-shot) ↑ higher is better · zero-shot · secondary	0.690	[0.676, 0.723]	500	—	0.764	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
ARI, k-means (zero-shot) ↑ higher is better · zero-shot · secondary	0.455	[0.426, 0.502]	500	—	0.534	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
Isolated-label ASW (zero-shot) ↑ higher is better · zero-shot · secondary	0.606	[0.596, 0.615]	500	—	0.681	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
cLISI (zero-shot) ↑ higher is better · zero-shot · secondary	0.909	[0.897, 0.915]	500	—	0.952	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
iLISI (zero-shot) ↑ higher is better · zero-shot · secondary	0.138	[0.121, 0.168]	500	—	0.245	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
kBET acceptance (zero-shot) ↑ higher is better · zero-shot · secondary	0.513	[0.479, 0.548]	500	—	0.632	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
scIB batch correction, unscaled, without PCR (zero-shot) ↑ higher is better · zero-shot · secondary	0.610	[0.599, 0.622]	500	—	0.673	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
scIB total, unscaled, without PCR (zero-shot) ↑ higher is better · zero-shot · secondary	0.630	[0.621, 0.642]	500	—	0.692	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board
PCR comparison (zero-shot) ↑ higher is better · zero-shot · secondary	0.097	[0.040, 0.143]	500	—	0.573	reproduced	overlap Train/test overlap declared by the evaluator: single-cell/cellxgene-census	Board