Protein subcellular localization (DeepLoc)

What is measured. Where in a eukaryotic cell a protein resides, from its amino-acid sequence alone, among ten locations: nucleus, cytoplasm, extracellular, mitochondrion, cell membrane, endoplasmic reticulum, plastid, Golgi apparatus, lysosome/vacuole, peroxisome.

Data. DeepLoc 1.0 (Almagro Armenteros et al., Bioinformatics 2017): Swiss-Prot proteins with experimentally supported locations, clustered at 30 % identity with every cluster kept inside one of five folds, one fold held out for testing. The AI4Protein/DeepLocMulti mirror keeps 13,724 of the 13,858 proteins; its test file is the test pool and its training file the train pool (its validation file is not used). Proteins annotated both cytoplasmic and nuclear carry the nucleus class, as in the source.

Items. atlas-full-v1: every protein of the test pool, 2,742 (nucleus 797 down to peroxisome 29), and 500 train proteins stratified by location (largest-remainder quotas, seed-hash rank inside each class). atlas-n100-v1: 100 test proteins stratified by location, at least 3 per location, and the same 500 train proteins. Sequences are full length; models with a length cap truncate as their adapter decides.

Test size. The leaderboard version is atlas-full-v1 on every profile: the whole DeepLoc test set (2,742 proteins) and the 500 train proteins. Embedding 2,742 sequences adds minutes per model; the probe's cost is its head, fit on the unchanged train set. The earlier atlas-n100-v1 (100 test items, drawn as described above) stays frozen and its cells stay current; it is a separate ranking, never mixed with the full one.

Probe (accuracy_probe, macro_f1_probe; secondary mcc_probe). Frozen embeddings (last layer, mean over residues); a multinomial logistic head is fit on the 500 train proteins and predicts every test protein. Macro F1 weighs the rare compartments as much as the nucleus. The multi-class MCC (Gorodkin's R_K) is DeepLoc 1.0's companion to accuracy.

Zero-shot (macro_f1_coarse5_zero_shot; secondary accuracy_coarse5_zero_shot). A model's own localization head predicts without any training here. Such heads rarely share DeepLoc's ten classes, so this metric is scored on a five-way grouping every one of them can be mapped onto (the label space of the hub's ESM_protein_localization classifier):

DeepLoc class	coarse-5
Cytoplasm	cytoplasmic
Mitochondrion	mitochondrial
Nucleus	nuclear
Extracellular	secreted
Cell membrane, Endoplasmic reticulum, Golgi apparatus, Lysosome/Vacuole, Peroxisome, Plastid	other

Five-class and ten-class numbers are not comparable with each other.

Uncertainty. 95 % bootstrap interval over test proteins, 1,000 resamples stratified by location.

Caveats. The rarest classes hold 29 (peroxisome) and 64 (lysosome/vacuole) test proteins (4–5 in atlas-n100-v1), so their per-class F1, and so macro F1, is noisier than accuracy; the interval shows how much. The probe trains on 500 proteins, not DeepLoc's 9,324: its numbers are below the literature's. Models trained on UniProt annotations may have seen these locations as text, not as labels; sequence models trained on UniRef have seen the sequences themselves, which is the usual pretraining overlap of protein probes.

Data

• deeploc: hf_dataset AI4Protein/DeepLocMulti at revision 1af82ded911c8b3faadf185d9357193de9f301c9, licence apache-2.0

Subset

• seed 20261006
• test: 100 item(s), stratified by label
• train: 500 item(s), stratified by label
• leaderboard version atlas-full-v1 on every profile: the whole eligible test pool (the sizes above define the n version)
• results at atlas-full-v1 use subset ce99ee830424cb7f
• results at atlas-n100-v1 use subset bbbffe95e32baaf3

Metrics

• accuracy_probe — Accuracy (logistic probe); protocol probe; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples, stratified by label
• macro_f1_probe — Macro F1 (logistic probe); protocol probe; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples, stratified by label
• mcc_probe — MCC (logistic probe); protocol probe; higher is better; range [-1.0, 1.0]; 95% bootstrap CI, 1000 resamples, stratified by label
• macro_f1_coarse5_zero_shot — Macro F1, 5 classes (native classifier); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples, stratified by label
• accuracy_coarse5_zero_shot — Accuracy, 5 classes (native classifier); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples, stratified by label

The leaderboard (atlas-full-v1) scores the whole DeepLoc test set, 2,742 proteins (100 in atlas-n100-v1). Three training proteins (input: the sequence; expected output: the location):

item_id	sequence	location	coarse-5
P0C893	DRDSCIDKSRCSKYGYYQECQDCCKKAGHNGGTCMFFKCKCA (42 aa)	Extracellular	secreted
O60356	MATFPPATSAPQQPPGPEDEDSSLDE…NSERKKRGARR (82 aa)	Nucleus	nuclear
O96011	MDAWVRFSAQSQARERLCRAAQYACS…LIYPWLRLKP (259 aa)	Peroxisome	other