RNA secondary structure (F1)

What is measured. How much base-pairing information a model's frozen per-nucleotide representation carries: secondary structure of natural RNAs recovered by a small 2-D convolutional pair head (the "frozen" protocol of the RiNALMo and RNA-FM papers), scored by base-pair F1.

Data. The bpRNA-1m split of SPOT-RNA (Singh et al., Nature Communications 2019), TR0 / VL0 / TS0 = 10,814 / 1,300 / 1,305 RNAs clustered at 80 % identity (CD-HIT-EST) with no cluster across sets, as packaged by genbio-ai/rna-secondary-structure-prediction (the hub's rna-ml/rna-secondary-structure-prediction; Apache-2.0 LICENSE in the repository) at a pinned commit. It holds TR0/TS0, so the alternative catalogue entry rna-ml/pdb-rna-secondary-structure (a small PDB-derived set, no bpRNA split) is not used. Structures are bpRNA dot-brackets with pseudoknots; sequences upper-cased, T read as U.

Items. RNAs of at most 200 nt: 100 test RNAs from TS0 (1,106 eligible) and 2,000 train RNAs from TR0 (9,268 eligible), seed-hash rank (seed 20261006); train RNAs identical in sequence to a test RNA are dropped first. VL0 is not used. (Definition version 1 drew the same 100 test RNAs and 500 of these train RNAs.)

Probe (bp_f1_probe, bp_precision_probe, bp_recall_probe, bp_mcc_probe; head pair_conv). The model returns one feature vector per nucleotide (embed.rna.tokens, last layer). Runner side, on the CPU, fixed seed (0) and deterministic algorithms:

1. Reduction. Standardise, PCA to 32 components (fit on the train nucleotides), each scaled to unit train variance.
2. Pair map. A learned linear map 32 → 16 per nucleotide (a 1×1 convolution), then for an RNA of length L the outer concatenation [h_i, h_j, h_i ⊙ h_j] (48 channels, L × L; the product makes base complementarity linear in the input).
3. 2-D ResNet. A 1×1 convolution to 32 channels, 4 bottleneck residual blocks (InstanceNorm, ELU, 1×1 to 16, 3×3 with dilation 1, 2, 4, 8, 1×1 back to 32: a 31 × 31 receptive field over neighbouring pairs, so helix stacking is visible), InstanceNorm, ELU, 1×1 to one logit, symmetrised. About 16 k parameters.
4. Training. Binary cross-entropy, paired vs unpaired, on every pair with j − i > 3; AdamW (learning rate 3·10⁻³, weight decay 0.01), batches of 8 RNAs (loss summed over their pairs, divided by their number), epoch order a fixed permutation per epoch. 10 % of the train RNAs (fixed hash) are a validation split, the other 1,800 are fit: at most 30 epochs, early stopping when the validation loss has not improved by 10⁻⁴ for 4 epochs, and the best epoch's weights are kept.
5. Decoding. The threshold is the one of 0.05 … 0.9 with the best mean F1 on the validation RNAs. Test RNAs: candidate pairs are canonical (AU, GC, GU, either orientation) with j − i > 3 (no sharp loop); those above the threshold are accepted greedily by decreasing probability, a pair clashing with an accepted one (a nucleotide already paired) is skipped. Pseudoknots can be predicted.

F1 per RNA between predicted and true pairs (exact pairs), mean over the 100 test RNAs; an RNA with neither true nor predicted pairs counts 1.

Uncertainty. 95 % bootstrap interval, 1,000 resamples of test RNAs, conditional on the trained head (one seed: the head's training variance is not in the interval).

Caveats. The head is deliberately small and identical for every model, so the number ranks representations; it is not a folding method (no energy model, no nesting constraint) and is below dedicated predictors and the papers' heads trained on all of TR0. Sequence alone carries part of the signal: a one-hot window of ±3 nucleotides through this head is the floor to read a model against. Non-canonical true pairs (a few % in bpRNA) are never predicted. RNA language models pretrained on RNAcentral (RNA-FM, RiNALMo) have likely seen many TS0 sequences, without their structures. Mostly Rfam-derived RNAs (88 of the 100 test RNAs).

Data

• bprna: hf_dataset genbio-ai/rna-secondary-structure-prediction at revision 4aaf6b325b2132cf73efd2ae0bff499113d18be4, licence apache-2.0, catalogued as rna-ml/rna-secondary-structure-prediction

Subset

• seed 20261006
• test: 100 item(s)
• train: 2000 item(s), disjoint by sequence
• results at atlas-n100-v2 use subset 6ef0396a484a4ad9

Metrics

• bp_f1_probe — Base-pair F1 (2-D pair head); protocol probe; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• bp_precision_probe — Base-pair precision (2-D pair head); protocol probe; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• bp_recall_probe — Base-pair recall (2-D pair head); protocol probe; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• bp_mcc_probe — Base-pair MCC (2-D pair head); protocol probe; higher is better; range [-1.0, 1.0]; 95% bootstrap CI, 1000 resamples

Three training RNAs (bpRNA TR0, truncated):

item_id	sequence	dot-bracket
bpRNA_CRW_16221	UUCUUGACGACCAUAGAACAUUGGAACCACCUGAUCCCAUCC…AUCGUCAAGAUU (120 nt)	((((((((((.....((((((((....(((((((........…).)))))))))).
bpRNA_CRW_16283	CUAGGUGGCUAUAGAGAGAGGGAAACGCCUGGUCCCAUUCCG…UCGCUGCCUAGU (116 nt)	((((((((.....((((((((....(((((((..........…...)))))))).
bpRNA_CRW_16541	CCUGAUGACCAUAGCAAGUUGGUCCCACUCCUUCCCAUCCCG…CCCAAACCGCCC (133 nt)	((((((((.....((((((((....(((((((..........…............

Input: the sequence. Expected output: the set of base pairs (any bracket notation or (i, j) list); here three Comparative RNA Website (CRW) entries.