PoseBusters PB-valid rate

What is measured. Whether a predicted protein-ligand pose is both right (within 2 Å RMSD of the crystal pose) and physically possible (the PoseBusters checks), the criterion of Buttenschoen et al. (Chem. Sci. 2024): docking methods that look accurate on RMSD alone often return poses with impossible geometry or steric clashes.

Data. The paper's archive on Zenodo (record 8278563, posebusters_paper_data.zip, md5-pinned): the PoseBusters Benchmark set, 428 PDB complexes released 2021 or later, one directory per <PDB>_<CCD> with the receptor (ligand of interest removed, cofactors kept), the crystal ligand, its copies and an RDKit start conformer. The Astex Diverse set in the same archive is not used.

Items (exc). Every complex with at most 400 amino-acid residues over all chains and a ligand of at most 50 heavy atoms: 214 of the 428 (the cut is almost entirely on protein size — 201 complexes are too large, 10 have too big a ligand, 3 both). All 214 are test items, rather than 100 of them, because a PB-valid rate is a property of a set: the published numbers are over a whole set, and halving the set would double the width of the interval for no saving (docking is seconds per complex, co-folding minutes). The subset version is therefore atlas-full-v1. Of the 214, 165 are single-chain and 49 have 2-4 chains.

Docking (pb_valid_rate_docking_zero_shot, capability generate.pose). The worker gets the receptor PDB, the ligand SMILES, the RDKit start conformer and pocket_center (the centroid of the crystal ligand's heavy atoms). PoseBusters defines the site by the crystal ligand for methods that need one; a centre is the least of that information a box-based program needs, and every method gets the same. The reply is an SDF in the receptor's frame.

Co-folding (pb_valid_rate_cofolding_zero_shot, capability generate.complex). The worker gets the protein sequences (one per chain, observed residues, in file order) and the SMILES, and returns its own protein PDB plus the ligand SDF in its own frame. The predicted protein is superposed onto the crystal protein on the Cα atoms of the pocket (crystal residues with a Cα within 10 Å of a crystal ligand atom; the whole protein when fewer than three qualify) and the ligand is moved with it — the standard way a co-folded complex is compared with a crystal one.

RMSD. Symmetry-aware heavy-atom RMSD with no superposition of the ligands themselves (RDKit CalcRMS); bond orders are re-assigned from the crystal ligand when the prediction's disagree.

PB-valid. RMSD ≤ 2 Å and every check of the posebusters package (redock configuration: chemical validity and consistency with the input molecule, internal geometry — bond lengths, angles, planarity, no internal clash — and intermolecular validity against the protein). That package is imported lazily; when it is not installed the evaluator scores the RMSD criterion alone, says so in notes_extra.checks (rmsd_only) and in CellKey.evaluator_version, so an RMSD-only number can never be compared with a fully checked one as if they were the same metric. Installing posebusters is a dependency request.

Metric shape. y_true is pb_valid on every item and y_pred is pb_valid or fail:<first failing check>, so accuracy is the paper's combined "RMSD ≤ 2 Å & PB-valid" rate (pb_valid_rate_*) and the stored predictions say why each failure failed. 95 % bootstrap interval, 1,000 resamples of complexes.

The two separate rates (secondary). rmsd_success_rate_* is the paper's "RMSD ≤ 2 Å" rate, from the ligand_rmsd column alone (share_below, 2 Å inclusive). pb_checks_pass_rate_* is PB-valid alone: the share of poses passing every check whatever their RMSD (checks_pass_rate on the per-check verdicts the evaluator keeps in y_score, every pose checked; an unreadable or uncheckable pose fails), with each check's pass rate as its breakdown. Without posebusters there are no verdicts and the evaluator omits that metric, with the reason.

Median ligand RMSD. The evaluator measures the symmetry-aware RMSD of every item and stores it in the predictions frame (ligand_rmsd, Å); ligand_rmsd_median_{docking,cofolding}_zero_shot is its median over the complexes (registry metric median_ligand_rmsd, lower is better), with the same bootstrap of complexes. It separates a method that misses by 2.1 Å from one that misses by 8 Å, which the PB-valid rate cannot.

Caveats. Protein ≤ 400 residues and ligand ≤ 50 heavy atoms make this an easier set than the full 428, so numbers are not comparable with the paper's. The 2 Å threshold is conventional and a pose at 2.1 Å is not qualitatively worse than one at 1.9 Å — the median RMSD metric is there for that reason. Complexes released 2021 or later postdate the training cutoff of the docking programs of the paper but not necessarily of a model evaluated here; cofactors and metals are kept in the receptor, which some methods ignore. A co-folding method is judged on a crystal conformation of the protein it never saw, which is part of the task.

Data

• posebusters: zenodo ?, licence CC-BY-4.0

Subset

• seed 20261006
• test: all item(s)
• train: all item(s)

Metrics

• pb_valid_rate_docking_zero_shot — RMSD ≤ 2 Å & PB-valid rate (docking); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• pb_valid_rate_cofolding_zero_shot — RMSD ≤ 2 Å & PB-valid rate (co-folding); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• rmsd_success_rate_docking_zero_shot — RMSD ≤ 2 Å rate (docking); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• pb_checks_pass_rate_docking_zero_shot — PB-valid rate, checks only (docking); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• rmsd_success_rate_cofolding_zero_shot — RMSD ≤ 2 Å rate (co-folding); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• pb_checks_pass_rate_cofolding_zero_shot — PB-valid rate, checks only (co-folding); protocol zero_shot; higher is better; range [0.0, 1.0]; 95% bootstrap CI, 1000 resamples
• ligand_rmsd_median_docking_zero_shot — Median ligand RMSD (docking); protocol zero_shot; lower is better; range ≥ 0.0; 95% bootstrap CI, 1000 resamples
• ligand_rmsd_median_cofolding_zero_shot — Median ligand RMSD (co-folding); protocol zero_shot; lower is better; range ≥ 0.0; 95% bootstrap CI, 1000 resamples

Three eligible complexes (input: receptor + SMILES + pocket centre, or sequences + SMILES; expected output: the crystal ligand pose):

item_id	ligand (CCD)	amino-acid residues	chains	ligand heavy atoms	SMILES
7UF2_5SP	5SP	209	1	14	O=C(CO)[C@@H](O)[C@H](O)COP(=O)(O)O
7L7C_XQ1	XQ1	376	2	21	COc1cccc(-c2ccc3c(CC(=O)O)coc3c2)c1
6VTA_AKN	AKN	361	2	40	NCC[C@H](O)C(=O)N[C@@H]1C[C@H](N)… (kanamycin)

For 7UF2_5SP the receptor is one 209-residue chain, the ligand has 14 heavy atoms and the pocket centre is (−5.141, −5.079, 12.266) Å. A docking method returns an SDF in the receptor's frame; a pose 1.4 Å from the crystal one that passes every PoseBusters check scores pb_valid, a pose at 1.4 Å with an internal clash scores fail:<that check>, and a pose at 3 Å scores fail:rmsd. 7L7C_XQ1 has two 188-residue chains, so a co-folding method is handed both sequences and its complex is superposed on the crystal protein's pocket Cα atoms before the ligand RMSD is measured.