Atlas
Sign in

Benchmarks

45 benchmarks in 8 domains

Literature

3 benchmarks · probe · 17 models
Protocol
Benchmarks: 3 of 3

Mean rank and coverage are recomputed over the benchmarks ticked here. None ticked means all.

Reset

17 models

 Ranked from 2 of 3 benchmarks (a third, at least 2)
# Model ↕ Mean rank ▲ Coverage ↕ BC5CDR chemical NER ↑ Chemical entity F1 (token probe) ↕ PubMedQA accuracy ↑ Accuracy (logistic probe) ↕ Spanish occupation entity recognition (ProfNER) ↑ Entity F1, PROFESION + SITUACION_LABORAL (token probe) ↕
1 bio-nlp/biomedlm overlap 0.20 3/3 0.852 (6) * 0.560 (4) * 0.603 (2)
2 bio-nlp/biomednlp-biomedbert-base-uncased-abstract-fulltext overlap 0.22 3/3 0.878 (3) * 0.554 (5) * 0.357 (5)
3 bio-nlp/biomedvlp-cxr-bert-specialized overlap 0.22 3/3 0.889 (1) * 0.564 (2) * 0.231 (10)
4 bio-nlp/roberta-base-biomedical-es 0.28 3/3 0.849 (7) 0.544 (8) 0.630 (1)
5 bio-nlp/biobert-base-cased-v1-2 overlap 0.31 3/3 0.878 (2) * 0.562 (3) * 0.083 (12)
6 bio-nlp/biogpt-large overlap 0.33 3/3 0.848 (8) * 0.576 (1) * 0.250 (9)
7 bio-nlp/scibert-scivocab-uncased 0.34 3/3 0.874 (4) 0.538 (9) 0.345 (6)
8 bio-nlp/clinical-assertion-negation-bert overlap 0.50 3/3 0.863 (5) * 0.548 (7) * 0.044 (14)
9 bio-nlp/clinicalbert 0.52 3/3 0.771 (12) 0.518 (11) 0.448 (4)
10 bio-nlp/sapbert-from-roberta-base-biomedical-clinical-es 0.56 3/3 0.821 (10) 0.504 (16) 0.575 (3)
11 bio-nlp/biomedical-ner-all 0.62 3/3 0.746 (14) 0.552 (6) 0.226 (11)
12 bio-nlp/pubmedbert-base-embeddings overlap 0.63 3/3 0.805 (11) * 0.516 (14) * 0.259 (7)
13 bio-nlp/bert-base-uncased-clinical-ner 0.63 3/3 0.752 (13) 0.518 (11) 0.258 (8)
14 bio-nlp/bio-clinicalbert overlap 0.74 3/3 0.829 (9) * 0.512 (15) * 0.045 (13)
15 bio-nlp/sapbert-from-pubmedbert-fulltext overlap 0.81 3/3 0.655 (16) * 0.532 (10) * 0.044 (14)
16 bio-nlp/s-pubmedbert-medquad 0.85 3/3 0.734 (15) 0.518 (11) 0.000 (16)
Low coverage: below 2 of 3, or nothing to compare with. Listed, not ranked.
— bio-nlp/biomedclip-pubmedbert-256-vit-base-patch16-224 overlap 1.00 1/3 · 0.500 (17) * ·
Reference floors: trivial baselines on the same items, not hub models, unranked.
— bow-text floor · 0.552 ·

Each cell is the model's value on that benchmark's board, with its rank there in brackets, as the benchmark's own leaderboard ranks it (its primary probe metric, at its default version); hover for the 95% interval, n and source, click to open that board. A * marks a declared train/test overlap. Rows are ordered by mean normalised rank, (rank − 1) / (board size − 1) averaged over the benchmarks the model has a result on (a board of one entry does not count): ranks, never values, so no average mixes units. Filtering hides rows without re-ranking anyone.