Atlas
Sign in
Models / bio-nlp / bert-base-uncased-clinical-ner / benchmarks

bio-nlp/bert-base-uncased-clinical-ner

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

BC5CDR chemical NER bio-nlp/bc5cdr-chemical-ner

Drawn from: bc5cdr

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Chemical entity F1 (token probe) ↑ higher is better · probe 0.752 [0.713, 0.788] 100 13 / 16 0.889 reproduced none declared Board
Chemical entity F1 (native NER head) ↑ higher is better · zero-shot 0.430 [0.377, 0.481] 100 2 / 2 0.541 reproduced none declared Board
Chemical entity precision (token probe) ↑ higher is better · probe 0.787 [0.736, 0.838] 100 12 / 16 0.870 reproduced none declared Board
Chemical entity recall (token probe) ↑ higher is better · probe 0.719 [0.673, 0.760] 100 16 / 16 0.914 reproduced none declared Board
Chemical entity precision (native NER head) ↑ higher is better · zero-shot 0.421 [0.368, 0.475] 100 2 / 2 0.725 reproduced none declared Board
Chemical entity recall (native NER head) ↑ higher is better · zero-shot 0.440 best [0.381, 0.502] 100 1 / 2 0.440 reproduced none declared Board

Clinical entity recognition (MACCROBAT) bio-nlp/clinical-entity-recognition

Drawn from: MACCROBAT_biomedical_ner

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Entity F1, 3 types (native NER head) ↑ higher is better · zero-shot 0.317 best [0.231, 0.391] 100 1 / 1 0.317 reproduced none declared Board
Entity F1, 3 types (token probe) ↑ higher is better · probe 0.510 [0.443, 0.572] 100 6 / 16 0.724 reproduced none declared Board
Entity precision, 3 types (native NER head) ↑ higher is better · zero-shot 0.287 best [0.207, 0.361] 100 1 / 1 0.287 reproduced none declared Board
Entity recall, 3 types (native NER head) ↑ higher is better · zero-shot 0.355 best [0.267, 0.436] 100 1 / 1 0.355 reproduced none declared Board
Entity precision, 3 types (token probe) ↑ higher is better · probe 0.462 [0.390, 0.531] 100 9 / 16 0.709 reproduced none declared Board
Entity recall, 3 types (token probe) ↑ higher is better · probe 0.568 [0.500, 0.634] 100 5 / 16 0.739 reproduced none declared Board
Entity macro F1, 3 types (native NER head) ↑ higher is better · zero-shot · secondary 0.357 best [0.267, 0.431] 100 — 0.357 reproduced none declared Board
Entity macro F1, 3 types (token probe) ↑ higher is better · probe · secondary 0.517 [0.439, 0.588] 100 — 0.728 reproduced none declared Board

PubMedQA accuracy bio-nlp/pubmedqa-accuracy

Drawn from: PubMedQA

atlas-full-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe) ↑ higher is better · probe 0.518 [0.482, 0.554] 500 11 / 17 0.576 reproduced none declared Board
Macro F1 (logistic probe) ↑ higher is better · probe 0.332 [0.303, 0.361] 500 17 / 17 0.444 reproduced none declared Board

Spanish occupation entity recognition (ProfNER) bio-nlp/spanish-occupation-entity-recognition

Drawn from: ProfNER_corpus_NER

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Entity F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe 0.258 [0.082, 0.425] 100 8 / 16 0.630 reproduced none declared Board
Entity precision, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe 0.421 [0.167, 0.632] 100 14 / 16 1.000 reproduced none declared Board
Entity recall, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe 0.186 [0.056, 0.334] 100 7 / 16 0.535 reproduced none declared Board
Entity macro F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe · secondary 0.136 [0.053, 0.407] 100 — 0.329 reproduced none declared Board