Atlas
Sign in
Models / bio-nlp / scibert-scivocab-uncased / benchmarks

bio-nlp/scibert-scivocab-uncased

Visibility: PUBLIC
Model cardFilesBenchmarks
Group by

BC5CDR chemical NER bio-nlp/bc5cdr-chemical-ner

Drawn from: bc5cdr

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Chemical entity F1 (token probe) ↑ higher is better · probe 0.874 [0.833, 0.908] 100 4 / 16 0.889 reproduced unknown Board
Chemical entity precision (token probe) ↑ higher is better · probe 0.860 [0.809, 0.911] 100 4 / 16 0.870 reproduced unknown Board
Chemical entity recall (token probe) ↑ higher is better · probe 0.887 [0.848, 0.920] 100 3 / 16 0.914 reproduced unknown Board

Clinical entity recognition (MACCROBAT) bio-nlp/clinical-entity-recognition

Drawn from: MACCROBAT_biomedical_ner

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Entity F1, 3 types (token probe) ↑ higher is better · probe 0.483 [0.417, 0.549] 100 9 / 16 0.724 reproduced unknown Board
Entity precision, 3 types (token probe) ↑ higher is better · probe 0.450 [0.384, 0.522] 100 12 / 16 0.709 reproduced unknown Board
Entity recall, 3 types (token probe) ↑ higher is better · probe 0.521 [0.449, 0.593] 100 8 / 16 0.739 reproduced unknown Board
Entity macro F1, 3 types (token probe) ↑ higher is better · probe · secondary 0.451 [0.382, 0.520] 100 — 0.728 reproduced unknown Board

PubMedQA accuracy bio-nlp/pubmedqa-accuracy

Drawn from: PubMedQA

atlas-full-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Accuracy (logistic probe) ↑ higher is better · probe 0.538 [0.500, 0.580] 500 9 / 17 0.576 reproduced unknown Board
Macro F1 (logistic probe) ↑ higher is better · probe 0.393 [0.351, 0.439] 500 8 / 17 0.444 reproduced unknown Board

Spanish occupation entity recognition (ProfNER) bio-nlp/spanish-occupation-entity-recognition

Drawn from: ProfNER_corpus_NER

atlas-n100-v1

Metric Value 95% CI n Rank Best in catalogue Source Overlap Board
Entity F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe 0.345 [0.161, 0.526] 100 6 / 16 0.630 reproduced none declared Board
Entity precision, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe 0.667 [0.400, 0.875] 100 6 / 16 1.000 reproduced none declared Board
Entity recall, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe 0.233 [0.100, 0.390] 100 5 / 16 0.535 reproduced none declared Board
Entity macro F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe · secondary 0.182 [0.095, 0.507] 100 — 0.329 reproduced none declared Board