Models / bio-nlp / biomedical-ner-all / benchmarks

bio-nlp/biomedical-ner-all

Visibility: PUBLIC

Model card FilesBenchmarks

Group by

bc5cdr bio-nlp/bc5cdr

BC5CDR chemical NER bio-nlp/bc5cdr-chemical-ner

Drawn from: bc5cdr

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Chemical entity F1 (token probe) ↑ higher is better · probe	0.746	[0.699, 0.786]	100	14 / 16	0.889	reproduced	none declared	Board
Chemical entity F1 (native NER head) ↑ higher is better · zero-shot	0.541 best	[0.473, 0.602]	100	1 / 2	0.541	reproduced	none declared	Board
Chemical entity precision (token probe) ↑ higher is better · probe	0.768	[0.719, 0.815]	100	14 / 16	0.870	reproduced	none declared	Board
Chemical entity recall (token probe) ↑ higher is better · probe	0.725	[0.671, 0.772]	100	15 / 16	0.914	reproduced	none declared	Board
Chemical entity precision (native NER head) ↑ higher is better · zero-shot	0.725 best	[0.667, 0.781]	100	1 / 2	0.725	reproduced	none declared	Board
Chemical entity recall (native NER head) ↑ higher is better · zero-shot	0.432	[0.362, 0.502]	100	2 / 2	0.440	reproduced	none declared	Board

MACCROBAT_biomedical_ner bio-nlp/maccrobat-biomedical-ner

Clinical entity recognition (MACCROBAT) bio-nlp/clinical-entity-recognition

Drawn from: MACCROBAT_biomedical_ner

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, 3 types (token probe) ↑ higher is better · probe	0.724 best	[0.652, 0.783]	100	1 / 16	0.724	reproduced	overlap Train/test overlap declared by the evaluator: bio-nlp/maccrobat-biomedical-ner	Board
Entity precision, 3 types (token probe) ↑ higher is better · probe	0.709 best	[0.627, 0.782]	100	1 / 16	0.709	reproduced	overlap Train/test overlap declared by the evaluator: bio-nlp/maccrobat-biomedical-ner	Board
Entity recall, 3 types (token probe) ↑ higher is better · probe	0.739 best	[0.678, 0.793]	100	1 / 16	0.739	reproduced	overlap Train/test overlap declared by the evaluator: bio-nlp/maccrobat-biomedical-ner	Board
Entity macro F1, 3 types (token probe) ↑ higher is better · probe · secondary	0.728 best	[0.643, 0.796]	100	—	0.728	reproduced	overlap Train/test overlap declared by the evaluator: bio-nlp/maccrobat-biomedical-ner	Board

ProfNER_corpus_NER bio-nlp/profner-corpus-ner

Spanish occupation entity recognition (ProfNER) bio-nlp/spanish-occupation-entity-recognition

Drawn from: ProfNER_corpus_NER

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.226	[0.061, 0.417]	100	11 / 16	0.630	reproduced	none declared	Board
Entity precision, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.600	[0.250, 0.889]	100	10 / 16	1.000	reproduced	none declared	Board
Entity recall, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.140	[0.034, 0.286]	100	10 / 16	0.535	reproduced	none declared	Board
Entity macro F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe · secondary	0.120	[0.035, 0.400]	100	—	0.329	reproduced	none declared	Board

PubMedQA bio-nlp/pubmedqa

PubMedQA accuracy bio-nlp/pubmedqa-accuracy

Drawn from: PubMedQA

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.552	[0.518, 0.586]	500	6 / 17	0.576	reproduced	none declared	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.383	[0.344, 0.430]	500	10 / 17	0.444	reproduced	none declared	Board