Models / bio-nlp / biomednlp-biomedbert-base-uncased-abstract-fulltext / benchmarks

bio-nlp/biomednlp-biomedbert-base-uncased-abstract-fulltext

Visibility: PUBLIC

Model card FilesBenchmarks

Group by

BC5CDR chemical NER bio-nlp/bc5cdr-chemical-ner

Drawn from: bc5cdr

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Chemical entity F1 (token probe) ↑ higher is better · probe	0.878	[0.841, 0.912]	100	3 / 16	0.889	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Chemical entity precision (token probe) ↑ higher is better · probe	0.861	[0.813, 0.912]	100	3 / 16	0.870	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Chemical entity recall (token probe) ↑ higher is better · probe	0.895	[0.864, 0.925]	100	2 / 16	0.914	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board

Clinical entity recognition (MACCROBAT) bio-nlp/clinical-entity-recognition

Drawn from: MACCROBAT_biomedical_ner

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, 3 types (token probe) ↑ higher is better · probe	0.538	[0.468, 0.601]	100	3 / 16	0.724	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board
Entity precision, 3 types (token probe) ↑ higher is better · probe	0.500	[0.433, 0.567]	100	3 / 16	0.709	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board
Entity recall, 3 types (token probe) ↑ higher is better · probe	0.581	[0.502, 0.650]	100	3 / 16	0.739	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board
Entity macro F1, 3 types (token probe) ↑ higher is better · probe · secondary	0.522	[0.443, 0.594]	100	—	0.728	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board

PubMedQA accuracy bio-nlp/pubmedqa-accuracy

Drawn from: PubMedQA

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.554	[0.518, 0.592]	500	5 / 17	0.576	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.399	[0.354, 0.445]	500	6 / 17	0.444	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board

Spanish occupation entity recognition (ProfNER) bio-nlp/spanish-occupation-entity-recognition

Drawn from: ProfNER_corpus_NER

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.357	[0.111, 0.559]	100	5 / 16	0.630	reproduced	none declared	Board
Entity precision, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.769	[0.375, 1.000]	100	2 / 16	1.000	reproduced	none declared	Board
Entity recall, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.233	[0.067, 0.413]	100	5 / 16	0.535	reproduced	none declared	Board
Entity macro F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe · secondary	0.189	[0.071, 0.546]	100	—	0.329	reproduced	none declared	Board