Models / bio-nlp / biomedlm / benchmarks

bio-nlp/biomedlm

Visibility: PUBLIC

Model card FilesBenchmarks

Group by

BC5CDR chemical NER bio-nlp/bc5cdr-chemical-ner

Drawn from: bc5cdr

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Chemical entity F1 (token probe) ↑ higher is better · probe	0.852	[0.806, 0.891]	100	6 / 16	0.889	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Chemical entity precision (token probe) ↑ higher is better · probe	0.820	[0.749, 0.883]	100	9 / 16	0.870	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Chemical entity recall (token probe) ↑ higher is better · probe	0.885	[0.859, 0.912]	100	4 / 16	0.914	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board

Clinical entity recognition (MACCROBAT) bio-nlp/clinical-entity-recognition

Drawn from: MACCROBAT_biomedical_ner

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, 3 types (token probe) ↑ higher is better · probe	0.464	[0.397, 0.537]	100	14 / 16	0.724	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board
Entity precision, 3 types (token probe) ↑ higher is better · probe	0.433	[0.367, 0.502]	100	14 / 16	0.709	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board
Entity recall, 3 types (token probe) ↑ higher is better · probe	0.500	[0.424, 0.585]	100	12 / 16	0.739	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board
Entity macro F1, 3 types (token probe) ↑ higher is better · probe · secondary	0.453	[0.381, 0.520]	100	—	0.728	reproduced	overlap Train/test overlap declared by the evaluator: PubMed Central full text	Board

PubMedQA accuracy bio-nlp/pubmedqa-accuracy

Drawn from: PubMedQA

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (option log-likelihood) ↑ higher is better · zero-shot	0.508	[0.478, 0.538]	500	2 / 2	0.622	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Macro F1 (option log-likelihood) ↑ higher is better · zero-shot	0.280	[0.255, 0.306]	500	2 / 2	0.430	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.560	[0.524, 0.598]	500	4 / 17	0.576	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.419	[0.372, 0.468]	500	2 / 17	0.444	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board

Spanish occupation entity recognition (ProfNER) bio-nlp/spanish-occupation-entity-recognition

Drawn from: ProfNER_corpus_NER

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.603	[0.427, 0.754]	100	2 / 16	0.630	reproduced	none declared	Board
Entity precision, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.733	[0.540, 0.879]	100	4 / 16	1.000	reproduced	none declared	Board
Entity recall, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.512	[0.340, 0.697]	100	2 / 16	0.535	reproduced	none declared	Board
Entity macro F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe · secondary	0.314	[0.233, 0.741]	100	—	0.329	reproduced	none declared	Board