Models / bio-nlp / biogpt-large / benchmarks

bio-nlp/biogpt-large

Visibility: PUBLIC

Model card FilesBenchmarks

Group by

bc5cdr bio-nlp/bc5cdr

BC5CDR chemical NER bio-nlp/bc5cdr-chemical-ner

Drawn from: bc5cdr

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Chemical entity F1 (token probe) ↑ higher is better · probe	0.848	[0.806, 0.887]	100	8 / 16	0.889	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Chemical entity precision (token probe) ↑ higher is better · probe	0.829	[0.767, 0.889]	100	7 / 16	0.870	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Chemical entity recall (token probe) ↑ higher is better · probe	0.867	[0.836, 0.895]	100	7 / 16	0.914	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board

MACCROBAT_biomedical_ner bio-nlp/maccrobat-biomedical-ner

Clinical entity recognition (MACCROBAT) bio-nlp/clinical-entity-recognition

Drawn from: MACCROBAT_biomedical_ner

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, 3 types (token probe) ↑ higher is better · probe	0.474	[0.396, 0.549]	100	13 / 16	0.724	reproduced	unknown	Board
Entity precision, 3 types (token probe) ↑ higher is better · probe	0.469	[0.385, 0.552]	100	7 / 16	0.709	reproduced	unknown	Board
Entity recall, 3 types (token probe) ↑ higher is better · probe	0.479	[0.396, 0.559]	100	14 / 16	0.739	reproduced	unknown	Board
Entity macro F1, 3 types (token probe) ↑ higher is better · probe · secondary	0.458	[0.378, 0.534]	100	—	0.728	reproduced	unknown	Board

ProfNER_corpus_NER bio-nlp/profner-corpus-ner

Spanish occupation entity recognition (ProfNER) bio-nlp/spanish-occupation-entity-recognition

Drawn from: ProfNER_corpus_NER

atlas-n100-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Entity F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.250	[0.109, 0.390]	100	9 / 16	0.630	reproduced	none declared	Board
Entity precision, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.538	[0.273, 0.800]	100	11 / 16	1.000	reproduced	none declared	Board
Entity recall, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe	0.163	[0.067, 0.269]	100	8 / 16	0.535	reproduced	none declared	Board
Entity macro F1, PROFESION + SITUACION_LABORAL (token probe) ↑ higher is better · probe · secondary	0.132	[0.062, 0.370]	100	—	0.329	reproduced	none declared	Board

PubMedQA bio-nlp/pubmedqa

PubMedQA accuracy bio-nlp/pubmedqa-accuracy

Drawn from: PubMedQA

atlas-full-v1

Metric	Value	95% CI	n	Rank	Best in catalogue	Source	Overlap	Board
Accuracy (option log-likelihood) ↑ higher is better · zero-shot	0.622 best	[0.582, 0.660]	500	1 / 2	0.622	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Macro F1 (option log-likelihood) ↑ higher is better · zero-shot	0.430 best	[0.401, 0.458]	500	1 / 2	0.430	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Accuracy (logistic probe) ↑ higher is better · probe	0.576 best	[0.538, 0.618]	500	1 / 17	0.576	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board
Macro F1 (logistic probe) ↑ higher is better · probe	0.444 best	[0.396, 0.495]	500	1 / 17	0.444	reproduced	overlap Train/test overlap declared by the evaluator: PubMed abstracts	Board