Datasets / protein-ml / proteingym-v1 / benchmarks

protein-ml/proteingym-v1

Visibility: PUBLIC

5 benchmarks · 400 results. Each benchmark shows the top of its default board (its first primary metric at its default version); ranks are the server's, ties share a rank, and "best" is the best visible result, not a state of the art.

Domain variant fitness, human PLM Clans-0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-05

Spearman ρ (zero-shot, domain input) · ↑ higher is better · zero-shot · atlas-n25-v2

#	Model	Value [95% CI]	Source	Overlap
1	protein-ml/progen2-base	0.463 [0.271, 0.612] best	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90
2	protein-ml/prot-bert	0.382 [0.184, 0.548]	reproduced	overlap Train/test overlap declared by the evaluator: UniRef100
3	protein-ml/esm2-t48-15b-ur50d	0.369 [0.163, 0.552]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
4	protein-ml/amplify-120m	0.367 [0.141, 0.541]	reproduced	overlap Train/test overlap declared by the evaluator: UR100P (UniRef100 + OAS + SCOP)
5	protein-ml/progen2-small	0.341 [0.136, 0.509]	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90

5 of 13 on this board. Full leaderboard

Domain variant fitness, human PLM Clans-0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-clan-09

Spearman ρ (zero-shot, domain input) · ↑ higher is better · zero-shot · atlas-n25-v2

#	Model	Value [95% CI]	Source	Overlap
1	protein-ml/progen2-base	0.362 [0.167, 0.522] best	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90
2	protein-ml/progen2-small	0.283 [0.079, 0.460]	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90
3	protein-ml/esm2-t6-8m-ur50d	0.281 [0.072, 0.455]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
4	protein-ml/protgpt2	0.277 [0.084, 0.463]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
5	protein-ml/esm2-t48-15b-ur50d	0.263 [0.059, 0.458]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50

5 of 13 on this board. Full leaderboard

Domain variant fitness, human PLM 0.5 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-05

Spearman ρ (zero-shot, domain input) · ↑ higher is better · zero-shot · atlas-n25-v2

#	Model	Value [95% CI]	Source	Overlap
1	protein-ml/progen2-base	0.476 [0.299, 0.608] best	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90
2	protein-ml/esm2-t48-15b-ur50d	0.385 [0.172, 0.547]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
3	protein-ml/esm2-t36-3b-ur50d	0.369 [0.159, 0.522]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
4	protein-ml/amplify-120m	0.351 [0.150, 0.501]	reproduced	overlap Train/test overlap declared by the evaluator: UR100P (UniRef100 + OAS + SCOP)
5	protein-ml/esm2-t6-8m-ur50d	0.347 [0.130, 0.506]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50

5 of 13 on this board. Full leaderboard

Domain variant fitness, human PLM 0.9 (ProteinGym x Pfam) protein-ml/protein-domain-variant-fitness-pfam-09

Spearman ρ (zero-shot, domain input) · ↑ higher is better · zero-shot · atlas-n25-v2

#	Model	Value [95% CI]	Source	Overlap
1	protein-ml/progen2-base	0.476 [0.299, 0.608] best	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90
2	protein-ml/esm2-t48-15b-ur50d	0.385 [0.172, 0.547]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
3	protein-ml/esm2-t36-3b-ur50d	0.369 [0.159, 0.522]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
4	protein-ml/amplify-120m	0.351 [0.150, 0.501]	reproduced	overlap Train/test overlap declared by the evaluator: UR100P (UniRef100 + OAS + SCOP)
5	protein-ml/esm2-t6-8m-ur50d	0.347 [0.130, 0.506]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50

5 of 13 on this board. Full leaderboard

ProteinGym DMS zero-shot (Spearman) protein-ml/proteingym-dms-zero-shot-spearman

Spearman ρ (zero-shot) · ↑ higher is better · zero-shot · atlas-n100-v1

#	Model	Value [95% CI]	Source	Overlap
1	protein-ml/amplify-120m	0.434 [0.376, 0.483] best	reproduced	overlap Train/test overlap declared by the evaluator: UR100P (UniRef100 + OAS + SCOP)
2	protein-ml/esm2-t33-650m-ur50d	0.425 [0.366, 0.472]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
3	protein-ml/progen2-base	0.424 [0.366, 0.471]	reproduced	overlap Train/test overlap declared by the evaluator: BFD, protein-ml/uniref90
4	protein-ml/esm2-t30-150m-ur50d	0.423 [0.361, 0.473]	reproduced	overlap Train/test overlap declared by the evaluator: protein-ml/uniref50
5	protein-ml/amplify-350m	0.416 [0.355, 0.464]	reproduced	overlap Train/test overlap declared by the evaluator: UR100P (UniRef100 + OAS + SCOP)

5 of 11 on this board. Full leaderboard