PROBE benchmarks LLM protein-function annotation across five organisms

PROBE is a benchmark for mapping experimentally characterized protein-function descriptions to Gene Ontology terms, using 1,000 UniProt Swiss-Prot proteins from five organisms and experimental-evidence-only annotations. The paper evaluates ten LLMs across five prompting formats and reports a 9.3% macro-F1 for its best model, describing current performance as insufficient and unreliable for autonomous annotation. The benchmark, evaluation pipeline and 150,000 response records are public; the repository states an MIT license, while the paper and derived dataset have separate license terms.