Stanford's Paper2Agent Turns Research Papers Into AI Agents That Reproduce Results

The tool addresses a familiar cost: computational papers ship code that readers must clone, install, configure and debug, which keeps methods locked inside PDFs. Paper2Agent is MIT-licensed and installs as a skill for Claude Code or Codex, according to MarkTechPost. Prebuilt AlphaGenome, Scanpy and TISSUE servers run on Hugging Face Spaces, and a hosted version is available at paper2agent.ai.

Paper2Agent runs on Claude Code's agent SDK. A central orchestrator dispatches specialized sub-agents through six steps: locating and downloading the codebase; building an isolated virtual environment; indexing usable tutorials; running them end to end and recording reference outputs; turning tutorials into parameterized MCP tools; and validating them. The orchestrator then assembles validated tools into one MCP server.

Validation is strict. A tool passes only when expected files appear and numbers match within 3 percent. Figures must also match references by perceptual hash, with Hamming distance under 20. The verifier gets up to six attempts per function, and tools that keep failing are excluded from the final server. Each server exposes MCP tools that wrap methods as executable functions, MCP resources that hold the manuscript, code links, datasets and figures, and MCP prompts that encode multi-step workflows such as the correct Scanpy preprocessing order. The research team used Claude Sonnet 4 for all Paper2Agent applications.

For AlphaGenome, Paper2Agent built 22 tools in about 45 minutes for US$14. All 22 passed validation without human intervention. MarkTechPost reported that in benchmarks spanning five runs and graded by two human experts with 96.7 percent inter-rater agreement, the agent scored 98.7 plus or minus 1.3 percent on 15 tutorial-derived queries, compared with 82.7 plus or minus 3.4 percent for Claude Code plus repository access and 37.3 plus or minus 4.0 percent for Biomni. It scored 100.0 plus or minus 0.0 percent on 15 novel queries, versus 78.7 plus or minus 4.4 percent and 56.0 plus or minus 3.4 percent, and 82.7 plus or minus 2.4 percent on 30 open-ended queries, versus 56.7 plus or minus 2.3 percent and 72.2 plus or minus 2.2 percent. On tutorial queries, median runtime fell 1.9 times versus Claude plus repository and 3.1 times versus Biomni. The gains persisted when the baseline was upgraded to Claude Opus 4.6.

The AlphaGenome agent also re-examined an LDL cholesterol variant, chr1:109274968:G>T, and ranked SORT1 as the likely causal gene. The original AlphaGenome paper emphasized CELSR2 and PSRC1. GTEx shows significant liver eQTLs for all three genes, and the research team says this shows how hard causal gene assignment is at such loci.

The Scanpy agent received seven validated tools in about 45 minutes for US$13. On four public datasets, it matched human researchers on cell counts, gene counts and top marker genes. A TISSUE agent reproduced human results on spatial transcriptomics data.

Scale tests covered three corpora with no manual cleanup. Among 100 bioRxiv computational biology papers, 74 were agentified and 593 of 599 proposed tools passed validation. On 300 questions, Paper2Agent scored 91.2 percent, versus 80.3 percent for Claude plus repository with Sonnet 4 and 86.3 percent with Sonnet 4.6. Cost per query was US$0.20 and 1.6 minutes, compared with US$0.38 and 4.3 minutes. For 10 non-biology papers including TabPFN, SAM 2 and SAELens, the system reached 98.1 percent accuracy on 42 execution tasks. For 26 data-focused papers, the resource layer scored 89.0 percent versus 82.0 percent for browser use, and was 34 times cheaper and 15 times faster. Paper2Agent also rejected 100 percent of out-of-scope queries in a permuted benchmark. It recovered from injected dependency, file-path, typo and deprecated API failures.

The research team connected three agents: AlphaGenome, an MPRA-coupled scCRISPRi screen and a CD4+ T cell Perturb-seq dataset. AlphaGenome flagged GPR137 at psoriasis locus rs887314, with an RNA-seq quantile score of 0.997. The AI co-scientist proposed 10 validation strategies, and a researcher picked signature correlation. Only GPR137 knockdown matched the CRE perturbation signature. The match appeared under stimulation: Spearman 0.613 at Stim8hr and 0.630 at Stim48hr. BAD and three other candidates showed no significant correlation. A second study paired AlphaGenome with an ADHD GWAS and nominated rs1626703 among 209 candidates. That hypothesis still needs experimental validation.

MarkTechPost reported that the code is MIT-licensed and installs as a skill for Claude Code or Codex, with prebuilt servers on Hugging Face Spaces and a hosted version at paper2agent.ai.