NVIDIA Details BioIR Inference Runtime: 2.90x Folding Throughput, 58.5K Residues per GPU-Hour on 8xH100

NVIDIA frames the problem as one of throughput rather than capability. Proteome-scale worklists move entire queues of independent targets through parsing, featurization, GPU inference and output writing, and BioIR targets operations that general-purpose inference stacks do not fully optimize, including Pairformer and Evoformer stacks, triangle operations, pairwise attention, diffusion transformers and atom-level modules. Models remain ordinary torch.nn.Module objects, with no engine build, export step or separate artifact between a checkpoint and a forward pass. The runtime can be used as an end-to-end processor that carries an InputRequest through parsing, tokenization, feature generation, GPU inference and PDB or mmCIF writing, or through direct PyTorch integration for developers who construct a supported model or reuse selected optimized modules inside custom code.

BioIR is available now as an open GitHub repository with a wheel containing precompiled CUBINs. Runtime use requires Python 3.12 or later, a compatible NVIDIA GPU and driver, a staged model checkpoint and per-chain A3M MSAs; it does not require nvcc, CUDA source, CMake or the CUDA toolkit. Paired or unpaired MSAs are accepted for inputs with multiple non-identical protein chains, and templates can be supplied manually because BioIR does not run HHsearch or HMMsearch. The processor supports ligand structure prediction but not ligand-affinity prediction.

Acceleration is applied at three layers. Kernel selection picks compatible BioIR custom, cuEquivariance or PyTorch fallback implementations based on model configuration, GPU, data type and tensor shape. A separate optimize() mechanism enables CUDA Graph capture for compatible modules to cut launch overhead. A Ray executor then places one complete model replica on each visible GPU in a node and distributes independent inputs among them, overlapping CPU stages such as parsing, featurization and writing with GPU folding. Ray does not split a single forward pass across GPUs, so replica mode scales worklists rather than individual targets; the support matrix lists context-parallel folding as planned but not yet available, and the capacity rule requires that engine_stage.compute multiplied by num_gpus does not exceed visible GPUs.

NVIDIA reports geometric-mean speedups at the model-forward level over an open-source torch.compile baseline on H100 of 1.55x for OpenFold3, 1.78x for Boltz-2 and 2.56x for OpenFold2 monomer, measured across 17 inputs spanning 29 to 1,734 residues. H200 figures are similar at 1.54x, 1.75x and 2.61x. At production scale, BioIR powered the recent expansion of the AlphaFold Database, generating protein-complex structures across 4,777 proteomes and about 31 million candidate complexes, of which 1.81 million were released as high-confidence predictions.

The performance figures are folding-stage measurements specific to the benchmark dataset and hardware. They exclude MSA generation, preprocessing CPU allocations, storage, data transfer and retries, and NVIDIA warns against generalizing them to all BioIR-supported models or datasets. In the 1,000-target run, the public implementation ran out of memory on 29 targets while BioIR completed all of them.

Extrapolating the benchmark linearly to 1 million comparable targets, NVIDIA estimates BioIR would need 11 MWh against 35 MWh for the public implementation using 8-GPU TDP equivalents, or 21 MWh against 64 MWh using full-node maximum-power equivalents. These are rated-power, folding-only estimates for IT equipment rather than metered measurements, and they exclude data center overhead such as PUE.

Editor's Summary

NVIDIA detailed BioIR, a Python inference runtime that accelerates Boltz-2, OpenFold2 and OpenFold3 folding without leaving the PyTorch workflow. A 1,000-target human dimer benchmark on eight H100s produced 58.5K folded residues per GPU-hour versus 20.2K for a torch-compiled open-source implementation, a 2.90x gain, while the public path ran out of memory on 29 targets. NVIDIA cautions that the numbers cover the folding stage only and should not be generalized, though the runtime is already available as an open repository.