Tutorial Details NVIDIA's cuDNN Frontend Graph API for Fusion, Autotuning and Plan Reuse

Every kernel in the tutorial is built the same way. Tensors are declared by dimensions and strides, operations are chained onto them, and a five-step pipeline is run: validate, build the operation graph, create execution plans, check support, and build plans. Execution then runs against a variant pack of pointers. According to the article, all examples run on a single Colab GPU, with each result checked against a PyTorch reference so readers can see both that the fusion is correct and what it costs.

The material is ordered so that each topic builds on the previous one. It starts with a single fused convolution and moves on to autotuning across engine configurations, FP8-style epilogues, attention, plan serialization, dynamic shapes and CUDA graph capture.

The first practical obstacle is making libcudnn.so visible to the frontend's dynamic loader after installing nvidia-cudnn-frontend with pip. The tutorial forces PyTorch to load its bundled cuDNN first by running a convolution, then preloads the shared objects explicitly with ctypes.CDLL in RTLD_GLOBAL mode, so the frontend's own dlopen call resolves against a library already resident in the process.

Environment reporting covers the GPU name, the compute capability mapped to an sm_ number, the torch and CUDA versions, and the cuDNN backend version and version string. The working dtype is bfloat16 on sm_80 or newer and float16 otherwise, and fused scaled dot-product attention is flagged as usable only on sm_80 and above. A cuDNN handle is created, and a dictionary maps the torch dtypes float16, bfloat16, float32, int32, int64, int8 and uint8 to their cuDNN counterparts. Two helpers construct graph tensors from a PyTorch tensor's size and stride, and build 1x1x1 float tensors marked pass-by-value for scalar inputs.

The build helper defaults to the heuristic modes A and FALLBACK, and can instead pass a policy to build_plans. Workspace is allocated from the size returned by get_workspace_size. Benchmarking uses 10 warmup iterations followed by 50 timed iterations bracketed by CUDA events, converting the result to TFLOP/s when a FLOP count is supplied. Each section of the notebook runs inside a wrapper that records either an "ok" result or a SKIPPED/FAILED string carrying the exception type, a structure that reflects the differing feature support across GPUs.

The article does not report independent performance comparisons against framework-native paths, so the timings shown are those produced by the notebook's own measurement loop on that single device.