torch.compile Methodology
Overview
The compiled benchmark suite measures eager vs torch.compile behavior for the
same operator registry used by the compile compatibility investigation. It is
separate from the SigPy comparisons and does not change the eager pages.
Suite levels
- reduced — quick compatibility sweep over every operator: forward and
adjoint, wrapper level (plus inner level for deep-dive operators), default
compile, CPU and CUDA, 5 warmup / 20 measured iterations. This runs as part of
just bench. - full — complete matrix: all methods, all natural dtypes, wrapper and inner
levels,
defaultandfullgraph, 20 warmup / 100 measured iterations, plus shape-recompilation, autograd, and pipeline checks. Run it withjust bench-compiled.
Results are written to level-specific files:
benchmarks/results/latest/compiled-reduced.jsonbenchmarks/results/latest/compiled-full.json
The docs page prefers full-level data when both exist. A compiled-only run never deletes eager result files.
Compile units: wrapper vs inner
- wrapper — compile the
NamedLinopcall path:A(x),A.H(y), orA.N(x). This includes shape checks, named-dimension bookkeeping, logging, and device context. - inner — compile the bare static math function:
A.fn,A.adj_fn, orA.normal_fn. This isolates the numerical kernel from the wrapper.
Functional kernels (unfold, fold, grid, ungrid) have no wrapper, so they
are always reported at the inner level.
Case construction
Each compiled case follows the repository's linop_input_output test paradigm:
building a case returns (linop, input, output). The output-shaped tensor is
constructed independently with torch.randn_like(linop(input)) rather than
being the forward result itself. That keeps adjoint inputs dense and avoids
inheriting forward views.
Timing
Steady-state per-call medians exclude one-time compilation cost. Inputs are tiny compared with production workloads, so speedup ratios are directional.
Graph breaks
Every default-config row runs torch._dynamo.explain after timing. The result
row records:
- the graph-break count,
- up to three formatted reason/user-stack locations.
fullgraph converts graph breaks into loud failures, so its rows record
graph_breaks = None rather than a misleading zero. If explain itself fails,
the row also records None and the monitoring tests fail closed.
Break counts are PyTorch-version dependent and are intended as a monitoring signal, not a standalone pass/fail criterion.
Full-suite phase coverage
The full-only phases are deliberately narrower than the sweep:
- CUDA only,
- recompilation is forward-only,
- autograd is deep-dive-only,
- each phase uses the case's first dtype.
Reduced runs do not collect autograd, recompilation, or pipeline results.
Charts
The generated charts are:
- dtype-labeled compiled speedups on the float-vs-complex page,
- wrapper-vs-inner speedups on the wrapper-vs-inner page,
- pipeline speedups on the chains page.
Running
# Reduced compiled suite + report
just bench-compiled-reduced
just bench-report
# Full compiled suite + report
just bench-compiled
just bench-report
# Eager + reduced compiled suite
just bench
just bench-report
Class glossary
Every result row gets one effective class. The classes are:
| Class | Meaning |
|---|---|
works |
Compiled and ran; output agreed within tolerance; no graph breaks or fallback warnings; speedup at least 0.9x. |
works_with_breaks |
Output agreed, but torch._dynamo.explain reported one or more graph breaks. |
works_with_unknown_breaks |
Output agreed, but graph-break measurement itself failed, so the break count is unknown rather than zero. |
compiles_slower |
Output agreed, but the compiled median was below 0.9x eager. |
silent_degradation |
Ran and agreed, but a known fallback warning was captured (for example, Inductor falling back for complex operators). This is flagged even when the timing looks normal. |
fails_loud |
Raised during eager or compiled execution; the traceback tail is recorded. |
fails_silent |
Returned output that disagreed with eager beyond the 1e-3 relative tolerance, or had a shape or non-finite-value mismatch. This is the red-flag class. |
n_a |
Not applicable in this environment, for example a CUDA-only path measured on CPU. |
blocked_out_of_scope |
Failed because of an excluded dependency path (cupy/SigPy), so it is not counted as a compile verdict. |
Supporting terms:
- Graph break: a point where Dynamo cannot continue tracing one graph and splits it into fragments.
fullgraph=Trueconverts graph breaks into loud failures. - Wrapper: the
NamedLinopcall path, including shape checks and context bookkeeping. - Inner: the bare
fn/adj_fn/normal_fnmath function. - Speedup: eager median time divided by compiled median time; values below 1.0 mean compiled is slower.
Caveats
- One GPU and one PyTorch version per run.
- Dynamic-shape behavior is sampled only by the full suite's recompile phase.
- Complex tensors use complex64; complex128 is not covered.
- The independent-output case construction avoids a Dynamo limitation with complex views over real storage; it is a benchmark-harness choice, not a change to the functional kernels.