An attention model, timed on the bench
forward only / bench
capataina.dev / research
Blaze
Contract-Guarded Runtime-Adaptive Kernel Fusion for Dynamic Tensor Execution
Burn merges neighbouring operations into single GPU kernels wherever it can. That makes some models much faster and others slower, so Blaze measures the alternatives and keeps whichever wins on your machine.
in short / the problem
Burn only ever has one plan
An MLP with an inner width of 1024 runs fastest when every neighbouring operation is merged into one GPU kernel. Widen it by one, to 1025, and the same network runs fastest with nothing merged at all: merging now costs it 19%. Burn, the deep learning framework running it, merges everything either way.
- boundary 1mergedleft apartnever tried
This is fusion. The two operations run as one kernel, so the write out to memory between them, and the read back, never happen. Removing those round trips is why merging is usually faster.
- boundary 2mergedleft apartnever tried
Burn takes the merged door here as well, for the same reason it took the first one: it is legal. Whether this particular merge helps or hurts is never asked.
- boundary 3mergedleft apartnever tried
A merged kernel is bigger, and past some size it spills. Then the spill costs more than the round trips it saved, and merging is the slower answer.
Burn takes the merged door at every boundary it legally can, every time. The doors it leaves are legal plans it never opens, because it has no way to rank a second plan against the first. Usually that is right. Sometimes it is badly wrong, and nothing in the framework can find out. Those unopened doors are what the next screen goes and opens.
The 19% is a forward pass. Under a real training step with a backward pass the same shape wants full fusion again; the sign flip between the two regimes is the finding rather than a discrepancy.
The two squares are the only plans Burn’s setting can reach. The ring is the plan Blaze kept here, and it is neither of them.
in short / what Blaze does
Blaze tries them all and keeps whichever wins
Every door Burn left shut is a candidate. Blaze opens them and races them against each other on the real GPU, during real training, and keeps whichever turns out to be fastest.
- 01
Enumerate the plans, including the ones between the two ends that Burn’s setting cannot express at all.
- 02
Collapse the duplicates. On the attention stack, 18 of 33 candidates turn out to be the same plan reached a different way, and they go before anything is timed.
- 03
Time what is left on real training steps. Those steps still update the model, so the measuring is not time taken out of training.
- 04
Before the winner is kept, check it computes the same numbers as Burn’s own plan. Keep it while the conditions it was measured under still hold; reopen if they drift.
18 of 33 is the attention stack under a backward pass. The dial is schematic: arc lengths depict relative measured cost rather than plotting a per-candidate series.
in short / what we found
What we found, and what we cannot claim
Opening those doors won twice, in two different ways: a default a user could fix, and a setting no user could pick. Both are per-step rates measured under real training with a backward pass, and neither is a claim about how a long run behaves.
19.6%
faster than Burn’s default, on a character-level LSTM. The default merges everything; here, not merging is the right answer. Blaze found that unaided on both runs, across 134 boundaries, the largest plan space measured.
4.9–13.2%
faster than the best setting Burn can reach at all, on workloads whose answer is a plan between the two ends: an attention layer at 4.9%, bit-identical across three of three repeats, and a mixture-of-experts model at 13.2% across two of two.
What we cannot claim
One machine
Every figure is one Apple M5 through Metal, on Burn 0.22.0-pre.2 at 2c06be35. Nothing has been reproduced on other hardware, due to hardware and budget limits.
No long run was performed
No full training run was executed. Every long-run figure on this page is arithmetic over measured per-step times, not a measurement of a long run.
Choosing is not free
Selection runs a one-off window, measured between about 10 seconds and about 320 seconds depending on how many plans there are to try. The steps inside that window still train the model.
What the two figures leave out
The 19.6% is against Burn’s default. Against someone who already knows to switch fusion off on that model, Blaze’s settled rate measures 3.4% above, and never crosses on these numbers. A second attention shape measured 2.0%, inside its own run-to-run noise and from a single run, so it is not counted in the range above.
1 / abstract
Abstract
Four claims, each with its number and the scope that number was taken under, all measured on one Apple M5 through Metal. The panel beside each one previews the section that proves it.
measuredread from sourcederived by arithmetic
proved in
0
alternative partitions ever costed
read from source
Burn never ranks one plan against another.
It fuses greedily, then picks which fuser implements each block. No alternative partition is ever costed.
The vendored revision, confirmed three times independently.
2 / background
+1411% to −19%
the value of fusing. Plus means fusing wins, minus means fusing loses.
measured
Fusing helps enormously on some models and hurts on others.
Refusing to fuse a long chain of elementwise operations costs 1411%. Insisting on fusing a residual MLP at width 1025 costs 19%. No fixed setting serves both.
Nothing reproduced on other hardware, due to hardware and budget limits.
3 / problem
4.9%
faster than the best stock setting
measured
Measuring on real training steps finds plans a formula cannot express.
On attention at model dimension 257, Blaze settles at 33.35 ms per step, against 35.07 for the best stock setting and 36.32 for the worst. The plan it keeps is between the two ends.
Character level language model, backward pass, Adam, real dataset. Same partition on 3 of 3 repeats, verified bit identical. The winning partition is interior and its mechanism is unattributed.
7 / results
320 s
the selection window on attention at 257
measured window, derived break even
The cost is bounded and paid once.
Selection spends one 320 second window, then keeps its winner. Against the best stock setting that puts adaptive ahead about 35 minutes in.
Attention at model dimension 257. The window is measured; break even is arithmetic over measured per step rates, and no full training run was executed.
8 / cost
2 / background
How Burn chooses a fusion plan
A GPU kernel is one program the GPU runs start to finish. Multiply, add a bias, take a ReLU: that is three kernels, and each one reads the whole tensor out of memory and writes it back. Fuse them into one kernel and the values stay in place, so two of those three trips to memory never happen. It is not free: a larger kernel holds more at once, and past a point that costs more than the traffic it saves.
Twelve operations in a row, schematic. The outline is one fusion plan.
fuse nothingfuse everything
Burn's default is the right-hand setting: it merges every pair it legally can, then picks an implementation for each merged block. It never compares one whole plan against another.
One setting, and it moves every boundary together.
~99places in one attention training step where two operations could be merged or left apart
2settings a user can actually reach: fuse everything, or fuse nothing at all
Every plan between the two settings is legal. None of them is reachable.
Boundaries, forward to backward, one run each: convolution 0 to 16, MLP 5 to 51, attention roughly 5 to 99.
3.1 / problem statement
One unit of width flips the right answer
1024 divides by 8, so every vector group is full. Fusing the whole block wins here, measured.
An MLP at width 1024 wants full fusion. At 1025 it wants none, and pays 19% for fusion instead.
Drag the width below and watch the winner change.
matmul inner dimension
the count a layer's matrix multiply sums over
1024
1024 mod 8=0
lattice aligned
winnerfull fusion
measured, forward regime
one residual block of that MLP. a vector group is the values the GPU runs at once
one fused kernellayernormmatmul + biasgelumatmul + biasresidual +
the final vector group
8 of 8 filled
drag the thumb, or focus it and use the arrow keys
lane width, illustrative1024 divides by all four, 1025 by none
- measured on hardware
- modelled from the divisibility rule
- ordering known, margin not published
3.2 / problem statement
Training and inference want different plans
Take one MLP at width 1025 and measure it twice. Merging its operations costs 19% when the model runs predictions, and saves 30% when it trains. Burn fuses everything it can, either way.
Inference
forward pass only
Five places in this MLP where neighbouring operations could be merged. None at all in a convolution network.
places to mergeconv 0 · MLP 5 · attention 5 to 8
Relative to this half's winner. No absolute rate.
MLP at width 1025 · every figure measured, none projected · each half scaled to its own winner, so the two are not to a common rate · merge counts are one run each · no repeat count or noise band
Training
forward and backward pass
The backward pass multiplies those places by roughly ten. A convolution goes from none at all to sixteen.
places to mergeconv 16 · MLP 51 · attention 99
Milliseconds per training step, both arms measured.
3.3 / problem statement
Changes we could not explain
Change one number in an attention model and the right answer flips, and we cannot say why. For a plain feed-forward model we can predict which way it goes: merging neighbouring operations into one GPU kernel pays when the matmul's inner dimension divides evenly by 8. Attention obeys no rule we have found.
Four widths, screened forward only, both plans forced by hand rather than searched.
384
Merging loses by 7.97%19.2× signal to noise
Widths that divide evenly by 8 want merging384 divides by 8, and by 128, and refuses anyway. Ruled out
258
Merging loses by 16.51%62.5× signal to noise
Even widths want merging, odd ones do not258 is even, and refuses anyway. Ruled out
256
Merging paysNo magnitude on this page
The width matching the length of text the model readsWidth is the model dimension. Every run reads 256 characters at a time, so only here do the two match. Still standing, never tested
That last row is the only explanation left standing, and it has never been tested. The Blaze repository records it as a hypothesis rather than a result, and so does this page.
Under real training, with the backward pass included, both ruled-out widths reverse. Neither refuses merging any more.
A phenomenon nobody can model is the clearest argument there is for measuring instead of predicting.
4 / method
How Blaze picks a plan, stage by stage
Blaze has to choose how to group a model’s operations into kernels, which are the units of work a GPU actually runs. It chooses by measuring rather than by guessing, and these six stages are how it gets from every legal grouping down to one. The test bench runs all six in full; a live training run cannot afford that, so it takes a cut-down path through the same code.
benchenumerateconfirmverifymeasureselectreport
runtimeenumerateconfirmverifymeasureselectreport
A filled dot means the stage runs in full on that path. A half dot means it runs cut down, and that stage’s own text says how. Every half dot is a deliberate omission, because a planner sitting inside somebody else’s training loop cannot start a clock or run an extra pass. Point at a stage, or tab to it, to read what it does.
Every legal way to cut the graph of operations into kernels. The bench lists all of them. A live training run cannot: the number of ways to cut grows exponentially with the number of places you could cut, so the runtime builds its list greedily, one round at a time.bench: the exhaustive set · runtime: greedy forward rounds · enumerate_partition_space.rs
Positive proof that a candidate really is a different plan, read from what the device executed rather than from what was asked for. It runs before any timing, because timing two identical plans and reporting the gap between them is this project’s defining failure.bench: every candidate, before timing · runtime: at the verify gate, after timing · confirm_plan_changed.rs
The candidate has to compute the same answer. Summary numbers are allowed a tolerance, but predictions are compared exactly, because rounding may move a sum and must never move the winning class. The gate has confirmed a winner on attention, on a mixture-of-experts model and on a character LSTM, and has never once rejected one. Its rejection branch is proven in unit tests rather than on hardware.bench: every candidate · runtime: the winner only · verify_plan_equivalence.rs
Time the survivors under one fixed protocol, then reduce each set of samples to a median and a spread. Numbers taken under a different protocol cannot be compared with these, which is why the protocol is changed with the care of a public interface.bench: one deliberate pass each · runtime: the training steps themselves · measure_candidate_plans.rs, summarise_timing_samples.rs
Take the fastest survivor, but only if it clears the wider of the two spreads, so that a quiet run cannot manufacture a win against a noisy baseline. A result inside the spread is recorded as unmeasured, never as a tie.bench: fastest, if it clears the band · runtime: its own settle gate · select_winning_plan.rs
The bench closes by reporting, and the losers stay in the report with their reasons, because a planner that shows only its winner cannot tell a searched space from a broken search. The runtime closes by not closing: it re-checks the settled winner on a widening delay.bench: report, losers kept · runtime: drift, on a widening delay · describe_plan_outcome.rs, track_selection_drift.rs
4.1Listing every plan, and proving they differ4.2Does the winner compute the same answers?4.4Picking a winner, and noticing it stop winning
4.1 / method
stageenumerateconfirmverifymeasureselectreport
Listing every plan, and proving they differ
Blaze picks how to split a model by timing the alternatives. This step writes that list, then proves its entries really differ: two plans can ask for different merges and run identically.
A plan decides which neighbouring operations get merged into one GPU kernel and which are left apart. Blaze wrote down 33 of them for one attention model, and 18 ran as a plan it had already tried.
Enumerate · write down every plan that is allowed
A cut point is a merge the planner is allowed to refuse, and a plan is a choice of which ones to refuse. Burn's own plan is always on the list, so the answer cannot come out worse than Burn's.
- cut points in one model
- running → training
- CNN
- 0 → 16
- MLP
- 5 → 51
- attention
- 5 to 8 → 99
Training adds the backward pass, and with it the plans.
- burnRefuses nothing, so every merge Burn would make is kept. This is Burn's own plan.
- Refuses merge 4, so the work runs as 2 blocks instead of 1.
- Refuses merge 3, so the work runs as 2 blocks instead of 1.
- Refuses merges 3 and 4, so the work runs as 3 blocks instead of 1.
- Refuses merge 2, so the work runs as 2 blocks instead of 1.
- Refuses merges 2 and 4, so the work runs as 3 blocks instead of 1.
- Refuses merges 2 and 3, so the work runs as 3 blocks instead of 1.
- Refuses merges 2, 3 and 4, so the work runs as 4 blocks instead of 1.
- Refuses merge 1, so the work runs as 2 blocks instead of 1.
- Refuses merges 1 and 4, so the work runs as 3 blocks instead of 1.
- Refuses merges 1 and 3, so the work runs as 3 blocks instead of 1.
- Refuses merges 1, 3 and 4, so the work runs as 4 blocks instead of 1.
- Refuses merges 1 and 2, so the work runs as 3 blocks instead of 1.
- Refuses merges 1, 2 and 4, so the work runs as 4 blocks instead of 1.
- Refuses merges 1, 2 and 3, so the work runs as 4 blocks instead of 1.
- Refuses merges 1, 2, 3 and 4, so the work runs as 5 blocks instead of 1.
Point at a plan to read what it refuses.
Confirm · watch each plan run before timing it
A plan is a request, and the engine does not always grant it. So Blaze runs each once and writes down what happened. Two plans with the same record are one plan.
- This one ran differently, so it is timed.
- This one ran differently, so it is timed.
- This one ran the same as the middle card, so it is not timed.
Point at a card: the two that ran the same light together.
Reject · drop the plans that turned out to be one plan
A duplicate is counted, kept out of the timing budget, and its share printed beside the result, because a reader shown only the survivors cannot judge how narrow a win really is.
on the bench every plan is tried, up to ten cut points, sampled by shape above that.
in live training Blaze adds one refused merge per round and stops when a round cannot beat the last: seven plans instead of 33.
4.2 / method
stageenumerateconfirmverifymeasureselectreport
Does the winner compute the same answers?
A plan that runs faster might be running faster because it is quietly computing something else. Before Blaze keeps a winner it re-runs the caller's probe under both plans and compares what came back.
Blaze re-runs the caller’s probe, a small sample computation, once under its own plan and once under Burn’s. Three things can happen.
nothing to check
Blaze kept Burn’s own plan, so there was no second version of the numbers to disagree with. Recorded as nothing to check, never as a pass.
did not settle
Something was there to check and the check did not settle it: no probe was supplied, or both runs dispatched identically, which is what comparing a plan against itself looks like.
verified
The two plans ran differently and produced the same answers. Predictions are compared exactly: rounding may move a number, never the answer the model picks.
- identicalkept
- same to roundingkept
- differentrefused
- brokenrefused
Matching plans give identical outputs. Differing plans differ only in the order f32 numbers were added: 1e-7 relative on the CNN, 9.51e-10 on the character LSTM.
what this does not prove
Verified on four workloads across three architectures: attention at 257 and at 258, a mixture of experts, and a character LSTM. Two came back unsettled. The bench checks every candidate; the live planner checks only the winner, and has only ever confirmed one. A wrong plan correctly refused has never happened on hardware, so that branch is proven in unit tests only.
verified / ran differently, same answers
4.3 / method
stageenumerateconfirmverifymeasureselectreport
Did the timings actually separate?
Three times in this project a plan looked 1.2% to 1.8% faster, and every one of those wins vanished on a re-run; one shrank to 0.19%. Time the same plan twice and you get two different numbers. So Blaze times both plans many times over and asks one question: did the two sets of timings actually come apart, or did they land on top of each other?
Attention at dimension 256, under real training
training / live path
If the two sets of timings overlap, there is no result.
both plans timed many times over / measured, not projected
4.4 / method
stageenumerateconfirmverifymeasureselectreport
Picking a winner, and noticing it stop winning
Blaze keeps whichever plan ran fastest. Timing on a real machine is noisy, so Blaze only calls a plan the winner when it beats the plan already running by more than that noise. And a plan that wins early can stop winning later, so Blaze keeps checking.
Kept Burn’s own planWhen nothing beats the plan already running by more than the noise, Blaze names no winner and keeps Burn’s plan. A candidate inside the noise band is recorded as unmeasured, never as a tie.
01 · Time them, and only believe a clear win
A candidate wins only by beating the plan already in use by more than the measurement noise, and the comparison uses the wider of the two noise bands, so a quiet run cannot look significant against a noisy one. Losers are kept with the reason they lost.
02 · Check again, and wait longer each time nothing has changed
Blaze is built to re-check its winner on a widening schedule: every clean check doubles the wait before the next one, so a workload that keeps agreeing with itself converges toward paying almost nothing. That schedule is written and unit tested, and it is not yet what runs. The shipped runtime checks on a fixed horizon of 5,000 steps, and a check that fires re-opens the whole list rather than comparing the top two. The ticks above are dotted for that reason: read from the source, not measured on hardware.
03 · When the winner stops winning, start over
Drift is the settled plan quietly stopping being the fastest one part way through a run. In a 6,000 step run Blaze settled at step 691, drift reopened selection at 5,691, it settled again at 5,996 and re-verified the numbers automatically. Forward passes only.
Measured where solid, read from the source where dotted. No projected figure appears in this pane.
Blaze is five separate Rust packages. Each box names what its package does, and the gap in its bottom edge names what it cannot see.
cargo workspacemembers = crates/*
5.1 / implementation
The five parts, and what each one does
uses
vendor/burn-blazea patched copy of Burn, not a fork. patches/burn.patch, 653 insertions across 9 files
outside the workspaceno edge arrives here
the spine stops here. the root Cargo.toml excludes this crate
vendor/burn-stocknever edited, so the patch can be priced
no path into vendor/, and no Burn on any arrow
assets/measured_rates.jsona committed file, not a crate
Structural counts, not measurements. Read from crates/, the root Cargo.toml, and patches/burn.patch.
5.2 / implementation
one of the five · blaze-core
The planner does not depend on Burn
Blaze decides which of Burn's operations get merged. A wrong decision never crashes; it only makes training slower, so its bugs stay invisible. The crate that decides imports nothing and never names Burn. Its 23 tests need no GPU and no Burn installed.
The world around it
- burn-fusionnot referencedthe Burn crate the first plan tried to avoid a cycle with
- burn-irnot referencedthe one dependency the first plan called for, then dropped
- Metal devicenot referencedwhere Burn actually runs the kernels
the only crossing
trait WorkloadRunner
Seven things the host must supply.
- apply
- run_once
- sync
- observe_plan
- fingerprint
- discover_cut_points
- describe
blaze-core
Cargo.toml
[dependencies]nothing follows
[dev-dependencies]no such section
No line of code in blaze-core/src names Burn.
23tests, no GPU, no Burn
counted as #[test] across blaze-core/src, measured
Who implements the trait · press to swap
any other frameworkOne trait is all it would take. None has been written.
The core panel does not change between these. Nothing inside it names a host.
5.3 / implementation
What patching Burn cost, and what it costs off
Blaze does not fork Burn. It is one patch file against a fixed version of Burn's source, kept beside an untouched copy of it.
vendor/burn-stock
2c06be35
Never edited. It is here so the change can be counted.
what changed
same version
vendor/burn-blaze
2c06be35
The same version, patched. Everything is built against this one.
8 files, untouched
exactly as Burn published them
Burn's files
+154 lines
14 hunks
8 files, with Blaze wired in
the largest single edit is 41 lines, in the fusion config
not there
nothing by this name exists in Burn
the new file
+499 lines
1 hunk
blaze_plan_control.rs
added under crates/burn-fusion/src/
none
the untouched copy, by definition
code removed
5 lines
4 files
5 lines gone
the whole of what the patch takes away
the whole change, totalled
653
lines added
5
lines removed
9
files touched
the entire user facing surface
burn.toml
[fusion]
planner = "adaptive"
Nothing this measurement can resolve.
Stock against patched Burn, same plan both sides, planner off. Overhead is 0.14% or less, under a run to run band of about 0.3% that did not tighten when rounds doubled. Re-priced whenever the patch changes.
how to check every number above
./scripts/setup-burn.sh
Clones Burn at the pin, patches a second copy, and refuses to run if the untouched one was edited.
git apply --stat patches/burn.patch
Prints 9 files changed, 653 insertions, 5 deletions. A --check run against the untouched tree exits 0.
6 / simulation
Blaze against both fixed settings
Burn, the framework this plugs into, gives you one switch for kernel fusion: merge the operations in your model into single GPU kernels, or leave them separate. Blaze times the plans your own machine allows and commits to the fastest. The race below is arithmetic, not a recording: it computes all three lanes from per-step rates measured on one Apple M5.
- rate measured on hardware
- the selection window. Its length and the steps it bought are both measured; the straight line across it is interpolation between them
- predicted crossing, arithmetic over those rates, drawn before the race reaches it
Measured under real training with a backward pass, Adam and a real dataset. Model dimension 257, sequence 256. Rates, fee and settle step are all medians of three repeats, which chose the same plan 3 times of 3, verified bit identical against Burn's own plan.
where each setting cuts the graph
- fusion offcut at every boundary
- fusion onBurn merges every boundary it can
- Blazethree blocks, the plan it timed fastest
Measured: the three per-step rates, and the 320.1 s Blaze spent measuring on this workload. Computed here in your browser: everything else.
workload
a character level language model, training
speed
drag, or use the arrow keys. Scrubbing pauses.
predictedBlaze draws level with fusion off at step 60,060, after 35.1 minutes of training. It passes fusion on first, at step 34,757.
fusion off
Burn, every operation on its own
35.07 ms/step
0
steps
fusion on
Burn's default, merging what it can
36.32 ms/step
0
steps
Blaze
times the plans, then commits
33.35 ms/step
0
ready
level with fusion off predicted here, step 60,060
level with fusion on, step 34,757
steps completed, shared axishorizon 95,000 steps
training clock0.0 minBlaze against fusion offnot started
Blaze wins here because it finds an intermediate plan, and neither of Burn's two settings can express one.
It settles 4.9% faster per step than the better of the two, and it is ahead on total training time 35 minutes in.
7 / results
The winning plan is one Burn cannot select
Fusing means running neighbouring operations as one pass over the GPU instead of one pass each.
Burn's only control over it is a switch with two positions: fuse nothing, or fuse everything. On the workload below the plan that ran fastest is neither, and no position of the switch asks for it. A user who guesses perfectly still cannot have it.
workloadmilliseconds per training step, lower is fastermargins only, no absolute speed on this axis
an attention language model over characters, model dimension 257
Measured. Real training with a backward pass, Adam, and a real dataset. Blaze picked the same plan on 3 of 3 repeats, bit identical to Burn's numbers.
the same model at model dimension 258, where the two settings trade places
Measured. Real training with a backward pass, Adam, and a real dataset. One run at this dimension rather than three, bit identical on that run.
Both workloads are the same model at two model dimensions: how many numbers wide the vectors inside it are, which sets the inner size of every matrix multiply. The sequence length is 256 in both, and never changes.
Burn's switch, and every position it has
fuse nothing35.07 ms/stepBlaze takes 4.9% less time per stepthe plan Blaze picked33.35 ms/stepno switch position reaches this planfuse everything36.32 ms/stepBlaze takes 8.2% less time per stepfuse nothingBlaze takes 4.7% less time per stepthe plan Blaze pickedno switch position reaches this planfuse everythingBlaze takes 2.0% less time per stepnothing fusedeverything fusedroughly 99 places to merge two neighbouring operations, against 5 to 8 with no backward pass
- measured on hardware
- candidate plan, schematic
- cost surface, schematic
mechanismWe do not know why this plan wins. The bench dumped the model's operations and looked at where the winning plan cut them: the cuts are scattered, and no single operation explains them. Finding the cause needs its own experiment, which has not been run.
8 / cost and break-even
What selection costs, and when it is repaid
Blaze spends about 103 seconds choosing a plan before it can save anything, then hands back 1.72 ms on every step after. Short runs lose, long runs win.
- run length
- 35 min
- steps, best stock arm
- 59,900
- vs best stock
- level
- vs framework default
- ahead by 1.1 min
projectedarithmetic over the measured rates. No run this long was executed; the longest that was is 4.9 min.
how to read this
- Below the line is behind.
- The shaded band is time spent and not yet earned back.
- The dip is the run that happened.
- Traced from a real run: it falls while Blaze times its candidates, and turns up the moment it settles.
- Zero is break even.
- Level with the framework default at 21 min, with the best stock setting at 35 min.
- Straight, because the saving never changes.
- Every later step saves 1.72 ms. A real run wanders off these lines by under 0.05 min at ten hours, thinner than the line itself.
the law
N* = excess / (slow_step - fast_step)
Excess is the 103 seconds over settling instantly. One step is worth 1.72 ms.
measured per step rates
- adaptive
- 33.35 ms
- best stock, no fusion
- 35.07 ms
- framework default, full fusion
- 36.32 ms
measured. attention at dimension 257, real training with a backward pass, Adam, a real dataset. Median of 3 repeats, verified bit identical.
drag the chart, or focus the slider and use the arrow keys
- measured, a run that happened
- arithmetic over the measured rates
- adaptive behind
A separate break even of about 11 seconds exists in the record. It is the forward only bench, which runs no backward pass, so it is a different measurement of a different thing and is not on this axis.
9.1 / discussion
The case for measuring
Burn speeds a model up by merging its operations, and it decides how by a fixed rule. Blaze times the alternatives and keeps the fastest. Four reasons that pays.
receipts filed
0 / 4
Blaze finds plans no setting can name, never settles slower than Burn’s own plan, beats the honest alternative rather than a straw man, and costs nothing while it is switched off.
Blaze found a faster plan than fusing everything or nothing.
- measured
- forward regime / bench
receipt
On attention at the gate shape, the middle plan beat full fusion by 4.22% and no fusion by 7.49%, at 7 to 17 times signal to noise, on bit-identical logits. Those two are the only plans a setting can name. Why the middle plan wins is unattributed: the winning positions are spread across the op stream with no dominant operation to name.
Blaze can never settle slower than Burn’s own plan.
- by construction
- window measured
- every regime
receipt
Burn’s own plan is always one of the candidates, so the settled rate cannot be worse than it. The measured part is the wait: on the four workloads compared here, selection ran about 10 to 320 seconds. That wait is not lost time. Selection times its candidates on real training steps, which still update the model.
It beats the honest alternative, which is testing both yourself.
- measured
- projected
- training regime
receipt
Testing both settings yourself costs two training runs before it answers. Against that, one Blaze run is 52% to 92% cheaper in wall clock, on four workloads at 1,200 steps. Against a lucky guess it needs a crossover, and that is where the evidence stops: about 35 minutes of training to overtake the best stock setting, on attention at model dimension 257. Arithmetic over measured per-step rates; no full training run was executed.
Switched off, Blaze costs nothing you can measure.
- measured
- patch priced separately
receipt
Overhead with the planner off is at most 0.14% in absolute value, either direction, and off is what a fresh checkout does. Turning it on is one line of burn.toml, planner = "adaptive". The Burn-side change is a tracked patch of 653 insertions and 5 deletions across 9 files, against a vendored checkout kept pristine so the patch can be priced.
9.2 / discussion
The case against
These are the four objections a sceptic would raise against Blaze, answered where there is an honest answer and left standing where there is not.
objections standing
2 in part · 1 conceded · 1 unanswered
Full fusion won on most of what we measured.
Training adds a backward pass, and with it far more places to merge. That buried every no-fusion winner the forward screens found. Training then found its own, an LSTM.
Answered in partmeasured · ten architectures classified under training
Where both settings tie, Blaze buys nothing.
On the CNN both settings land within 4% of each other. Blaze ties the winner, 12.53 against 12.34 ms per step, and spends 9.74 s selecting. Burn’s own plan is always a candidate, so the settled rate is never worse than stock.
Answered in partmeasured · cnn on MNIST, three repeats
Below the crossover, short runs gain nothing.
On attention at model dimension 257, Blaze takes the lead over the best stock setting about 35 minutes in, and over Burn’s default at about 21. Choose a short run by hand.
Concededprojected · arithmetic over measured per-step rates, no full run executed
The selection window is not predictable from the graph.
Selection ran from 9.74 s on the CNN to 1037 s on the mixture-of-experts. Size does not explain that spread: the LSTM offered the largest space and still paid less.
Not answeredmeasured · selection window, adaptive arm, capped runs excluded
convolution0 to 16
MLP5 to 51
attention5 to 99
The extra surface is what buried the forward-only no-fusion winners. The class moved rather than emptied: an LSTM under training is a no-fusion winner no forward screen flagged.
Blaze12.53
best stock12.34
Blaze spends 9.74 s selecting to arrive where stock already was. It cannot lose here either: Burn’s own plan is always one of the candidates.
21 minahead of Burn’s default35 minahead of the best stock setting60 min
The shaded region is where this objection is simply correct. Its scope is one workload: on the mixture-of-experts there is no crossover to wait for, because Blaze is ahead of the best stock setting before it has finished selecting.
cnn 16moe 75attention 99lstm 134boundaries, 0 to 1400 to 1100 s
The LSTM offers 134 boundaries, the largest plan space measured here, and settles faster than the mixture-of-experts at 75. Size does not order these.
10.1 / limitations
How we measured, and where that falls short
Everything above the line was timed on a real machine. Everything below it was calculated from those timings, not measured. None of it is a table of speeds to copy for your own machine.
Measured directly
- per-step ratesboth arms timed, ms per step
- repeatsattention at model width 257: same plan 3 of 3, bit-identical
- noise bandreported as signal-to-noise per result
- window biassome runs hit their cap mid-search, so part of it lands inside the window: 124.4 ms against 122.6 for a clean one, biased against Blaze
- selection feeabout 10 s to about 1,040 s, by workload
Every figure on this page: one Apple M5 through Metal, Burn 0.22.0-pre.2 at 2c06be35. Nothing reproduced on other hardware, due to hardware and budget limits.
Measured aboveProjected below
No full training run was executed. We timed how long a single training step takes, then calculated the rest: the ten-hour projections, the minutes saved and the takeover points. It was built on one personal machine with no budget for rented GPUs.
The simulation is a model
The comparison in section 6 computes what training would look like from the measured rates. It replays no recorded traces and runs no network. It is a model, not a recording of a run.
Computed, not run · attention at model width 257
- break-evenfee divided by the per-step saving
- takeoverpulls ahead 35 min into training vs the best fixed setting, 21 min vs Burn's default
- ten hourssaves 28 min against the best fixed setting, 47 min against Burn's default
Every figure on this page traces to a results file in the Blaze repository.
10.2 / limitations
What we checked, and what we did not
Blaze promises that the faster plan it picks gives the same answers as the plan Burn would have used. A gate checks that promise, and this pane is how far the check reaches.
Proven insideUnproven outside
Outer boundary
The gate has never thrown a plan out.
Every plan we measured gave the same answers, so the code that rejects a fast but wrong plan has only ever run in unit tests. We know the gate confirms a good winner. We have not watched it catch a bad one.
Never litunit tests only, zero rejections on hardware
Every gate run on record
0/4
verified0/2
inconclusive0/0
rejectedSix runs are on record, all of them under a backward pass. Step through them and watch which column can never fill.
Attention, dimension 257Same numbers to every digit, with the two plans observed to run differently.
Attention, dimension 258Same numbers to every digit, with the two plans observed to run differently.
Mixture of expertsSame numbers to every digit, on both repeats, from a plan space of 75 boundaries.
Character LSTMAgreed within rounding, 9.5e-10 and 1.2e-10 apart on two repeats, across 134 boundaries. That gap is what f32 addition does in a different order.
CNNThe numbers agreed, but both probes ran the same plan, so the agreement proves nothing.
Small GPTDropout put the two probes on the same plan, so there was nothing to compare.
Middle boundary
You only get the check if you ask for it.
The application hands Blaze the probe the gate runs. An application that hands it none still gets the faster plan, and Blaze reports that choice as unverified rather than as passed.
Lit only when a probe arrivesper caller, per workload
Inner boundary
Only the winner is checked.
The bench checks every candidate it enumerates before timing it. A live training run cannot afford that, so it checks the plan it settled on and no other.
Lit on every settlethe settled plan, each time selection closes
What the guarantee actually covers
The plan Blaze settled on, checked against Burn's own plan, on the probe the caller supplied. Where the two plans match, the loss curves agree to every digit recorded. Where they differ, the widest gap measured anywhere in the project is one unit in the seventh significant figure, on the CNN loss trace; the character LSTM's two plans came about a hundred times closer. Both are the signature of f32 addition happening in a different order, not of a different computation.
measured
One more thing this guarantee does not cover
Blaze has a module that works out when a choice stays valid and how to reuse it cheaply. It is written and it is tested, and nothing in the project imports it yet, so the reuse economics this paper describes are designed rather than proven.
11 / conclusion
Conclusion
The right fusion plan changes with the device, the shape and the regime, and on one measured shape nothing we can name explains it. Two positions cannot say that.
Ninety-nine mergeable boundaries on attention, under a backward pass. The pattern is schematic.
no fusionfull fusion
The switch above is a picture, not a control. Burn's setting reaches two states, all of them or none of them, and a settled plan sets them differently along the block
+1411% to -19%what fusing is worth, across architectures. Refusing to fuse the elementwise chain is catastrophic; insisting on fusing the MLP at width 1025 is actively harmful. One Apple M5 through Metal, one Burn revision
10 s to 17 minhow long selection runs before it settles: 10 seconds on a convolutional net, 17 minutes on a mixture-of-experts, across 24 adaptive runs. Its steps are real training steps, so the cost is the slower candidates, not time taken from training
The interior results on this page were measured on attention, and attention is the layer the widely deployed language models are built from.
the planner
blaze-coreempty dependency list, 23 tests against fake runners, no GPU
the traces
results/every figure on this page is read from a file here
the reproduction
vendor/burn-stockkept pristine beside the patch: 653 insertions, 5 deletions, 9 files
turning it on
one line of burn.toml, off by default. Idle overhead within 0.14%
Measured on one Apple M5 through Metal, one framework revision, not reproduced on other hardware. No full training run was executed.
capataina.dev / research
Who made this, and where it lives
I build systems that measure themselves.
I’m Caner. Most of what I build is infrastructure that has to prove its own claims: compilers, engines, and the harnesses that decide whether a thing got faster or only looked like it.
Blaze is the largest of them. Roughly half of it is not the planner at all: it is the gate, the significance test and the bench that stop it believing its own results. Three findings here are wins that evaporated under checking, and those taught more than the ones that survived.
The repository is private, for now.
Every figure here traces to a file in it: a trace, a commit, a results file. Several were corrected while this page was written, because rendering a number forces you to name its scope.
It opens when the write-up does, and the traces stay with it, so these claims remain checkable rather than becoming history. Until then this page is the honest version: the fee, the nulls and the unexplained results sit alongside the wins.