feat(speculation): generator contract and bestfirst impl

[behinddwalls]

Add submitqueue/extension/speculation/generator, the candidate-stream composition point (Generator/PathIterator) the standard Speculator pulls from, plus the bestfirst implementation and mocks.

Open takes only the queue's batches. The generator offers every coherent path in the space, including paths whose builds already ran — suppressing finished paths belongs to the Allocator, the piece that already reconciles candidates against the stored path sets by ID. Keeping the generator ignorant of path sets leaves it a pure enumerator and removes the expand-before-skip subtlety from its iterator: Next pops, expands, returns, with no filtering loop. Ranking stays implementation-defined at the contract; only bestfirst commits to a scale.

bestfirst ranks each path by how likely all its assumptions are to hold and builds paths lazily. A single heap holds every path built but not yet handed out, across all heads; handing one out builds only the one or two paths that come after it within that same head. Pulling k candidates builds at most 1+2k paths however large the space behind them, so a head with twelve unresolved dependencies costs no more at the front of the queue than one with two.

Scores are summed log probabilities, which preserves ordering without underflow. Multiplying the probabilities instead loses the ordering outright: two heads waiting on 1200 and 1300 coin-flip dependencies both round to exactly 0.0, tie, and the heap falls back to comparing path IDs — so it can emit the 1300-dependency path first, one that is 2^100 times less probable. The ranking does not degrade, it inverts. Summing logs keeps the arithmetic in range at any depth, preserves the order because log is monotonic, and gives an assumption that cannot happen a score of negative infinity, which compares correctly against every finite score. A wide-head test pins this, asserting both that two 7100-dependency heads order correctly and that exponentiating either score yields zero.

Scores from the injected scorer are validated on the way in. Everything downstream treats a score as a probability — its log is the ranking key, its complement is the other side's probability — so a value outside [0, 1] would not fail loudly, it would quietly produce a positive log, an inverted ordering, or a NaN that makes every comparison false.

Generation honors context cancellation. Open checks before each head and newHead before each dependency, since a head can carry hundreds; Next checks before each pull, which is where a caller that has stopped consuming stops paying. All return the context error with no iterator and no candidate.

Each unresolved dependency has a preferred assumption — the one with the higher probability — and an unpreferred one. The head's best path takes preferred everywhere and its score is computed once; every other path takes some subset of the head's alternatives, each adding its relativeScore. The heap stores only paths: which alternatives a path takes is read back off its assumptions, so there is no parallel representation to keep in sync, and pathFor and takenAlternatives are exact inverses. Subsets are walked by a canonical expansion under which every subset has exactly one parent, so each path is built exactly once with no visited set, and a child never outranks its parent — which is what lets the heap hand paths out in true descending order. Equal scores break on path ID among candidates resident in the heap together, which makes a run repeatable rather than globally ID-sorted.

Both sides of each unresolved dependency are carried through as computed rather than derived from one another: 1-(1-score) is not score in floating point, and deriving one side would perturb every score below the cutoff.

There is no depth bound. It existed to cap 2^n enumeration, and lazy generation removed that cost, so a head may now have as many unresolved dependencies as it likes.

A dependency absent from the live batches cannot be scored, and gets a high default rather than an even split. The cutoff used to pick which side is preferred is a separate constant from that default, so the two move independently — sharing one constant would have silently inverted the preferred assumption for every dependency below the default.

Also sharpens scorer.Scorer's contract: it returns the probability that a batch's build succeeds, not "the likelihood of a successful land", which conflated the build with the merge. That is exactly the quantity bestfirst needs, since an assumption that a dependency succeeds is refuted when its builds cannot pass. The scorer README is rewritten around entity.Batch, which the interface has taken for some time.

The package is three files by phase: bestfirst.go holds the generator and scoring done at Open, head.go holds a head and everything derived from it including path materialization, and iterator.go holds the heap and the pull-time walk.

Stack

1. refactor(entity)!: speculation paths carry assumptions, not bets #473
2. @ feat(speculation): generator contract and bestfirst impl #446
3. feat(speculation): allocator contract and sticky impl #450
4. feat(speculation): standard composed speculator #451