v0.17.0 Migration Guide

Runs-store schema moves to version 11

Honest capacity measurement under contention and pipeline change advances the local runs-store schema from version 10 to version 11. The migration is purely additive: it adds five columns to the pipeline_profiles table -- plan_hash (the DAG-topology hash a profile is measured against), floor_cores / floor_memory_bytes (the per-pipeline demand floor a contended run raises without setting the measured peak), and prev_peak_cores / prev_peak_memory_bytes (the predecessor plan's peak a re-measuring version is charged 2x of). No column is dropped or rewritten and no existing row is lost.

The store migrates a database forward the first time a newer binary opens it; migration is one-way, so a database opened by v0.17.0 no longer opens under an older binary. A binary older than the version a database needs refuses it by name (the database stamps the minimum sparkwing version it requires), rather than reporting a bare schema number:

this state database needs sparkwing >= v0.17.0; you have <older>; run sparkwing version update --cli

Upgrade every pinned binary on a machine in one sitting

Admission lives in the compiled pipeline binary, not in the CLI. Each repo's .sparkwing/ pins a sparkwing SDK version, and the pinned version is what decides how that repo's runs get admitted and what schema it expects. Once any binary migrates a shared state database to schema 11, every repo on the machine still pinned to an older sparkwing refuses that database until its pin moves.

So when you upgrade a machine, bump the .sparkwing sparkwing pin for every repo that runs on it, in one sitting -- do not leave one repo on an old pin "for now." Run sparkwing queue or sparkwing doctor to find stragglers; both name a legacy-pinned binary admitting outside the daemon.