v0.16.0 Migration Guide
Local concurrency is rebuilt around a single per-host admission daemon
(sparkwingd). It measures the machine, owns host admission and the
queue, and reacts to a run's death through the kernel closing its socket
-- so there are no heartbeats, no leases to tune, and no manual sweeps.
The old box-slot semaphore, the store-side local admission slots, and the
destructive verbs that let a single run destabilize a shared machine are
all removed in this release. There is no daemonless fallback: a fallback
path is how a system grows two admission models again.
This is a hard cut. Removed symbols are gone, not aliased. Upgrade every sparkwing binary that shares a machine or a state database together -- see Upgrade every pinned binary on a machine in one sitting, which is the one step most likely to bite.
Removed CLI verbs, flags, and environment variables
| Removed | Replacement |
|---|---|
sparkwing box-slots show / list | sparkwing queue -- holders, waiters, positions, and ETAs, served by the daemon |
sparkwing box-slots set --to N | off | default | Nothing. The daemon measures real host capacity; there is no cap to set. Override reserved headroom in config if you must. |
sparkwing box-slots release [--force] | sparkwing doctor clears a lock file whose owner is gone; a live owner is reported, never killed. |
sparkwing box-slots sweep [--reap] | sparkwing doctor. The daemon reacts to a run's death immediately, so wedged holders do not accumulate; sparkwing queue flags an alive-but-idle holder with the exact non-destructive recovery command. |
sparkwing maintenance | sparkwing doctor. State converges without a scheduled sweep; doctor clears only provably-dead leftovers. |
SPARKWING_BOX_SLOTS (cap baseline) | Nothing. Capacity is measured, not configured. |
SPARKWING_BOX_SLOT_STALL_TTL (stall threshold) | Nothing. Stall flagging is measured CPU over a window, not an mtime timeout. |
--sw-box-slots / --sw-no-wait (on sparkwing run) | Nothing. Runs queue in the daemon and Ctrl-C cancels the wait cleanly. |
SPARKWING_BOX_SLOTS_PIN / SPARKWING_BOX_NO_WAIT | Nothing, as above. |
SPARKWING_BOX_ID is unaffected: it still sets the identity a
box-scoped .Concurrency() group keys on.
If a script or CI job invoked any removed verb as "remediation," delete that step. The daemon needs no babysitting; when you do need to look or repair, the whole operational surface is two verbs:
sparkwing queue-- read the truth: every holder, every waiter, every position and ETA.sparkwing doctor-- repair only provably-dead state, idempotently, and report it. Safe to run at any time.
HostAdmission removed from the SDK
Host admission is now universal and implicit -- every local run is
admitted against measured host capacity -- so there is no per-group host
flag. ConcurrencyLimit.HostAdmission and Plan.HostAdmission() are
removed, and ScopeBox means locality only (which machine shares the
group), never "also take a host slot."
Before:
grp := sw.NewConcurrencyGroup("build", sw.ConcurrencyLimit{
Capacity: 2,
Scope: sw.ScopeBox,
HostAdmission: true,
})
After:
// Host CPU/memory admission is automatic. Keep .Concurrency() only for a
// logical lock (a deploy gate, a shared fixture). Declare an optional
// cold-start cost hint if the pipeline is heavy and unmeasured.
grp := sw.NewConcurrencyGroup("build", sw.ConcurrencyLimit{
Capacity: 2,
Scope: sw.ScopeBox,
})
plan.Resources(sparkwing.Cores(4), sparkwing.MemoryGB(8))
A pin set with .Resources() is authoritative but policed: when it
drifts from what the pipeline actually uses, sparkwing queue flags the
gap. The posture is declare nothing and let sparkwing measure; pin
sparingly, and sparkwing polices the pin.
Interrupted runs finalize themselves
A local run now installs a signal handler: Ctrl-C (SIGINT) or SIGTERM
cancels the run cleanly and finalizes its row as cancelled, naming the
signal, instead of leaving the row stuck running. When a run is killed
outright (SIGKILL, panic, power loss), the kernel closes its daemon
connection and the daemon finalizes the orphaned row as interrupted. No
action is required; if an older interrupted row is still stuck
running after upgrading, sparkwing doctor finalizes it.
The trigger API drops plan_admission
The controller trigger API no longer carries a plan_admission request
block; spawned children inherit admission by attaching to the parent
run's daemon lease (SPARKWING_LEASE_TOKEN) rather than through the
controller. Callers that built trigger payloads by hand should drop that
block; the SDK spawn helpers (RunAndAwait, PipelineRef) handle
inheritance transparently.
Runs-store schema moves to version 10
The concurrency rebuild advances the local runs-store schema from version 6 to version 10: the daemon's admission ledger, the measured per-command CPU and memory columns, the queue-wait and contended-run bookkeeping, and a stamp recording the minimum sparkwing version a database requires all land here. The store migrates a database forward the first time a newer binary opens it; migration is one-way, so a database opened by v0.16.0 no longer opens under an older binary.
A binary older than the version a database needs now refuses it by name:
this state database needs sparkwing >= v0.16.0; you have <older>; run sparkwing version update --cli
instead of reporting a bare schema number. That is the same failure a
mixed machine hits when one repo lags on an old .sparkwing pin, which is
why the pin bump below is done for every repo on the machine at once.
Upgrade every pinned binary on a machine in one sitting
This is the step that bites, so do it deliberately.
Admission lives in the compiled pipeline binary, not in the CLI. Each
repo's .sparkwing/ pins a sparkwing SDK version, and the pinned version
is what decides how that repo's runs get admitted. A repo still pinned to
an older SDK keeps running the old box-slot admission -- and that
admission is invisible to the daemon. The daemon cannot see a
legacy-pinned run's box-slot hold, and the legacy run cannot see the
daemon's leases, so the two admit against each other's blind spots and
can oversubscribe the machine.
So: when you migrate a machine, bump the .sparkwing sparkwing pin for
every repo that runs on it, in one sitting. Do not leave one repo on
an old pin "for now."
To find the stragglers, run sparkwing queue (or sparkwing doctor) on
the machine: both warn when a legacy-pinned binary is admitting outside
the daemon, for example:
warning: 2 legacy-pinned pipelines running outside daemon admission -- bump their sparkwing pins
The warning clears once the last old pin is bumped and its runs have turned over. Until then, treat the machine as mixed and expect the daemon to under-account for the legacy runs' CPU.