Migrating to v0.47.0

Sparkwing v0.47.0 teaches a pipeline's on.schedule trigger a mapping form and ships sparkwing crons, which evaluates those cadences on the machine that arms them. Two edits are mechanical for Go callers: the schedule trigger became a struct, and cache-key callbacks return an error. The runs store advances to schema 32 additively, so nothing needs to happen on a host.

on.schedule takes a mappingSection anchor link

  • Before: on.schedule was a cron string that sparkwing recorded and displayed. Nothing evaluated it, so the cadence came from an external timer. In Go, pipelines.Triggers.Schedule was a string.

  • After: on.schedule still accepts the bare cron string, and also a mapping:

    on:
      schedule:
        cron: "0 3 * * *"
        tz: America/Denver
        overlap: queue
        catch_up: 6h
    

    tz defaults to UTC and takes local for the host's own zone. overlap is skip (default) or queue, and decides a fire that comes due while the previous scheduled run is still running. catch_up defaults to 1h with a 2m floor, and bounds how late a due minute may still fire. Sparkwing validates the cron expression when the config loads, so a malformed one now fails the command that reads the config.

    In Go, pipelines.Triggers.Schedule is a *pipelines.ScheduleTrigger.

  • Migration: YAML needs no change. Go callers read t.Schedule.Cron where they read t.Schedule, and test t.Schedule != nil where they tested t.Schedule != "".

  • Why: A recorded-but-unevaluated field could not say which host runs the cadence, what a late tick should do, or what an overlapping run should do. The mapping carries those answers, and arming a host is a separate, explicit step on that host: sparkwing crons install.

  • Older SDK pins: a pipeline binary built against an SDK before this release reads schedule: as a string, so a repo pinned to one must keep the scalar form until it bumps; the mapping form makes that binary warn that the project config is unreadable and run with the caller's arguments only.

Cache-key callbacks return errorsSection anchor link

CacheKeyFn now returns (CacheKey, error). Add nil to successful returns, and propagate failures from key inputs:

node.Memoize(func(ctx context.Context) (sparkwing.CacheKey, error) {
    key, err := inputs.RepoFiles()(ctx)
    if err != nil {
        return "", err
    }
    return sparkwing.Key("example-build", key), nil
})

Return sparkwing.NoCache, nil for an explicit bypass. Replace empty-key bypasses with that sentinel; errors, panics, empty keys, and expired resolution deadlines now fail the node before dispatch.

The inputs helpers return the same two-result callbacks. Update direct calls to check their errors. inputs.Compose propagates an input error and stops at an explicit NoCache; it rejects empty input keys.

Scheduled pipelines and runs-store schema 32Section anchor link

  • Before: The runs store was at schema 31 and had nowhere to keep a pipeline schedule; nothing in state.db described one.
  • After: Schema 32 adds two tables. cron_schedules holds one row per armed repository checkout and pipeline -- the cron expression, zone, overlap policy, and catch-up window the repository declares, plus the host state around them: paused, still declared, when it was armed and by whom, the cursor naming the last due instant resolved, the last fire, and the next matching instant. cron_fires holds one row per resolved instant (fired, skipped_overlap, missed, or failed), pruned to the newest 200 per schedule. Tick bookkeeping lives in sparkwing_meta under the crons.last_tick_at, crons.last_tick_host, crons.last_tick_version, and crons.last_tick_error keys.
  • Migration: None to perform. The migration is additive: it declares no schema requirement and alters no existing table, so a binary built before it opens and writes the same database exactly as it did, never reading the two new tables. The store creates them on the next open.