Migrating to v0.9.0
v0.9.0 splits the overloaded .Cache() modifier into two independent
primitives:
Cache-- content-addressed result memoization. "Same work: compute once, reuse." No scope, no group.Concurrency-- a named budget a set of nodes shares. "Different work competing for a shared budget: everyone runs, we only bound how many at once."
The old Cache(CacheOptions{Namespace, Max, OnLimit, ContentHash, ...})
overloaded one Namespace field to mean both a memoization identity and
a concurrency-coordination scope, which collided: throttling distinct
nodes together required one shared namespace, but that made their
content hashes collide too, so one node replayed another's result.
The signature change is a compile error at every call site, which walks you through the migration. There is no compatibility shim.
What disappears
| v0.8 | v0.9 |
|---|---|
Cache(CacheOptions{...}) | Cache(key, TTL(...)) + Concurrency(group, cost...) |
CacheOptions struct | Removed |
CacheOptions.Namespace | A ConcurrencyGroup name (concurrency) and/or a content CacheKey (cache) |
CacheOptions.Max | ConcurrencyLimit.Capacity |
CacheOptions.OnLimit | ConcurrencyLimit.OnLimit |
CacheOptions.ContentHash | The CacheKeyFn passed positionally to Cache |
CacheOptions.CacheTTL | TTL(d) option |
CacheOptions.QueueTimeout | ConcurrencyLimit.QueueTimeout |
CacheOptions.CancelTimeout | ConcurrencyLimit.CancelTimeout |
OnLimit: Coalesce | Removed (in-flight dedupe is automatic in Cache) |
OnLimitPolicy type | OnLimit type |
Plan.Cache(CacheOptions{...}) | Plan.Concurrency(group) |
node.CacheOpts() accessor | node.CacheConfig() + node.ConcurrencyGroupRef() / node.ConcurrencyCost() |
Cache: content key plus options, no more CacheOptions
Memoization now takes a content key directly plus functional options.
The key is keyed on content alone; in-flight dedupe (what the old
OnLimit: Coalesce provided) is automatic.
Before:
build := plan.Add("build", &Build{}).Cache(sparkwing.CacheOptions{
Namespace: "build",
ContentHash: func(ctx context.Context) sparkwing.CacheKey {
return sparkwing.Key("build", target, sourceDigest.Get(ctx))
},
CacheTTL: 24 * time.Hour,
})
After:
build := sparkwing.Job(plan, "build", &Build{}).Cache(
func(ctx context.Context) sparkwing.CacheKey {
return sparkwing.Key("build", target, sourceDigest.Get(ctx))
},
sparkwing.TTL(24*time.Hour),
)
TTL is optional (defaults to sparkwing.DefaultCacheTTL, 7 days).
Return sparkwing.NoCache from the key fn to skip caching for an
invocation. The JobGroup mirror is group.Cache(key, opts...).
Concurrency: a named group, not a Cache namespace
Throttling, mutexes, and gates move to Concurrency. Define the group
once and pass the handle to each member node. The scheduling fields that
lived on CacheOptions (Max, OnLimit, QueueTimeout,
CancelTimeout) live on ConcurrencyLimit.
Before (a semaphore of 3, queue the rest):
plan.Add("index", &Index{}).Cache(sparkwing.CacheOptions{
Namespace: "es-writer",
Max: 3,
OnLimit: sparkwing.Queue,
})
After:
esWriter := sparkwing.NewConcurrencyGroup("es-writer", sparkwing.ConcurrencyLimit{
Capacity: 3,
OnLimit: sparkwing.Queue,
})
sparkwing.Job(plan, "index", &Index{}).Concurrency(esWriter)
New in v0.9: Capacity/cost are author-defined units, so a member can
draw more than one unit (.Concurrency(group, 4)); Scope
(ScopeRun / ScopeBox / ScopeGlobal) selects how far the budget
reaches. Count-limiting is the cost-1 case. See
sdk.md.
Before (mutex -- one at a time):
plan.Add("deploy", &Deploy{}).Cache(sparkwing.CacheOptions{Namespace: "deploy-prod"})
After:
deployGate := sparkwing.NewConcurrencyGroup("deploy-prod", sparkwing.ConcurrencyLimit{
Capacity: 1,
})
sparkwing.Job(plan, "deploy", &Deploy{}).Concurrency(deployGate)
OnLimit: Coalesce is gone
OnLimit: Coalesce (one leader runs, the rest inherit its output) is
removed. That behavior is now automatic in Cache and keyed on content
instead of the group, which also fixes the old hash-blind coalescing.
Before:
plan.Add("image", &Build{}).Cache(sparkwing.CacheOptions{
Namespace: "build",
OnLimit: sparkwing.Coalesce,
ContentHash: func(ctx context.Context) sparkwing.CacheKey {
return sparkwing.Key("image", repo.Ref().Get(ctx).SHA)
},
})
After -- just Cache; concurrent identical content dedupes on its own:
sparkwing.Job(plan, "image", &Build{}).Cache(func(ctx context.Context) sparkwing.CacheKey {
return sparkwing.Key("image", repo.Ref().Get(ctx).SHA)
})
If you used Coalesce purely to collapse a thundering herd onto one
execution, Cache already does that. If you used it to also throttle a
group, add a separate .Concurrency(group).
Plan.Cache becomes Plan.Concurrency
Plan-level .Cache() was concurrency-only already (a plan has no single
output to memoize), so it becomes Plan.Concurrency.
Before:
plan.Cache(sparkwing.CacheOptions{Namespace: "prod-deploys", Max: 1, OnLimit: sparkwing.Fail})
After:
plan.Concurrency(sparkwing.NewConcurrencyGroup("prod-deploys", sparkwing.ConcurrencyLimit{
Capacity: 1,
OnLimit: sparkwing.Fail,
}))
Reading modifiers back
If you inspected node.CacheOpts(), the split surfaces two accessors:
cfg := node.CacheConfig() // *CacheConfig (Key, TTL); nil if no Cache()
g := node.ConcurrencyGroupRef() // *ConcurrencyGroup; nil if no Concurrency()
cost := node.ConcurrencyCost() // admission cost, 0 if no membership