D-0003 — Codec-Declared Middlewares — a general pattern for reusable middleware/policy values¶
Status: Implemented — architectural foundation. All 7 design decisions (D1-D7, see "Resolved design decisions") are shipped and verified across
api/rest(RouteANDSSERoute),api/events,api/reqreply, and every pub/sub + request-reply adapter (adapters/mqtt5,adapters/mqtt,adapters/zeromq). This remains the accurate, current description of shipped code — UPDATED by Addendum 3 below, which folds in the middleware-consolidation effort's outcome: presence-only (non-merged) param declarations are now FULLY part of this mechanism (closing this doc's own §2 gap), andmiddleware.Middlewareis no longer unchanged — it was DELIBERATELY shrunk to{Name, Security}(see Addendum 3 for the full reasoning and what was deleted).Addendum 4 below further UPDATES
api/restspecifically:Transform/ClientTransform/TransformSSE/ClientTransformSSE(the route/channel-BOUND attachment mechanism this doc's main body and earlier addenda describe) were REMOVED — folded intoHandleMW/ClientMWvia reflection-based shape detection (docs/design/d-0007-declarative-middleware-layering.md's Rollout Phase A). Everyrest.Transform(...)/rest.ClientTransform(...)code sample below is HISTORICAL — no longer accurate forapi/rest. See Addendum 4 for the currentapi/restmechanism, theGrantedScopesSecurity convention, and thecodex.EncodeMergeVars/omit-empty constructor family.Addendum 5 below further UPDATES
api/eventsspecifically (Rollout Phase B):events.Transform/events.ClientTransformwere REMOVED the SAME way — folded intoSubscriber.SubscribeMW/Publisher.PublishMW. Everyevents.Transform(...)/events.ClientTransform(...)code sample below is now ALSO HISTORICAL. See Addendum 5 for events' own version of the reflection-based shape detection (a genuine extra wrinkle vs. REST's), the generalizedevents.SecurityMiddleware[In,Out], andClient.AddConnectSecurityScheme.Addendum 6 below further UPDATES
api/reqreplyspecifically (Rollout Phase C — the FINAL of the 3 planned phases):reqreply.Transform/reqreply.ClientTransformwere REMOVED the SAME way — folded intoRoute.HandleMW/Route.ClientMW. Everyreqreply.Transform(...)/reqreply.ClientTransform(...)code sample below is now ALSO HISTORICAL. See Addendum 6 for reqreply's own generalizedSecurityMiddleware[In,Out], its fully-duplexContextFieldwiring,Server.AddConnectSecurityScheme, and a confirmed cross-package.Use()+bound-HandleMWgap shared withapi/rest(tracked at the time, FIXED by Addendum 7 below — see there).Addendum 7 below REPLACES the reflection-based bound-attachment mechanism Addenda ⅘/6 each describe, across ALL 3 APIs — the SAME
Middleware[In,Out]type serving both the agnostic AND bound roles, discriminated viaisBoundHandleMWShape-style Fn-shape reflection, is SUPERSEDED by two explicit, compile-time-distinct types per package (the reusableMiddleware[In,Out], UNCHANGED,.Use()-only; a NEWBoundMiddleware/BoundSubscribeMiddleware/BoundPublishMiddlewarefamily, Fn embedded at construction, attached via a dedicated method). This ALSO fixes Addendum 6's own tracked.Use()+bound gap (DuplicateMiddlewareNameError) — now structurally impossible, since neither class can satisfy the other's attach-interface. D7 (theAmbiguousMiddlewareAttachmentErrordesign decision referenced in the status line above) is DELETED outright by Addendum 7, not merely "shipped" — it is now structurally unreachable, not a check that still runs. EveryisBoundHandleMWShape/isBoundClientMWShape/isBoundSubscribeMWShape/isBoundPublishMWShapemention in Addenda ⅘/6 below is HISTORICAL — see Addendum 7 for the current mechanism.Forward-looking note (not a status change): Feature (a broader roadmap-stage redesign, idea only, no code written — formerly titled "Feature/Provider") has RESOLVED its relationship to this doc's
Declaration[In,Out]/Middleware[In,Out]mechanism (that doc's §3, via a confirmed 4-stage lifecycle model, tested with a throwaway Go prototype): both mechanisms occupy the SAME declare-time, spec-contributing lifecycle stage as that doc's sealed, per-adapterCapabilitymechanism, WITHOUT merging into one Go type — each keeps its own runtime-enforcement path (Middleware[In,Out]: reflection-basedTransform/.Use(mw), unchanged;Capability: sealed, compile-timeAttach-time supply). This is NOT the earlier sub-round's "yes, subsume" conclusion (reached against a since-superseded, open string-ID-based primitive) — that conclusion no longer applies; the current resolution keeps the two mechanisms distinct. Nothing in THIS doc or in shipped code changes as a result of that roadmap doc alone; any actual migration remains explicitly deferred to a separate, dedicated implementation-planning round. Until that round ships, this doc's design remains final and unchanged.Codec-backed, per-pattern middleware declaration (
middleware.Declaration[In,Out]+rest.Middleware[In,Out]/events.Middleware[In,Out]) forapi/rest(RouteANDSSERoute) andapi/events, fully additive — no breaking changes tomiddleware.Middleware/SecurityScheme/existing.Use(...)call sites. Two attachment styles, one shared vocabulary across ALL THREE surfaces:Transform/ClientTransform(route/channel-BOUND,fngetsreq *Req/msg *Taccess for reading/enriching, plusInfor vars the route/channel doesn't model) and plain.Use(mw)(route/channel-AGNOSTIC, viaWithReceive/WithSend-bundled Fns, reused verbatim across many routes/channels). SSE gets its ownTransformSSE/ClientTransformSSE; events adopts REST's naming exactly. Deliberately NO raw*http.Requestanywhere — everything stays codec-defined, a stricter position than net/http's/chi's own middleware convention. A feasibility analysis confirms extending this toports(File/Cache/Dir) later is realistic, not aspirational (see "Feasibility" below);ports.SQLis a confirmed, separate exception.Headline decisions from the D1-D7 resolution pass: D2 makes a middleware
fnerrorErrorPattern-eligible, falling back to a NEWrest.MiddlewareError(notSecurityError); D6(b) ENFORCESDeclaration.Nameuniqueness per route/channel (NEWrest.DuplicateMiddlewareNameError), deliberately diverging from the legacy mechanism's looser precedent; D7 REJECTS combining both attachment styles on oneMiddleware[In,Out]value as ambiguous (NEWrest.AmbiguousMiddlewareAttachmentError).Supersedes
docs/roadmap/common-middleware-architecture.md(now deleted, its finding fully absorbed here) — see "Relationship to other roadmap docs" below. ← Back to Design Documents
Lessons learned (implementation, Phase 1)¶
-
RouteMiddleware's marker method must be EXPORTED, not unexported. Every code snippet in this doc originally showedisRouteMiddleware()(unexported), mirroringports.Pattern's own technique. This compiles right up until an ACTUAL cross-package value (rest.Middleware[In,Out]) is passed to.Use(...)— Go's unexported-method interface satisfaction is scoped PER PACKAGE (confirmed with a minimal reproduction), so a method literally namedisRouteMiddlewaredeclared in packageapi/restis a DIFFERENT identifier frommiddleware.RouteMiddleware's own unexportedisRouteMiddleware, and never satisfies it.ports.Patternavoids this because EVERYPatternimplementation (RESTPattern,EventPattern, etc.) is declared INSIDE theportspackage itself — never cross-package. Fixed by renaming the marker method to the EXPORTEDRouteMiddlewareMarker()everywhere (packagemiddleware's interface +Middleware's implementation, andrest.Middleware[In,Out]'s own). All code snippets in this doc below reflect the corrected, exported name. -
ResponseDepositor(examples/rest-api) is NOT a candidate for aTransform-based refactor — confirmed during Phase 4, not a gap in the implementation. The original Motivation section's premise (POST /users'ssessioncookie needing declarativeMaxAge/Insecure) is real, butResponseDepositor.SetHeader/SetCookieare called from INSIDEMakeCreateUserHandler, AFTERstore.Save(record)runs — i.e. the Location header and session cookie VALUE are derived from data the HANDLER ITSELF computes (a newly created user's ID), not from the route's ownReq.Transform's dispatch is deliberately PRE-HANDLER-ONLY (D1, confirmed and not revisited: "no second, post-handler attachment point... a concern that genuinely does already have a mechanism: the EXISTING wrapping-shaped Fn") — aTransform-attachedfnruns BEFORE the handler and has no access to whatever the handler will go on to compute. The §7 worked example (sessionCookiePolicy) that motivated this whole design works ONLY because it hardcodesnewSessionToken()independent of any handler-computed value — it was never actually a fit forexamples/rest-api's OWNCreateUserRoute, whose session value depends onuser.ID. Conclusion:ResponseDepositor/chiResponseDepositor/nethttpResponseDepositorremain the CORRECT, intentional escape hatch for this specific case (post-handler-computed response header/cookie values) — not a gap this design was ever meant to close, and no code change is made here. A genuinelyTransform-suited case is one where the response header/cookie value derives ONLY from the route's ownReq/topic vars/a middleware's ownIn— e.g. this doc's §7 API-key/session-attribute sketches — never from the handler's own return value.
Motivation¶
examples/rest-api's POST /users handler needs a session response cookie with
custom attributes (MaxAge: 3600, Insecure: true). Investigating this surfaced a real
gap: rest.NewRequiredResponseCookieParam already merges a cookie's NAME/VALUE
declaratively, but merge-derived cookies always get DEFAULT PendingCookie.Opts
(confirmed in code: both adapters/nethttp/serve.go and adapters/chi/serve.go build
PendingCookie{Name: k, Value: v} with a zero-value Opts for every merge-derived
cookie) — there is no way today to declare MaxAge/Secure/SameSite/Path/Domain
declaratively. A caller who needs those must drop to the adapter-specific
WithResponseCookies/PendingCookie.Opts escape hatch (exactly what
examples/rest-api's ResponseDepositor/chiResponseDepositor/
nethttpResponseDepositor does today).
Investigating this surfaced a SECOND, independently-known gap, already tracked (see
"Relationship to other roadmap docs" below): middleware.Middleware is a single flat
struct with 5 REST-only param-contribution fields, imported directly by api/rest,
api/reqreply, AND api/events — even though only REST uses those fields. api/events
already carries an INTERIM fix for this (checkUnsupportedMiddlewareParams in
api/events/builder.go, confirmed via code — eagerly rejects any .Use()-attached
middleware whose REST-only fields are non-empty).
Both gaps are resolved by ONE mechanism: giving middleware itself an Input/Output codec pair, exactly like a route/channel declares its Req/Resp (or Item) shape via a codec — and making that mechanism PER-PATTERN (REST's own merge vocabulary vs. events' own), not a single shared shape.
Guiding principle: middleware IS a spec-layering mechanism (non-spec is the degenerate case, not a separate mechanism)¶
Before designing anything new, it's worth stating explicitly what every EXISTING
middleware kind already does, since this design must fit the SAME model, not invent a
parallel one. Middleware, as a whole, is fundamentally a spec-layering
mechanism: a declare-time value, attached via .Use(...), resolved ONCE (at
Register/ValidateRoute time for anything affecting the spec; at dispatch time for
anything affecting runtime behavior). Some attachments layer something into the final
OpenAPI/AsyncAPI spec; some have dispatch-time runtime behavior; some do both. A
middleware contributing NOTHING to the spec — an Observer, a retry wrapper, a raw
enrichment step — is not a DIFFERENT mechanism from spec-layering; it is simply the
case where the spec column of the SAME mechanism is empty. Every existing and
newly-designed middleware kind already fits this ONE model, confirmed via code:
| Middleware kind | Spec effect (layers into the final spec) | Runtime effect (dispatch-time behavior) |
|---|---|---|
middleware.SecurityScheme, attached alone (no HandleMW ever paired) |
Layers a Security requirement + scheme | None — documents an externally-enforced requirement |
middleware.SecurityScheme + paired HandleMW/ClientMW |
Layers a Security requirement + scheme | Verifies (server) / supplies (client) the credential |
FromHeaderParam/FromCookieParam/FromQueryParam/response siblings |
Layers a header/cookie/query param | None (spec-only, unless separately paired with a general-purpose Fn reading the same value) |
Security-shaped Fn, attached UNPAIRED (mw nil or Security nil) |
None — the degenerate case | Direct write access into *Req/*T — Transform's enrichment use case, confirmed "free" per d-0002 |
Wrapping-shaped Fn (func(http.Handler) http.Handler, pub/sub's func(next) next) |
None — the degenerate case | Wraps the entire dispatch/round-trip — Observer, retries, tracing |
rest.Middleware[In,Out] (this doc's design) + Transform/ClientTransform (route-BOUND) or .Use(mw) (route-AGNOSTIC) |
Layers header/cookie/query/response params (§3) — the SAME conflict-detection the legacy path already uses | Transform/ClientTransform: reads/enriches the route's own req *Req (or msg *T) PLUS decodes In from vars it doesn't model; .Use(mw): decodes In only, NO req/msg access, reusable verbatim across routes (§4) |
rest.PathPrefix/events.TopicPathPrefix (sketch, §"Extending the model") |
Layers a path/topic-string transform | None — the mirror-image degenerate case |
ports.FilePathPrefix/DirPathPrefix (sketch, §"Extending the model") |
None — ports has no spec to layer into | None — resolved once into internal config at declare time, not per-call dispatch (see the refinement discussed where this row is sketched) |
This table is the reason Middleware[In,Out] retaining the FULL declared param
(spec metadata AND merge field — see §3) isn't a bespoke fix bolted onto a
narrower design — it is simply this middleware kind populating BOTH columns at
once, exactly like paired Security already does. It is also why the existing
non-spec Fn shapes (§9) need no special justification: they were ALWAYS spec-layering
middleware with an empty spec column, whether or not that was stated explicitly
before.
This is the guiding design principle for REST and events middleware today, and
should guide any FUTURE ports middleware mechanism too. No ports.Middleware
concept exists yet (ports.Pattern is a different thing — binding metadata, not
cross-cutting-concern attachment); docs/roadmap/mcp-ports-declarative-middleware.md
(formerly declarative-middleware.md) already sketches/proves ports.File's own
decorator shapes as Phase 2 prior art.
Feasibility of a full .Use()-based ports.Middleware[In,Out] — a grounded analysis, not just a placeholder¶
The long-term goal is for ports to adopt the SAME .Use()+codec-backed model this
doc designs for REST/events, not a permanently separate mechanism. Investigated
whether the building blocks this design depends on ALREADY exist in ports —
they do, more concretely than expected.
The prerequisite this whole design leans on is already present in ports. Every
Middleware[In,Out] variant in this doc reuses ALREADY-GENERIC param constructors
(rest.NewRequiredHeaderParam[T,V], events.NewTopicParam[T,V]) instead of inventing
new ones — retargeting T from a Route's/Channel's own type to a Middleware's In/
Out. Confirmed via code: ports already has the IDENTICAL shape of reusable,
generic constructor — ports.NewCacheKeyParam[T, V any](...) MergedCacheKeyParam[T]
and ports.NewFilePathParam[T, V any](...) are BOTH already generic over T,
mirroring rest.NewRequiredHeaderParam[T,V] exactly. A ports.Middleware[In,Out]
reusing WithCacheKey(p MergedCacheKeyParam[In])/WithFilePathVar(p
MergedFilePathParam[In]) is the SAME architectural move already proven twice (REST,
events), not a speculative extension.
Two port shapes need two different treatments, both achievable:
- Pattern-bound ports (
SourcePort/SinkPort/IOPort/ToolPortwithRESTPattern/EventPattern) already build a realrest.Route/events.Channelinternally — confirmedRESTPattern.Opts []rest.RouteOpt/EventPattern.Opts []events.ChannelOptalready pass straight through to it, meaning.Use(securityScheme)ALREADY reaches these ports today via the Pattern'sOptsfield. A future.Use()-on-the-port-itself is mostly SUGAR here: accumulate attachedmiddleware.RouteMiddlewarevalues and feed them into the Pattern'sOptsbefore the port constructs its handle — no new middleware TYPE needed, since the port already inherits REST's/events' real spec unchanged. - Pattern-less ports (
File/Cache/Dir, confirmed to have NO spec at all) — aports.Middleware[In,Out]here occupies ONLY the runtime column, exactly as the guiding-principle table predicts for a spec-less concern.NewFile(...)/NewCache(...)ARE already declare-once values (mirrorsNewRoute/NewChannel's own two-phase shape: declare once, dispatch per-operation viaRead/Write/Get/Set) — contrary todeclarative-middleware.md's OLDER decorator sketch, which attachesmws ...middleware.Middlewareas a CALL-TIME variadic parameter and concluded "there is only ONE attachment point for ports" — that conclusion describes ITS OWN specific design choice, not an inherent limitation ofports.File's structure. Under a.Use()-based redesign,File[T].Use(mws ...middleware.RouteMiddleware) File[T]at declare time + dispatch insideRead/Writemirrors REST/events' declare/dispatch split cleanly.
The "security via an HTTP auth endpoint" example maps directly onto this, with
In = struct{}: mirrors this doc's OWN REST session-cookie worked example (§7),
which also uses struct{} for a concern with no var-boundary to decode. fn(ctx,
struct{}{}) reads a credential from ctx (via middleware.ContextField, already
confirmed to compose for free) or an explicit param, calls the HTTP endpoint itself
(arbitrary I/O — exactly what the older decorator sketch already assumed a Fn's
body could do), and returns Out (e.g. granted scopes) or an error gating the real
Read/Write. Observer middleware for ports maps onto the SAME wrapping-shaped,
runtime-only category §9 already covers for REST/events — no new mechanism category
needed, though stats.WithObserver(ctx, obs)/ObserverFromContext already solve this
independently today via ctx injection, so a .Use()-attached observer middleware
would be an ADDITIONAL declare-once attachment style, not a required replacement.
Concrete, confirmed implementation costs (not just abstract feasibility):
ports.File[T].Read(vars, opts)/.Write(vars, v, opts)have NOctx context.Contextparameter today — confirmed via code, only an OPTIONALopts.Contextfield used solely for observer lookup. Any middlewarefnneeding realctx(the HTTP auth call's timeout/cancellation, orContextFieldaccess) requiresRead/Writeto gain a properctxparameter — a genuine signature change (additive variants, or breaking the existing ones) to plan for explicitly, not free.ports.SQLis confirmed metadata-only by design — no handle, noRoute/Channel-equivalent value exists to attach.Use()to at all (SQL query text/ placeholders are driver-specific closures owned by the adapter constructor directly). This is a REAL LIMIT CASE, not just an edge case: unlike File/Cache/Dir, SQL has no natural "declare once" value under the CURRENT architecture — it would need its own separate design (a different attachment point, e.g. at the adapter constructor call) or an accepted, permanent exception, mirroring howSQLPatternis already a deliberately different, metadata-only shape vs.RESTPattern/EventPattern/FilePattern.
Verdict: realistic, not merely aspirational — the core reusable-constructor
prerequisite already exists for File/Cache/Dir; the two-phase declare/dispatch shape
already exists structurally (just not wired to .Use() yet); the "arbitrary I/O in a
middleware's fn body" pattern this doc already designs for REST (struct{}-typed
In) directly covers the HTTP-auth-endpoint case with no new mechanism. What remains
is real but bounded: wiring .Use()/dispatch onto File/Cache/Dir, threading a proper
ctx through Read/Write, and either solving or explicitly accepting SQL's
metadata-only exception. Not designed in full here — recorded as the grounded
starting point for that future work, not a placeholder. See also
ports.FilePathPrefix/DirPathPrefix in "Extending the model" below — a concrete,
confirmed-needed driver (neither File nor Dir has a general workspace-root-prefix
mechanism today) for exactly this same "Pattern-less ports get .Use() at declare
time" mechanism, applied to a template/config transform rather than a runtime In/
Out dispatch.
The design¶
1. middleware.RouteMiddleware — additive marker interface, zero breaking changes¶
// package middleware
// RouteMiddleware is a marker interface any attach-time middleware value
// can implement to become passable to a route/channel's .Use(...) method.
// The marker method is EXPORTED (RouteMiddlewareMarker, not an unexported
// isRouteMiddleware) — Go's unexported-method interface satisfaction is
// scoped PER PACKAGE, so a type declared in api/rest/api/events could
// NEVER satisfy an interface whose unexported method lives in package
// middleware, regardless of name (confirmed the hard way during
// implementation — see "Lessons Learned"). ports.Pattern's own
// unexported-method sealing technique works ONLY because every Pattern
// implementation is declared IN the ports package itself.
type RouteMiddleware interface{ RouteMiddlewareMarker() }
// Middleware gains ONE new, trivial method — every EXISTING
// .Use(someSecurityScheme)-style call site keeps compiling unchanged
// (Go's structural interface satisfaction).
func (Middleware) RouteMiddlewareMarker() {}
Route.Use/SSERoute.Use (confirmed at api/rest/middleware.go:130,137) and
Subscriber.Use/Publisher.Use (confirmed at api/events/builder.go:1713,1793)
widen their parameter type from ...middleware.Middleware to
...middleware.RouteMiddleware. No existing call site anywhere in the repo needs to
change.
2. middleware.Declaration[In, Out] — new, minimal, pattern-agnostic shared core¶
// package middleware
type Declaration[In, Out any] struct {
Name string
InCodec codex.Codec[In]
OutCodec codex.Codec[Out]
// Security, when non-nil, is a COMPLETE security scheme + requirement
// declaration for the attaching route/channel. Folded in here (not
// re-declared per-pattern) so rest.Middleware[In,Out]/
// events.Middleware[In,Out]/reqreply.Middleware[In,Out] all gain it
// for free via embedding — see Addendum 3 below for why this WAS
// retrofitted after all, correcting this section's original claim.
Security *SecurityDeclaration
}
func NewDeclaration[In, Out any](name string, inCodec codex.Codec[In], outCodec codex.Codec[Out]) Declaration[In, Out] {
return Declaration[In, Out]{Name: name, InCodec: inCodec, OutCodec: outCodec}
}
Correction (Addendum 3, below): this section originally claimed SecurityDeclaration
was intentionally NOT retrofitted onto Declaration. That held true for a while,
but a later, separate round (breaking changes pre-approved) DID add a Security
*SecurityDeclaration field directly to Declaration[In,Out] — each pattern's own
SecurityMiddleware constructor builds a Middleware[struct{},struct{}] carrying
ONLY this field (In=Out=struct{}, no var-boundary to decode), attachable via the
SAME .Use(...)/HandleMW(...)/ClientMW(...) vocabulary as any other codec-backed
value, for the SAME-API-only case. The ORIGINAL legacy middleware.Middleware{Name,
Security} type (this section's own middleware.Common-proposal precedent, from the
now-deleted common-middleware-architecture.md) still survives, PERMANENTLY, for
the one case the codec-backed fold cannot replicate — see Addendum 3 for the full
reasoning (a genuinely cross-API-shared Security declaration — one Go value usable
on REST + events + reqreply at once — cannot be built from a per-pattern generic
type, since each pattern's internal dispatch only recognizes its own concrete type).
3. rest.Middleware[In, Out] — REST's own per-pattern derived type, populating BOTH the spec and runtime columns¶
Per the guiding principle above, Middleware[In,Out] is designed to occupy BOTH
table columns at once: its declared params layer into the route's spec (exactly like
middleware.Middleware's existing RequestHeaderParams/etc already do), AND its
merge fields drive its runtime dispatch (§4). This requires retaining the FULL
declared param — spec metadata (Name/Description/Required/Codec) AND the
merge-capable FieldCodec — not just the merge field alone (an earlier draft of this
design discarded the spec half, which is what originally caused §9's "declare twice,
no conflict check" gap; retaining both halves resolves it as a natural consequence of
fitting this table, not a bespoke fix).
Route/channel-AGNOSTIC vs. route/channel-BOUND — a real design gap, resolved by
splitting the ATTACHMENT point, not by splitting Middleware[In,Out] itself (§4
covers this in full; summarized here since it changes what Middleware[In,Out]
itself needs to carry). A Middleware[In,Out] whose runtime fn needs req
*Req/msg *T access (the enrichment case) is inherently BOUND to one route/channel
— Go's type system forces Req/T into fn's own type, so the literal same fn
value cannot be reused across routes with different Req types (confirmed: the
ALREADY-SHIPPED security-shaped Fn has this SAME constraint). But a Middleware[In,Out]
whose fn genuinely does NOT need Req/T access (pure In→Out logic — API-key
verification with no enrichment, a rate-limit check, a cookie policy) can be
ATTACHED, verbatim, to arbitrarily many routes/channels — mirroring how
middleware.SecurityScheme/Observer/PathPrefix already achieve this today, simply
because THEIR Fn types never mention Req/T either. Middleware[In,Out] therefore
gains an OPTIONAL way to carry such a Req/T-free fn DIRECTLY as a field — see
the receiveFn/sendFn fields and WithReceive/WithSend methods in the struct
below, after the following clarification of In's own scope.
In's scope, stated precisely (a critical review round clarified this): In is
NOT a general-purpose replacement for the route's own Req. It exists SPECIFICALLY
to give a middleware declarative, codec-validated access to header/cookie/query
values the route's OWN Req does not model at all — e.g. an API key the route never
declares as part of its own request shape. Whatever the route's Req ALREADY models
(including any header/cookie/query the ROUTE ITSELF declared as a merge field) is
read/enriched DIRECTLY via req *Req, passed alongside In to Transform's
fn (§4) — mirroring the ALREADY-ESTABLISHED, ALREADY-PROVEN security-shaped Fn
precedent (func(ctx, *http.Request, *Req) (...), confirmed via code to already
have POINTER/write access, and confirmed to already cover nethttp.Transform's
ENRICHMENT use case "for free" — see §9). A middleware can therefore both READ
whatever the route's Req already carries AND WRITE new, derived data into it (a
user ID resolved from an auth token, say) that the wire request never carried at
all — the actual handler then sees the enriched Req transparently. In/req *Req
are deliberately NOT alternatives to choose between — a middleware needing BOTH
kinds of data (something the route already models, AND something declaratively new)
uses both parameters together (§4).
No raw *http.Request anywhere in this design, by deliberate choice. An earlier
draft of this design considered exposing the raw *http.Request directly to a
middleware's fn, mirroring net/http's/chi's own middleware convention (func(http.Handler)
http.Handler, confirmed via adapters/nethttp/adapter.go's applyGeneralMiddleware
to expose ONLY the raw request/response, zero codec knowledge) — REJECTED for THIS
mechanism specifically: everything a Middleware[In,Out]-based fn touches must be
codec-defined (Req via the route's own codec, In/Out via the middleware's own
Declaration[In,Out] codecs), with zero untyped, unvalidated escape hatches — the
SAME "declare once, validated, schema'd" trade-off go-codex already makes everywhere
else (routes/channels never expose raw bytes to a handler either). This is a
STRICTER position than net/http's/chi's own convention, adopted deliberately — the
codebase's EXISTING wrapping-shaped Fn (func(http.Handler) http.Handler, §9) still
covers the "I genuinely need raw, untyped request/response access" case for callers
who want it; Middleware[In,Out] is a DIFFERENT, more declarative option for callers
who don't.
// package api/rest
type Middleware[In, Out any] struct {
middleware.Declaration[In, Out]
reqHeaderParams []MergedHeaderParam[In] // spec metadata + merge field, together
reqCookieParams []MergedCookieParam[In]
reqQueryParams []MergedQueryParam[In]
respHeaderParams []MergedResponseHeaderParam[Out]
respCookieParams []MergedResponseCookieParam[Out]
// receiveFn/sendFn, when set (via WithReceive/WithSend below), carry a
// Req/Resp-FREE runtime Fn directly on the value itself — enabling
// route/channel-AGNOSTIC attachment via plain .Use(mw) (see the new
// subsection in §4). Left nil for the route/channel-BOUND case, where
// Transform/ClientTransform supply an Req/T-accessing fn separately
// instead (never both — see §4's D7: combining both is rejected as
// ambiguous via rest.AmbiguousMiddlewareAttachmentError).
receiveFn func(ctx context.Context, in In) (Out, error)
sendFn func(ctx context.Context) (In, error)
}
func NewMiddleware[In, Out any](decl middleware.Declaration[In, Out]) Middleware[In, Out]
// WithRequestHeader/WithRequestCookie/WithRequestQuery populate the In side;
// WithResponseHeader/WithResponseCookie populate the Out side. Each takes
// the SAME MergedHeaderParam[In]/MergedCookieParam[In]/MergedQueryParam[In]/
// MergedResponseHeaderParam[Out]/MergedResponseCookieParam[Out] values
// already returned by rest.NewRequiredHeaderParam[In,V]/
// NewRequiredCookieParam[In,V]/NewRequiredQueryParam[In,V]/
// NewRequiredResponseHeaderParam[Out,V]/NewRequiredResponseCookieParam[Out,V]
// (and their Optional siblings) — confirmed these constructors are ALREADY
// GENERIC over any T, not hardcoded to a route's own Req/Resp. NO new
// REST-side param constructors are introduced by this design.
func (m Middleware[In, Out]) WithRequestHeader(p MergedHeaderParam[In]) Middleware[In, Out]
func (m Middleware[In, Out]) WithRequestCookie(p MergedCookieParam[In]) Middleware[In, Out]
func (m Middleware[In, Out]) WithRequestQuery(p MergedQueryParam[In]) Middleware[In, Out]
func (m Middleware[In, Out]) WithResponseHeader(p MergedResponseHeaderParam[Out]) Middleware[In, Out]
func (m Middleware[In, Out]) WithResponseCookie(p MergedResponseCookieParam[Out]) Middleware[In, Out]
// WithReceive/WithSend attach a route/channel-AGNOSTIC runtime Fn directly to
// mw — NEITHER signature mentions Req/Resp, so the returned Middleware[In,Out]
// value (fn included) can be passed to .Use(...) verbatim, on as many
// different routes as needed (see §4's new subsection). Use Transform/
// ClientTransform instead (§4) when fn genuinely needs req/msg access.
func (m Middleware[In, Out]) WithReceive(fn func(ctx context.Context, in In) (Out, error)) Middleware[In, Out]
func (m Middleware[In, Out]) WithSend(fn func(ctx context.Context) (In, error)) Middleware[In, Out]
// RouteMiddlewareMarker makes Middleware[In,Out] satisfy middleware.RouteMiddleware
// — NEW in this round (an earlier draft left this unimplemented, since
// Middleware[In,Out] was ONLY ever attached via Transform/ClientTransform
// directly, never via .Use()). Required now so .Use(mw) can recognize and
// dispatch a route/channel-agnostic mw carrying a WithReceive/WithSend fn.
// EXPORTED, not isRouteMiddleware — see §1's RouteMiddleware note on why an
// unexported marker method can never be satisfied from another package.
func (Middleware[In, Out]) RouteMiddlewareMarker() {}
How the spec half layers in — reusing the EXISTING conflict-detection pass, not a
parallel one. Whichever attachment point is used — plain .Use(mw) (route/channel-
agnostic, this section) or Transform/ClientTransform (route/channel-bound, §4) —
feeds mw's retained MergedHeaderParam[In].HeaderParam/etc (the embedded spec
structs) into the SAME per-name contribution map applyParamDeclarations/
checkParamConflicts already build from middleware.Middleware.RequestHeaderParams/
etc and the route's own manual params — collected from ALL sources, conflict-checked
ONCE, then layered into rb.headerParams/etc exactly as today. A manual route-level
declaration with the same param name still wins (unchanged precedence); the
middleware's OWN merge field keeps decoding the raw header/cookie/query value
regardless of which spec entry "won," since merge decode operates on raw vars,
independent of spec bookkeeping. middleware.Middleware (the legacy type) is
unaffected — it keeps flowing through its own existing, unchanged code path; this is
purely a SECOND (and now THIRD) source feeding the SAME resolution pass, mirroring
how the pass already tolerates an arbitrary number of contributing
middleware.Middleware values today. Exact plumbing (e.g. whether routeBuilder
gains one new accumulator field, or Middleware[In,Out]'s spec-contribution is
captured directly inside the RouteOpt each attachment point returns) is a small
implementation detail, not resolved further here — the CONTRACT (one declaration,
both effects, one shared conflict-check, REGARDLESS of attachment style) is what
this section commits to.
4. rest.Transform (route/channel-BOUND) + .Use(mw) (route/channel-AGNOSTIC) — two attachment points for one Middleware[In,Out] type¶
Both are free functions, not methods (Go disallows new type params on a method —
the SAME constraint nethttp.Transform[Req] itself is already subject to).
// package api/rest — SERVER attachment (RECEIVING role)
func Transform[Req, Resp, In, Out any](
r Route[Req, Resp],
mw Middleware[In, Out],
fn func(ctx context.Context, req *Req, in In) (Out, error),
) Route[Req, Resp]
// package api/rest — CLIENT attachment (SENDING role) — NEW
func ClientTransform[Req, Resp, In, Out any](
r Route[Req, Resp],
mw Middleware[In, Out],
fn func(ctx context.Context, req Req) (In, error),
) Route[Req, Resp]
Naming, adopted deliberately, not invented: adapters/nethttp.Transform[Req
any](fn func(ctx, r *http.Request, req *Req) error) any ALREADY EXISTS in this
codebase — confirmed via code — as exactly today's route-BOUND, Req-enriching
mechanism (generic over Req, instantiated per route, consumed via
route.HandleMW(nil, nethttp.Transform(...))). rest.Transform/rest.ClientTransform
adopt this ALREADY-ESTABLISHED name for the SAME concept — "a Req-bound enrichment
Fn, one instantiation per route" — rather than inventing new vocabulary, living in
api/rest (transport-agnostic) rather than the nethttp adapter package (which
targets *http.Request specifically) since this mechanism has no adapter dependency.
fn's req parameter (§3's clarification, made concrete here): Transform's
req *Req is the route's OWN already-decoded value — POINTER, so fn can both READ
whatever the route's Req already models AND WRITE derived/enriched data into it
(e.g. a user ID resolved from an auth token) before the actual handler runs, exactly
mirroring the security-shaped Fn's existing *Req access. ClientTransform's req
Req (value, not pointer — the caller already owns and can mutate its own Req
directly before calling Call at all, so write access through the middleware isn't
the only path to enrichment on this side) is available for fn to INSPECT when
deciding what In to produce (e.g. deriving an outgoing trace header FROM a field
already present in Req). Neither req parameter is a substitute for In — a
middleware needing data the route's Req does NOT model at all still declares it via
Middleware[In,Out]'s own merge fields (§3), decoded/encoded independently.
Transform registers (mw, fn) as a type-erased MiddlewareHandler entry on
routeBuilder (mirrors exactly how today's HandleMW/ServerImplementation already
erase Req — same erasure technique, one more entry). Unlike HandleMW (whose Fn
shape is checked via runtime reflection), both Transform's and ClientTransform's
Fn shapes are checked AT COMPILE TIME — In/Out are inferred from mw/fn
directly. Both are FREE FUNCTIONS, not methods — Go disallows type parameters on a
method beyond its receiver's own, the SAME constraint nethttp.Transform[Req] itself
is already subject to.
Why ClientTransform exists — confirmed via code, not designed from scratch:
CallWithHandle/CallOptions already auto-derive QueryParams/HeaderParams/
CookieParams from a route's OWN Req merge fields (ENCODE direction, via
codex.EncodeVars), and DecodeMergedResponse decodes a route's OWN Resp merge
fields back from the HTTP response (DECODE direction) — this is the EXISTING
client-side "one struct, one call" symmetry for a route's own Req/Resp. Security has
the SAME split: Route.HandleMW (server: verify) + Route.ClientMW (client: supply —
mergeCredentialHeaders runs a func(ctx, secReqs) (http.Header, error) Fn,
producing headers attached to the outgoing request). The original version of this doc
designed ONLY the server half — an asymmetry inconsistent with BOTH of these existing
precedents. ClientTransform closes it: fn PRODUCES an In value (mirrors the
credential Fn producing a value to send, rather than decoding one that arrived); the
middleware's OWN request header/cookie/query merge fields — the SAME fields
WithRequestHeader/etc. declare — are reused for EncodeIn, merged into
CallOptions alongside the route's own derived values. After the response,
DecodeOut runs mechanically (no Fn — mirrors DecodeMergedResponse) against the
middleware's OWN response header/cookie merge fields, returning Out to the caller
alongside Resp.
This closes the loop: the SAME rest.Middleware[In,Out] declaration now has BOTH a
server attachment (Transform) and a client attachment (ClientTransform), exactly
mirroring HandleMW/ClientMW's existing split for security. A middleware can be
attached via EITHER, BOTH, or neither, per route — e.g. an API-key middleware
declared once, ClientTransform-attached on the calling application's Route value
(to send the key) and Transform-attached on the serving application's OWN copy of
that same Route value (to verify it) — mirroring exactly how a shared
middleware.SecurityScheme value is attached via HandleMW server-side and ClientMW
client-side today.
Attaching BOTH on the SAME Route value, chained (a critical review round flagged
this scenario as unaddressed) — confirmed ALLOWED, not designed against. Since
Transform/ClientTransform both take a Route[Req,Resp] and return a
Route[Req,Resp], ClientTransform(Transform(r, mw, fnServer), mw, fnClient) is
ordinary, legal Go — e.g. a single process acting as BOTH server and proxying client
for the same route. This is not the doc's typical worked-example shape (server/
client attachment on two SEPARATE Route values, in different applications), but
nothing in this design prevents or specially handles it either. §3's spec-layering
conflict-detection (feeding mw's param structs into checkParamConflicts) already
tolerates this correctly, mirroring how it already tolerates an arbitrary number of
contributing middleware.Middleware values today — N4's already-added-names guard
applies uniformly regardless of WHICH free function (Transform or
ClientTransform) contributed a given name.
Route/channel-AGNOSTIC attachment — plain .Use(mw), for Fns that never need req/msg access¶
The problem this solves, stated precisely. Transform/ClientTransform's fn
signature FORCES the concrete Req type into fn's own Go type — the literal SAME
fn value therefore CANNOT be passed to two different routes with different Req
types (Go's type system won't allow it). Confirmed this is NOT a new limitation this
design introduces — the ALREADY-SHIPPED security-shaped Fn has the IDENTICAL
constraint (impl.Fn.(func(context.Context, *http.Request, *Req) (map[string][]string,
error)) is type-asserted PER ROUTE in runSecurityMiddlewareReflect). But NOT every
middleware needs Req access — confirmed every EXISTING route/channel-AGNOSTIC
middleware kind achieves reuse specifically by making sure its Fn's type NEVER
MENTIONS Req/Resp/T at all: the wrapping-shaped Fn (func(http.Handler)
http.Handler, no Req/Resp anywhere — the literal same nethttp.Observability(obs)
value is reusable across every route verbatim), the client-side credential Fn
(func(ctx, []route.SecurityRequirement) (http.Header, error), no Req either —
confirmed adapters/nethttp.NewCachingCredentialFunc builds exactly ONE such value,
reused across every route needing that credential), and PathPrefix (no Fn at all,
just a value).
The fix: Middleware[In,Out]'s WithReceive/WithSend (§3) bundle a Req/T-free
Fn DIRECTLY onto the value itself, attached via the EXISTING, UNWIDENED .Use(mw)
call — no new attachment method to learn beyond .Use(), which callers already
know from SecurityScheme/PathPrefix. Since Middleware[In,Out] now implements
middleware.RouteMiddleware (§3), route.Use(mw) recognizes it directly: it layers
mw's declared params into the spec (§3, IDENTICAL mechanism regardless of
attachment style) AND, when mw carries a WithReceive/WithSend Fn, registers the
runtime dispatch automatically — in the SAME call, for as many DIFFERENT routes as
mw is .Use()'d on, with the literal SAME mw value (Fn included) reused
verbatim:
rateLimitPolicy := rest.NewMiddleware(
middleware.NewDeclaration[struct{}, RateLimitOut]("rate-limit", codex.Struct[struct{}](), rateLimitOutCodec),
).WithReceive(func(ctx context.Context, _ struct{}) (RateLimitOut, error) {
return checkRateLimit(ctx)
})
// Attached verbatim to N different routes — the SAME rateLimitPolicy value,
// NO per-route Req-typed closure needed, unlike Transform:
routeA = routeA.Use(rateLimitPolicy)
routeB = routeB.Use(rateLimitPolicy)
routeC = routeC.Use(rateLimitPolicy)
This is a MATERIAL WIDENING of .Use()'s existing contract, stated explicitly
rather than silently changed. Today, .Use() is declare-time-ONLY — confirmed via
WithMiddleware's own doc comment: "the actual Security/RequestParams/ResponseParams
application... happens ONCE, at Register/ValidateRoute time" — it never runs
anything at request-dispatch time. Bundling a runtime Fn onto a .Use()-attached
Middleware[In,Out] value means .Use() can now ALSO cause runtime dispatch, for
this ONE new attaching type — middleware.Middleware (legacy) is UNAFFECTED, since
it never carries a Fn at all (Security's runtime Fn is ALWAYS supplied separately,
via HandleMW/ClientMW, matched by scheme name). Worth flagging as a genuine
precedent change to .Use()'s documented behavior, not merely an additive feature.
D7 — combining both attachment styles on the SAME Middleware[In,Out] value:
RESOLVED, REJECTED as ambiguous. A mw carrying a bundled WithReceive/
WithSend Fn, if ALSO passed to Transform/ClientTransform (which supplies its
OWN, separate fn), leaves it genuinely unclear WHICH Fn should run — unlike
header/cookie merge fields (where multiple contributing sources simply compose),
there is only ONE dispatch point per role, so two candidate Fns for the same role
cannot both apply. Decision: reject this combination at Register/Handle time
with a new, typed error — rest.AmbiguousMiddlewareAttachmentError{Name string}
(events mirror: events.AmbiguousMiddlewareAttachmentError{Name string}) — checked
in the SAME resolution pass as D6(b)'s uniqueness check. A Middleware[In,Out]
value is therefore either BUNDLED-Fn (agnostic, .Use()-only) or BOUND-Fn (supplied
separately via Transform/ClientTransform) for a given role — never both on the
SAME value. Nothing prevents declaring TWO SEPARATE Middleware[In,Out] values
(even representing "the same logical concern") where one is bundled and the other
is bound — this restriction applies only to combining both styles on ONE value.
5. Adapter dispatch wiring — server (adapters/nethttp/serve.go, adapters/chi/serve.go) and client (adapters/nethttp/client.go)¶
Server side — confirmed dispatch-ordering finding that makes this safe: the
existing security-middleware dispatch (runSecurityMiddlewareReflect) already runs
AFTER Req is fully decoded+merged, receiving BOTH the raw *http.Request and the
merged *Req — at that exact point, headerVars/cookieVars/queryVars (the raw
string maps used to build Req) are ALREADY in scope. A Transform-registered
entry:
- Derives
Infrom those SAME raw var maps via its OWN independentcodex.DecodeVarscall (usingmw's ownreqHeaderMergeFields/etc.) plus a fullInCodec.Validatepass — completely independent of Req's own decode, no ordering conflict, no shared-state hazard. - Calls
fn(ctx, reqPtr, in)→(out, err), passing the SAME already-decoded*Reqpointer the handler will ALSO receive (confirmed via code:serve.goalready holds this exactreqPtrin scope at this dispatch point, used identically byrunSecurityMiddlewareReflecttoday) — any enrichmentfnwrites into*reqis visible to the actual handler afterward, transparently. An error short-circuits the request (same error-response handlingrunSecurityMiddlewareReflectalready does). - On success, derives response header/cookie values from
outviacodex.EncodeVars(usingmw's ownrespHeaderMergeFields/respCookieMergeFields), merging them into the SAME pending-header/cookie collection the route's OWNEncodeMerged/EncodeResponseMergeFieldsalready populates — middleware-produced and route-produced response header/cookie values compose into ONE final HTTP response. Conflict precedence (same header/cookie name from both sources): registration-order, last-applied-wins — consistent with how chainedOpts already behave elsewhere in this codebase.
Client side — confirmed via adapters/nethttp/client.go's CallWithHandle/call:
a ClientTransform-registered entry runs at the SAME point mergeCredentialHeaders
already does (before the HTTP request is built):
- Calls
fn(ctx, req)→(in, err), passing the caller's OWNReqvalue (already fully built at this point, beforeCallWithHandle's own derivation runs) forfnto inspect if useful. An error aborts the call before any network activity (status 0 per the existing client observer convention — seestats.Observer's "status 0 is the sentinel for 'call attempted but no HTTP request reached the network'" rule). - On success, derives header/cookie/query values from
inviacodex.EncodeVars(usingmw's ownreqHeaderMergeFields/etc.), merged intoCallOptionsalongside the route's own req-derived values (sameoverrideDerived-style precedenceCallWithHandlealready uses for the route's own merge fields — caller-suppliedCallOptionsentries still win over BOTH the route's own derived values and a middleware's). - After the response is received, derives
Outviacodex.DecodeVars(usingmw's ownrespHeaderMergeFields/respCookieMergeFields) against the response's actual headers/cookies — mechanical, no Fn — returned to the caller ALONGSIDE the decodedResp(exact return-shape mechanism — e.g. an additional out-param onCall, vs. a separateCallTransform-scoped accessor — is a small signature detail to finalize during implementation, not a design blocker).
Route/channel-AGNOSTIC dispatch (.Use(mw), §4's new subsection) reuses the
IDENTICAL dispatch steps above — just without a req/msg parameter to pass. A
.Use()-attached Middleware[In,Out] carrying a WithReceive Fn dispatches at the
SAME server-side point as Transform (step 1-3 above, with fn(ctx, in) instead of
fn(ctx, reqPtr, in)); a WithSend Fn dispatches at the SAME client-side point as
ClientTransform (fn(ctx) instead of fn(ctx, req)). No new dispatch MECHANISM is
introduced — only a narrower fn signature at the SAME two call sites.
SSE — migrated onto the SAME model, via TransformSSE/ClientTransformSSE (UN-DEFERRED — a critical review round confirmed neither blocker is fundamental)¶
An earlier draft of this doc scoped SSE OUT entirely ("plain Route/Client.Call
only"), reasoning that SSERouteHandle lacks response merge fields and
consumeSSE's client implementations recognize only the credential shape. A
further review confirmed BOTH gaps are small, concrete, sketchable additions — not
fundamental blockers — and un-deferring them is REQUIRED, not optional, for this
design to deliver a genuinely "simple, declarative, consistent workflow": a caller
attaching the SAME cookie-policy/rate-limit middleware to a plain Route and an
SSERoute should use the SAME two verbs, not learn a second vocabulary for SSE.
In-side — CONFIRMED wireable with ZERO new prerequisite (upgraded from the
earlier draft's "likely"): adapters/nethttp/serve_sse.go ALREADY builds
headerVars/cookieVars/queryVars and calls runSecurityMiddlewareReflect(ctx, r,
reqPtr, impls, secReqs) at the EXACT SAME dispatch timing plain Route already
uses (confirmed via code, lines ~143-191) — TransformSSE's DecodeIn→fn step
slots in at this SAME point, unchanged from Transform's own mechanism.
Out-side — CONFIRMED solvable via a small, concrete addition, not a redesign:
confirmed via code an EXACT gap exists in serve_sse.go between
runSecurityMiddlewareReflect returning and SSE's own headers being committed
(w.Header().Set("Content-Type", "text/event-stream"), marked "must be set before
WriteHeader" in the code's own comment) — this is PRECISELY where Out's
response-header/cookie dispatch slots in: derive header/cookie values from Out via
codex.EncodeVars, call w.Header().Set(...)/SetCookie(...), THEN commit SSE's
own headers. The only missing piece: SSERouteHandle[Req,Event] needs its own
responseHeaderMergeFields/responseCookieMergeFields fields + an
EncodeResponseMergeFields-equivalent method, mirroring RouteHandle's existing
ones — a small, well-understood addition.
Client-consume — CONFIRMED solvable via one new recognized Fn shape, same
category of change Call already has TWICE: consumeSSE's
validateClientImplementationShapes recognizes ONLY the credential shape today, but
Call's OWN validateCallImplementationShapes ALREADY recognizes TWO shapes
(credential + general-purpose wrapping) — proving "recognize more than one Fn shape"
is an established mechanism, just not yet extended to SSE consume. Adding a THIRD
recognized shape (func(ctx, req Req) (In, error), ClientTransformSSE's shape) is
the SAME category of change.
One genuinely NEW decision this surfaces (no existing precedent to mirror):
unlike Call (one request → one response, Out decodes once from THAT response),
SSE's consume delivers MANY events over ONE long-lived connection — Out should
decode ONCE, from the INITIAL SSE connection's response headers/cookies at
connection-open time, BEFORE the first event arrives, analogous to a single
Response's headers being set once per connection rather than per event. Sketched
in principle here; the exact mechanical hook (e.g. whether consumeSSE needs a
new "connection opened" callback distinct from its existing per-event delivery) is
not designed in full mechanical detail in this round.
New free functions, mirroring Transform/ClientTransform but for SSERoute:
// api/rest — SSE server attachment (RECEIVING role)
func TransformSSE[Req, Event, In, Out any](
s SSERoute[Req, Event],
mw Middleware[In, Out],
fn func(ctx context.Context, req *Req, in In) (Out, error),
) SSERoute[Req, Event]
// api/rest — SSE client attachment (SENDING role, consume-side)
func ClientTransformSSE[Req, Event, In, Out any](
s SSERoute[Req, Event],
mw Middleware[In, Out],
fn func(ctx context.Context, req Req) (In, error),
) SSERoute[Req, Event]
Why TWO SEPARATE pairs of names, not one shared pair — confirmed via code:
HandleMW/ClientMW are METHODS today, defined separately on BOTH Route and
SSERoute (identical logic, different receiver — no naming collision, since Go
allows the same method name on different types). Transform/ClientTransform
(and their SSE counterparts) MUST be FREE FUNCTIONS (Go disallows type parameters
on a method beyond its receiver's own) — free functions CANNOT be "overloaded" by
receiver type the way methods can, so Route and SSERoute genuinely need
DISTINCT function names. Out on SSERoute targets response headers/cookies only
(no body) — the SAME scope Out already has on plain Route (it never touches
Resp's body codec there either), so this is not a new asymmetry, just consistently
applied to a second route kind.
Route/channel-AGNOSTIC attachment needs NO new naming for SSE at all —
SSERoute.Use ALREADY EXISTS (confirmed via code) and needs only the SAME
middleware.RouteMiddleware-widening Route.Use already gets (§1) to recognize a
Middleware[In,Out] carrying a bundled WithReceive/WithSend Fn — the identical
mechanism, the identical call (sseRoute.Use(mw)), no SSE-specific verb needed.
6. Structured errors¶
rest.MiddlewareInputError{Name string, Err error}(mirrorsPathParamError- style wrapping, implementsslog.LogValuer) — returned when a middleware'sInfails to decode/validate. Events mirror:events.MiddlewareInputError{Name string, Err error}, returned fromevents.Transform's topic-varIndecode failure.rest.MiddlewareError{Name string, Err error}(NEW, per D2) — the fallback when aTransform/ClientTransformfn's own returned business error does NOT match any declaredErrorPattern(checked FIRST, viahandle.ErrorResponseFor, mirroring how a handler error is already treated). Default status 400. Implementsslog.LogValuer. Events mirror:events.MiddlewareError{Name string, Err error}(D2's events extension), the fallback whenevents.Transform'sfnerror doesn't match a declaredErrorChannelpattern.rest.DuplicateMiddlewareNameError{Route string, Name string}(NEW, per D6(b)) — returned at Register/Handle time when twoMiddleware[In,Out]values attached to the SAME route (via ANY combination of.Use()/Transform/ClientTransform) share the sameDeclaration.Name. Implementsslog.LogValuer. Events mirror:events.DuplicateMiddlewareNameError{Topic string, Name string}.rest.AmbiguousMiddlewareAttachmentError{Name string}(NEW, per D7) — returned at Register/Handle time when a SINGLEMiddleware[In,Out]value carries a bundledWithReceive/WithSendFn AND is ALSO passed toTransform/ClientTransform(which supplies its own, separatefn) — ambiguous, since only one Fn can run per role. Implementsslog.LogValuer. Events mirror:events.AmbiguousMiddlewareAttachmentError{Name string}.
All four are distinct from rest.SecurityError (reserved for the EXISTING
middleware.SecurityScheme mechanism, unchanged).
7. Worked example — cookie attributes, the original motivating gap¶
type CookieAttrs struct {
Value string
MaxAge int
SameSite string // "default"|"lax"|"strict"|"none" — validate.OneOf-constrained
Insecure bool
}
var cookieAttrsCodec = codex.Struct[CookieAttrs](
codex.RequiredField("value", codex.String(), ...),
codex.OptionalField("maxAge", codex.Int(), ...),
codex.OptionalField("sameSite", codex.String().Refine(validate.OneOf("default", "lax", "strict", "none")), ...),
codex.OptionalField("insecure", codex.Bool(), ...),
)
sessionCookiePolicy := rest.NewMiddleware(
middleware.NewDeclaration[struct{}, CookieAttrs]("session-cookie-policy", codex.Struct[struct{}](), cookieAttrsCodec),
).WithResponseCookie(rest.NewRequiredResponseCookieParam("session", codex.String(),
func(a CookieAttrs) string { return a.Value },
func(a *CookieAttrs, v string) { a.Value = v },
))
// SERVER side — sets the cookie. This example's In is struct{} (no header/cookie/
// query the route doesn't already model is needed here), but req *LoginReq is
// still passed — unused in this particular example, present because Transform's
// signature always offers it (see §4). Since this fn never reads req, it could
// ALSO have been declared via .Use(sessionCookiePolicy.WithReceive(...)) instead
// (§4's route/channel-agnostic path) — Transform is used here only to show the
// full signature; either mechanism works for THIS particular example.
route = rest.Transform(route, sessionCookiePolicy,
func(ctx context.Context, req *LoginReq, _ struct{}) (CookieAttrs, error) {
return CookieAttrs{Value: newSessionToken(), MaxAge: 3600, Insecure: true}, nil
})
This closes the ORIGINAL gap (declarative MaxAge/Insecure for a response cookie)
with ZERO new REST-side param constructors — only rest.NewRequiredResponseCookieParam,
already existing, reused against Out = CookieAttrs instead of a route's own Resp.
Note only the cookie's VALUE is merge-derived here (a Path/Domain/SameSite
extension follows the identical pattern via additional WithResponseCookie-style
merge fields once a concrete need for those specific attributes as SEPARATE Set-Cookie
attributes — vs. bundled inside Out, as sketched above — is confirmed; not designed
further here since it is a mechanical extension, not a new mechanism).
Client side — a DIFFERENT middleware attaches the SENDING role (an API-key
example is clearer than resending a session cookie, which is more naturally an
application-level cookie-jar concern outside a single stateless Call):
type APIKeyIn struct{ Key string }
var apiKeyInCodec = codex.Struct[APIKeyIn](
codex.RequiredField("key", codex.String().Refine(validate.NonEmptyString), ...),
)
apiKeyPolicy := rest.NewMiddleware(
middleware.NewDeclaration[APIKeyIn, struct{}]("api-key-policy", apiKeyInCodec, codex.Struct[struct{}]()),
).WithRequestHeader(rest.NewRequiredHeaderParam("X-API-Key", codex.String(),
func(in APIKeyIn) string { return in.Key },
func(in *APIKeyIn, v string) { in.Key = v },
))
// SERVER side — verifies the key AND enriches the route's own Req with a
// resolved user ID the wire request never carried, demonstrating req *Req
// (enrichment) and in In (a header the route itself doesn't model) used
// TOGETHER — the actual handler afterward sees req.UserID already populated,
// with zero extra plumbing:
route = rest.Transform(route, apiKeyPolicy,
func(ctx context.Context, req *GetProfileReq, in APIKeyIn) (struct{}, error) {
userID, err := lookupUserIDForKey(in.Key)
if err != nil {
return struct{}{}, ErrInvalidAPIKey
}
req.UserID = userID // enrichment — GetProfileReq's own codec never decodes this
return struct{}{}, nil
})
// CLIENT side — supplies the key on every call, no manual CallOptions.HeaderParams.
// req is available to inspect (e.g. deriving the key from a field already on
// Req) but unused in this example — the key comes from the environment instead:
route = rest.ClientTransform(route, apiKeyPolicy,
func(ctx context.Context, req GetProfileReq) (APIKeyIn, error) {
return APIKeyIn{Key: os.Getenv("MY_API_KEY")}, nil
})
The SAME apiKeyPolicy declaration (built once, e.g. as a shared package-level var)
is attached via Transform on the SERVING application's Route value and via
ClientTransform on the CALLING application's Route value — exactly mirroring how
a shared middleware.SecurityScheme value is used today. GetProfileReq.UserID
would be an ordinary field the route's OWN codec never decodes from the wire (no
RequiredField/OptionalField for it) — populated ONLY via this middleware's
enrichment, exactly like nethttp.Transform's existing enrichment pattern (§9)
already proves works for the security-shaped Fn precedent this design's req *Req
parameter mirrors.
8. events.Middleware[In, Out] — the events-side derived type, adopting the SAME naming and BOTH attachment styles as REST¶
Naming, RESOLVED this round (previously left open): an earlier draft left
events using the PRE-rename names (HandleMiddleware/ClientMiddleware), flagging
whether it should follow REST's Transform/ClientTransform rename as an open
question. Resolved in favor of ONE SHARED vocabulary across both patterns — the
explicit goal is a "simple, declarative, consistent workflow" for the user across
REST and events, which a per-pattern-different verb would directly undermine.
Events therefore renames HandleMiddleware/ClientMiddleware to events.Transform/
events.ClientTransform, mirroring REST's naming exactly (this is a rename WITHIN
an as-yet-unimplemented design, not a breaking change to shipped code — neither name
was ever implemented). The route/channel-agnostic vs. bound tension applies
identically to events: a subscribe/publish-side concern whose fn doesn't need
msg *T access is just as reusable across multiple channels as REST's .Use(mw)
case. Subscriber[T].Use/Publisher[T].Use ALREADY EXIST (confirmed via code) and
need only the SAME middleware.RouteMiddleware-widening + bundled-Fn dispatch
REST's .Use() gets — no new events-specific verb needed for the agnostic case,
exactly mirroring SSERoute.Use's situation (see "SSE" above).
Confirmed via code: api/events' ONLY merge vocabulary today is topic-var merge
(NewTopicParam/MergedTopicParam[T], ChannelHandle.EncodeVars) — there is NO
header/cookie concept on pub/sub's wire at all. Events' security uses a completely
different mechanism (Decision 3's in-payload *T-write access), not merge fields.
Corrected role mapping (an earlier version of this doc got this backwards) —
confirmed via adapters/mqtt5/adapter.go:
- Subscribe (runSubscribeSecurityImpls) runs AFTER topic vars are merged into
the decoded value, receiving both the raw message and the merged value — exactly
mirrors REST's SERVER dispatch. This is the RECEIVING role: derive In from
incoming topic vars → call fn(ctx, msg, in) → error only. Pub/sub has NO
response/reply channel, so there is nothing to encode an Out INTO on subscribe —
unlike REST's server side, subscribe's Out is unused. Mirrors REST's req *Req
addition exactly: msg *T is the channel's OWN already-decoded value (confirmed
via runSubscribeSecurityImpls's existing *T-write-access precedent — already
proven for enrichment), letting fn read/enrich it, alongside in for anything
the channel's topic template doesn't model.
- Publish (runPublishSecurityImpls, a ClientImplementation) produces a value
([]UserProperty in mqtt5's case) attached to the OUTGOING message — exactly
mirrors REST's CLIENT dispatch (mergeCredentialHeaders producing headers to
attach). This is the SENDING role: fn(ctx, msg) → (out, error) → encode Out
into ADDITIONAL topic vars merged into the outgoing publish, with msg T (the
caller's own already-built value, mirrors REST's client-side req Req) available
for fn to inspect. In is unused on publish (mirrors REST client:
events.ClientTransform doesn't decode anything either, it only produces/encodes).
// package api/events
type Middleware[In, Out any] struct {
middleware.Declaration[In, Out]
topicMergeFieldsIn []codex.FieldCodec[In] // used by Transform (subscribe)
topicMergeFieldsOut []codex.FieldCodec[Out] // used by ClientTransform (publish)
// receiveFn/sendFn — the SAME route/channel-AGNOSTIC bundling mechanism
// as rest.Middleware[In,Out] (§3): a Req/T-free Fn attached directly to
// the value, dispatched automatically via plain .Use(mw) on a
// Subscriber[T]/Publisher[T], reusable verbatim across many channels.
receiveFn func(ctx context.Context, in In) error
sendFn func(ctx context.Context) (In, error)
}
func NewMiddleware[In, Out any](decl middleware.Declaration[In, Out]) Middleware[In, Out]
// WithSubscribeTopic/WithPublishTopic populate the In/Out topic-var merge
// vocabulary respectively, reusing the SAME events.NewTopicParam[T,V]
// constructor already used for a channel's own Item — no new events-side
// param constructor either.
func (m Middleware[In, Out]) WithSubscribeTopic(p MergedTopicParam[In]) Middleware[In, Out]
func (m Middleware[In, Out]) WithPublishTopic(p MergedTopicParam[Out]) Middleware[In, Out]
// WithReceive/WithSend mirror rest.Middleware[In,Out]'s own methods (§3) —
// bundling a route/channel-agnostic Fn for attachment via plain .Use(mw).
func (m Middleware[In, Out]) WithReceive(fn func(ctx context.Context, in In) error) Middleware[In, Out]
func (m Middleware[In, Out]) WithSend(fn func(ctx context.Context) (In, error)) Middleware[In, Out]
// RouteMiddlewareMarker makes events.Middleware[In,Out] satisfy
// middleware.RouteMiddleware — mirrors rest.Middleware[In,Out]'s own (§3).
// EXPORTED, not isRouteMiddleware — see §1.
func (Middleware[In, Out]) RouteMiddlewareMarker() {}
// Transform (subscribe, RECEIVING role, route/channel-BOUND) — mirrors
// rest.Transform, minus the Out/response half (no reply channel to encode
// into). msg *T is the channel's OWN already-decoded value (enrichable,
// mirrors req *Req exactly). Renamed from HandleMiddleware (RESOLVED naming
// question — see this section's header note).
func Transform[T, In, Out any](
s Subscriber[T],
mw Middleware[In, Out],
fn func(ctx context.Context, msg *T, in In) error,
) Subscriber[T]
// ClientTransform (publish, SENDING role, route/channel-BOUND) — mirrors
// rest.ClientTransform exactly. msg T is the caller's OWN already-built
// value, available to inspect. Renamed from ClientMiddleware.
func ClientTransform[T, In, Out any](
p Publisher[T],
mw Middleware[In, Out],
fn func(ctx context.Context, msg T) (Out, error),
) Publisher[T]
In/Out's scope here, stated as precisely as REST's §3 clarification (a critical
review round found this was missing). Unlike a REST header/cookie/query — a
free-standing name a middleware can introduce independently of the route — a pub/sub
topic var is constrained by the CHANNEL's OWN topic TEMPLATE: a middleware cannot
invent a NEW topic var out of thin air, since a var only exists if the topic string
declares a matching {varName} placeholder in the first place. WithSubscribeTopic/
WithPublishTopic therefore only make sense for a topic var the channel's template
DOES declare, but that the channel's own Item (T) does NOT already merge via its
OWN NewTopicParam — e.g. a topic "sensors/{sensorID}/{region}/data" where the
channel's own T merges only sensorID, leaving region unclaimed for a
region-based authorization middleware to merge into its OWN In instead. This is the
DIRECT events-side analogue of REST's "In is for vars the route's Req doesn't
model" scope (§3) — restated here explicitly since events' constraint is sourced
differently (a shared topic template vs. REST's free-standing header namespace).
Both dispatch at whatever point api/events' own subscribe/publish handling already
resolves topic vars today (same "derive/encode from vars already in scope" principle
as REST's dispatch wiring) — Transform's DecodeIn runs at the SAME point
runSubscribeSecurityImpls already runs (after topic vars are merged into the
decoded value); ClientTransform's EncodeOut runs at the SAME point
runPublishSecurityImpls already runs (before the message is sent, merged into the
SAME additional-vars mechanism that mqtt5's UserProperty attachment already uses as
precedent — exact per-adapter wiring, e.g. whether zeromq/mqtt(v3) have an
equivalent attachment point, TBD during implementation).
This directly resolves common-middleware-architecture.md's original finding for
events specifically: events.Middleware[In,Out] carries ONLY topic-var merge
vocabulary — no unused header/cookie/query fields — making
checkUnsupportedMiddlewareParams's INTERIM eager-rejection check obsolete once this
ships (a legacy middleware.Middleware with REST-only fields attached to an events
.Use() call is STILL correctly rejected the same way; the NEW events.Middleware[In,Out]
path simply has no such fields to reject in the first place).
Extending the model — rest.PathPrefix/events.TopicPathPrefix/ports.FilePathPrefix/DirPathPrefix (sketch, illustrates the model generalizes beyond REST/events)¶
The guiding-principle table's PathPrefix row is a SKETCH, included to demonstrate
that the model generalizes beyond params — not a fully-resolved design this round.
The idea: a declare-time-only value, attached via .Use(...) exactly like
SecurityScheme/Middleware[In,Out], whose effect is a TEMPLATE/CONFIG TRANSFORM
resolved exactly ONCE — prefixing a route's/channel's final path or topic string
before the descriptor is built (e.g. mounting the SAME declared route/channel set
under /api/v1 vs /api/v2, or a different topic namespace per deployment, without
redeclaring every route/channel):
// api/events — sketch, not finalized
func TopicPathPrefix(prefix string) middleware.RouteMiddleware
A concrete third driver confirms this generalizes past REST/events, to Pattern-less
ports too: mounting a workspace-folder ROOT prefix for ports.File/Dir (e.g.
per-tenant or per-environment base directories) is the SAME shape, reusing the
"Pattern-less ports get .Use() at declare time" mechanism from the ports
feasibility analysis above:
// ports — sketch, not finalized
func FilePathPrefix(prefix string) middleware.RouteMiddleware
func DirPathPrefix(prefix string) middleware.RouteMiddleware
Confirmed via code this closes a REAL, currently-unfilled gap: ports.Dir's existing
WithBaseDir(path string) DirOpt is narrower than it sounds — its own doc comment
states it affects ONLY Dir.List's GLOB-DISCOVERY scan anchor, explicitly having "no
effect on Dir.BuildPath/MatchPath, which already resolve relative to cwd exactly
as today." ports.File has no base-path concept at all. Neither port type has a
GENERAL root-prefix mechanism today, independent of this whole middleware design.
A refinement the guiding-principle table doesn't fully capture, surfaced by this
third driver. The table's two columns (spec effect / runtime-dispatch effect)
happen to coincide for REST/events specifically because a route's/channel's
path/topic string IS both the spec's displayed value AND the literal runtime routing
key, computed once at Register/descriptor-build time — mutating it pre-freeze
determines runtime routing with no separate dispatch step, which is why "spec-only,
no runtime Fn" accurately described PathPrefix/TopicPathPrefix there.
ports.File/Dir have no spec at all (confirmed, intentional) — so
FilePathPrefix/DirPathPrefix cannot be "spec-only" in that same sense; there is no
spec column to populate. But the SAME mechanical shape still applies: attached via
.Use() at NewFile(...)/NewDir(...)-construction time, the prefix is baked into
the File[T]/Dir value's own internal template/baseDir field EXACTLY ONCE —
BuildPath/MatchPath/List/Delete never need a separate per-call dispatch step,
mirroring REST's PathPrefix needing none. The TRUE common shape across all four
sketches is "declare-time-only, resolved-once template/config transform" — REST/
events render this as a spec effect because path/topic IS the spec; ports render the
IDENTICAL mechanical shape as a resolved-once CONFIG transform with NO spec effect at
all (both landing in the guiding-principle table's "spec: None" cell — but for a
different reason than Observer/wrapping-shaped Fns land there: not because it's
per-call runtime-dispatched, but because ports has no spec to populate, full stop).
Worth stating this distinction explicitly rather than letting "mirror image of
Observer" imply a resemblance that isn't really there.
Why a .Use()-attached, reusable prefix value earns its keep over a plain
constructor option (not previously stated): the value is REUSE ACROSS MULTIPLE,
ALREADY-DECLARED values, toggled together as a unit — e.g. a SET of File[T]/Dir
declarations (a config file, a log dir, a data dir) built ONCE with relative
templates, then mounted under DIFFERENT tenant/environment workspace roots by
attaching a DIFFERENT FilePathPrefix value per deployment, without redeclaring each
file/dir. This is the same justification REST's /api/v1-vs-/api/v2 example
gestures at but never states plainly: a reusable prefix middleware is worth a
dedicated .Use()-attached type specifically when SEVERAL declarations need to move
together, not for a single one-off path (which a plain constructor argument already
handles more simply).
Left open, not resolved this round: composition semantics for MULTIPLE attached
PathPrefix/TopicPathPrefix/FilePathPrefix/DirPathPrefix values on the same
route/channel/file/dir (error on conflict, concatenate in attachment order, or
last-attached-wins) — a separate, smaller question from the guiding principle itself,
deferred to a future Refine pass once a concrete need for this specific middleware
surfaces (following this repo's own "don't design past a second concrete driver"
precedent, same as elsewhere in this doc). The ports-side sketch adds a CONCRETE new
wrinkle to this same open question: if a .Use()-attached FilePathPrefix AND an
inline WithBaseDir-equivalent constructor option are BOTH present on the same
File[T]/Dir value, which wins, or do they compose (e.g.
filepath.Join(prefix, baseDir, template))? Not resolved — named explicitly here so
it isn't rediscovered as a surprise later. Whether this generalized attachment
mechanism should ever be EXPORTED for third-party custom spec-layering types, or stay
closed to api/rest's/api/events'/ports' own shipped types, is likewise left open —
leaning closed/sealed for now, consistent with RouteMiddleware's/ports.Pattern's
existing sealed-marker-interface precedent elsewhere in this codebase.
9. How the existing non-spec-adding Fn shapes fit the guiding principle's runtime column¶
The guiding principle's table already places the two EXISTING non-spec-adding Fn
shapes in the runtime-only column. This section grounds that placement against real
code — docs/design/d-0002-pubsub-workflow-simplification.md already asked and
answered this exact question for pub/sub's OWN middleware concept — a dedicated
review confirmed BOTH REST and events, on BOTH roles (server/subscribe,
client/publish), recognize TWO such shapes each, validated eagerly via reflection:
- Security-shaped, attached UNPAIRED (
mwnil ormw.Security == nil) — direct WRITE ACCESS to the route's/channel's own shared, already-merged*Req/*T(func(ctx, *http.Request, *Req) (map[string][]string, error)for REST server; the credential shape for REST client; the mirrored security shape for events). Confirmed (d-0002): this shape already coversnethttp.Transform's enrichment use case "for free" — an unpaired security-shaped Fn can decode/derive a value and write it DIRECTLY into*Req/*Tbefore the handler runs, no new mechanism needed. - Wrapping-shaped (most general — wraps the ENTIRE dispatch/round-trip):
func(http.Handler) http.Handler(REST server),func(next func(ctx,Req)(Resp,error)) func(ctx,Req)(Resp,error)(REST client, shipped per d-0001 Addendum 3),func(next func(ctx,T) error) func(ctx,T) error(events, both roles). Used for observability, retries, tracing, body manipulation.
Middleware[In,Out] is a NEW way to populate BOTH columns together (per §3), not a
fourth, unrelated Fn-shape category competing with these two. Both existing shapes
were ALWAYS spec-layering middleware with an empty spec column (per the guiding
principle above) — this design doesn't replace them, it adds a mechanism suited to a
different sub-case: typed extraction from vars, optionally typed production of
outgoing vars, WITH an optional spec contribution, rather than arbitrary Req-mutation
or full-dispatch wrapping with none.
Comparison against the two existing shapes (updated: Middleware[In,Out]'s fn ALSO
gets req *Req/msg *T access per §3/§4 — the SAME enrichment capability
security-shaped-unpaired already has; that row is no longer a differentiator between
the two shapes, so the table below reflects this honestly rather than the earlier,
now-corrected claim that Middleware[In,Out] never touches Req/T):
| Security-shaped-unpaired | Wrapping-shaped | Middleware[In,Out] (this design) |
|
|---|---|---|---|
| Touches route's own Req/Resp (or T) | Directly, by mutation | Indirectly (wraps the whole handler) | Yes — read AND enrich, via req/msg (§3/§4), SAME access as security-shaped-unpaired |
ALSO gets typed access to vars Req/T doesn't model |
No (raw string extraction only, into Req) | No | Yes — In, codec-validated (§3) |
| Spec contribution | None | None | Optional — layers into the route's spec (§3) |
| Response-side declarative output | No (write-only into Req) | No (must hand-roll) | Yes, for REST only (Out → response merge fields) |
| Shape safety | Runtime reflection check | Runtime reflection check | Compile-time (generic free functions) |
The REAL remaining differentiators, now that Req/T access is shared, are: (1) In's
codec-validated access to vars the route doesn't model at all (security-shaped-
unpaired only ever extracts raw strings, with no schema/validation); (2) the optional
spec contribution; (3) compile-time Fn-shape safety instead of runtime reflection.
Best suited when the concern's natural shape is "typed extraction from vars,
optionally typed production of outgoing vars, ALONGSIDE enrichment of the route's own
Req/T" rather than arbitrary Req-mutation with no schema, or full-dispatch wrapping —
an ADDITIONAL option, not a consolidation of the other two.
ContextField composes with this design FOR FREE — confirmed via dispatch-order
code, mirroring d-0002's own nethttp.Transform finding exactly (a naming
collision worth flagging explicitly now that this design's OWN mechanism is ALSO
named Transform, per §4 — the TWO are related in spirit but distinct symbols in
DIFFERENT packages: nethttp.Transform[Req] is the pre-existing adapter-level
helper wrapping the security-shaped Fn; rest.Transform/rest.ClientTransform are
this doc's NEW, Middleware[In,Out]-based mechanism — a future doc reader must not
conflate the two despite the shared name). middleware.EnsureContextFields(ctx) is
called at the very TOP of adapters/nethttp/serve.go's (and chi's) dispatch —
BEFORE Req decode, BEFORE runSecurityMiddlewareReflect, and therefore BEFORE the
point this design places Transform's dispatch. An implementer's fn passed to
rest.Transform can ALREADY call someContextField.Set(ctx, value) today with ZERO
new wiring — the actual business handler retrieves it via someContextField.Get(ctx),
exactly like any OTHER Fn shape already can.
Two caveats, not design blockers:
- ContextField.Set(ctx, raw any) decodes raw via its OWN codec — whether an
ALREADY-DECODED Out/In Go value can be passed directly (vs. needing Out's own
codec reused as the ContextField's codec) is codec-dependent (many Struct codecs
expect a map[string]any-shaped raw) — a small implementation detail to confirm,
not resolved here.
- events has NO ContextField wiring at all today (confirmed: its own doc
comment states pub/sub's security-shaped Fns get direct *T write access instead,
serving the same need via a different mechanism). If events.Transform's
fn ever needs to publish a value for the actual subscribe handler to read
(mirroring REST's ContextField use), EnsureContextFields would need to be added
to pub/sub's own dispatch first — a new, small prerequisite, not previously called
out anywhere in this doc.
RESOLVED (previously an open gap in an earlier draft of this doc): no more
"declare twice." An earlier version of Middleware[In,Out] discarded a declared
param's spec metadata, keeping only its merge field — meaning a caller wanting
"X-API-Key" BOTH documented in the OpenAPI spec AND merge-decoded into a typed In
had to declare it twice, with no conflict check between the two declarations. Per §3's
fix, Middleware[In,Out] now retains BOTH halves of every declared param, and its
spec half feeds the SAME checkParamConflicts pass the legacy middleware.Middleware
path already uses — one declaration, both effects, one shared conflict-check. This
isn't a bespoke patch; it falls directly out of the guiding principle above: a
middleware occupying both table columns needs a single source of truth for its spec
contribution, exactly like paired Security already has one.
Resolved design decisions (previously N1-N6, now closed)¶
A dedicated review traced this design's proposed dispatch points against the REAL
code they would hook into (adapters/nethttp/serve.go, adapters/nethttp/client.go,
adapters/mqtt5/adapter.go) and found six open questions beyond SSE/PathPrefix/
third-party-extensibility above. ALL SIX are now resolved — reconsidered once more,
explicitly, under the freedom to make breaking changes where it improves
maintainability (uncomplicated here: this is all still-unimplemented design, so
"breaking" only means picking the best shape, not migrating real callers).
D1 (was N1/N7) — pre-handler-only timing: RESOLVED, stays pre-handler-only.
Confirmed: Transform's ENTIRE dispatch (DecodeIn → call fn → EncodeOut) is
placed at the SAME point runSecurityMiddlewareReflect/runSubscribeSecurityImpls
already run — BEFORE the actual handler is invoked, unconditionally, mirroring the
already-proven security-shaped Fn's own timing exactly. Decision: no second,
post-handler attachment point — no worked example in this doc needs post-handler
Resp access, and a concern that genuinely does already has a mechanism: the
EXISTING wrapping-shaped Fn (func(http.Handler) http.Handler) wraps the ENTIRE
dispatch, handler included. Adding a second attachment point now would be designing
past a driver that doesn't exist — this repo's own established discipline. Applies
identically to REST and events (events' subscribe side has the identical
constraint; publish has no handler to depend on, so it's unaffected).
D2 (was N2) — ErrorPattern integration: RESOLVED, fn errors ARE eligible.
Confirmed: runSecurityMiddlewareReflect's own Fn errors always produce a FIXED
http.StatusUnauthorized + rest.SecurityError{Err: err}, never routed through the
route's own declared ErrorPattern/ErrorStatus rules. Decision: a Transform/
ClientTransform fn error is now run through handle.ErrorResponseFor(err) FIRST
(confirmed generic enough to reuse — the SAME mechanism a handler error already
uses) — a middleware's own business error (e.g. "malformed API key") can be matched
against a declared ErrorPattern exactly like a handler error can. Only when NO
pattern matches does it fall back to a NEW, dedicated error type —
rest.MiddlewareError{Name string, Err error}, default status 400 — replacing
the previous "reuse SecurityError" approach, which was semantically wrong for a
non-security middleware (a cookie-policy middleware's business error mislabeled as
a "security" failure). rest.SecurityError remains reserved for the EXISTING
middleware.SecurityScheme mechanism, unchanged. Events mirror, confirmed
extendable (not REST-only): api/events.ChannelHandle.ErrorResponseFor ALREADY
EXISTS (api/events/error_pattern.go), mirroring REST's ErrorResponseFor exactly
— events.Transform's (subscribe) fn error is matched against a declared
ErrorChannel pattern FIRST, the SAME way, falling back to a NEW
events.MiddlewareError{Name string, Err error} (mirrors rest.MiddlewareError)
when unmatched. events.ClientTransform (publish) errors get the same treatment if
and when events' publish path gains its own error-response concept — not a blocker
to stating the RECEIVING (subscribe) side's resolution now.
D3 (was N3) — client-side precedence: RESOLVED, three tiers. Confirmed:
adapters/nethttp/client.go's overrideDerived establishes a clean 2-tier
precedence today — explicit CallOptions always wins over the route's own
Req-derived merge values. Decision: explicit CallOptions > middleware-derived
(ClientTransform's In) > route-own-derived — middleware is the MORE SPECIFIC,
LATER-ATTACHED concern (ClientTransform chains onto an already-built Route
value), mirroring this doc's OWN already-established server-side rule for response
header/cookie conflicts ("registration-order, last-applied-wins," §5) applied
symmetrically to the client side. Fills the previously-unstated middle tier.
Events mirror: the SAME 3-tier PRINCIPLE (explicit > middleware-derived >
channel-derived) applies to events.ClientTransform's publish side — confirmed
events' publish adapters already have their OWN explicit-override mechanism (e.g.
adapters/mqtt5/binding.go's Vars map[string]string) to sit at the TOP tier,
mirroring CallOptions's role. Exact per-adapter wiring (whether mqtt(v3)/
zeromq expose the same override shape) is an implementation detail, same category
as REST's own "exact Out-return-shape" detail (§5) — not a design blocker.
D4 (was N4) — mandatory implementation requirement, confirmed, not a decision
with two answers. A separate review round found and fixed a real, shipped bug:
applyParamDeclarations's layering loop could double-append a param name agreed
upon by 2+ LEGACY middleware.Middleware values, absent a guard tracking
already-added names (not just manually-declared ones). §3's "layering" mechanism
(Middleware[In,Out] retaining full param structs, feeding the SAME conflict-
detection pass) MUST extend the SAME already-added-names guard to cover
contributions from MULTIPLE Middleware[In,Out] instances (or a Middleware[In,Out]
alongside a legacy middleware.Middleware) agreeing on the same param name — stated
here as a MANDATORY requirement for whoever implements §3, not an open question.
D5 (was N5) — Observer/stats integration: RESOLVED, the new mechanism improves on
the legacy silence. Confirmed: runSecurityMiddlewareReflect itself calls no
observer at all for its OWN Fn errors today (a pre-existing gap in the legacy
mechanism, not introduced by this design). Decision: Transform/ClientTransform
call stats.ReportErrors(obs, "middleware:in", err) on an In-decode failure
(MiddlewareInputError) and stats.ReportErrors(obs, "middleware:fn", err) on the
fn's own returned business error — distinct location strings for the two
failure kinds, mirroring this codebase's existing stats.ReportErrors(obs, "path",
err)-style call sites elsewhere. No backward-compat cost to doing this properly,
since this is entirely new code — deliberately NOT mirroring the legacy mechanism's
silence. Events mirror, confirmed NOT REST-specific: stats.Observer/
stats.ReportErrors are ALREADY pattern-agnostic (used identically across REST and
events elsewhere in this codebase) — events.Transform/events.ClientTransform
get the SAME stats.ReportErrors(obs, "middleware:in"/"middleware:fn", err) calls,
no adaptation needed beyond calling the same function from events' own dispatch
points.
D6 (was N6) — multiple Middleware[In,Out] attached to ONE route/channel:
RESOLVED, three sub-cases.
- (a) Two different
Middleware[In,Out]values both reading the SAME header name into their OWN, independentIntypes — VALID, no conflict. Each decodes its OWN typed copy from the same underlying raw value; no shared mutable state, no ambiguity. This is a DIFFERENT case from §3's spec-layering conflict-detection (which is about DECLARED SPEC METADATA agreement, not independent runtime reads) — no new check needed. - (b)
Declaration.Nameuniqueness — ENFORCED, per attaching route/channel. Reconsidered explicitly under the freedom to diverge from the legacy mechanism's looseness (middleware.ServerImplementation.Name/ClientImplementation.Nameare NEVER uniqueness-checked today) — for THIS brand-new mechanism, whereNamenow matters more (D5's Observer attribution; D2'sMiddlewareError.Name), enforcing uniqueness is a genuine maintainability win: it catches a real copy-paste mistake (accidentally attaching the sameDeclarationtwice, or two different declarations sharing aName) at Register/Handle time instead of silently producing ambiguous observability/error data. New error:rest.DuplicateMiddlewareNameError{Route string, Name string}(events mirror:events.DuplicateMiddlewareNameError{Topic string, Name string}), checked in the SAME resolution pass that already collectsmwcontributions for spec-layering (§3) — no new pass, just an added check in the existing one. The LEGACYmiddleware.ServerImplementation.Name/ClientImplementation.Nameremain exactly as they are — this enforcement applies ONLY to the NEWMiddleware[In,Out]/Declarationmechanism, not retroactively to the old one. - © Two middlewares both WRITE to the SAME
*Req/*Tfield via enrichment — attachment-order, last-applied-wins, NOT flagged as a conflict/error. Mirrors §5's already-established response header/cookie precedent, applied consistently here too. Not a compat question either way — a fundamental Go-language limitation (arbitrary runtime field writes inside a closure can't be statically tracked without disproportionate reflection machinery) independent of what's safe to break. A legitimate use case (two independent enrichment steps, one a fallback for the other) isn't necessarily wrong either — no error, just a defined order.
Explicitly out of scope for this "ready to implement" pass: M2/M3, from an
EARLIER, separate critical review of the ALREADY-SHIPPED legacy mechanism (Fn
fail-fast ordering; events' CheckCoverage eager-vs-deferred timing) — a different
topic (existing, shipped code behavior), tracked as its own separate SQL todos, not
bundled into this design's readiness.
Test plan (once implementation begins)¶
middleware.Declaration/NewDeclaration— construction + codec validation.rest.Middleware[In,Out]/events.Middleware[In,Out]— merge-field registration tests mirroringRouteHandle's/ChannelHandle's own existing merge-field tests.rest.Transform/events.Transform(RECEIVING role, both packages) — registration + dispatch: happy path, In-decode failure (MiddlewareInputError), fn error (short-circuits before handler/delivery), Out response-merge composing correctly alongside the route's own merge fields (REST only — events' Out is unused on subscribe, see §8).rest.ClientTransform/events.ClientTransform(SENDING role, both packages) — registration + dispatch: happy path (fn's produced value correctly encoded into outgoing headers/cookies/query or topic vars, merged alongside the route/channel's own derived values with correct precedence), fn error (aborts before any network/ publish activity), Out decode from the response composing correctly (REST only — events' publish has no response to decode from).- Backward-compatibility regression: an EXISTING
.Use(securityScheme)call site continues to pass unchanged afterUse's parameter type widens tomiddleware.RouteMiddleware. - A test attaching the SAME
Middleware[In,Out]value via BOTHTransformandClientTransformon two differentRoute/Subscriber/Publishervalues (the API-key worked example) — confirming the "one declaration, two attachment points" contract end-to-end. - Route/channel-AGNOSTIC (
.Use(mw)) coverage: aMiddleware[In,Out]carrying aWithReceive/WithSendFn, attached verbatim (the LITERAL samemwvalue) to TWO OR MORE routes with DIFFERENTReqtypes, confirming both routes dispatch correctly and independently — the test that actually proves route-agnostic reuse, not just that the mechanism compiles. Repeat forSSERoute.Use/Subscriber[T].Use/Publisher[T].Useto confirm the SAME reuse across all attachment surfaces (the point of this round's SSE/events migration). rest.TransformSSE/rest.ClientTransformSSE— the SAME test shapes asTransform/ClientTransformabove, PLUS:Outresponse-merge composing correctly alongside SSE's own headers (Content-Type: text/event-streametc., confirmed set AFTER middleware dispatch inserve_sse.go);ClientTransformSSE'sOutdecoding ONCE at connection-open time, confirmed NOT re-decoded per event.- A test confirming
req *Req/msg *TENRICHMENT — a value written byfninto*req/*msgis visible to the actual handler afterward, unchanged from whateverfnset it to (the core new capability this design round added; not previously named as its own test case). - D6©: two attached middlewares both writing the SAME
*Req/*Tfield — confirms the RESOLVED rule (attachment-order, last-applied-wins, no error). - D2: a middleware
fnerror matching a declaredErrorPatternproduces the PATTERN's structured response (not a generic status); afnerror matching NO pattern falls back torest.MiddlewareErrorwith status 400. Repeat forevents.Transform/ErrorChannel/events.MiddlewareError. - D3: explicit
CallOptionswins overClientTransform-derived values, which in turn win over the route's own req-derived values, for the SAME header/cookie/ query name — all three tiers exercised in one test. Repeat forevents.ClientTransform's publish-side precedence (explicit adapter Vars > middleware-derived > channel-derived). - D5:
stats.ReportErrors(obs, "middleware:in", err)called onIn-decode failure;stats.ReportErrors(obs, "middleware:fn", err)called on thefn's own error. Repeat forevents.Transform/ClientTransform. - D6(b): attaching two
Middleware[In,Out]values with the SAMEDeclaration.Nameto one route/channel (via any combination of.Use()/Transform/ClientTransform) returnsrest.DuplicateMiddlewareNameError(events.DuplicateMiddlewareNameErrorfor events) at Register/Handle time. - D7: a
Middleware[In,Out]carrying a bundledWithReceive/WithSendFn, ALSO passed toTransform/ClientTransform, returnsrest.AmbiguousMiddlewareAttachmentErrorat Register/Handle time; amwused ONLY via.Use()(bundled) or ONLY viaTransform/ClientTransform(bound) succeeds normally, confirming the rejection is specific to COMBINING both styles on one value, not either style alone.
Deferred, not part of this design¶
- Refactoring
examples/rest-api'sResponseDepositor/chiResponseDepositor/nethttpResponseDepositorto use this mechanism — a separate, later round, once this ships. - A full
ports.Middleware[In,Out]/.Use()implementation forports— see "Feasibility of a full.Use()-basedports.Middleware[In,Out]" above for the grounded analysis (confirmed reusable building blocks, the Pattern-bound vs. Pattern-less split, thectx-parameter cost, and SQL's metadata-only exception). Out of scope for THIS implementation round, but no longer just a placeholder. - SSE is NO LONGER deferred (see "SSE" above) —
SSERouteHandle's new response merge fields,TransformSSE/ClientTransformSSE's implementation, andconsumeSSE's new recognized Fn shape are all IN SCOPE, sketched at the confirmed dispatch gap-points. What remains genuinely deferred: the EXACT mechanical hook forClientTransformSSE's Out-decode-once-at-connection-open timing (e.g. whetherconsumeSSEneeds restructuring to expose a distinct "connection opened" callback) — sketched in principle only, not in full mechanical detail. - Per-adapter wiring details for
events.ClientTransform's "additional topic vars" mechanism beyond mqtt5's confirmedUserProperty-attachment precedent (whetherzeromq/mqtt(v3) have an equivalent attachment point for OUT-side data) — a mechanical per-adapter detail, not a design blocker, to resolve during implementation. PathPrefix/TopicPathPrefix's own composition semantics for multiple attachments (error on conflict, concatenate, last-wins) — sketched under "Extending the model," explicitly not resolved, deferred until a concrete driver surfaces.- Whether the generalized spec-layering attachment mechanism should ever be EXPORTED for third-party custom middleware types, or stay closed/sealed to api/rest's/ api/events' own shipped types ("Extending the model") — leaning sealed, not decided.
- Adding
middleware.EnsureContextFields-equivalent wiring to pub/sub's own dispatch — needed only ifevents.Transform'sfnwants to publish a value for the actual subscribe handler to read, mirroring REST's already-freeContextFieldcomposition (§9). Not needed for anything else this design ships.
Relationship to other roadmap docs¶
- Supersedes
docs/roadmap/common-middleware-architecture.md(now deleted, its finding fully absorbed here): that doc's core finding (a single sharedmiddleware.Middlewarestruct carrying REST-only fields unused byapi/events/api/reqreply) is what this design fixes — but via an ADDITIVE marker interface + NEW per-pattern generic types (rest.Middleware[In,Out]/events.Middleware[In,Out]), not that doc's original proposal to retrofit/splitmiddleware.Middlewareitself (which would have been breaking).middleware.Middleware/SecuritySchemeremained completely unchanged at the time this design shipped — since then, Addendum 3 (below) records a LATER, separate round that DID changemiddleware.Middleware(breaking changes pre-approved by that point): it was shrunk to{Name, Security}, fully absorbingdocs/roadmap/middleware-consolidation.md's own evaluation and outcome (that roadmap doc is now ALSO deleted, its content folded into Addendum 3 the same waycommon-middleware-architecture.md's was folded in here). - Update — relationship RESOLVED, via a confirmed 4-stage lifecycle
model. An earlier version of this section described
Feature (then titled
"Protocol-Native Feature Declarations," later "Feature/Provider") as a
separate, complementary axis — opaque protocol capability flags (MQTT5
Shared Subscriptions, Message Expiry, ZeroMQ Conflate/HWM) evaluated via
adapter type-switching, distinct from THIS doc's structured,
codec-backed Input/Output data. That framing went through TWO further
rounds — a "subsume this doc's mechanism" conclusion (reached against an
open, string-ID-based primitive later REJECTED for compile-time-safety
reasons), then reopened — before landing on its CURRENT, confirmed
resolution:
Middleware[In,Out]and the sealed, per-adapterCapabilitymechanism both occupy the SAME declare-time, spec-contributing lifecycle stage, WITHOUT merging into one Go type — each keeps its own distinct runtime-enforcement path. See that doc's §3 for the full resolution and its supporting prototype evidence. THIS doc remains the accurate, unchanged description of SHIPPED code; the roadmap doc's chosen direction is idea-only, not yet implemented, and any actual migration is deferred to a separate implementation-planning round. - Distinct from the ALREADY-SHIPPED REST/events
HandleMW/ClientMW/SubscribeMW/PublishMWsecurity+observer+general-purpose split (docs/design/d-0001-rest-middleware-workflow-simplification.md/docs/design/d-0002-pubsub-workflow-simplification.md) — a different, already-resolved topic, untouched by this design.docs/roadmap/declarative-middleware.md(the doc that ORIGINALLY proposed that split, before d-0001/d-0002 shipped it) has since been deleted, its remaining MCP/ports scope moved todocs/roadmap/mcp-ports-declarative-middleware.md, which THIS design's own "Feasibility" section above builds directly on.
Next steps¶
Historical note: items 1-4 below describe the ORIGINAL rollout plan (REST → SSE → events) — all shipped, in this order, confirmed via the "Addendum:
api/reqreplyandapi/events' property axis..." section below, which covers the LAST major follow-on round (bringingapi/reqreplyup to this same parity, plus 2 bug fixes discovered in events' already-shipped mechanism). Item 5 (portsmiddleware) remains future work, unaffected by any of this.
- Implement in the order:
middleware.Declaration/RouteMiddleware(packagemiddleware) →rest.Middleware[In,Out]+ BOTHTransform(server, bound) ANDClientTransform(client, bound) together, PLUSWithReceive/WithSend+.Use(mw)'s widened dispatch (route/channel-agnostic) (packageapi/rest, with adapter wiring inadapters/nethttp/adapters/chiserver dispatch ANDadapters/nethttp/client.goclient dispatch, covering BOTH attachment styles) →rest.TransformSSE/ClientTransformSSE+SSERoute.Use's widened dispatch (SSE, no longer deferred — see "SSE" above), includingSSERouteHandle's new response merge fields andconsumeSSE's new recognized Fn shape → RESOLVED naming:events.Middleware[In,Out]+ BOTHevents.Transform(subscribe) ANDevents.ClientTransform(publish) together, PLUSSubscriber[T].Use/Publisher[T].Use's widened dispatch (packageapi/events, with adapter wiring in whichever pub/sub adapters need it, starting withmqtt5which already has the confirmedUserProperty-style precedent). Sequencing, explicit: REST plainRouteFIRST (it has the concrete cookie-attrs AND API-key drivers) → RESTSSERouteSECOND → events THIRD. SSE precedes events (not merely listed first by convention) for a concrete, code-grounded reason:TransformSSE/ClientTransformSSElive in the SAMEapi/restpackage asTransform/ClientTransform, and SSE's adapter wiring lives in the SAME adapter files (adapters/nethttp/serve_sse.go, alongsideserve.go) — reusing infrastructure just built for plainRoute, zero new package. Events requires an ENTIRELY SEPARATE package (api/events) built from scratch, PLUS separate adapter packages (mqtt5/mqtt/zeromq) — a genuinely bigger increment, best attempted onceRoute's (and SSE's) pattern is proven out, not interchangeably with SSE. Ship each pattern's receiving AND sending role TOGETHER, not staggered, AND ship BOTH attachment styles (Transform-family and.Use()) together per surface — shipping only one half is exactly the asymmetry earlier review rounds found and corrected, twice now (roles, then attachment styles). - Full verification each step:
gofmt/go build/go vet/go test/just check/just examples. - Update
.github/instructions/go-codex.instructions.mdand fold intodocs/design/d-0001-rest-middleware-workflow-simplification.md(REST, including SSE) anddocs/design/d-0002-pubsub-workflow-simplification.md(events) as new Addenda. - Revisit
examples/rest-api'sResponseDepositoronce REST's half ships. ports(File/Cache/Dir/SQL) middleware remains its own, separate, later implementation effort — see "Feasibility of a full.Use()-basedports.Middleware[In,Out]" above; UNAFFECTED by this round's REST/SSE/events naming and attachment-style resolution, since it was already scoped as future work, not blocked on any decision made this round.
Addendum: api/reqreply and api/events' "property" vocabulary axis, bringing both up to full parity with this design¶
Added after docs/roadmap/reqreply-codec-declared-middleware.md shipped
(19 review rounds, 8 numbered design decisions, implemented in a later
session as a 13-phase, two-track parallel effort) — recorded here so the
lineage is discoverable from this document, since the newer feature's own
roadmap doc did not itself narrate where its core mechanism came from. That
roadmap doc has SINCE BEEN DELETED (per its own 3-way delete/keep/promote
graduation policy — a single-feature roadmap doc, fully shipped, with a
confirmed zero-gap review, and no lasting cross-cutting design value of its
own beyond what is captured here); this addendum is the durable record of
the design lineage that remains after that deletion.
What shipped¶
Before this round, Middleware[In,Out] (REST/events, per D1-D7 above) only
had a "topic"/"path" vocabulary axis (WithRequestTopic/WithResponseTopic
for reqreply, channel-var equivalents for events) — no way to declare a
merge field against MQTT5 User Properties/AMQP-style headers, "named
metadata separate from payload." This round added a SECOND, orthogonal
vocabulary axis, applied identically to BOTH api/reqreply (new package,
brought up to D-0003 parity for the first time in this round) and
api/events (already D-0003-compliant for the topic axis; this round added
property-axis parity):
PropertyParam/MergedPropertyParam[T](per-package, duplicatingTopicParam's existing shape exactly — same reasoning asTopicParamitself: two independently-defined thin wrappers over the SAME sharedcodex.Param/MergedParam[T]/NewParam[T,V]primitives, not one cross-package type).NewPropertyParam[T,V](required) /NewOptionalPropertyParam[T,V](optional — a NEW capability added to the sharedcodexprimitives' surface, viacodex.OptionalField, which already existed internally but was never exposed throughNewParam's API before this round).Required boollives onPropertyParam/MergedPropertyParam[T]themselves (mirrorsrest.HeaderParam.Required— NOT added to sharedcodex.Param, keepingTopicParam's own no-Required-field design untouched).WithRequestProperty/WithResponseProperty(reqreply),WithSubscribeProperty/WithPublishProperty(events) — newMiddleware[In,Out]attachment methods, alongside the existing topic-axis ones.buildDecodeIn/buildEncodeOutextended to take TWO SEPARATE map parameters (topic vars, property vars) — never combined into one map, mirroring REST's own realbuildDecodeIn(headerVars, cookieVars, queryVars map[string]string)multi-axis pattern exactly (confirmed against REST's real code, not assumed).- AsyncAPI spec rendering: the property axis's contributions unify into
Phase 1b's existing
applyParamDeclarationsmechanism for reqreply (events renders its own standalone contribution, since it had no pre-existing flat mechanism to unify with);Requiredcorrectly propagates into the rendered schema'srequiredarray, not just runtime validation. - Conflict detection: a NEW, uniform algorithm applies to ALL contributions
(reqreply's flat Phase-1b mechanism AND the new axis; a brand-new
events.ConflictingParamContributionError+checkEventsParamConflictsfor events, which had zero pre-existing conflict machinery at all). Topic-vars and properties are tracked in INDEPENDENT namespaces (never cross-checked against each other — a stronger boundary than REST's own looser header/cookie/query shared-namespace precedent, justified by the two axes coming from genuinely different wire locations). Contributions additionally compareSchema(viareflect.DeepEqual, nil-vs-non-nil treated as a mismatch) — a deliberate DIVERGENCE beyond REST's own realcheckParamConflicts, which never compares codecs at all.
Accepted breaking change (deliberate, not silently introduced)¶
Unifying conflict-detection onto ONE uniform algorithm retired reqreply's
Phase 1b's OLD silent-first-seen-wins dedupe behavior for MISMATCHED
declarations sharing a name — two contributions disagreeing on
Required/codec now ALWAYS error, regardless of origin (Phase 1b's flat
mechanism or the new axis). Scope of the break is narrow: only routes with
two PRE-EXISTING Phase-1b-only declarations for the same property name with
differing Required/codec (previously silently tolerated) would newly fail
to Register. Chosen deliberately over preserving the old laxer behavior —
"one algorithm, one mental model" was judged simpler and more maintainable
than tracking contribution origin through the comparison, even at the cost
of a breaking change to already-shipped behavior.
Write-side wiring — the most significant implementation finding¶
Tracing property values all the way to the wire (not just the type-level
API) surfaced a REAL, pre-existing capability gap: adapters/mqtt5's
reqreply server-reply path had ZERO mechanism to write ANY outgoing User
Property onto its own reply message (ServeOptions.UserPropertyParams was
validate-only, checked against the INCOMING request only). This round added
the missing capability — both success-reply and error-reply .Publish(...)
call sites in adapters/mqtt5/reqreply_transport.go now write
propertyVars from WithResponseProperty into the outgoing
PublishProperties.User. The client-request side and events' publish side
both had existing write-targets already (t.opts.UserProperties/
PublishOptions.UserProperties) and only needed wiring, not new capability.
2 bugs found and fixed in events' ALREADY-SHIPPED D-0003 mechanism (pre-existing, not introduced by this round)¶
Discovered while scoping full reqreply/events parity, both confirmed real via code tracing, fixed in the same round since it already touched the exact code paths:
- Value-precedence bug — events' publish-side
(
adapters/mqtt5/adapter.go/adapters/zeromq/adapter.go, identical pattern in both) had channel-own-derived vars winning over middleware-derived vars — the OPPOSITE of this doc's own D3 (explicit > middleware-derived > route/channel-own-derived, confirmed via REST's realoverrideDerivedchain, and via this doc's own "Test plan" wording verbatim). Fixed via a minimalisExplicitVars boolflag throughpublish()'s call chain (simpler than an earlier 2-map-parameter sketch — the explicit-vs-channel-own cases are already mutually exclusive at the call site). - Missing Observer integration — events'
dispatchSubscribeMiddlewareHandlers/dispatchPublishMiddlewareHandlers(adapters/mqtt5/transform_dispatch.go) had ZEROstats.ReportErrorscalls for middleware decode/business-error failures, contradicting this doc's own D5 (which explicitly states "Events mirror, confirmed NOT REST-specific"). Fixed: both now callstats.ReportErrors(obs, "middleware:in"/"middleware:fn", err), matching REST's real call sites exactly.
Validation¶
A dedicated post-implementation gap review (comparing the roadmap doc's
"Files to create" table and all 54 planned unit tests against the real,
shipped codebase) found and fixed 4 residual gaps (a missing unit test, a
missing docs/features/reqreply-middleware.md page, 2 stale
cross-references in companion roadmap docs) — confirmed CLOSED, zero
functional gaps remained in shipped code. Full verification (go build
./..., go test -count=1 ./... repo-wide, just check, gofmt -l .) —
all clean.
Addendum 2: REST conflict-detection alignment + "middleware:out" cross-adapter parity¶
Spun out of docs/roadmap/rest-middleware-conflict-detection-improvements.md
(now shipped and deleted per its own graduation policy) — 3 candidate
improvements found while tracing REST's real checkParamConflicts/
applyParamDeclarations/runMiddlewareHandlersReflect code as the
reference model for the ADDENDUM ABOVE's own property-axis design
decisions. Where that prior addendum brought api/events/api/reqreply
up to parity with REST's mechanism, THIS round is the mirror image: REST's
OWN conflict-detection caught up to api/events/api/reqreply's
already-stricter, unconditional default — REST was the outlier this time,
confirmed via code (checkEventsParamConflicts/checkReqReplyParamConflicts
have been independent-per-axis and codec-schema-comparing since the
property-axis round above, with no opt-in/toggle in either).
1. Cross-kind namespace independence (previously shared)¶
REST's checkParamConflicts previously compared header/cookie/query
contributions in ONE combined map keyed by name only — a header named "X"
and a query also named "X" were rejected as conflicting, even though they
are semantically unrelated values from different parts of an HTTP request.
Fixed: applyParamDeclarations now builds 5 INDEPENDENT per-kind maps
(header, cookie, query, response-header, response-cookie); a conflict is
only raised between two contributions of the SAME kind. paramContribution
dropped its now-redundant kind field (every entry in a per-kind map
already shares one). A confirmed real test,
TestWithMiddleware_ConflictingParamContribution (declared a header AND a
cookie both named "X-Trace" and asserted this conflicted), was rewritten to
use two same-kind headers instead, preserving genuine same-kind-conflict
coverage.
Dependent bug found and fixed in the SAME change (not previously
scoped): paramNameSet/paramNameSetResponse (used to build
addedRequestNames/addedResponseNames, the "don't double-add the same
spec entry when multiple middlewares agree on a name" dedup guard) were
ALSO flat/kind-agnostic — harmless before this round only because
checkParamConflicts rejected any cross-kind same-name pair before the
dedup logic ever ran. Once cross-kind pairs became valid, the flat dedup
guard would have silently DROPPED a legitimate second contribution sharing
a name across kinds (e.g. a middleware-contributed header "X" and a
separate middleware-contributed query "X" — only the header would have
been applied to the spec). Fixed in the same change: these guards are now
5 independent per-kind sets too, mirroring the conflict-detection maps.
2. Codec-schema comparison (previously absent)¶
REST's paramContribution had no Codec field at all — two contributions
sharing a name/kind/required-ness but with WILDLY different codecs (one
validating a UUID, the other a free-form string) passed silently. Fixed:
paramContribution gained a codec *codex.Codec[string] field, wired from
every contribution site; checkParamConflicts now also rejects on a schema
mismatch.
Shared-helper extraction (recommended in the roadmap doc, accepted):
the schema-mismatch comparison was ALREADY duplicated twice — api/events'
codecSchemaMismatch and api/reqreply's inline
checkReqReplyContributionMap comparison. Adding REST's own copy would
have made a 3rd near-identical implementation. Instead, extracted
route.CodecSchemaMismatch(a, b *codex.Codec[string]) bool (mirroring the
established route.FirstSchemeName extraction precedent — moved there from
5 duplicated per-adapter copies) and refactored ALL THREE APIs to call it,
eliminating 3 copies down to 1. api/events/api/reqreply both lost their
local reflect import as a result (no longer needed once the comparison
moved out).
3. "middleware:out" — a new observer location, plus 4 real bugs it surfaced¶
REST's middleware output-encode (EncodeOut/EncodeOutCookieAttrs)
failure path had NO stats.ReportErrors call at all, unlike the
input-decode ("middleware:in") and handler-fn ("middleware:fn") paths —
confirmed in 4 places, not just the one file originally suspected:
adapters/nethttp/serve.go, adapters/nethttp/serve_sse.go,
adapters/chi/serve.go, adapters/chi/serve_sse.go (8 total patch points
— 2 errFn sites per file). Fixed: all 8 sites now report
stats.ReportErrors(rest.DiagnosticObserver{Ctx: ctx}, "middleware:out",
err), mirroring the existing "middleware:in"/"middleware:fn" calls'
shape exactly.
Reviewing whether this same gap existed in api/events/api/reqreply's
own adapter dispatch (mqtt5/zeromq/mqtt) surfaced 4 REAL, CONFIRMED bugs in
ALREADY-SHIPPED D-0003 code — not just an analogous additive gap:
- mqtt5/zeromq events-publish:
dispatchPublishMiddlewareHandlersreturned a raw, un-wrappedEncodeOuterror on failure; its callers unconditionally reported EVERY such failure as"middleware:fn"— mislabeling an encode failure as a business-logic error. Fixed: the dispatch functions now wrapEncodeOutfailures distinguishably (middlewareDispatchError.isEncodeErr), and callers 3-way switch to"middleware:out"for that case. adapters/mqtt(v3): had ZERO"middleware:in"/"middleware:fn"reporting anywhere in its subscribe/publish dispatch — a strictly BIGGER, pre-existing gap than REST's original one (mqtt5/zeromq at least attempted"middleware:fn", wrongly forEncodeOut; mqtt v3 had nothing). Fixed: added the missing calls, mirroring mqtt5/zeromq's pattern, plus the SAME"middleware:out"fix.- mqtt5/zeromq reqreply server-side:
dispatchServerMiddlewareHandlersdecodes the request's In, runs Fn, THEN encodes the REPLY's Out — but only returned a 2-wayisFnErr bool, collapsing DecodeIn failures AND EncodeOut failures (building the reply) into the same bucket, both reported as"middleware:in". Fixed: changed to a 3-wayfailKind string("in"/"fn"/"out"), updated all 3 call sites (mqtt5 ×1; zeromq ×2, router + plain socket variants) to report the correct location. - mqtt5/zeromq reqreply client-side:
dispatchClientMiddlewareOut(client reading the REPLY's Out, no Fn involved) was also reported as"middleware:in". Fixed: pure string-literal change to"middleware:out"at all 3 call sites (mqtt5 ×1; zeromq ×2) — no signature change needed, since this function's failure is unambiguously always an Out-decode failure.
"middleware:out" is symmetric with "middleware:in"'s existing
dual-direction convention — "middleware:in" already covers BOTH the
server DECODING the request's In (subscribe/reqreply-server) and the
client ENCODING the request's In before sending (reqreply-client's
dispatchClientMiddlewareIn); "middleware:out" now covers BOTH the
server ENCODING the reply's Out (reqreply-server, events-publish) and the
client DECODING the reply's Out (reqreply-client) — the location names the
STRUCT, not the direction, exactly like its sibling.
Validation¶
An independent code-review pass (fresh eyes, no assumptions) verified: no
error-wrapping bugs, correct observer location strings at every call site,
no fallthrough errors in the 3-way discriminator switches, errors.As
unwrap paths intact, correct per-kind map/set usage in
applyParamDeclarations (no header/cookie/query mixups), and clean
route.CodecSchemaMismatch extraction (no dangling reflect imports).
Full verification (gofmt -l ., go build ./..., go test -count=1
./... — 55 packages, all pass, just check — zero issues,
events-api/reqreply-api/rest-api examples all exit 0) — all clean.
Addendum 3: middleware-consolidation — folding in the legacy middleware.Middleware type's presence-only params, Security's permanent split¶
Absorbs docs/roadmap/middleware-consolidation.md in full (now deleted,
per this repo's standard "merge the finding, drop the now-empty roadmap
shell" convention). That roadmap doc was itself the SECOND doc to notice
this tension — the FIRST was docs/roadmap/common-middleware-architecture.md
(deleted earlier, see "Relationship to other roadmap docs" above),
spun out of D-0002's "F6" critical-review finding that
middleware.Middleware is a single flat struct carrying REST-only
fields api/events/api/reqreply import unused. That doc's own
proposed fix (retrofit middleware.Middleware into a breaking
Common-base + per-pattern DERIVED STRUCTS) was superseded by THIS
design's additive, non-breaking RouteMiddleware marker + generic
Declaration[In,Out] family instead — middleware-consolidation.md
picked up where that left off, evaluating the REMAINING tension this
design's own shipped mechanism didn't fully close. This Addendum
evaluates — and resolves —
the tension §2's original text glossed over: this design's own
Declaration[In,Out]/Middleware[In,Out] family was never actually a
drop-in superset of the legacy middleware.Middleware type it was meant
to (eventually) replace. Two real capability gaps existed, confirmed via
code, not assumed:
- Security had no codec-backed equivalent at all.
middleware.SecurityScheme(...)/rest.FromSecurityScheme(...)both returned the legacy type;Middleware[In,Out]had noSecurityfield, andHandleMW/ClientMW/SubscribeMW/PublishMW(all 3 APIs) were hard-coded to the CONCRETE legacy type, not the sharedRouteMiddlewareinterface both families implement. - Presence-only (non-merged) header/cookie/query/property
declarations had no codec-backed equivalent either — an ACTIVELY USED
pattern, not a hypothetical.
mqtt5.FromUserPropertyParam/FromResponseUserPropertyParambuilt a legacymiddleware.Middleware{RequestHeaderParams: [...]}value with NO merge field at all — pure spec+validation (an MQTT5 User Property needing validation + AsyncAPI rendering, but no corresponding Go struct field to decode into).Middleware[In,Out].WithRequestHeaderREQUIRES a merge field; there was no presence-only variant.
What shipped — a deliberate hybrid, not a full fold¶
Breaking changes were pre-approved, so this went further than a "drop-in-only-if-free" evaluation — but landed on a hybrid, not a clean single-mechanism fold:
- Presence-only params fully consolidated (gap 2, closed in full).
Middleware[In,Out]gainedWithRequestHeaderSpec/WithRequestCookieSpec/WithRequestQuerySpec/WithResponseHeaderSpec/WithResponseCookieSpec(REST) andWithRequestPropertySpec/WithResponsePropertySpec(events/reqreply) — presence-only siblings of the existing merge-field methods, feeding the SAMEapplyParamDeclarationsconflict-detection/spec-layering pass the merge-field contributions already use (§3 above). The legacy type'sRequestHeaderParams/RequestCookieParams/RequestQueryParams/ResponseHeaderParams/ResponseCookieParamsfields, and the bridge constructors that built them (rest.FromHeaderParam/FromCookieParam/FromQueryParam/FromResponseHeaderParam/FromResponseCookieParam,mqtt5.FromUserPropertyParam/FromResponseUserPropertyParam), were DELETED entirely — fully replaced, zero remaining callers (confirmed via repo-wide grep before deletion).events. UnsupportedMiddlewareParamsError(which guarded against a legacy value's REST-only param fields leaking into a pub/sub channel) was ALSO deleted — the condition it guarded against can no longer occur once those fields don't exist. - Security followed a DIFFERENT path than gap 2 — discovered DURING
implementation, not anticipated up front. A genuinely
cross-API-shared Security declaration (one Go value, attached via
.Use()to a REST route, an events channel, AND a reqreply route — a real, documented, tested capability; seeexamples/reqreply-api/demo_cross_api_oauth2_sharing.goanddocs/features/security.md's "Sharing a security SCHEME across REST/events/reqreply" section) CANNOT be replicated by a per-pattern codec-backedMiddleware[In,Out]value. Each pattern's internal dispatch recognizes values via its OWN unexported, package-private interface (routeMiddlewareContributor-shaped in REST; the identically-shaped-but-separately-named equivalent in events/reqreply), whose method takes THAT package's own concrete, unexported*routeBuildertype as its parameter. A foreign pattern's codec-backed value satisfies NEITHER pattern's own dispatch interface (Go requires an EXACT parameter-type match), so it would COMPILE — both the native and foreign values structurally satisfy the genericRouteMiddlewaremarker interface.Use()accepts — but be SILENTLY DROPPED at attachment time: no error, no spec contribution, the Security requirement simply vanishes, exactly the "declared but not enforced, nobody notices" failure classd-0006-protocol-native-capabilities.md's entire capability-coverage mechanism was built to eliminate elsewhere. Reintroducing a NEW instance of that failure mode to achieve "one mechanism" was judged worse than keeping Security on its own dedicated, permanent type. middleware.Middlewaretherefore SURVIVES, permanently, SHRUNK to{Name string; Security *SecurityDeclaration}— its former RequestParams/ResponseParams fields removed (fully absorbed by the codec-backed presence-only methods above), but its Security half kept exactly as-is.middleware.SecurityScheme/rest.FromSecurityScheme/events.FromSecuritySchemeall remain, unchanged — alongside each API's ownrest.SecurityMiddleware/events.SecurityMiddleware/reqreply.SecurityMiddleware(added per §2's correction above), for the SAME-API-only case.- The cross-API sharing GUARANTEE narrowed slightly, where it actually
mattered.
examples/reqreply-api/routes/middleware.go'sOAuthMw(the one real call site needing genuine cross-API sharing) became 2 values (OAuthMwReqreply/OAuthMwREST) built from ONE sharedroute.SecurityScheme/credential-codec config, rather than 1 literal shared Go value — "same config, declared twice" instead of "one value, attached twice". Seedocs/features/security.md's rewritten section for the full guarantee as it stands today. - The 8 pairing methods (
Route.HandleMW/SSERoute.HandleMW/Route.ClientMW/SSERoute.ClientMW(REST),Subscriber.SubscribeMW/Publisher.PublishMW(events),Route.HandleMW/Route.ClientMW(reqreply)) were widened from the concrete legacy type to the sharedmiddleware.RouteMiddlewareinterface — via a newmiddleware.SecurityCarrierinterface (duck-typedSecurityDeclaration()method) both the legacy AND codec-backed types implement, letting dispatch extract Security UNIFORMLY. Confirmed: this widening required ZERO changes to any of the ~52 pre-existing call sites (Go's structural interface satisfaction + the legacy type's value-receiverRouteMiddlewareMarker()method, promoted to the pointer type too, means every existingroute.HandleMW(&someMw, fn)/route.HandleMW(nil, fn)call site kept compiling unchanged) — exactly as this section's original cost-correction (above) predicted.
Tests migrated, not just deleted¶
Every test exercising a deleted symbol was PORTED onto its codec-backed
replacement rather than simply removed, closing 2 real pre-existing test
gaps found along the way: REST's presence-only methods
(WithRequestHeaderSpec/etc., shipped earlier but never covered by a
dedicated test) and reqreply's RESPONSE-side presence-only property
enforcement (WithResponsePropertySpec's runtime enforcement had no
regression test until this round — only the request side did).
Validation¶
gofmt -l . clean; go build ./.../go vet ./... clean; go test
./... — 56 packages, zero failures; just check — 501 files, 0 issues;
every example under examples/*/ re-run to exit 0; a final repo-wide
grep confirmed zero remaining non-historical references to any deleted
symbol (FromHeaderParam/FromCookieParam/FromQueryParam/
FromResponseHeaderParam/FromResponseCookieParam/
FromUserPropertyParam/FromResponseUserPropertyParam/
UnsupportedMiddlewareParamsError).
Addendum 4: api/rest's Transform/ClientTransform folded into HandleMW/ClientMW — unified reflection dispatch (Rollout Phase A)¶
Absorbs the REST-specific outcome of
docs/design/d-0007-declarative-middleware-layering.md's Rollout Phase A (that
roadmap doc stays in docs/roadmap/ — Phase B/C, api/events/
api/reqreply, remain open). This addendum documents a REAL change to
shipped api/rest code — unlike Addendum 3's additive consolidation,
this one REMOVES 4 previously-shipped free functions
(Transform/ClientTransform/TransformSSE/ClientTransformSSE,
documented throughout this doc's main body and Addendum ⅔ above as the
route/channel-BOUND attachment mechanism). Every code sample above
showing rest.Transform(route, mw, fn)/rest.ClientTransform(route, mw,
fn) is now HISTORICAL — accurate as a description of the architecture
AT THE TIME this doc's main body/earlier addenda were written, not of
current code. The CONCEPT (route-BOUND, *Req-aware dispatch for a
codec-backed Middleware[In,Out]) is unchanged; only the SURFACE moved.
What changed¶
Route.HandleMW/SSERoute.HandleMW/Route.ClientMW/SSERoute.ClientMW —
previously the LEGACY-only attachment methods (middleware.Middleware/
nil, building a middleware.ServerImplementation/ClientImplementation
unconditionally) — now ALSO recognize a codec-backed Middleware[In,Out]
and dispatch it through the SAME route-BOUND mechanism Transform/
ClientTransform used to provide, via fn's REFLECTED signature (not
mw's type):
HandleMW's bound shape:func(ctx context.Context, req *Req, in In) (Out, error)— 3-in/2-out, 2nd param type*Req(the route's OWN type). A legacy credential shape (func(ctx, *http.Request, *Req) (map[string][]string, error)) is ALSO 3-in/2-out but its 2nd param is*http.Request, not*Req— the two are distinguished by that 2nd-param-type check alone, confirmed NECESSARY (not merely sufficient) via a real bug this phase's own migration work caught:rest.SecurityMiddleware[In, Out](see below) can ALSO return a codec-backedMiddleware[In,Out]used PURELY as a legacy credential-shape carrier — mw's TYPE alone cannot distinguish the two cases.ClientMW's bound shape:func(ctx context.Context, req Req) (In, error)— 2-in/2-out, 2nd param typeReqBY VALUE. The legacy credential shape (func(ctx, []route.SecurityRequirement) (http.Header, error)) is ALSO 2-in/2-out, distinguished the same way (2nd param typeReqvs.[]route.SecurityRequirement).
MiddlewareHandler/ClientMiddlewareHandler gained a Satisfies
[]string field (mirroring middleware.ServerImplementation.Satisfies/
ClientImplementation.Satisfies exactly), and CheckCoverage gained a
4th parameter (handlers []MiddlewareHandler) to check BOTH lists —
closing the gap where a codec-backed Security middleware's coverage was
previously invisible to this check.
Security folded onto the SAME mechanism — the GrantedScopes convention¶
Security-carrying codec-backed middleware (built via
rest.SecurityMiddleware[In, Out], now GENERALIZED over In/Out — away
from a hardcoded Middleware[struct{}, struct{}]) dispatches through the
IDENTICAL HandleMW/ClientMW mechanism as any other codec-backed
middleware. adapters/nethttp/adapters/chi's own separate
runSecurityMiddleware/httpsecurity.RunSecurityMiddlewareReflect code
path (which only ever consulted the legacy
middleware.ServerImplementation shape) is RETIRED — replaced by:
httpsecurity.CollectGrantsReflect— runs the legacy security-shaped impls, returning merged grants WITHOUT callingmiddleware.CheckScopesyet.rest.DispatchMiddlewareHandlersruns (as it already did for ordinary middleware) — producing each handler's decodedOut.httpsecurity.MergeMiddlewareHandlerGrantsmergesGrantedScopesfrom every Security-Satisfying handler'sOutinto the SAME grants map — read viaelem.FieldByName("GrantedScopes"), the RESOLVED "conventional field" design (Option 3 from this roadmap's own design review):Outkeeps the EXACT uniformfunc(ctx, In) (Out, error)signature every other middleware uses; a SecurityOuttype is simply EXPECTED to carry a field namedGrantedScopes map[string][]string.- ONE final
middleware.CheckScopescall combines both sources.
A FAILING Security-Satisfying handler's own Fn error keeps Security's
distinct fallback (401, rest.SecurityError) — looked up via the failing
handler's Name/Satisfies, NOT a separate dispatch path — an ordinary
(non-Security) middleware Fn error keeps the generic fallback (400,
rest.MiddlewareError).
adapters/nethttp/client.go's mergeCredentialHeaders (the CLIENT-side
legacy-credential-shape dispatcher) was investigated for the SAME
retirement and found NOT to need it: it has no separate early
CheckScopes-style call to reorder/unify (the client never judges its
own authorization), and its shape genuinely needs secReqs itself (to
decide which scheme's credential to produce) — a capability the bound
ClientMW shape (func(ctx, Req) (In, error), no secReqs) does not
provide and was never meant to replace. It composes correctly, side by
side with the new bound ClientMiddlewareHandler dispatch, via the SAME
D3 precedence chain (explicit > middleware-derived > route-own-derived).
codex.EncodeMergeVars + the omit-empty constructor family¶
A NEW sibling of codex.EncodeVars — codex.EncodeMergeVars — omits a
merge field's key entirely from the encoded map when the field (declared
via codex.OmitEmptyField/OmitEmptyFieldFunc/OmitDefaultField)
reports itself not present, rather than always writing it unconditionally
(EncodeVars's own, DELIBERATELY unchanged behavior — still used for
path/topic/dotted-key building, where a missing segment would corrupt the
structure). Every REST merge-field location gained an omit-empty
constructor pairing its existing Required/Optional pair:
NewOmitEmptyQueryParam/CookieParam/HeaderParam (request side),
NewOmitEmptyResponseHeaderParam/ResponseCookieParam (response side) —
plus events/reqreply's own NewOmitEmptyPropertyParam (no topic-param
equivalent — a topic template var is positional/structural, no "optional"
concept at all).
The motivating real case: examples/go-edge-models/app/registry/auth.go's
newAuthCredentialFunc previously hand-built an http.Header value
directly, returning nil for the anonymous-access (no-token) case.
Migrated onto examples/go-edge-models/models/docker/registry/security.go's
BearerAuthDeclaration (now Middleware[BearerCredential, struct{}],
generalized from Middleware[struct{}, struct{}]) — BearerCredential's
Token field is declared via NewOmitEmptyHeaderParam, so an empty
token OMITS the Authorization header entirely, matching the prior
hand-built behavior exactly, declaratively. This migration ALSO caught a
real bug: SecurityMiddleware[In, Out]'s generalization left
InCodec/OutCodec at their Go zero value, which PANICS the first time
dispatch calls InCodec.Validate for any non-struct{} In (a
zero-value codex.Codec's Encode/Decode funcs are nil). Fixed:
InCodec/OutCodec default to codex.Struct[In]()/codex.Struct[Out]()
(a fieldless struct codec — confirmed safe, a no-op round trip) instead.
middleware.ContextFieldSetter — fixing a sketch that didn't compile¶
The roadmap doc's own Phase 3 design review caught, via an ACTUAL compile
check, that its original sketch —
func (m Middleware[In, Out]) SetContextFieldFromIn(field
middleware.ContextField[V], get func(In) any) Middleware[In, Out] — does
NOT compile (undefined: V; Go forbids a method introducing a type
parameter beyond its receiver's own). Fixed with a new exported interface,
middleware.ContextFieldSetter ({ Set(ctx context.Context, raw any)
error }), which every ContextField[V] already satisfies regardless of
V (Set's own signature never references V). Middleware[In,Out] gained
SetContextFieldFromIn/SetContextFieldFromOut, dispatched automatically
— FromIn right after DecodeIn/alongside client-side EncodeIn,
FromOut alongside EncodeOut/client-side DecodeOut — publishing into
the SAME middleware.ContextField box middleware.EnsureContextFields
pre-allocates. EnsureContextFields is now ALSO called by
clienttransport.go's Call/Consume (previously server-adapter-only),
closing the client-side gap this new mechanism would otherwise have hit
immediately.
Validation¶
gofmt -l . clean; go build ./.../go vet ./... clean; go test
./... clean across every package (including the real
examples/go-edge-models migration, which caught the SecurityMiddleware
codec panic described above); every example under examples/*/ re-run to
exit 0; a repo-wide grep confirmed zero remaining non-historical
references to rest.Transform/rest.ClientTransform/rest.TransformSSE/
rest.ClientTransformSSE (this doc's own code samples above are the
historical exception, left as-is per this doc's standing "frozen design
record" convention).
Addendum 5: api/events's Transform/ClientTransform folded into SubscribeMW/PublishMW — unified reflection dispatch (Rollout Phase B)¶
Absorbs the EVENTS-specific outcome of docs/roadmap/declarative-
middleware-layering.md's Rollout Phase B (that roadmap doc stays in
docs/roadmap/ — Phase C, api/reqreply, remains open). Mirrors
Addendum 4's REST change in structure and intent, applied to events' own
Transform/ClientTransform/SubscribeMW/PublishMW — every code
sample above showing events.Transform(subscriber, mw, fn)/
events.ClientTransform(publisher, mw, fn) is now HISTORICAL.
What changed¶
Subscriber.SubscribeMW/Publisher.PublishMW — previously the
LEGACY-only attachment methods (middleware.Middleware/nil, building a
middleware.ServerImplementation/ClientImplementation
unconditionally) — now ALSO recognize a codec-backed Middleware[In,Out]
and dispatch it through the SAME channel-BOUND mechanism Transform/
ClientTransform used to provide, via fn's REFLECTED signature (not
mw's type):
SubscribeMW's bound shape:func(ctx context.Context, msg *T, in In) error— 3-in/1-out, 2nd param type*T. A subtlety events has that REST does NOT:adapters/mqtt's/adapters/zeromq's OWN legacy subscribe- security Fn shape —func(ctx, *T, []route.SecurityRequirement) error— shares the EXACT SAME 3-in/1-out arity AND the EXACT SAME 2nd-param type (*T) as the bound shape; REST's "2nd param differs" check alone is insufficient here. Distinguished instead by the 3RD param's type: legacy is always[]route.SecurityRequirement, the bound shape's isIn(confirmed via an actual test exercising this exact edge case — not just code review).adapters/mqtt5's OWN legacy subscribe-security Fn (func(ctx, *pahomqtt5.Publish, *T) (map[string][]string, error)) already differs in both 2nd-param type AND arity (1-out vs. 2-out), so needs no special-casing.PublishMW's bound shape:func(ctx context.Context, msg T) (Out, error)— 2-in/2-out,TBY VALUE. EVERY confirmed legacy publish-security Fn shape (adapters/mqtt5: 3-in/2-out;adapters/mqtt/adapters/zeromq: 3-in/1-out) is 3-in, distinguished from the bound shape's 2-in by arity ALONE — no param-type inspection needed here, simpler than Subscribe's case.
MiddlewareHandler/ClientMiddlewareHandler gained a Satisfies
[]string field (mirroring REST's identical Phase A addition), populated
via ONE shared pair of builders (buildMiddlewareHandlerAny/
buildClientMiddlewareHandlerAny) reused by BOTH the agnostic (.Use())
AND bound (SubscribeMW/PublishMW) paths — a DRY consolidation REST's
OWN Phase A also converged on. One events-specific wrinkle this
consolidation surfaced, caught via an actual test failure:
dualAttached (D7's ambiguous-dual-attachment marker) must NEVER be set
inside the shared builder, since mw.isBundled() is naturally true for a
LEGITIMATE agnostic-only attachment (bundled IS the whole point of that
style) — setting it there unconditionally misfired D7 against pure
agnostic usage. Fixed: only the BOUND call sites (SubscribeMW/
PublishMW's bound branch) set dualAttached explicitly, after calling
the shared builder.
events.CheckCoverage gained a 4th parameter (handlers
[]MiddlewareHandler) to check BOTH lists — identical to REST's Phase A
extension. checkEventsMiddlewareNameUniquenessAndAttachment (D6(b)/D7)
needed ZERO changes — confirmed already generic over both attachment
styles, since it already took []MiddlewareHandler/
[]ClientMiddlewareHandler directly.
Security generalized the SAME way — GrantedScopes, applied from the start¶
events.SecurityMiddleware[In, Out any] is now GENERALIZED over In/Out
(was a hardcoded Middleware[struct{}, struct{}]) — InCodec/OutCodec
default to codex.Struct[In]()/codex.Struct[Out]() from the FIRST
version shipped, not discovered as a panic later the way REST's
generalization was. Security dispatches through the SAME
SubscribeMW/PublishMW mechanism as any other codec-backed middleware
— no separate httpsecurity-equivalent retirement was needed for events,
since adapters/mqtt5/adapter.go's existing DispatchSubscribeMiddlewareHandlers/
DispatchPublishMiddlewareHandlers calls were ALREADY the ONE AND ONLY
per-message dispatch pipeline (confirmed no REST-style duplicate-path
risk — events never had a separate reflect-based Attach loop vs. a
ports-facing generic one the way REST's serve.go/handlerFunc split
did).
Phase 1 generality test — DecodeLayer/EncodeLayer proven on a SECOND consumer¶
api/events/transform.go's buildDecodeIn/buildEncodeOut were
migrated from hand-written sequential codex.DecodeVars/EncodeVars/
EncodeMergeVars calls onto the SAME shared middleware.DecodeLayer/
EncodeLayer mechanism REST's Phase A extracted — confirming the
mechanism generalizes to events' 2-axis (topic, property) shape, not just
REST's 3-axis (header, cookie, query) one. Zero behavior change,
confirmed via the full pre-existing test suite passing unchanged before
any other Phase B work began.
SetContextFieldFromIn/SetContextFieldFromOut — the SAME ContextFieldSetter fix, zero NEW implementation needed¶
events.Middleware[In,Out] gained SetContextFieldFromIn/
SetContextFieldFromOut, dispatched from buildDecodeIn/buildEncodeOut
respectively — confirmed requiring ZERO changes to
middleware.ContextFieldSetter/EnsureContextFields themselves (both
were ALREADY V-free and package-agnostic, per Addendum 4's own fix).
SetContextFieldFromOut is Publish-only — events' Subscribe
(WithReceive) produces no Out at all (the one confirmed, genuine
asymmetry vs. REST/reqreply), so this method is only ever meaningfully
called on a Publish-side Middleware. middleware.EnsureContextFields
is now wired into adapters/mqtt5/mqtt/zeromq's per-message
Subscribe/Publish dispatch — fully greenfield (confirmed zero prior
usage anywhere in events' adapters).
Connection-level auth — Client.AddConnectSecurityScheme, closing the spec-registration gap¶
A NEW, small, additive method — Client.AddConnectSecurityScheme(name
string, scheme route.SecurityScheme) *Client — registers a
connection-level scheme directly into components/securitySchemes,
independent of any channel's own declaration, closing a confirmed gap:
AsyncAPISpec() previously aggregated securitySchemes ONLY from
per-channel declarations, meaning a scheme referenced solely via a
Server.Security list (never by any individual channel) could not be
spec-registered at all. Registered BEFORE the per-channel aggregation
loop (same last-registered-wins collision policy, now a 3rd contributor
alongside AddSchema/AddServer). adapters/mqtt5.ConnectError gained
ReasonCode byte/ReasonString string fields, populated from the
*paho.Connack Connect already received from the broker on a
CONNECT-time rejection (previously silently discarded — only a generic
wrapped error string survived); ConnectOptions gained an Observer
stats.Observer field, reporting RecordSecurityRejection("connect",
...) on an auth-rejected CONNACK (reason code >= 0x80), mirroring
NewSecuredClient's existing WithObserver pattern.
Validation¶
go build ./.../go vet ./... clean; go test ./... clean across
every package; the full examples/*/ sweep re-run to exit 0; a
repo-wide grep confirmed zero remaining non-historical references to
events.Transform/events.ClientTransform (this doc's own code samples
above are the historical exception, per this doc's standing "frozen
design record" convention); the 2 real, shipping motivating examples
(examples/events-api/routes/routes.go's APIKeyAuthMW,
examples/api-events/main.go's bearerAuthMW) were migrated onto the
generalized SecurityMiddleware[In, Out] signature early, per this
doc's own established "migrate the real motivating case early" discipline.
Addendum 6: api/reqreply's Transform/ClientTransform folded into HandleMW/ClientMW — unified reflection dispatch (Rollout Phase C)¶
Absorbs the REQREPLY-specific outcome of docs/roadmap/declarative-
middleware-layering.md's Rollout Phase C — the FINAL of the 3 planned
phases (REST=A, events=B, reqreply=C), completing the pattern across all
3 messaging APIs. Mirrors Addenda ⅘'s structure and intent, applied to
reqreply's own Transform/ClientTransform/HandleMW/ClientMW —
every code sample elsewhere in this doc showing
reqreply.Transform(route, mw, fn)/reqreply.ClientTransform(route, mw,
fn) is now HISTORICAL.
What changed¶
Route.HandleMW/Route.ClientMW — previously the LEGACY-only
attachment methods (middleware.Middleware/nil) — now ALSO recognize
a codec-backed reqreply.Middleware[In,Out] and dispatch it through the
SAME route-BOUND mechanism Transform/ClientTransform used to
provide, via fn's REFLECTED signature (not mw's type):
isBoundHandleMWShape[Req]/isBoundClientMWShape[Req] mirror REST's
functions exactly. Both reqreply adapters' legacy Fn shapes were
pre-verified safe against collision with the new bound shapes before
implementation began (not discovered mid-implementation the way some of
Phase A/B's edge cases were): mqtt5's legacy 2nd param is a concrete
adapter type (*paho.Publish), and zeromq's legacy shape differs in
return arity — neither collides with the bound shape's *Req/In
2nd/3rd params.
MiddlewareHandler/ClientMiddlewareHandler gained a Satisfies
[]string field (mirroring REST's/events' identical addition), populated
via ONE shared builder pair (buildMiddlewareHandlerAny/
buildClientMiddlewareHandlerAny) reused by BOTH the agnostic (.Use())
AND bound (HandleMW/ClientMW) paths. dualAttached (D7's
ambiguous-dual-attachment marker) is, per Phase A/B's hard-won learning,
computed ONLY at the bound call site, never inside the shared builder.
reqreply.CheckCoverage gained a handlers []MiddlewareHandler
parameter — identical extension to REST's/events' Phase A/B.
Security generalized the SAME way — GrantedScopes, applied from the start¶
reqreply.SecurityMiddleware[In, Out any] is now GENERALIZED over In/Out
(was a hardcoded Middleware[struct{}, struct{}]) — InCodec/OutCodec
default to codex.Struct[In]()/codex.Struct[Out]() from the FIRST
version shipped (events' Phase B discipline applied up front, not
rediscovered as a panic the way REST's Phase A generalization was). Fixed
the 2 real call sites this broke
(examples/reqreply-api/routes/middleware.go's BearerAuthMw/
OAuthMwReqreply) with explicit [struct{}, struct{}] instantiation.
GrantedScopes merge-and-enforce is wired RECEIVING (Serve)-side ONLY
— confirmed via REST's own adapters/nethttp/clienttransport.go that
this merge-and-enforce concept is receiving-side-only everywhere in the
codebase (ClientMW's sending-side dispatch never does it either); the
roadmap doc's own initial draft incorrectly scoped this as "both
directions" for both events and reqreply, corrected once this was
confirmed by code inspection rather than assumption. Reuses
adapters/internal/scopesmerge.MergeHandlerGrants/HasSatisfyingHandler
— a NEW package (NOT adapters/internal/httpsecurity, which is
explicitly scoped by its own doc comment to net/http-family adapters
only; initially named eventssecurity since it was built for events'
own Phase-B-adjacent prerequisite patch, RENAMED to scopesmerge once
reqreply needed the identical logic too — confirming it was genuinely
generic, not events-specific). mqtt5's and zeromq's runServerSecurity*
functions were refactored to return (map[string][]string, error)
instead of calling CheckScopes internally, so ONE unified CheckScopes
call (after merging legacy + bound grants) replaces what used to be
scattered per-mechanism calls.
zeromq's legacy security mechanism is PURE binary accept/reject (no
grants concept) — for BOTH events and reqreply. Unconditionally gating
the new CheckScopes call on len(secReqs) > 0 alone breaks zeromq's
pre-existing passing tests (a route relying solely on the legacy
mechanism's genuine success got incorrectly rejected, since granted
stayed empty). Fixed by gating zeromq's (only) CheckScopes call on
len(secReqs) > 0 && scopesmerge.HasSatisfyingHandler(satisfiesOfBound
HandlersOnly) — checked ONLY against bound MiddlewareHandler.Satisfies,
never legacy Fns. mqtt5's simpler unconditional len(secReqs) > 0 gate
remained correct and unchanged — its legacy mechanism genuinely produces
real grants on success.
SetContextFieldFromIn/SetContextFieldFromOut — fully symmetric, unlike events¶
reqreply.Middleware[In,Out] gained SetContextFieldFromIn/
SetContextFieldFromOut, dispatched from the (now 4, fully duplex)
buildDecodeIn/buildEncodeIn/buildEncodeOut/buildDecodeOut
functions — reqreply is fully symmetric with REST here (unlike events'
Subscribe/Publish asymmetry): reqreply's Serve side already returns
(Out, error), symmetric with Call's own (Out, error), so BOTH
directions meaningfully use SetContextFieldFromOut, not just one.
middleware.EnsureContextFields is wired into
adapters/mqtt5/reqreply_transport.go's and
adapters/zeromq/reqreply_transport.go's Serve AND Call dispatch entry
points (6 call sites total across both adapters' REP/ROUTER variants) —
fully greenfield, confirmed zero prior usage.
Connection-level auth — Server.AddConnectSecurityScheme¶
Server.AddConnectSecurityScheme(name string, scheme route.
SecurityScheme) *Server mirrors events.Client.AddConnectSecurityScheme
byte-for-byte (reqreply.Builder is a deprecated type alias for
Server — the method lives on Server, both names pick it up
automatically), registered before AsyncAPISpec()'s per-route
aggregation loop, same last-registered-wins collision policy. Drop-in
confirmation tests prove mqtt5.Connect's/mqtt.Connect's plain client
and *SecuredClient work UNMODIFIED as the ServerTransportOptions/
ClientTransportOptions fields reqreply's Serve/Call take — the
roadmap's own "zero adapter change needed" claim, now PROVEN by a
compile-time type-assignability test rather than merely asserted.
Known gap surfaced during test-writing — NOT fixed, tracked for follow-up¶
Writing reqreply's end-to-end GrantedScopes test surfaced a confirmed,
pre-existing, cross-package bug: .Use(mw).HandleMW(&mw, boundFn) — the
EXACT pattern this doc's own worked examples show for a bound Security
middleware — fails with DuplicateMiddlewareNameError when mw carries
ONLY a Security declaration (no merge fields). Root cause: both
.Use(mw) (applyAgnosticRoute) and the bound HandleMW/ClientMW
path (applyBoundRoute) unconditionally append a spec contribution
under the SAME mw.Name, and the name-uniqueness check treats this as a
genuine duplicate — distinct from D7's legitimate dualAttached case
(which is about a BUNDLED Fn, not this .Use()+bound combination).
Confirmed to ALSO affect api/rest (identical
checkMiddlewareNameUniquenessAndAttachment structure; no existing REST
test exercises this exact .Use()+bound-shape combination either — only
the legacy-shape combination is tested there). api/events does NOT
have this issue (it bundles spec metadata directly on the handler
struct, no separate spec-contribution list with an unconditional
.Use()-path append). Workaround used in the new reqreply/zeromq test:
declare RouteMeta.Security directly, skip .Use(mw) — sufficient for
CheckCoverage/CheckScopes correctness, though AsyncAPISpec() won't
auto-register the scheme via that path. Left unfixed — correctly
distinguishing "legitimate .Use()+bound combo for a no-merge-field
Security mw" from "genuine duplicate name collision" is a nontrivial
design question touching REST too, out of scope for a reqreply-only
pass.
Validation¶
go build ./.../go vet ./... clean; go test ./... clean across all
57 packages; gofmt -l . clean; a repo-wide grep confirmed zero
remaining non-historical references to reqreply.Transform/
reqreply.ClientTransform (this doc's own code samples above are the
historical exception, per this doc's standing "frozen design record"
convention); the real motivating examples
(examples/reqreply-api/routes/middleware.go) were migrated onto the
generalized SecurityMiddleware[In, Out] signature as part of pc-11,
not deferred to last.
Addendum 7: splitting Middleware[In,Out]'s reusable/bound duality into two explicit, compile-time-distinct types — removing isBoundHandleMWShape-style reflection detection entirely (all 3 APIs)¶
Absorbs docs/design/d-0003-codec-declared-middlewares.md's Addendum 7's full design and
rollout, folded in per this doc's own established "promote once shipped,
retire the roadmap doc" convention. Addenda ⅘/6 (above) each documented
a SINGLE Middleware[In,Out] type per package serving TWO roles —
route/channel-AGNOSTIC (attached via .Use(), reusable across any
number of routes/channels) and route/channel-BOUND (the SAME type,
fn's shape REFLECTED to detect which role applied, e.g.
isBoundHandleMWShape[Req]/isBoundClientMWShape[Req]/
isBoundSubscribeMWShape/isBoundPublishMWShape). This Addendum
replaces that ENTIRE reflection-based mechanism with two separate,
explicit Go types — see D-0007's
own SUPERSEDED callout (added alongside this Addendum) for the full
historical narrative of the bugs that motivated this.
What changed¶
The reusable class, Middleware[In, Out], is UNCHANGED in public surface
(Declaration[In,Out] + merge-field vocabulary +
WithReceive/WithSend, attached via .Use(...) only) but internally
simplified: applyBoundRoute/applyBoundClientRoute/
applyBoundSubscriber/applyBoundPublisher methods and ALL
isBound*Shape reflection detectors are REMOVED outright from all 3
packages — confirmed via repo-wide grep, zero live code remains (only
historical "NOTE: removed" doc comments). HandleMW/ClientMW/
SubscribeMW/PublishMW now ACTIVELY REJECT a codec-backed
Middleware[In,Out] value with a NEW MiddlewareMisattachedError,
regardless of the paired fn's shape — no more silent promotion to a
different dispatch path based on guessing fn's signature.
A NEW, separate type family replaces the bound role, Fn embedded AT CONSTRUCTION (never supplied separately later):
api/rest:BoundMiddleware[Req,In,Out]/BoundClientMiddleware[Req,In,Out], viaNewBoundMiddleware/BoundSecurityMiddleware/NewBoundClientMiddleware/BoundSecurityClientMiddleware, attached viaRoute.HandleBoundMW/Route.ClientBoundMW(ANDSSERoute's own copies —SSERoutehas its OWNHandleMW/ClientMWmethods, textually distinct fromRoute's, needing the identical treatment).api/events:BoundSubscribeMiddleware[T,In,Out]/BoundPublishMiddleware[T,In,Out], viaBoundSecuritySubscribeMiddleware/similar, attached viaSubscriber.SubscribeBoundMW/Publisher.PublishBoundMW.api/reqreply:BoundMiddleware[Req,In,Out]/BoundClientMiddleware[Req,In,Out](mirrors REST almost exactly, since reqreply'sRoute[Req,Resp]carries BOTH server/client roles on ONE shared value, like REST — NOT events' independent Subscriber/Publisher split), attached viaRoute.HandleBoundMW/Route.ClientBoundMW.
INTERNAL layout decision, found significant during implementation:
each bound type holds a NAMED (never embedded/anonymous) mw
Middleware[In,Out] field. Embedding was the naive first instinct but is
WRONG — Go promotes ALL of an embedded field's methods, including
Middleware's own applyAgnosticRoute, which would make the bound type
ALSO (accidentally) satisfy the reusable class's own attach-interface,
reintroducing the exact ambiguity this split exists to eliminate. A
named field avoids this entirely while still letting the bound type's 12
merge-field builder methods forward one-line onto the named field's own
existing method, and letting applyBoundRoute/etc. call the EXISTING 9
transform.go dispatch helpers UNCHANGED (passing bm.mw) — zero
duplication of those helpers across the split.
A second significant internal-design bug found only via a FAILING
TEST, not via 3 rounds of doc review: the naive
boundContributor[Req] interface { applyBoundRoute(rb *routeBuilder) }
sketch is VACUOUS — its method signature never mentions Req at all, so
EVERY BoundMiddleware[X,...] would satisfy boundContributor[Y] for
ANY X,Y, completely defeating the Req-mismatch detection the whole
mechanism exists to provide. Fixed by adding a deliberate, never-called
boundReqWitness(Req) method to both the interface and every concrete
bound type, forcing Go's interface satisfaction to actually discriminate
on the generic parameter — applied retroactively to REST (where it was
found), then proactively from the FIRST commit for events' and
reqreply's own bound types.
New structured errors¶
BoundMiddlewareReqMismatchError{Route, Got} (nil-Got-guarded
LogValue() — found via a repro, reflect.TypeOf(nil) panics on
.String() if not guarded) and MiddlewareMisattachedError{Route, Name}
— both construction-time (Register/ClientHandle/Handle), never
reaching Observer/ErrorPattern (confirmed structurally impossible to
reach Serve-time dispatch). A routeBuilder/Subscriber/Publisher-
level buildErr error field (new, since no prior mechanism stashed a
configuration mistake for later surfacing this way) carries the error
from the opt that detected it through to Register's return — AND,
confirmed as a REAL bug found in REST first (via a repro, not
inspection) and fixed PROACTIVELY for reqreply, ClientHandle()'s own
infallible-by-design signature (no error return, REST/reqreply only —
events' Subscriber.Handle/Publisher.Handle are already fallible) now
PANICS on a non-nil buildErr rather than silently swallowing it —
reusing the exact FormatOptError panic convention already present.
Confirmed, via code: the legacy bare middleware.Middleware{Name,Security} type's raw-adapter-Fn PAIRING was deliberately KEPT, not dropped — do not confuse this with the removed reflection-detection mechanism¶
HandleMW/ClientMW/SubscribeMW/PublishMW reject ONLY a
codec-backed Middleware[In,Out]/BoundMiddleware/
BoundClientMiddleware value (via the routeMiddlewareContributor-style
interface check) — a BARE middleware.Middleware{Name,Security} value
PAIRED with a raw-transport-shaped Fn (e.g. func(ctx,
*pahomqtt5.Publish, []route.SecurityRequirement) (map[string][]string,
error)) continues to be fully accepted and dispatched, UNCHANGED, via
buildServerImplementation/buildClientImplementation-equivalent
helpers — confirmed via this project's own preserved, passing legacy
tests across all 3 patterns (TestErrorPattern_SecurityMiddlewareFn_
Matched_Publishes_ReqReply, TestErrorChannel_SecurityImplFn_Matched_
Publishes_ViaSubscribeHandle, etc.). This is the ONE remaining tool for
a genuinely Req/T-type-generic Fn (reused across many different
route/channel Req/T types via a free function, REST's
ScopesImpl[Req any]-style) or a scheme shared across multiple
patterns — kept permanently, by design, not an oversight. (An earlier
draft of docs/design/d-0003-codec-declared-middlewares.md's Addendum 7 itself overclaimed this
closure as unconditional in two places; corrected there once the
discrepancy against HandleMW's own code/doc comment was found.)
Cross-pattern consistency fix found and shipped during this rollout: Security-carrying Fn failures now uniformly wrap as SecurityError, not MiddlewareError, across ALL 3 patterns' adapters¶
REST's adapters (nethttp/chi) already had a deliberate, TESTED
special case: a FAILING MiddlewareHandler whose own Satisfies is
non-empty (Security-carrying) wraps its business error as
rest.SecurityError (not the generic rest.MiddlewareError), via an
isSecuritySatisfyingHandler helper, and records a SecurityObserver
rejection. Confirmed via direct code comparison: BOTH api/events (all
3 adapters: mqtt, mqtt5, zeromq) and api/reqreply (both adapters:
mqtt5, zeromq, including zeromq's REQ/REP AND ROUTER/DEALER
sub-variants) LACKED this special case entirely — always wrapping as the
generic MiddlewareError, with no RecordSecurityRejection call for
this specific failure mode. examples/events-api's own shipped code had
already discovered and documented this exact gap (a code comment
explicitly described working around it via a "zero granted scopes"
trick instead of a direct Fn error) — but the dispatch code itself was
never fixed until this rollout. Fixed by adding
reqreply.IsSecuritySatisfyingHandler/events.IsSecuritySatisfyingHandler
(centralized per-package helpers, mirroring REST's own per-adapter
isSecuritySatisfyingHandler) and wiring them into every adapter's
dispatch error-handling (1 call site each for mqtt/mqtt5/events'
zeromq, reqreply's mqtt5; 2 call sites for reqreply's zeromq,
REQ/REP + ROUTER/DEALER) — all 3 patterns' adapters now share IDENTICAL
Security-Fn-error-wrapping behavior, each independently verified via a
dedicated regression test per adapter (proactively, for every adapter in
one pass, once the gap was found for one).
Confirmed, via code: BoundClientMiddleware/BoundPublishMiddleware cannot write into the request/message BODY on any transport — only derive wire-level (topic/property) values from it¶
Found while migrating examples/reqreply-api's zeromq in-payload OAuth
credential demo: the bound CLIENT-side type's Fn
(func(ctx, req Req) (In, error)) only ever produces topic/property
VARS via EncodeIn — there is no mechanism, on ANY transport, for the
returned In to overlay/merge into the actual outgoing Req's own
payload fields. For a transport with a genuine property/header side
channel (mqtt5), this is sufficient — credentials travel as wire
properties. For zeromq's in-payload model (no side channel at all, the
credential must live ON the request's own decoded struct), there is NO
client-side bound-middleware equivalent — the caller simply sets the
field directly as an ordinary part of the request value passed to
Call(), no middleware involved. Corrected in docs/features/security.md
and in docs/design/d-0003-codec-declared-middlewares.md's Addendum 7's own text (both
previously implied a BoundClientMiddleware attachment was the right
pattern here) before this doc absorbed their content.
Final adapter-parity audit (all 3 APIs, confirmed via direct code comparison, not re-assertion)¶
- REST (
nethttpvschi): byte-for-byte identical dispatch logic, confirmed via diff. - events (
mqttv3 vsmqtt5vszeromq): all 3 dispatchMiddlewareHandlers/ClientMiddlewareHandlers(bound and agnostic classes indistinguishable at this layer) identically; all 3 now share the SAME Security-Fn-error-wrapping fix (above), each with its own regression test. - reqreply (
mqtt5vszeromq, including zeromq's REQ/REP and ROUTER/DEALER sub-variants): confirmed exactly 4 top-level Serve/Call implementations exist repo-wide needing this mechanism, all 4 consistently fixed; client-side dispatch correctly has NO Security-error special case on either adapter (matches REST's/events' own client-side design — the client side only ever SUPPLIES credentials, never REJECTS based on them). - D-0006's sealed
Capabilitymechanism (QoS/Retained for mqtt-family, HWM/Conflate for zeromq) remains fully orthogonal and unaffected by this Addendum — its own per-protocol capability-set asymmetry is separately confirmed intentional, unrelated to this split.
Validation¶
go build ./.../go vet ./.../gofmt -l . clean across all packages;
go test ./... clean, zero failures, across every round of this
rollout (3 implementation phases + 7 cross-phase review rounds); every
touched example (examples/rest-api, examples/events-api,
examples/api-events, examples/reqreply-api, plus the Security-
specific examples/mutable-security-keys/adapters-nethttp-client/
adapters-sse/go-edge-models) re-run end-to-end with exit 0 at each
phase's completion, confirming real security enforcement continues to
be exercised (rejections, scope checks, credential errors) throughout.
Addendum 8: middleware package relocated under internal/middleware (docs/design/d-0009-internalize-shared-mechanics.md)¶
Every middleware.X reference throughout this document (Middleware,
Declaration[In,Out], SecurityDeclaration, ContextField[V],
RouteMiddleware, ServerImplementation, ClientImplementation,
CheckScopes, etc.) describes vocabulary that has since been relocated
from the public top-level middleware package to internal/middleware
— the package NAME is unchanged (still package middleware, so code
examples above spelling middleware.X remain byte-accurate for
maintainers working inside the go-codex module), only its import path
changed, and Go's internal/ import rule now makes it uninstallable by
an external consumer of go-codex. Each pattern package (api/rest/
api/events/api/reqreply) exposes the identical vocabulary under its
own name via a thin wrapper. UPDATE: the SharedMiddleware public
alias (rest.SharedMiddleware/events.SharedMiddleware/
reqreply.SharedMiddleware, IDENTICAL type aliases to
internal/middleware.Middleware) and FromSecurityScheme were
subsequently REMOVED entirely — a from-scratch investigation found zero
real callers needing a literal shared Go value across patterns (see
Addendum 9 of docs/design/d-0001-rest-middleware-workflow-simplification.md
and docs/design/d-0009-internalize-shared-mechanics.md); the underlying
internal/middleware.Middleware TYPE remains, unchanged, as internal-only
bridging currency. Disposition/ResolveDisposition/EnsureDispositionBox/
SetDisposition/DispositionFromContext moved OUT to stats instead
(not internal/middleware), since stats.DispositionObserver is a
PUBLIC, user-implementable extensibility interface that must be able to
reference Disposition by name. See
docs/concepts/ports-and-adapters.md's "Guardrail: shared cross-pattern
MECHANICS belong in internal/, not a public package" section for the
general rule this establishes for future shared mechanics.