Internal-service auth: signed-claims rollout (#543)

The OWUI tools authenticate to the marimo proxy (k8s/marimo_proxy.py) with an HMAC in X-Internal-Service-Auth. Until #543 that HMAC covered only X-Internal-User-Id; X-Internal-User-Email and X-Internal-User-Role were copied into the claims unsigned, so anyone holding a valid signature for some id — a captured header set, a replayed request, a compromised low-privilege caller — could bolt an admin role (shared writes) or another user's email (legacy email-prefix directory) onto it. v2 binds the claims to the signature. Residual risk, unchanged: a holder of INTERNAL_SERVICE_SECRET itself can still mint any claims; the HMAC proves possession of the shared secret, nothing more.

Formats

X-Internal-Service-Auth	Signs	Email / role headers
v2:<hex>	HMAC-SHA256(secret, "{user_id}\n{email}\n{role}")	trusted — covered by the signature
<hex> (legacy, no prefix)	HMAC-SHA256(secret, user_id)	ignored — claims become email="", role="user"; the proxy logs a WARNING

Client side: marimo_auth.build_marimo_auth_headers (inlined into each tool by scripts/bundle_tool.py). The proxy mirrors the canonical message in _internal_auth_message because its image ships only k8s/marimo_proxy.py; tests/unit/test_internal_auth_signed_claims.py pins the two copies.

Rollout order

The tool re-save and the proxy image are separate deploys, so the sequence matters:

1. Deploy the proxy (accepts both formats). Tools still sending legacy signatures keep working for identity, but their email/role headers stop being honoured — admin-only shared writes from a legacy-signed call return 403 until step 2, and the user's legacy email-prefix directory is not listed.
2. Re-bundle and re-save the OWUI tools so they sign v2: — python scripts/bundle_tool.py <tool>.py for each tool that imports marimo_auth (marimo_tools.py, gdsfactory_photonic_tools.py, vvuq_notebook_verifier_tool.py), then re-save each tool in the OWUI admin UI (tool code is frozen at save time). Email/role are honoured again.

Other legacy callers (not tools, but they sign the same header): - openclaw/openwebui_pipe_standalone.py — a pasted OWUI Pipe; identity only (sends no email/role), so it keeps working, but a pasted copy stays legacy and keeps the legacy counter nonzero until it is re-pasted. - .claude/skills/seed-notebooks/SKILL.md — hand-rolls the HMAC and sends X-Internal-User-Role: admin for shared writes; after step 1 those writes 403 unless the recipe signs v2 (the recipe now does). - tests/contracts/issue_244_notebook_generation/test_contract.py — integration test against the live proxy; signs legacy with role=admin (its save is non-shared, so the role is silently ignored). 3. Watch the rollout signal — with a positive control. kubectl logs shows only the current container since its last restart, and the proxy emits several lines per request (readiness probes every 10 s), so an empty grep can mean "no legacy callers" or "pod restarted" or "log rotated". Count both formats so a zero legacy is trusted only next to a nonzero v2:

kubectl logs --since=24h -n openweb-marimo deploy/marimo -c marimo-proxy \
  | grep -o 'sig=[a-z0-9]*' | sort | uniq -c
# and, independent of log retention (counts since the proxy process started):
kubectl exec -n openweb-marimo deploy/marimo -c marimo-proxy -- \
  python -c "import urllib.request,json; print(json.load(urllib.request.urlopen('http://127.0.0.1:8001/health'))['internal_auth'])"

A pod restart resets both the log and the counters. Rejected signatures are logged at WARNING (REJECTED v2 signature …) and counted under rejected — a nonzero rejected after step 2 means a tool bundle drifted from the proxy's canonical message or was saved with a stale secret.

Once legacy has stayed at zero for a day with v2 climbing, drop legacy acceptance in a follow-up: make _verify_internal_signature reject any value without the v2: prefix, delete the legacy-branch tests, and update the tests that still inline the id-only HMAC (they go red, loudly): tests/unit/test_workspace_autoopen.py, tests/unit/test_list_notebooks_multi_dir.py, tests/unit/test_check_notebook_endpoint.py, tests/unit/test_marimo_proxy_auth_check.py, tests/goal/test_g4_features.py, tests/goal/test_g6_education.py, tests/contracts/issue_244_notebook_generation/test_contract.py.