cto/.sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-ISSUES.md
2026-05-31 21:42:58 -04:00

5.6 KiB

title status lifecycle_classification owner created last_reviewed core_promotion_status route
CTO Case Provider Build Issues draft sot jp 2026-05-31 2026-05-31 not-promoted cto

CTO Case Provider Build Issues

Local planning SOT only. Not a Core Protocol. Not active Core authority.

CTO-WORK-015 - Durable Case Provider Build PRD

Status: validated.

Register the provider build route that separates source discovery from executable admission.

Acceptance:

  • Records WorkOS Case source https://github.com/workos/case.git.
  • Records pinned commit 7959ac917cdeb0983b4aaa20bb9f42021747fed8.
  • Records local build dependency status: Node v20.19.5 present, Bun missing.
  • Requires bun run build:binary to produce dist/ca before real Stage 2 can pass.
  • Requires SHA-256 digest for admitted dist/ca.
  • Keeps /tmp clone as discovery evidence only.

CTO-WORK-016 - Real Case Provider Stage 2 Run

Status: blocked.

Build or supply a durable admitted WorkOS Case ca executable, then run the existing Hermes CTO Stage 2 artificial fixture through real Case.

Acceptance:

  • Bun is available, or a durable admitted dist/ca path is supplied with source pin and digest.
  • CTO_HARNESS_CASE_BIN points to the admitted ca executable.
  • Stage 2 invokes ca run --task <task-file> --mode unattended.
  • Real Case execution produces a Stage 2 pass report through the Harness Evidence Interface.
  • Provider-unavailable fail-closed behavior remains tested.
  • Fake remains the default validation lane.
  • No Target Repository path is inspected or copied.

Blocked by:

  • Missing local bun executable, unless a prebuilt admitted dist/ca is supplied.

Provider Build Evidence - 2026-05-31

  • Reversible Bun bootstrap succeeded under /tmp/cto-bun; no global Bun install was required.
  • /tmp/workos-case source pin remained 7959ac917cdeb0983b4aaa20bb9f42021747fed8.
  • bun run build:binary produced /tmp/workos-case/dist/ca.
  • Built binary digest: 9811f870af2f85616e359d42ba70566c9af08ca20d8660456929a56ec761513f.
  • /tmp/workos-case/dist/ca --help identified the executable as Case ca.
  • /tmp/workos-case remains discovery/build evidence only because build modified src/generated/package-assets.ts.
  • Real Stage 2 run with CTO_HARNESS_CASE_BIN=/tmp/workos-case/dist/ca hung and was terminated. It created artifact directory /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T233721Z-r1-string-slugify-1814067 without report.json.
  • CTO-WORK-016 remains blocked because no real Case Stage 2 pass report exists.

CTO-WORK-017 - Case Provider Timeout Fail-Closed Route

Status: validated.

Harden the Hermes CTO Case adapter so a hung real Case provider fails closed with normal evidence instead of leaving an orphan artifact directory without report.json.

Acceptance:

  • Case provider execution is bounded by CTO_HARNESS_CASE_TIMEOUT_SECONDS.
  • Timeout writes backend/provider-timeout.txt.
  • Timeout still emits report.json through the Harness Evidence Interface.
  • validate-case-provider-adapter.py checks a sleeping provider path.
  • harness/evals/health.sh --json passes after merge and includes provider_timeout_fail_closed.
  • This does not mark real Case Stage 2 as passed.

Validation Evidence:

  • Hermes commit: d23c492 Fail closed on Case provider timeout.
  • Post-merge harness/evals/health.sh --json passed.

Real Provider Runtime Evidence - 2026-05-31

  • Real Case Stage 2 run with /tmp/workos-case/dist/ca and CTO_HARNESS_CASE_TIMEOUT_SECONDS=8 produced report /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T234205Z-r1-string-slugify-1834617/report.json.
  • Case started provider anthropic with model claude-sonnet-4-6.
  • Adapter-generated backend/case-data/config.json contained no model registry, so Case used its built-in default model.
  • The run timed out before patch application; tests failed because strings.py was unchanged.
  • This is an unadmitted external model path for CTO harness proof and must be blocked before Case process start.

Admitted Provider Runtime Evidence - 2026-06-01

  • Real Case Stage 2 run with /tmp/workos-case/dist/ca and admitted openai-codex / gpt-5.5 produced report /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T013918Z-r1-string-slugify-2381028/report.json.
  • Case process started after admission passed.
  • Backend exit code was 1.
  • The harness recorded no changed files.
  • The patch artifact was empty.
  • Tests failed because the artificial fixture bug remained unchanged.
  • Case stderr recorded AGENT_RESULT start delimiter not found.
  • Case stderr classified the retry as agent-protocol-error.
  • CTO-WORK-016 remains blocked because no real Case Stage 2 pass report exists.
  • Current downstream blocker is tracked by CTO-WORK-028.

CTO-WORK-018 - Case Model Provider Admission Gate

Status: validated.

Harden the Hermes CTO Case adapter so real Case execution cannot silently fall back to its built-in Anthropic default model.

Acceptance:

  • Real Case execution requires CTO_HARNESS_CASE_MODEL_PROVIDER and CTO_HARNESS_CASE_MODEL.
  • Missing model admission writes backend/provider-model-not-admitted.txt.
  • Missing model admission blocks before case_process_started.
  • Adapter writes models.default into isolated CASE_DATA_DIR/config.json.
  • validate-case-provider-adapter.py checks model_provider_gate_blocks.
  • harness/evals/health.sh --json passes after merge.
  • This does not mark real Case Stage 2 as passed.

Validation Evidence:

  • Hermes commit: 4500082 Gate Case execution on admitted model.
  • Post-merge harness/evals/health.sh --json passed.