178 lines
8.8 KiB
Markdown
178 lines
8.8 KiB
Markdown
---
|
|
title: CTO Case Provider Build Issues
|
|
status: draft
|
|
lifecycle_classification: sot
|
|
owner: jp
|
|
created: 2026-05-31
|
|
last_reviewed: 2026-06-01
|
|
core_promotion_status: not-promoted
|
|
route: cto
|
|
---
|
|
|
|
# CTO Case Provider Build Issues
|
|
|
|
Local planning SOT only. Not a Core Protocol. Not active Core authority.
|
|
|
|
## CTO-WORK-015 - Durable Case Provider Build PRD
|
|
|
|
Status: validated.
|
|
|
|
Register the provider build route that separates source discovery from executable admission.
|
|
|
|
Acceptance:
|
|
|
|
- Records WorkOS Case source `https://github.com/workos/case.git`.
|
|
- Records pinned commit `7959ac917cdeb0983b4aaa20bb9f42021747fed8`.
|
|
- Records local build dependency status: Node `v20.19.5` present, Bun missing.
|
|
- Requires `bun run build:binary` to produce `dist/ca` before real Stage 2 can pass.
|
|
- Requires SHA-256 digest for admitted `dist/ca`.
|
|
- Keeps `/tmp` clone as discovery evidence only.
|
|
|
|
## CTO-WORK-016 - Real Case Provider Stage 2 Run
|
|
|
|
Status: blocked.
|
|
|
|
Build or supply a durable admitted WorkOS Case `ca` executable, then run the existing Hermes CTO Stage 2 artificial fixture through real Case.
|
|
|
|
Acceptance:
|
|
|
|
- Bun is available, or a durable admitted `dist/ca` path is supplied with source pin and digest.
|
|
- `CTO_HARNESS_CASE_BIN` points to the admitted `ca` executable.
|
|
- Stage 2 invokes `ca run --task <task-file> --mode unattended`.
|
|
- Real Case execution produces a Stage 2 pass report through the Harness Evidence Interface.
|
|
- Provider-unavailable fail-closed behavior remains tested.
|
|
- Fake remains the default validation lane.
|
|
- No Target Repository path is inspected or copied.
|
|
|
|
Blocked by:
|
|
|
|
- Missing local `bun` executable, unless a prebuilt admitted `dist/ca` is supplied.
|
|
|
|
## Provider Build Evidence - 2026-05-31
|
|
|
|
- Reversible Bun bootstrap succeeded under `/tmp/cto-bun`; no global Bun install was required.
|
|
- `/tmp/workos-case` source pin remained `7959ac917cdeb0983b4aaa20bb9f42021747fed8`.
|
|
- `bun run build:binary` produced `/tmp/workos-case/dist/ca`.
|
|
- Built binary digest: `9811f870af2f85616e359d42ba70566c9af08ca20d8660456929a56ec761513f`.
|
|
- `/tmp/workos-case/dist/ca --help` identified the executable as Case `ca`.
|
|
- `/tmp/workos-case` remains discovery/build evidence only because build modified `src/generated/package-assets.ts`.
|
|
- Real Stage 2 run with `CTO_HARNESS_CASE_BIN=/tmp/workos-case/dist/ca` hung and was terminated. It created artifact directory `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T233721Z-r1-string-slugify-1814067` without `report.json`.
|
|
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
|
|
|
|
## CTO-WORK-017 - Case Provider Timeout Fail-Closed Route
|
|
|
|
Status: validated.
|
|
|
|
Harden the Hermes CTO Case adapter so a hung real Case provider fails closed with normal evidence instead of leaving an orphan artifact directory without `report.json`.
|
|
|
|
Acceptance:
|
|
|
|
- Case provider execution is bounded by `CTO_HARNESS_CASE_TIMEOUT_SECONDS`.
|
|
- Timeout writes `backend/provider-timeout.txt`.
|
|
- Timeout still emits `report.json` through the Harness Evidence Interface.
|
|
- `validate-case-provider-adapter.py` checks a sleeping provider path.
|
|
- `harness/evals/health.sh --json` passes after merge and includes `provider_timeout_fail_closed`.
|
|
- This does not mark real Case Stage 2 as passed.
|
|
|
|
Validation Evidence:
|
|
|
|
- Hermes commit: `d23c492 Fail closed on Case provider timeout`.
|
|
- Post-merge `harness/evals/health.sh --json` passed.
|
|
|
|
## Real Provider Runtime Evidence - 2026-05-31
|
|
|
|
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and `CTO_HARNESS_CASE_TIMEOUT_SECONDS=8` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T234205Z-r1-string-slugify-1834617/report.json`.
|
|
- Case started provider `anthropic` with model `claude-sonnet-4-6`.
|
|
- Adapter-generated `backend/case-data/config.json` contained no model registry, so Case used its built-in default model.
|
|
- The run timed out before patch application; tests failed because `strings.py` was unchanged.
|
|
- This is an unadmitted external model path for CTO harness proof and must be blocked before Case process start.
|
|
|
|
## Admitted Provider Runtime Evidence - 2026-06-01
|
|
|
|
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and admitted `openai-codex` / `gpt-5.5` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T013918Z-r1-string-slugify-2381028/report.json`.
|
|
- Case process started after admission passed.
|
|
- Backend exit code was `1`.
|
|
- The harness recorded no changed files.
|
|
- The patch artifact was empty.
|
|
- Tests failed because the artificial fixture bug remained unchanged.
|
|
- Case stderr recorded `AGENT_RESULT start delimiter not found`.
|
|
- Case stderr classified the retry as `agent-protocol-error`.
|
|
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
|
|
- Current downstream blocker is tracked by `CTO-WORK-028`.
|
|
|
|
## Qwen Local Provider Runtime Evidence - 2026-06-01
|
|
|
|
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and admitted `qwen-local` / `qwen3.6-35b-a3b` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T015208Z-r1-string-slugify-2478256/report.json`.
|
|
- Case process started after admission passed.
|
|
- Backend exit code was `1`.
|
|
- The harness recorded no changed files.
|
|
- The patch artifact was empty.
|
|
- Tests failed because the artificial fixture bug remained unchanged.
|
|
- Case stderr recorded `AGENT_RESULT start delimiter not found`.
|
|
- Case stderr classified the retry as `agent-protocol-error`.
|
|
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
|
|
- Current downstream blocker remains `CTO-WORK-028`.
|
|
|
|
## Classified Agent Protocol Runtime Evidence - 2026-06-01
|
|
|
|
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and admitted `qwen-local` / `qwen3.6-35b-a3b` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020117Z-r1-string-slugify-2566310/report.json`.
|
|
- Case process started after admission passed.
|
|
- Backend exit code was `1`.
|
|
- Failure reason was `case agent result protocol failed`.
|
|
- Protocol marker was recorded at `backend/provider-agent-protocol.txt`.
|
|
- The harness recorded no changed files.
|
|
- The patch artifact was empty.
|
|
- Tests failed because the artificial fixture bug remained unchanged.
|
|
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
|
|
- Current downstream blocker remains `CTO-WORK-028`.
|
|
|
|
## Isolated Pi Config Runtime Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `09b5851 Isolate Case Pi provider config`.
|
|
- The Case harness now sets an isolated `PI_CODING_AGENT_DIR` for the Case process.
|
|
- The Case harness no longer relies on ambient `~/.pi/agent` model/auth state for local provider routing.
|
|
- Local providers require explicit `CTO_HARNESS_CASE_LOCAL_BASE_URL`.
|
|
- Missing local provider config writes `backend/provider-local-config-unavailable.txt`.
|
|
- Real Qwen local config-gate proof artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020847Z-r1-string-slugify-2619644`.
|
|
- Report status was `blocked`.
|
|
- Failure reason was `provider unavailable`.
|
|
- Case process started was `false`.
|
|
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
|
|
- Current downstream blocker remains `CTO-WORK-028`.
|
|
|
|
## CTO-WORK-018 - Case Model Provider Admission Gate
|
|
|
|
Status: validated.
|
|
|
|
Harden the Hermes CTO Case adapter so real Case execution cannot silently fall back to its built-in Anthropic default model.
|
|
|
|
Acceptance:
|
|
|
|
- Real Case execution requires `CTO_HARNESS_CASE_MODEL_PROVIDER` and `CTO_HARNESS_CASE_MODEL`.
|
|
- Missing model admission writes `backend/provider-model-not-admitted.txt`.
|
|
- Missing model admission blocks before `case_process_started`.
|
|
- Adapter writes `models.default` into isolated `CASE_DATA_DIR/config.json`.
|
|
- `validate-case-provider-adapter.py` checks `model_provider_gate_blocks`.
|
|
- `harness/evals/health.sh --json` passes after merge.
|
|
- This does not mark real Case Stage 2 as passed.
|
|
|
|
Validation Evidence:
|
|
|
|
- Hermes commit: `4500082 Gate Case execution on admitted model`.
|
|
- Post-merge `harness/evals/health.sh --json` passed.
|
|
|
|
## Spark Endpoint Config Blocker - 2026-06-01
|
|
|
|
- `CTO-WORK-030` tracks the missing Spark endpoint config for the selected `qwen-local` / `qwen3.6-35b-a3b` path.
|
|
- The Hermes harness requires `CTO_HARNESS_CASE_LOCAL_BASE_URL` before local Case provider execution can start.
|
|
- Missing endpoint config blocks before `case_process_started` and writes `backend/provider-local-config-unavailable.txt`.
|
|
- `CTO-WORK-016` remains blocked until real Case Stage 2 pass evidence exists through the Harness Evidence Interface.
|
|
|
|
## Case Qwen Loop Entrypoint Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `6c453ee Add Case Qwen loop entrypoint`.
|
|
- New command: `harness/evals/case-qwen-loop.sh --json`.
|
|
- The command gives the selected Qwen local path a standard Case harness entrypoint.
|
|
- It does not remove the `CTO_HARNESS_CASE_LOCAL_BASE_URL` blocker.
|
|
- `CTO-WORK-016` remains blocked until real Case Stage 2 pass evidence exists.
|