hermes/cto
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
99b5353e51
cto/.sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-ISSUES.md
Svrnty 99b5353e51 Record Spark vLLM Case evidence
2026-05-31 22:26:39 -04:00

192 lines
9.6 KiB
Markdown
Raw Blame History

---
title: CTO Case Provider Build Issues
status: draft
lifecycle_classification: sot
owner: jp
created: 2026-05-31
last_reviewed: 2026-06-01
core_promotion_status: not-promoted
route: cto
---
# CTO Case Provider Build Issues
Local planning SOT only. Not a Core Protocol. Not active Core authority.
## CTO-WORK-015 - Durable Case Provider Build PRD
Status: validated.
Register the provider build route that separates source discovery from executable admission.
Acceptance:
- Records WorkOS Case source `https://github.com/workos/case.git`.
- Records pinned commit `7959ac917cdeb0983b4aaa20bb9f42021747fed8`.
- Records local build dependency status: Node `v20.19.5` present, Bun missing.
- Requires `bun run build:binary` to produce `dist/ca` before real Stage 2 can pass.
- Requires SHA-256 digest for admitted `dist/ca`.
- Keeps `/tmp` clone as discovery evidence only.
## CTO-WORK-016 - Real Case Provider Stage 2 Run
Status: blocked.
Build or supply a durable admitted WorkOS Case `ca` executable, then run the existing Hermes CTO Stage 2 artificial fixture through real Case.
Acceptance:
- Bun is available, or a durable admitted `dist/ca` path is supplied with source pin and digest.
- `CTO_HARNESS_CASE_BIN` points to the admitted `ca` executable.
- Stage 2 invokes `ca run --task <task-file> --mode unattended`.
- Real Case execution produces a Stage 2 pass report through the Harness Evidence Interface.
- Provider-unavailable fail-closed behavior remains tested.
- Fake remains the default validation lane.
- No Target Repository path is inspected or copied.
Blocked by:
- Missing local `bun` executable, unless a prebuilt admitted `dist/ca` is supplied.
## Provider Build Evidence - 2026-05-31
- Reversible Bun bootstrap succeeded under `/tmp/cto-bun`; no global Bun install was required.
- `/tmp/workos-case` source pin remained `7959ac917cdeb0983b4aaa20bb9f42021747fed8`.
- `bun run build:binary` produced `/tmp/workos-case/dist/ca`.
- Built binary digest: `9811f870af2f85616e359d42ba70566c9af08ca20d8660456929a56ec761513f`.
- `/tmp/workos-case/dist/ca --help` identified the executable as Case `ca`.
- `/tmp/workos-case` remains discovery/build evidence only because build modified `src/generated/package-assets.ts`.
- Real Stage 2 run with `CTO_HARNESS_CASE_BIN=/tmp/workos-case/dist/ca` hung and was terminated. It created artifact directory `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T233721Z-r1-string-slugify-1814067` without `report.json`.
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
## CTO-WORK-017 - Case Provider Timeout Fail-Closed Route
Status: validated.
Harden the Hermes CTO Case adapter so a hung real Case provider fails closed with normal evidence instead of leaving an orphan artifact directory without `report.json`.
Acceptance:
- Case provider execution is bounded by `CTO_HARNESS_CASE_TIMEOUT_SECONDS`.
- Timeout writes `backend/provider-timeout.txt`.
- Timeout still emits `report.json` through the Harness Evidence Interface.
- `validate-case-provider-adapter.py` checks a sleeping provider path.
- `harness/evals/health.sh --json` passes after merge and includes `provider_timeout_fail_closed`.
- This does not mark real Case Stage 2 as passed.
Validation Evidence:
- Hermes commit: `d23c492 Fail closed on Case provider timeout`.
- Post-merge `harness/evals/health.sh --json` passed.
## Real Provider Runtime Evidence - 2026-05-31
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and `CTO_HARNESS_CASE_TIMEOUT_SECONDS=8` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T234205Z-r1-string-slugify-1834617/report.json`.
- Case started provider `anthropic` with model `claude-sonnet-4-6`.
- Adapter-generated `backend/case-data/config.json` contained no model registry, so Case used its built-in default model.
- The run timed out before patch application; tests failed because `strings.py` was unchanged.
- This is an unadmitted external model path for CTO harness proof and must be blocked before Case process start.
## Admitted Provider Runtime Evidence - 2026-06-01
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and admitted `openai-codex` / `gpt-5.5` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T013918Z-r1-string-slugify-2381028/report.json`.
- Case process started after admission passed.
- Backend exit code was `1`.
- The harness recorded no changed files.
- The patch artifact was empty.
- Tests failed because the artificial fixture bug remained unchanged.
- Case stderr recorded `AGENT_RESULT start delimiter not found`.
- Case stderr classified the retry as `agent-protocol-error`.
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
- Current downstream blocker is tracked by `CTO-WORK-028`.
## Qwen Local Provider Runtime Evidence - 2026-06-01
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and admitted `qwen-local` / `qwen3.6-35b-a3b` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T015208Z-r1-string-slugify-2478256/report.json`.
- Case process started after admission passed.
- Backend exit code was `1`.
- The harness recorded no changed files.
- The patch artifact was empty.
- Tests failed because the artificial fixture bug remained unchanged.
- Case stderr recorded `AGENT_RESULT start delimiter not found`.
- Case stderr classified the retry as `agent-protocol-error`.
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
- Current downstream blocker remains `CTO-WORK-028`.
## Classified Agent Protocol Runtime Evidence - 2026-06-01
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and admitted `qwen-local` / `qwen3.6-35b-a3b` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020117Z-r1-string-slugify-2566310/report.json`.
- Case process started after admission passed.
- Backend exit code was `1`.
- Failure reason was `case agent result protocol failed`.
- Protocol marker was recorded at `backend/provider-agent-protocol.txt`.
- The harness recorded no changed files.
- The patch artifact was empty.
- Tests failed because the artificial fixture bug remained unchanged.
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
- Current downstream blocker remains `CTO-WORK-028`.
## Isolated Pi Config Runtime Evidence - 2026-06-01
- Hermes commit: `09b5851 Isolate Case Pi provider config`.
- The Case harness now sets an isolated `PI_CODING_AGENT_DIR` for the Case process.
- The Case harness no longer relies on ambient `~/.pi/agent` model/auth state for local provider routing.
- Local providers require explicit `CTO_HARNESS_CASE_LOCAL_BASE_URL`.
- Missing local provider config writes `backend/provider-local-config-unavailable.txt`.
- Real Qwen local config-gate proof artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020847Z-r1-string-slugify-2619644`.
- Report status was `blocked`.
- Failure reason was `provider unavailable`.
- Case process started was `false`.
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
- Current downstream blocker remains `CTO-WORK-028`.
## CTO-WORK-018 - Case Model Provider Admission Gate
Status: validated.
Harden the Hermes CTO Case adapter so real Case execution cannot silently fall back to its built-in Anthropic default model.
Acceptance:
- Real Case execution requires `CTO_HARNESS_CASE_MODEL_PROVIDER` and `CTO_HARNESS_CASE_MODEL`.
- Missing model admission writes `backend/provider-model-not-admitted.txt`.
- Missing model admission blocks before `case_process_started`.
- Adapter writes `models.default` into isolated `CASE_DATA_DIR/config.json`.
- `validate-case-provider-adapter.py` checks `model_provider_gate_blocks`.
- `harness/evals/health.sh --json` passes after merge.
- This does not mark real Case Stage 2 as passed.
Validation Evidence:
- Hermes commit: `4500082 Gate Case execution on admitted model`.
- Post-merge `harness/evals/health.sh --json` passed.
## Spark Endpoint Config Blocker - 2026-06-01
- `CTO-WORK-030` tracks the missing Spark endpoint config for the selected `qwen-local` / `qwen3.6-35b-a3b` path.
- The Hermes harness requires `CTO_HARNESS_CASE_LOCAL_BASE_URL` before local Case provider execution can start.
- Missing endpoint config blocks before `case_process_started` and writes `backend/provider-local-config-unavailable.txt`.
- `CTO-WORK-016` remains blocked until real Case Stage 2 pass evidence exists through the Harness Evidence Interface.
## Case Qwen Loop Entrypoint Evidence - 2026-06-01
- Hermes commit: `6c453ee Add Case Qwen loop entrypoint`.
- New command: `harness/evals/case-qwen-loop.sh --json`.
- The command gives the selected Qwen local path a standard Case harness entrypoint.
- It does not remove the `CTO_HARNESS_CASE_LOCAL_BASE_URL` blocker.
- `CTO-WORK-016` remains blocked until real Case Stage 2 pass evidence exists.
## Spark vLLM Runtime Evidence - 2026-06-01
- Runtime supplied a Spark OpenAI-compatible vLLM endpoint without recording the endpoint value in SOT.
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and admitted `qwen-local` / `qwen3.6-35b-a3b` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T022535Z-r1-string-slugify-2731603/report.json`.
- Case process started after admission passed.
- Backend exit code was `1`.
- Failure reason was `case agent result protocol failed`.
- Protocol marker was recorded at `backend/provider-agent-protocol.txt`.
- The harness recorded no changed files.
- The patch artifact was empty.
- Tests failed because the artificial fixture bug remained unchanged.
- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists.
- Current downstream blocker remains `CTO-WORK-028`.
Reference in New Issue View Git Blame Copy Permalink