213 lines
11 KiB
Markdown
213 lines
11 KiB
Markdown
---
|
|
title: CTO Case Agent Protocol Blocker
|
|
status: draft
|
|
lifecycle_classification: sot
|
|
owner: jp
|
|
created: 2026-06-01
|
|
last_reviewed: 2026-06-01
|
|
core_promotion_status: not-promoted
|
|
route: cto
|
|
---
|
|
|
|
# CTO Case Agent Protocol Blocker
|
|
|
|
Local planning SOT only. Not a Core Protocol. Not active Core authority.
|
|
|
|
## CTO-WORK-028 - Case Agent Result Protocol Blocker
|
|
|
|
Status: blocked.
|
|
|
|
Record the first admitted real Case Stage 2 run after OpenAI Codex model admission.
|
|
The run proves that provider/model admission now reaches Case execution, but does
|
|
not prove Stage 2. Case failed before producing a workspace diff because its
|
|
implementer agent result did not satisfy the Case result-envelope contract.
|
|
|
|
The later admitted Qwen local run reproduced the same result-envelope failure
|
|
after Case process start. This makes the active blocker the Case agent-result
|
|
protocol seam, not model admission.
|
|
|
|
Acceptance:
|
|
|
|
- Real Case Stage 2 remains blocked until Case produces a Harness Evidence Interface pass report.
|
|
- The admitted provider/model pair evidence includes `openai-codex` / `gpt-5.5` and `qwen-local` / `qwen3.6-35b-a3b`.
|
|
- The admission files remain `.sot/03-PROTOCOLS/CTO-CASE-MODEL-PROVIDER-ADMISSION.openai-codex-gpt-5.5.json` and `.sot/03-PROTOCOLS/CTO-CASE-MODEL-PROVIDER-ADMISSION.qwen-local-qwen3.6-35b-a3b.json`.
|
|
- Evidence must show `case_process_started: true` before this blocker is accepted as the current blocker.
|
|
- Evidence must show `case_model_admission_status: admitted`.
|
|
- Evidence must show no target repository path was inspected or copied.
|
|
- Evidence must show no workspace patch was produced.
|
|
- Evidence must show tests did not pass.
|
|
- The next implementation route must happen through the Hermes CTO harness seam, a Case-compatible provider adapter seam, or an external compatibility layer.
|
|
- The next implementation route must not mutate Cortex Core, vendor Case source, or external developer repositories.
|
|
- No real-repo, copied-repo, sandbox-repo, owned-repo, default-candidate, or Core promotion stage may use this failed run as pass evidence.
|
|
|
|
## Evidence - 2026-06-01
|
|
|
|
- Harness command class: real Case Stage 2 artificial fixture.
|
|
- Run artifact directory: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T013918Z-r1-string-slugify-2381028`.
|
|
- Case binary path used by harness: `/tmp/workos-case/dist/ca`.
|
|
- Case source pin for the built binary: `7959ac917cdeb0983b4aaa20bb9f42021747fed8`.
|
|
- Report status: `fail`.
|
|
- Backend: `case`.
|
|
- Backend exit code: `1`.
|
|
- Case process started: `true`.
|
|
- Case model provider: `openai-codex`.
|
|
- Case model: `gpt-5.5`.
|
|
- Case model admission status: `admitted`.
|
|
- Source admission status: `not_admitted`.
|
|
- No target inspection proof: `stage2-no-target-inspection.json`.
|
|
- Changed files: none.
|
|
- Patch artifact: `patch.diff`.
|
|
- Patch digest: `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`.
|
|
- Tests command: `python3 -m pytest -q`.
|
|
- Tests passed: `false`.
|
|
- Required events passed: `false`.
|
|
- Report blocker: `case engine failed with exit code 1`.
|
|
- Case stderr evidence: implementer failed with `AGENT_RESULT start delimiter not found`.
|
|
- Case stderr evidence: retry classified the failure as `agent-protocol-error`.
|
|
- Case stdout evidence: unattended mode auto-selected `Abort`.
|
|
- Result: Stage 2 is still blocked.
|
|
|
|
## Qwen Local Evidence - 2026-06-01
|
|
|
|
- Harness command class: real Case Stage 2 artificial fixture.
|
|
- Run artifact directory: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T015208Z-r1-string-slugify-2478256`.
|
|
- Case binary path used by harness: `/tmp/workos-case/dist/ca`.
|
|
- Case source pin for the built binary: `7959ac917cdeb0983b4aaa20bb9f42021747fed8`.
|
|
- Report status: `fail`.
|
|
- Backend: `case`.
|
|
- Backend exit code: `1`.
|
|
- Case process started: `true`.
|
|
- Case model provider: `qwen-local`.
|
|
- Case model: `qwen3.6-35b-a3b`.
|
|
- Case model admission status: `admitted`.
|
|
- Source admission status: `not_admitted`.
|
|
- No target inspection proof: `stage2-no-target-inspection.json`.
|
|
- Changed files: none.
|
|
- Patch artifact: `patch.diff`.
|
|
- Patch digest: `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`.
|
|
- Tests command: `python3 -m pytest -q`.
|
|
- Tests passed: `false`.
|
|
- Required events passed: `false`.
|
|
- Report blocker: `case engine failed with exit code 1`.
|
|
- Case stderr evidence: implementer failed with `AGENT_RESULT start delimiter not found`.
|
|
- Case stderr evidence: retry classified the failure as `agent-protocol-error`.
|
|
- Case stdout evidence: unattended mode auto-selected `Abort`.
|
|
- Result: Stage 2 is still blocked.
|
|
|
|
## Current Interpretation
|
|
|
|
This is a protocol compatibility blocker, not a provider approval or model
|
|
admission blocker.
|
|
|
|
Two admitted provider/model paths reached Case. Case then failed because the
|
|
implementer agent did not return output framed by the Case `AGENT_RESULT`
|
|
delimiter contract. The evidence does not prove whether the defect is Case
|
|
provider configuration, provider adapter behavior, model output framing, or
|
|
harness invocation shape.
|
|
|
|
## Narrowed Interpretation - 2026-06-01
|
|
|
|
Hermes commit `5db23c7 Fail closed on Case Codex auth gap` narrows the current
|
|
OpenAI Codex path blocker.
|
|
|
|
The current known OpenAI Codex blocker is an auth-bridge gap:
|
|
|
|
- Case's pipeline SDK path constructs its Pi Agent runtime directly.
|
|
- That path does not pass Pi AuthStorage OAuth headers into `streamSimple`.
|
|
- Pi env API-key lookup does not map `openai-codex` to an environment API key.
|
|
- Hermes now blocks `openai-codex` before `case_process_started` unless an explicit non-vendor auth bridge is proven.
|
|
|
|
This does not prove that any local provider path passes Stage 2. It only prevents
|
|
repeating the misleading malformed-output run for the `openai-codex` path.
|
|
|
|
## Required Next Route
|
|
|
|
The next useful route is a small Case agent protocol compatibility investigation.
|
|
It should answer only this question:
|
|
|
|
```text
|
|
What minimal non-vendor seam makes admitted Case execution return the required
|
|
AGENT_RESULT envelope and produce a Stage 2 artificial fixture diff?
|
|
```
|
|
|
|
Allowed next actions:
|
|
|
|
- Inspect Case provider adapter behavior read-only.
|
|
- Inspect Hermes CTO Case invocation behavior.
|
|
- Add fail-closed classification in Hermes CTO harness if needed.
|
|
- Add a compatibility shim only outside vendor Case source.
|
|
- Admit and test the existing Pi local provider id `qwen-local` only through the Harness Evidence Interface.
|
|
- Re-run real Case Stage 2 only after a specific protocol compatibility change exists.
|
|
|
|
Forbidden next actions:
|
|
|
|
- Do not edit `/tmp/workos-case` as the durable solution.
|
|
- Do not mark Stage 2 validated from this run.
|
|
- Do not promote Case to copied repo, sandbox repo, owned repo, or default candidate.
|
|
- Do not write provider secrets to SOT, argv, task files, backend logs, reports, traces, or commits.
|
|
|
|
## Hermes Classifier Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `48d487a Classify Case agent protocol failures`.
|
|
- Hermes commit: `798fb5a Harden Case protocol failure marker`.
|
|
- `48d487a` adds a CTO Harness `AGENT_RESULT` protocol appendix to the Case task markdown.
|
|
- `48d487a` adds a fail-closed protocol marker path: `backend/provider-agent-protocol.txt`.
|
|
- `798fb5a` hardens report-time classification so missing `AGENT_RESULT` evidence is recorded even if backend marker creation is missed.
|
|
- Focused validator passed: `python3 harness/runner/validate-case-provider-adapter.py --harness-root harness --json`.
|
|
- Aggregate validator passed: `harness/evals/health.sh --json`.
|
|
- Post-merge aggregate validator passed: `harness/evals/health.sh --json`.
|
|
|
|
## Qwen Local Classified Runtime Evidence - 2026-06-01
|
|
|
|
- Run artifact directory: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020117Z-r1-string-slugify-2566310`.
|
|
- Report status: `fail`.
|
|
- Backend: `case`.
|
|
- Backend exit code: `1`.
|
|
- Case process started: `true`.
|
|
- Case model provider: `qwen-local`.
|
|
- Case model: `qwen3.6-35b-a3b`.
|
|
- Case model admission status: `admitted`.
|
|
- Failure reason: `case agent result protocol failed`.
|
|
- Protocol marker: `backend/provider-agent-protocol.txt`.
|
|
- Changed files: none.
|
|
- Patch artifact: `patch.diff`.
|
|
- Patch digest: `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`.
|
|
- Tests passed: `false`.
|
|
- Required events passed: `false`.
|
|
- Result: Stage 2 is still blocked.
|
|
- Current next route remains a Case/Pi runtime protocol compatibility fix, not another admission record.
|
|
|
|
## Isolated Pi Config Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `09b5851 Isolate Case Pi provider config`.
|
|
- The Hermes CTO Case harness now sets `PI_CODING_AGENT_DIR` under the run artifact directory before invoking Case.
|
|
- The harness writes isolated Pi auth state under `backend/case-data/pi-agent/auth.json`.
|
|
- Local Case providers now require explicit `CTO_HARNESS_CASE_LOCAL_BASE_URL`.
|
|
- Missing local provider config writes `backend/provider-local-config-unavailable.txt`.
|
|
- Missing local provider config blocks before `case_process_started`.
|
|
- Focused validator passed: `python3 harness/runner/validate-case-provider-adapter.py --harness-root harness --json`.
|
|
- Focused validator artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020732Z-r1-string-slugify-2609546`.
|
|
- Aggregate validator passed before merge: `harness/evals/health.sh --json`.
|
|
- Aggregate validator artifact before merge: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020741Z-r1-string-slugify-2611203`.
|
|
- Post-merge aggregate validator passed: `harness/evals/health.sh --json`.
|
|
- Post-merge aggregate validator artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020801Z-r1-string-slugify-2613843`.
|
|
- Real Qwen local config-gate proof artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020847Z-r1-string-slugify-2619644`.
|
|
- Real Qwen local config-gate status: `blocked`.
|
|
- Real Qwen local config-gate failure reason: `provider unavailable`.
|
|
- Real Qwen local config-gate Case process started: `false`.
|
|
- This removes the ambient `~/.pi/agent` dependency from the harness path.
|
|
- This does not prove Case can produce the required `AGENT_RESULT` envelope.
|
|
- `CTO-WORK-028` remains blocked until a configured local provider or another admitted provider returns a valid result envelope and produces a Stage 2 artificial fixture diff.
|
|
|
|
## Spark Endpoint Config Dependency - 2026-06-01
|
|
|
|
- `CTO-WORK-030` must be resolved before another configured Qwen local run can retest the Case `AGENT_RESULT` protocol path.
|
|
- Until `CTO_HARNESS_CASE_LOCAL_BASE_URL` is supplied, the harness blocks before Case starts.
|
|
- The agent protocol blocker remains unproven for the isolated Spark endpoint path until Case reaches execution again and returns or fails the required result envelope.
|
|
|
|
## Case Qwen Loop Entrypoint Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `6c453ee Add Case Qwen loop entrypoint`.
|
|
- The new `harness/evals/case-qwen-loop.sh --json` command is the next standard path for retesting the Case `AGENT_RESULT` protocol after Spark endpoint config is supplied.
|
|
- This does not resolve the protocol blocker because Case has not reached execution through the configured isolated Spark endpoint path yet.
|