From fca300afa79d6a11c4e7daa588af59a4865ca643 Mon Sep 17 00:00:00 2001 From: Svrnty Date: Sun, 31 May 2026 22:33:12 -0400 Subject: [PATCH] Record Case response shape blocker --- .../CTO-CASE-AGENT-PROTOCOL-BLOCKER.md | 63 +++++++++++++++++++ .../CTO-CASE-PROVIDER-BUILD-ISSUES.md | 14 +++++ .../CTO-CASE-SPARK-ENDPOINT-CONFIG-ISSUES.md | 18 ++++++ WORKBOARD.yaml | 5 ++ 4 files changed, 100 insertions(+) diff --git a/.sot/03-PROTOCOLS/CTO-CASE-AGENT-PROTOCOL-BLOCKER.md b/.sot/03-PROTOCOLS/CTO-CASE-AGENT-PROTOCOL-BLOCKER.md index 11f6a12..8d9722a 100644 --- a/.sot/03-PROTOCOLS/CTO-CASE-AGENT-PROTOCOL-BLOCKER.md +++ b/.sot/03-PROTOCOLS/CTO-CASE-AGENT-PROTOCOL-BLOCKER.md @@ -234,3 +234,66 @@ Forbidden next actions: - Required events passed: `false`. - Result: Stage 2 is still blocked. - Current next route is a Case/Pi result-envelope compatibility fix outside vendor Case source. + +## Narrowed Response Shape Interpretation - 2026-06-01 + +Hermes commit `974813b Block Case on reasoning-only local provider` narrows the +Qwen local blocker. + +Observed: + +- Spark vLLM accepts OpenAI-compatible chat-completions requests for `qwen3.6-35b-a3b`. +- The response can contain a `reasoning` field with `content: null`. +- Case/Pi only turns assistant text deltas into the raw text parsed by `AGENT_RESULT`. +- A reasoning-only response therefore reaches no valid `AGENT_RESULT` envelope. + +Harness effect: + +- The Case Qwen loop now probes local provider response shape before Case process start. +- Reasoning-only local responses write `backend/provider-reasoning-only.txt`. +- Reasoning-only local responses report `failure_reason: provider response shape unavailable`. +- Reasoning-only local responses block with `case_process_started: false`. + +Latest evidence: + +- Run artifact directory: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T023119Z-r1-string-slugify-2759949`. +- Report status: `blocked`. +- Failure reason: `provider response shape unavailable`. +- Marker: `backend/provider-reasoning-only.txt`. +- Case process started: `false`. +- Case model provider: `qwen-local`. +- Case model: `qwen3.6-35b-a3b`. +- Case model admission status: `admitted`. +- Result: Stage 2 is still blocked. + +## CTO-WORK-031 - Case Local Provider Response Shape Shim + +Status: blocked. + +Create a non-vendor compatibility route that makes the admitted local Qwen path +return assistant content usable by Case/Pi result parsing, without weakening +Harness evidence gates. + +Acceptance: + +- No vendor Case source is mutated as the durable solution. +- No endpoint value or credential value is recorded in SOT, argv examples, task files, backend logs, reports, traces, generated config, or commits. +- The shim is outside Cortex Core and outside target repositories. +- Reasoning-only responses remain fail-closed before Case process start. +- A configured local provider can produce assistant content, not only reasoning. +- Case reaches execution only after admission and response-shape checks pass. +- Real Case Stage 2 pass evidence exists through the Harness Evidence Interface. +- Same-run fake baseline comparison remains required for any pass claim. +- No copied-repo, sandbox-repo, owned-repo, default-candidate, or Core promotion stage uses a response-shape blocked run as pass evidence. + +Allowed routes: + +- vLLM serving configuration that disables reasoning-only output for this model. +- A local OpenAI-compatible proxy that converts or requests usable assistant content. +- A Hermes CTO harness adapter setting that is proven by focused validator and aggregate health. + +Forbidden routes: + +- Do not patch `/tmp/workos-case` as the durable fix. +- Do not make Case default before Stage 2 pass evidence. +- Do not treat reasoning text as a completed `AGENT_RESULT` unless a governed adapter proves the result envelope and file diff. diff --git a/.sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-ISSUES.md b/.sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-ISSUES.md index dc5b3d1..aa15143 100644 --- a/.sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-ISSUES.md +++ b/.sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-ISSUES.md @@ -126,6 +126,20 @@ Validation Evidence: - `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists. - Current downstream blocker remains `CTO-WORK-028`. +## Response Shape Gate Evidence - 2026-06-01 + +- Hermes commit: `974813b Block Case on reasoning-only local provider`. +- The standard Case Qwen loop now probes the local OpenAI-compatible endpoint before starting Case. +- Real Case Stage 2 retry with admitted `qwen-local` / `qwen3.6-35b-a3b` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T023119Z-r1-string-slugify-2759949/report.json`. +- Report status was `blocked`. +- Failure reason was `provider response shape unavailable`. +- Marker was recorded at `backend/provider-reasoning-only.txt`. +- Case process started was `false`. +- The harness recorded no changed files. +- The patch artifact was empty. +- `CTO-WORK-016` remains blocked because no real Case Stage 2 pass report exists. +- Current downstream blocker is `CTO-WORK-031`. + ## Isolated Pi Config Runtime Evidence - 2026-06-01 - Hermes commit: `09b5851 Isolate Case Pi provider config`. diff --git a/.sot/03-PROTOCOLS/CTO-CASE-SPARK-ENDPOINT-CONFIG-ISSUES.md b/.sot/03-PROTOCOLS/CTO-CASE-SPARK-ENDPOINT-CONFIG-ISSUES.md index 65e84e9..2c7d6b4 100644 --- a/.sot/03-PROTOCOLS/CTO-CASE-SPARK-ENDPOINT-CONFIG-ISSUES.md +++ b/.sot/03-PROTOCOLS/CTO-CASE-SPARK-ENDPOINT-CONFIG-ISSUES.md @@ -69,6 +69,24 @@ Current evidence: - This does not validate `CTO-WORK-016`, `CTO-WORK-020`, `CTO-WORK-022`, or `CTO-WORK-028`. - Current active blocker is the Case agent-result protocol seam. +## Spark Response Shape Gate Evidence - 2026-06-01 + +- Hermes commit: `974813b Block Case on reasoning-only local provider`. +- The Hermes Case Qwen loop now enables a local provider response-shape probe before Case process start. +- The probe classifies OpenAI-compatible responses that contain reasoning but no assistant content. +- Runtime endpoint value was supplied only through environment and is not recorded in SOT. +- Run artifact directory: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T023119Z-r1-string-slugify-2759949`. +- Report status: `blocked`. +- Failure reason: `provider response shape unavailable`. +- Marker: `backend/provider-reasoning-only.txt`. +- Case process started: `false`. +- Case model provider: `qwen-local`. +- Case model: `qwen3.6-35b-a3b`. +- Case model admission status: `admitted`. +- This proves Spark endpoint config is reachable but response-shape incompatible with Case/Pi result parsing. +- `CTO-WORK-030` remains blocked until a configured endpoint can support a real Stage 2 pass. +- Current downstream blocker is `CTO-WORK-031`. + ## Hermes Case Qwen Loop Evidence - 2026-06-01 - Hermes commit: `6c453ee Add Case Qwen loop entrypoint`. diff --git a/WORKBOARD.yaml b/WORKBOARD.yaml index ad33ff8..62da33d 100644 --- a/WORKBOARD.yaml +++ b/WORKBOARD.yaml @@ -150,3 +150,8 @@ items: status: blocked source: .sot/03-PROTOCOLS/CTO-CASE-SPARK-ENDPOINT-CONFIG-ISSUES.md owner: jp + - id: CTO-WORK-031 + title: Case Local Provider Response Shape Shim + status: blocked + source: .sot/03-PROTOCOLS/CTO-CASE-AGENT-PROTOCOL-BLOCKER.md + owner: jp