140 lines
7.7 KiB
Markdown
140 lines
7.7 KiB
Markdown
---
|
|
title: CTO Case Spark Endpoint Config Issues
|
|
status: draft
|
|
lifecycle_classification: sot
|
|
owner: jp
|
|
created: 2026-06-01
|
|
last_reviewed: 2026-06-01
|
|
core_promotion_status: not-promoted
|
|
route: cto
|
|
---
|
|
|
|
# CTO Case Spark Endpoint Config Issues
|
|
|
|
Local planning SOT only. Not a Core Protocol. Not active Core authority.
|
|
|
|
## CTO-WORK-030 - Spark Local Provider Endpoint Config
|
|
|
|
Status: blocked.
|
|
|
|
Supply explicit Spark OpenAI-compatible endpoint config for the selected `qwen-local` / `qwen3.6-35b-a3b` Case path, without storing endpoint secrets in SOT or commits.
|
|
|
|
Acceptance:
|
|
|
|
- `CTO_HARNESS_CASE_LOCAL_BASE_URL` is supplied at runtime for the real Qwen Case Stage 2 retry.
|
|
- Missing endpoint config blocks before `case_process_started`.
|
|
- Missing endpoint config writes `backend/provider-local-config-unavailable.txt`.
|
|
- Endpoint values are not written to SOT, task file, argv, backend logs, report, trace, generated config, or commit.
|
|
- Harness report proves `case_model_provider`: `qwen-local`.
|
|
- Harness report proves `case_model`: `qwen3.6-35b-a3b`.
|
|
- Harness report proves `case_model_admission_status`: `admitted`.
|
|
- Harness report proves no fallback to `anthropic` or `claude-sonnet-4-6`.
|
|
- Real Case Stage 2 pass evidence exists only through the Harness Evidence Interface.
|
|
- A configured endpoint alone does not validate `CTO-WORK-016`, `CTO-WORK-020`, `CTO-WORK-022`, or `CTO-WORK-028`.
|
|
- No Target Repository path may be inspected or copied.
|
|
- Same-run fake baseline comparison remains required for any pass claim.
|
|
- `CTO-WORK-016` remains blocked until real Case Stage 2 pass evidence exists.
|
|
- `CTO-WORK-022` remains blocked until explicit endpoint config and real Case Stage 2 pass evidence exist.
|
|
|
|
Blocked by:
|
|
|
|
- Real Case Stage 2 pass evidence.
|
|
- Case agent-result protocol compatibility.
|
|
|
|
Current evidence:
|
|
|
|
- Non-secret readiness check on 2026-06-01 showed `CTO_HARNESS_CASE_LOCAL_BASE_URL=missing`.
|
|
- Existing Hermes harness blocks missing local provider config before `case_process_started`.
|
|
|
|
## Spark vLLM Runtime Evidence - 2026-06-01
|
|
|
|
- Spark1 was reachable over Tailscale and SSH.
|
|
- A vLLM Docker container exposed an OpenAI-compatible model route for `qwen3.6-35b-a3b`.
|
|
- The Spark endpoint value was supplied only through runtime environment and is not recorded in this SOT.
|
|
- The standard command class was `harness/evals/case-qwen-loop.sh --json`.
|
|
- Run artifact directory: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T022535Z-r1-string-slugify-2731603`.
|
|
- Report status: `fail`.
|
|
- Backend: `case`.
|
|
- Backend exit code: `1`.
|
|
- Case process started: `true`.
|
|
- Case model provider: `qwen-local`.
|
|
- Case model: `qwen3.6-35b-a3b`.
|
|
- Case model admission status: `admitted`.
|
|
- Source admission status: `not_admitted`.
|
|
- Failure reason: `case agent result protocol failed`.
|
|
- The harness recorded no changed files.
|
|
- The patch artifact was empty.
|
|
- Tests failed because the artificial fixture bug remained unchanged.
|
|
- This proves the Spark endpoint blocker was crossed for one runtime attempt.
|
|
- This does not validate `CTO-WORK-016`, `CTO-WORK-020`, `CTO-WORK-022`, or `CTO-WORK-028`.
|
|
- Current active blocker is the Case agent-result protocol seam.
|
|
|
|
## Spark Response Shape Gate Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `974813b Block Case on reasoning-only local provider`.
|
|
- The Hermes Case Qwen loop now enables a local provider response-shape probe before Case process start.
|
|
- The probe classifies OpenAI-compatible responses that contain reasoning but no assistant content.
|
|
- Runtime endpoint value was supplied only through environment and is not recorded in SOT.
|
|
- Run artifact directory: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T023119Z-r1-string-slugify-2759949`.
|
|
- Report status: `blocked`.
|
|
- Failure reason: `provider response shape unavailable`.
|
|
- Marker: `backend/provider-reasoning-only.txt`.
|
|
- Case process started: `false`.
|
|
- Case model provider: `qwen-local`.
|
|
- Case model: `qwen3.6-35b-a3b`.
|
|
- Case model admission status: `admitted`.
|
|
- This proves Spark endpoint config is reachable but response-shape incompatible with Case/Pi result parsing.
|
|
- `CTO-WORK-030` remains blocked until a configured endpoint can support a real Stage 2 pass.
|
|
- Current downstream blocker is `CTO-WORK-031`.
|
|
|
|
## Spark Response Probe Budget Correction - 2026-06-01
|
|
|
|
- Hermes commit: `bbe7c72 Use realistic Case local response probe`.
|
|
- Direct Spark probe showed `qwen3.6-35b-a3b` returns reasoning only at `max_tokens=32`.
|
|
- Direct Spark probe showed the same route returns assistant content at `max_tokens=256` and `max_tokens=1024`.
|
|
- The Hermes response-shape probe now uses `max_tokens=1024`.
|
|
- Focused validator added `local_provider_delayed_content_allows_case`.
|
|
- Runtime endpoint value was supplied only through environment and is not recorded in SOT.
|
|
- Real Case Qwen loop artifact after the correction: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T023532Z-r1-string-slugify-2776187`.
|
|
- Report status: `fail`.
|
|
- Failure reason: `case agent result protocol failed`.
|
|
- Case process started: `true`.
|
|
- Case model provider: `qwen-local`.
|
|
- Case model: `qwen3.6-35b-a3b`.
|
|
- Case model admission status: `admitted`.
|
|
- Result: Spark endpoint availability is no longer the current unknown; Stage 2 remains blocked by the Case agent-result protocol seam.
|
|
|
|
## Spark OpenAI-Compatible Runtime Bridge Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `5c5448b Bridge Case Qwen through OpenAI-compatible runtime`.
|
|
- The harness reached Spark through an OpenAI-compatible Case runtime provider bridge.
|
|
- CTO admission identity stayed `qwen-local` / `qwen3.6-35b-a3b`.
|
|
- Runtime endpoint value was supplied only through environment and is not recorded in SOT.
|
|
- Focused validator artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T024443Z-r1-string-slugify-2817037`.
|
|
- Real Case Qwen loop artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T024456Z-r1-string-slugify-2819659`.
|
|
- Report status: `fail`.
|
|
- Failure reason: `case engine failed with exit code 124`.
|
|
- Case process started: `true`.
|
|
- Case runtime model provider: `openai`.
|
|
- Tests passed: `true`.
|
|
- Patch artifact was non-empty.
|
|
- This proves endpoint config and runtime provider bridging are sufficient for Case to produce a fixture patch.
|
|
- This does not validate `CTO-WORK-016`, `CTO-WORK-020`, `CTO-WORK-022`, `CTO-WORK-028`, or `CTO-WORK-032`.
|
|
- Current active blocker is Case lifecycle timeout after valid patch evidence.
|
|
|
|
## Hermes Case Qwen Loop Evidence - 2026-06-01
|
|
|
|
- Hermes commit: `6c453ee Add Case Qwen loop entrypoint`.
|
|
- New command: `harness/evals/case-qwen-loop.sh --json`.
|
|
- The command sets fixed non-secret Case/Qwen routing knobs for `qwen-local` / `qwen3.6-35b-a3b`.
|
|
- The command requires `CTO_HARNESS_CASE_BIN` and `CTO_HARNESS_CASE_MODEL_ADMISSION_FILE` from the caller.
|
|
- The command delegates to `run-case.sh --engine case` so runtime artifacts keep the Harness Evidence Interface shape.
|
|
- The command must not print `CTO_HARNESS_CASE_LOCAL_BASE_URL`.
|
|
- Focused validator passed: `python3 harness/runner/validate-case-provider-adapter.py --harness-root harness --json`.
|
|
- Focused validator artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T022022Z-r1-string-slugify-2694767`.
|
|
- Pre-merge aggregate validator passed: `harness/evals/health.sh --json`.
|
|
- Pre-merge aggregate validator artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T022036Z-r1-string-slugify-2695716`.
|
|
- Post-merge aggregate validator passed: `harness/evals/health.sh --json`.
|
|
- Post-merge aggregate validator artifact: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T022100Z-r1-string-slugify-2697229`.
|
|
- `CTO-WORK-030` remains blocked until `CTO_HARNESS_CASE_LOCAL_BASE_URL` is supplied at runtime and a real Stage 2 pass report exists.
|