cto/.sot/03-PROTOCOLS/CTO-CASE-MODEL-PROVIDER-ADMISSION-ISSUES.md
2026-05-31 22:09:44 -04:00

11 KiB

title status lifecycle_classification owner created last_reviewed core_promotion_status route
CTO Case Model Provider Admission Issues draft sot jp 2026-05-31 2026-06-01 not-promoted cto

CTO Case Model Provider Admission Issues

Local planning SOT only. Not a Core Protocol. Not active Core authority.

CTO-WORK-019 - Case Model Provider Admission PRD

Status: validated.

Extract the existing CTO-WORK-018 harness gate into a first-class model provider admission route. This is the SOT route for deciding which provider/model pair may power real Case Stage 2.

Acceptance:

  • Records observed fallback provider anthropic.
  • Records observed fallback model claude-sonnet-4-6.
  • Requires explicit admitted provider and exact model ID before real Case starts.
  • Requires CTO_HARNESS_CASE_MODEL_PROVIDER and CTO_HARNESS_CASE_MODEL to match the admission record.
  • Requires backend/provider-model-not-admitted.txt when admission is missing.
  • Requires isolated CASE_DATA_DIR/config.json to contain admitted models.default.
  • Requires negative gates for missing provider/model and unadmitted provider/model.
  • Requires no secrets in task file, argv, report, trace, backend logs, SOT, or commits.
  • Keeps Case as candidate execution backend, not CTO authority.

CTO-WORK-020 - Admit Case Model Provider For Real Stage 2

Status: blocked.

Choose and admit the exact provider/model path for real Case Stage 2, then rerun Stage 2 through the Harness Evidence Interface.

Acceptance:

  • Admission record names provider, exact model ID, credential source class, allowed network class, approval source, admission timestamp, review trigger, and evidence expectations.
  • No provider/model is admitted by default.
  • No secret is written to SOT, argv, task file, backend logs, report, trace, or commit.
  • CTO_HARNESS_CASE_MODEL_PROVIDER and CTO_HARNESS_CASE_MODEL match the admission record.
  • Missing or unadmitted provider/model blocks before case_process_started.
  • Report records case_model_provider, case_model, and case_model_admission_status.
  • Real Case Stage 2 produces a pass report only if the admitted provider/model was used.
  • Same-run fake baseline comparison remains required.
  • No Target Repository path is inspected or copied.

Blocked by:

  • Human provider approval if an external provider such as Anthropic is selected.
  • A Case-compatible local provider route if external providers are not approved.

Hermes Implementation Evidence - 2026-05-31

  • Hermes commit: f39d8ab Require admitted Case model pair.
  • f39d8ab proves admission gating implementation only; it is not a real Case Stage 2 pass.
  • The Hermes adapter now requires CTO_HARNESS_CASE_MODEL_ADMISSION_FILE.
  • Env provider/model is only the requested pair; the admission JSON is the authority.
  • Missing admission blocks before case_process_started.
  • Mismatched admission blocks before case_process_started.
  • Report evidence records case_model_provider, case_model, and case_model_admission_status.
  • Status vocabulary includes admitted, missing_admission, mismatch, invalid_admission, and not_admitted.
  • Secret scan covers report.json, report.md, trace.jsonl, backend logs, Case stdout/stderr, and generated Case config.
  • Focused validator passed: python3 harness/runner/validate-case-provider-adapter.py --harness-root harness --json.
  • Aggregate validator passed: harness/evals/health.sh --json.
  • Focused validator artifact: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T235421Z-r1-string-slugify-1875638.
  • Aggregate validator artifact: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T235448Z-r1-string-slugify-1876884.
  • CTO-WORK-020 remains blocked until a provider/model is explicitly approved and real Case Stage 2 produces a pass report.

CTO-WORK-020 Decision Record Template

This template belongs to CTO-WORK-020; it is not a new provider approval.

Required fields:

  • decision_status: not_decided, external_provider_approved, or local_provider_required.
  • provider_class: external_anthropic, external_openai_codex, or local_case_compatible.
  • provider: exact provider string, or empty while blocked.
  • model: exact model string, or empty while blocked.
  • approval_source: JP approval reference or governed Core route reference.
  • credential_source_class: credential class only; no secret value.
  • allowed_network_class: allowed network class for this provider.
  • review_trigger: expiry, date, or condition that forces review.
  • evidence_sources: references to existing admission/build evidence, not copied runtime evidence.
  • effect: CTO-WORK-020 remains blocked until admitted provider/model and real Stage 2 pass report exist.

CTO-WORK-027 - OpenAI Codex Model Admission JSON

Status: validated.

Record the exact non-secret admission JSON required by CTO_HARNESS_CASE_MODEL_ADMISSION_FILE for the approved openai-codex / gpt-5.5 primary provider path.

Acceptance:

  • Admission file path is .sot/03-PROTOCOLS/CTO-CASE-MODEL-PROVIDER-ADMISSION.openai-codex-gpt-5.5.json.
  • Admission JSON has status: admitted.
  • Admission JSON has provider: openai-codex.
  • Admission JSON has model: gpt-5.5.
  • Admission JSON has credential_source_class: hermes-openai-codex-oauth-and-local-vllm-config.
  • Admission JSON has allowed_network_class: codex-oauth-hosted-model-plus-local-vllm-fallback.
  • Admission JSON has approval_source: JP chat approval on 2026-05-31.
  • Admission JSON has admission_timestamp.
  • Admission JSON has review_trigger.
  • Admission JSON contains no secret keys or secret values.
  • Fallback to vllm / qwen3.6-35b-a3b remains explicit decision-record context and must be represented in runtime evidence before it may count as a Case provider/model path.
  • CTO-WORK-020 remains blocked until real Case Stage 2 produces a Harness Evidence Interface pass report using this admission file.
  • Real Case Stage 2 command must set CTO_HARNESS_CASE_MODEL_ADMISSION_FILE to this admission JSON path.

Post-Admission Runtime Evidence - 2026-06-01

  • Run artifact directory: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T013918Z-r1-string-slugify-2381028.
  • Report status: fail.
  • Backend: case.
  • Case process started: true.
  • Case model provider: openai-codex.
  • Case model: gpt-5.5.
  • Case model admission status: admitted.
  • The admitted provider/model path reached Case execution.
  • Case failed before producing a diff.
  • Case stderr recorded AGENT_RESULT start delimiter not found.
  • Case retry classified the failure as agent-protocol-error.
  • CTO-WORK-020 remains blocked because no real Case Stage 2 pass report exists.
  • Current downstream blocker is tracked by CTO-WORK-028.

Hermes Auth Preflight Evidence - 2026-06-01

  • Hermes commit: 5db23c7 Fail closed on Case Codex auth gap.
  • 5db23c7 proves fail-closed classification only; it is not a real Case Stage 2 pass.
  • The Hermes adapter now blocks openai-codex before case_process_started unless CTO_HARNESS_CASE_OPENAI_CODEX_AUTH_BRIDGE=1.
  • The block writes backend/provider-auth-unavailable.txt.
  • The reason is specific: Case's pipeline SDK path constructs its Pi Agent runtime directly and does not pass Pi AuthStorage OAuth headers into streamSimple.
  • Pi env API-key lookup does not map openai-codex to an environment API key.
  • Focused validator passed: python3 harness/runner/validate-case-provider-adapter.py --harness-root harness --json.
  • Aggregate validator passed: harness/evals/health.sh --json.
  • Focused validator artifact: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T014803Z-r1-string-slugify-2448172.
  • Aggregate validator artifact: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T014805Z-r1-string-slugify-2449212.
  • CTO-WORK-020 remains blocked because no real Case Stage 2 pass report exists.

CTO-WORK-029 - Qwen Local Case Model Admission JSON

Status: validated.

Record the exact non-secret admission JSON required by CTO_HARNESS_CASE_MODEL_ADMISSION_FILE for the existing Pi local provider path qwen-local / qwen3.6-35b-a3b.

Acceptance:

  • Admission file path is .sot/03-PROTOCOLS/CTO-CASE-MODEL-PROVIDER-ADMISSION.qwen-local-qwen3.6-35b-a3b.json.
  • Admission JSON has status: admitted.
  • Admission JSON has provider: qwen-local.
  • Admission JSON has model: qwen3.6-35b-a3b.
  • Admission JSON has credential_source_class: pi-models-json-local-provider-no-secret.
  • Admission JSON has allowed_network_class: local-tailscale-vllm-spark1.
  • Admission JSON has approval_source: JP chat approval on 2026-05-31.
  • Admission JSON has admission_timestamp.
  • Admission JSON has review_trigger.
  • Admission JSON contains no secret keys or secret values.
  • This admission does not change Hermes primary model policy.
  • This admission does not approve real-repo, copied-repo, sandbox-repo, owned-repo, default-candidate, or Core promotion stages.
  • CTO-WORK-020 remains blocked until real Case Stage 2 produces a Harness Evidence Interface pass report using this admission file.
  • Real Case Stage 2 command must set CTO_HARNESS_CASE_MODEL_ADMISSION_FILE to this admission JSON path.

Qwen Local Runtime Evidence - 2026-06-01

  • Run artifact directory: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T015208Z-r1-string-slugify-2478256.
  • Report status: fail.
  • Backend: case.
  • Case process started: true.
  • Case model provider: qwen-local.
  • Case model: qwen3.6-35b-a3b.
  • Case model admission status: admitted.
  • The admitted Qwen local provider/model path reached Case execution.
  • Case failed before producing a diff.
  • Case stderr recorded AGENT_RESULT start delimiter not found.
  • Case retry classified the failure as agent-protocol-error.
  • CTO-WORK-020 remains blocked because no real Case Stage 2 pass report exists.
  • Current downstream blocker remains CTO-WORK-028.

Qwen Local Classified Runtime Evidence - 2026-06-01

  • Run artifact directory: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020117Z-r1-string-slugify-2566310.
  • Report status: fail.
  • Backend: case.
  • Case process started: true.
  • Case model provider: qwen-local.
  • Case model: qwen3.6-35b-a3b.
  • Case model admission status: admitted.
  • Failure reason: case agent result protocol failed.
  • Protocol marker: backend/provider-agent-protocol.txt.
  • CTO-WORK-020 remains blocked because no real Case Stage 2 pass report exists.
  • Current downstream blocker remains CTO-WORK-028.

Qwen Local Isolated Pi Config Gate Evidence - 2026-06-01

  • Hermes commit: 09b5851 Isolate Case Pi provider config.
  • The admission file remained valid for qwen-local / qwen3.6-35b-a3b.
  • Real Qwen local config-gate proof artifact: /home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260601T020847Z-r1-string-slugify-2619644.
  • Report status was blocked.
  • Case model admission status was admitted.
  • Case process started was false.
  • The block reason was missing explicit local provider config, not missing model admission.
  • The required unblock variable is CTO_HARNESS_CASE_LOCAL_BASE_URL.
  • CTO-WORK-020 remains blocked because no real Case Stage 2 pass report exists.