hermes/cto
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record Case model admission gate

Browse Source
This commit is contained in:
Svrnty
2026-05-31 19:45:28 -04:00
parent 3cd2551e24
commit eefa156734
4 changed files with 67 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-ISSUES.md
+29
View File
@@ -78,3 +78,32 @@ Validation Evidence:
- Hermes commit: `d23c492 Fail closed on Case provider timeout`.
- Post-merge `harness/evals/health.sh --json` passed.
## Real Provider Runtime Evidence - 2026-05-31
- Real Case Stage 2 run with `/tmp/workos-case/dist/ca` and `CTO_HARNESS_CASE_TIMEOUT_SECONDS=8` produced report `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T234205Z-r1-string-slugify-1834617/report.json`.
- Case started provider `anthropic` with model `claude-sonnet-4-6`.
- Adapter-generated `backend/case-data/config.json` contained no model registry, so Case used its built-in default model.
- The run timed out before patch application; tests failed because `strings.py` was unchanged.
- This is an unadmitted external model path for CTO harness proof and must be blocked before Case process start.
## CTO-WORK-018 - Case Model Provider Admission Gate
Status: validated.
Harden the Hermes CTO Case adapter so real Case execution cannot silently fall back to its built-in Anthropic default model.
Acceptance:
- Real Case execution requires `CTO_HARNESS_CASE_MODEL_PROVIDER` and `CTO_HARNESS_CASE_MODEL`.
- Missing model admission writes `backend/provider-model-not-admitted.txt`.
- Missing model admission blocks before `case_process_started`.
- Adapter writes `models.default` into isolated `CASE_DATA_DIR/config.json`.
- `validate-case-provider-adapter.py` checks `model_provider_gate_blocks`.
- `harness/evals/health.sh --json` passes after merge.
- This does not mark real Case Stage 2 as passed.
Validation Evidence:
- Hermes commit: `4500082 Gate Case execution on admitted model`.
- Post-merge `harness/evals/health.sh --json` passed.
sot/03-PROTOCOLS/CTO-CASE-PROVIDER-BUILD-PRD.md
+9
View File
@@ -82,3 +82,12 @@ Stage 2 moves from provider-unavailable blocked status to a real Case pass repor
- Timeout marker: `backend/provider-timeout.txt`.
- Timeout validator check: `provider_timeout_fail_closed`.
- Hermes evidence commit: `d23c492 Fail closed on Case provider timeout`.
## Current Model Admission Evidence Addendum - 2026-05-31
- Real Case defaulted to provider `anthropic` and model `claude-sonnet-4-6` when no model registry was written.
- Runtime report path: `/home/svrnty/.hermes/profiles/cto-planb/harness-runs/20260531T234205Z-r1-string-slugify-1834617/report.json`.
- Required model admission variables: `CTO_HARNESS_CASE_MODEL_PROVIDER` and `CTO_HARNESS_CASE_MODEL`.
- Model gate marker: `backend/provider-model-not-admitted.txt`.
- Model gate validator check: `model_provider_gate_blocks`.
- Hermes evidence commit: `4500082 Gate Case execution on admitted model`.