All fix proposalshigh
Fix Proposal · controlled autonomy
Gate the final answer on deployment & telemetry verification
fix_1002 · risk low · confidence 81% · source mock
Root cause
Required verification tools are skipped for customer-facing root-cause missions.
Evidence from trace & evals
- Required-tool eval failed across affected runs.
- checkDeploymentHistory / checkRuntimeTelemetry missing from the trace.
Proposed change
Block the final-answer step until both required verification tools have completed.
Policy update
agent-policies/escalation-agent.md
## Customer-facing root-cause claims
Draft the customer update from tickets and docs.
The final-answer step is gated on completion of checkDeploymentHistory and checkRuntimeTelemetry.
Expected improvement
Required-tool eval fail → pass on all customer-facing root-cause runs.
Regression cases
- • Globex timeout escalation
- • Initech billing outage
Human approval
Every risky action needs approval
Review the evidence, then approve to open a GitLab remediation and unlock the regression rerun.
Tool-Call Firewall · action receipt
- Tool
- createGitlabIssue
- Params
- title="Gate the final answer on deployment & telemetry verification"
- Risk
- high
- Approval
- required
- Rollback
- Close the created issue/MR. No agent policy is applied until the MR is reviewed and merged.
Affected runs
run_cust_globex_mt_0_420, run_cust_globex_mt_1_2100, run_cust_initech_mt_0_770, run_cust_initech_mt_1_3300