The system proposes, a human promotes. A weekly job drafts a new rubric version from disagreements, but drafts never become active on their own.
Weekly · 4 steps
- Collect: human verdicts, AI verdicts and QA comments
- Weekly job: Monday 03:30 IST, last 7 days of disagreements, runs at 5+ per rubric
- Draft version: rewritten skill file, inactive, rationale posted to Slack
- Human review: reject sends it back to draft; promote makes it active
Versioning · 3 rules
- Nothing takes effect until a human promotes it
- Resolves workflow → workspace → global; the judge uses the new version on the next SKU
- Every evaluation records its version; promoting an older version is the rollback