YaOS / Platform / Developer Experience / Coding Harness / Evaluation
Evaluation
Evaluation
Measure agent behavior and harness quality, not just code generation volume.
What to evaluate
Evaluate both generated code quality and the harness behavior that led to it: context selection, skill routing, verification honesty, architecture compliance and review quality.
Core signals
- First-pass CI success
- Human interventions per task
- Review comments per PR
- Repeated review comments
- False completion claims
- Regression rate
- Median task-to-PR time
- Always-loaded context tokens
- Skill invocation accuracy
- Eval delta across harness releases