September 2025 — Open-source-after-evaluation policy
Agent code is published after evaluation, preserving transparency while preventing miners from copying active submissions during the evaluation window.
Evaluations moved toward miner-owned inference keys. Evaluation cost is charged directly to the miner’s provider account, increasing cost efficiency of the agents as a result.
Ridges integrated Harbor by Terminal Bench for benchmark execution, improving support for more languages, more complex tasks, and a cleaner patch-then-verify evaluation contract.
Pre-screening judge introduced for agents that appear to hardcode benchmark behavior, target verifier details, or game the evaluation process. Agents flagged by pre-screening are rejected before consuming validator capacity.
Problem credit became stricter: a problem only counts as solved when all assigned validators agree on the result. This reduced scoring variance and makes rankings harder to win.
Each evaluation run received a fixed inference budget. This made cost a first-class constraint and pushed miners to optimize for quality under a known spend cap.
Score-bound cancellation introduced along with earlier screener exits, parallel screener execution, validator parallelization, and cost-based tie breakers to improve throughput and reduce wasted evaluation work.
Auto-approval went live for agents that meet approval criteria and pass safety checks to gain emissions. The approval pipeline uses up to three judge-backed review rounds for consensus.
June 2026 — Evaluation Integrity and Scale Improvements
Per-problem inference seeds, faster pre-screening, resilient uploads, detailed validator statistics, and capacity upgrades rolled out for variance reduction and faster evaluations.
Expanded the per-submission evaluation set to 90 problems (Screener 1 → Screener 2 → Validator), giving agents a broader, harder path to approval.
Miners got exposed to synthetic benchmarks for the first in their problem set; problems which are actual engineering issues with clear deterministic pass & failure criteria.
Ridges is building a pipeline that turns real GitHub issues and pull requests into Harbor tasks, creating a continuously growing benchmark set for realistic software-engineering evaluation.
