Commit History
-
Rename fixture-named result keys to sample-named keys
Michael Rabinovich +1: CursorRead/write per_sample_scores, per_sample_breakdown, n_samples, per_sample_results across submit/gallery/admin and tools. Also link the canonical submission docs from the Submit tab. Co-authored-by: Cursor <cursoragent@cursor.com>be6fa3d
-
leaderboard: admin rescore (selected + all) reusing the eval pipeline
Michael Rabinovich +1: CursorAdd a maintainer-only rescore that re-evaluates submissions against the current ground truth/data, reusing the submit worker end to end: each target row is reset to the pending regime (scores cleared, submitted_at preserved) and _spawn_worker is re-dispatched, which re-renders the gallery, regenerates reports/<id>.{html,json}, and recomputes the score. - rescore_rows(ids): selected rows; LookupError on unknown id before any write. rescore_all(): every row with a stored zip that isn't pending (skips legacy zip-less seed rows + in-flight evals). - Bulk dispatch is staggered on a background thread so a board-wide rescore doesn't fire N run_job calls at once; fully re-runnable, so an interrupted rescore converges on a retry. - UI in the Admin tab: "Rescore selected" behind a confirm checkbox and "Rescore ALL" behind a type-to-confirm phrase, both admin-gated with a server-side re-check and disarmed after firing / on load. Co-authored-by: Cursor <cursoragent@cursor.com>2893b22 -
leaderboard: serve renders from the public bucket, not the dataset proxy
Michael Rabinovich +1: CursorCandidate renders (turntables + edit-diff WebP) now live in the public HF Storage Bucket under renders/<id>/, uploaded once by the eval job. The gallery grid and the hosted report reference them by anonymous bucket URL, so the Space is out of the render read path and the binary renders stop bloating the submissions dataset / its commit queue. - leaderboard: HF_RENDER_BUCKET + render URL/path/prefix helpers. - app: render resolvers return public bucket URLs; the two render proxy routes + fetchers are retired. Only the private GT render keeps its token-held proxy. - submit: merge passes render_base_url to generate_html and commits report-only; dispatched job gets CADGENBENCH_RENDER_BUCKET/HF_ENDPOINT. - admin: delete also purges the bucket renders/<id>/ prefix (previously orphaned). - Dockerfile: bump cadgenbench pin to 3d49822 (render_base_url support). Co-authored-by: Cursor <cursoragent@cursor.com>d2161b1
-
admin: stop pending jobs before delete
Michael RabinovichAdd an admin action that cancels matching in-flight HF Jobs before deleting submissions, and keep the admin table refreshed so pending rows stay visible.c4b5d70
-
admin: bulk promote/demote/delete + admin table loader
Michael Rabinovichadds load_admin_table to the leaderboard reader: a single flat frame of every row (both tiers) with a leading editable select column, for the admin tab to act on. reworks admin.py around bulk helpers (promote_rows, demote_rows, delete_rows) that each do one results.jsonl write for the whole selection; delete also best-effort removes the companion zip and report artifacts. the singular promote_row/demote_row stay as thin wrappers. tests cover the bulk paths, a missing-id abort, empty-selection guards, and artifact deletion, all with the hub mocked.0957a56 -
tests: cover the admin gate and promote/demote helpers
Michael Rabinovichfour unit tests for admin.py: the CADGENBENCH_ADMINS gate (in-set, out-of-set, logged-out, empty set), promote and demote happy paths, and idempotent re-promotion. the results.jsonl read and write that _hub_rmw_results drives are mocked, so the suite makes no hub calls.007a50e