Skip to content
Coder Eval

Tutorial 03 — Browsing Results Locally with evalboard

coder-eval writes results as JSON, markdown, and logs under runs/. evalboard is a small local web UI that renders those runs — pass rates, per-task detail, tool and message timelines, artifact downloads — so you can explore a run in a browser instead of reading raw files. It reads straight from your filesystem: no database, no backend, nothing leaves your machine.

Prerequisites

Section titled “Prerequisites”
  • A completed run (do Tutorial 01 first — you need a runs/<timestamp>/ directory with a run.json).
  • Node.js 20+ and pnpm. The repo pins pnpm via packageManager, so the simplest way is Corepack:
    Terminal window
    corepack enable    # provides the pinned pnpm

1. Install dependencies

Section titled “1. Install dependencies”
Terminal window
cd evalboard
pnpm install

2. Point evalboard at your runs and start it

Section titled “2. Point evalboard at your runs and start it”
Terminal window
# Use coder_eval's own ./runs directory (shortcut):
pnpm dev:local


# ...or point at any runs directory (absolute path):
EVALBOARD_LOCAL_RUNS_DIR=/absolute/path/to/runs pnpm dev

Then open http://localhost:3030.

Only directories that contain a run.json appear in the index — empty shells and the latest symlink are filtered out. If your run list is empty, confirm the run actually completed and that EVALBOARD_LOCAL_RUNS_DIR points at the parent runs/ folder (not a single run).

3. Explore

Section titled “3. Explore”
PageWhat it shows
/The 20 most recent runs, one clickable row each
/runs/latestRedirects to the newest run
/runs/<run-id>Run summary — pass rate, cost, duration — plus one row per task (green = pass, red = fail). “Download run (.zip)” bundles the whole folder
/runs/<run-id>/<task-id>Per-task detail: success-criteria cards, the tool timeline, the message timeline (per-message tokens, expandable into thinking/tool/text), artifact downloads, and the tail of task.log
/trendsPer-task pass rate and average duration/cost/turns across recent runs

<task-id> matches the task_id in the results and the subdirectory name under <run-id>/<variant>/ — the same layout described in the Output Structure reference.

How it works (and why it’s safe to run locally)

Section titled “How it works (and why it’s safe to run locally)”
  • Filesystem-only in local mode. Setting EVALBOARD_LOCAL_RUNS_DIR makes evalboard read runs from that directory and never reach out to any remote storage.
  • OSS edition by default. evalboard runs in its open-source edition with no configuration; UiPath-internal surfaces are opt-in via EVALBOARD_EDITION=internal.
  • No database. Run listings and detail pages are rendered on the fly from the files Coder Eval already wrote.

Next steps

Section titled “Next steps”