source-reader
reads the implicated source file
waiting...
kevinmurphywebdev.com · live demo
A multi-agent debugging concierge. Paste a stack trace; four specialist subagents fan out in parallel, each with a focused tool set, then a coordinator merges their structured findings into ranked, calibration-aware hypotheses.
The LLM calls, structured outputs, parallel fan-out, abort plumbing, calibration log, prompt caching, and cost math are all real. The data sources the subagents call into are fixture-backed for this public demo:
src/auth/session.ts — a small TypeScript snippet shaped like a real auth handler with a null-deref bug. Returned by fetch_file when source-reader navigates to it.a3f1c92, 1b8e44d) and their diffs. Returned by git_log and git_diff when blame-correlator queries the history.reads the implicated source file
waiting...
ranks suspect commits
waiting...
quantifies blast radius
waiting...
drafts a minimal repro
waiting...
Forge ships with a CLI eval harness (scripts/eval.ts) that runs each of 5 reproducible bug scenarios N times and grades the output against a per-lane rubric (lib/eval/rubric.ts). The rubric scores correctness components (file match, line-range intersection-over-union, top suspect match, severity match) plus process components (snippet present, reasoning length, candidate explanations). The runner reports mean and stddev per scenario so prompt changes can be compared for statistical significance, not single-run noise. Snapshots write to .forge/evals/<timestamp>.json.
snapshot below is illustrative · from a 3-run sweep on 2026-05-20 · run pnpm eval --runs=3 to regenerate
Why an eval harness matters: LLM output is non-deterministic. A single "correct" answer is a sample from a distribution. n-runs aggregation makes variance visible — a v2 that scores 17/20 once isn't proof of improvement if v1's mean is 15 ± 2. Without this layer, every prompt change is a vibes-based judgment.