← Field Notes
SEP 10 · Clipped · via Promptfoo on GitHub Release DecisionEval Run Setup

Promptfoo lets a pass percentage turn the release check red

A threshold is a decision made once, by whoever wrote the workflow file. The product does not ask whether this regression should block or only warn; that question needs a screen, not a setting.

Read the source ↗ attached to Release Decision →
watching

watching → emerging → established

A candidate behavior, not yet a published pattern. Early signal — worth tracking, not yet worth standardizing on.

7 documented sightings since May 2026 — every one traces to a dated source.

See the evidence →

Machine summary of the source

The GitHub Action posts a comment with pass and fail counts and a link to the web viewer. A fail-on-threshold setting requires a pass percentage, and a repeat setting handles judges that flip between runs. The viewer shows the grader's reasoning for rubric checks and lets the person filter failures and compare outputs. Release 0.123.0 shipped on 2026-09-10.

The summary above is generated; the note at the top is the editorial judgment. Primary source ↗