Skip to content

Report ​

bash
npx monkeytest report --json

The file under .monkeytest/runs/ is a stored run: { schema, id, createdAt, engine, planRef?, run }. schema is 1. run is a RunResult.

RunResult ​

FieldValues
outcomebugs_found, clean, inconclusive, could_not_test
outcome_reasonlogin_required, blocked_by_bot_protection, site_unreachable, invalid_url, llm_error, all_flows_skipped, timeout, automation_error, not_attempted, nothing_tested, no_assertion, assertion_unverifiable, budget_exhausted, contact_submit_blocked, request_blocked, and the auth reasons auth_mfa_required, auth_expired, auth_origin_rejected, auth_rejected, auth_not_authorized, auth_target_rejected, auth_storage_unsafe
authPublic sign-in metadata when a session was attempted. No password and no cookie jar. Omitted on anonymous runs. privateDataScrubbed is false.
outcome_detailOne or two sentences that match the bug count, plus a next step
public_screensCrawled pages that are not auth-gated. Omitted when the run did not crawl.
engine_versionENGINE_VERSION, for example the package version
signup_attempted, signup_verifiedBooleans from the one sign-up attempt
signup_startedtrue once filling a sign-up form begins. Omitted otherwise.
signup_statusverified, unverified, not_tested, blocked
signup_reasonno_inbox, captcha, phone, payment, timeout, cap_reached, no_signup_form, automation_error
pagesExploredUnique pages that loaded. The same screen at two URLs counts once.
suggestionsSuggestion[]. Style notes and page errors that are not site bugs. Omitted when empty. Not in stats or the verdict.
warningsModel errors and a planner-fallback line. Not bugs.
flowsFlowResult[]
bugsBugReport[]. Confirmed site bugs only. Suggestions are not in this list.
statsflowsTotal, flowsPassed, flowsFailed, flowsBlocked, flowsInconclusive, step counts, bugsCritical, bugsMajor, bugsMinor

could_not_test means nothing was tested. inconclusive means the site was reached and a real check did not finish. login_required is also set when outcome is clean or bugs_found and public_screens is 0.

When one confirmed bug is filed and the site is not login-only, outcome_detail is:

text
Found 1 confirmed bug. Open the top one and confirm it on the site before treating this run as healthy.

FlowResult ​

FieldMeaning
statuspassed, failed, blocked, error, inconclusive
outcome_reason, outcome_detailSet when this flow did not produce a confirmed site bug
assertionOutcome-check result. verified is required for passed on runs from 0.7.0-beta.3.
stepsEach step has status, observation, label, durationMs, screenshotPath, console and network evidence
traceWhat rerun replays. Absent on runs recorded before traces existed.
videoPathWebM for the flow
bugIdsBugs produced from this flow
severityIfFailsCopied from the plan so triage can rank a bug

passed is not "the steps ran". no_assertion, assertion_unverifiable, contact_submit_blocked, and request_blocked are never paired with passed. contact_submit_blocked means a contact, enquiry, feedback, support, quote, or get-in-touch submit was not sent. It is not a site bug. request_blocked means some other request was aborted by the runner (a sensitive submit, or net::ERR_BLOCKED_BY_CLIENT) and the browser showed an error page. It is not a site bug.

BugReport ​

FieldMeaning
title, severity, observed, expectedseverity is critical, major, or minor
fingerprintLegacy id. Still written so older hosts keep matching.
fingerprintV2, fingerprintPartsStable id from category, URL template, role, name, and error signal. Not an LLM tool name.
reproStepsSteps to repeat
evidencescreenshots, consoleErrors, networkFailures, optional domSnapshot
traceActions from the start of the flow through the failing step
impactOne line. The polish model may supply it. Otherwise the engine writes a short line. Omitted before 0.6.1.
page_urlURL where the step failed. Omitted before 0.6.1.
step_index0-based index of the failing step. Omitted before 0.6.1.
flow_idFlowResult.flowId. Omitted before 0.6.1.
elementOptional selector and bounding_box (x, y, width, height) for a ring on the screenshot. A miss omits the field.
affectedPagesPages where the same axe rule fired. One bug per rule.
seenInFlows{ flowId, name }[] when more than one flow hit the same HTTP failure (method + path + status).
axeRulesOn the step, not the bug. The rule id, impact, and page.

Suggestion ​

FieldMeaning
pageUrlPage the note is about. Group by this for a per-page list.
title, detailWhat was observed
kindautocomplete, viewport, description, overflow, third-party-error, unattributed-error, auth-redirect
severityAlways info

color-contrast and scrollable-region-focusable are bugs of severity minor at most, rolled up by rule id. Other serious axe rules stay bugs, also one per rule.

TestPlan ​

FieldMeaning
plannerFallbacktrue only when the grounded plan replaced a failed model call. A login wall that correctly returns no journeys leaves this unset.
plannerFallbackReasonShort reason. Safe to show next to the flag.

fingerprint and fingerprintV2 do not include impact, page_url, step_index, flow_id, or element. The CLI treats every severity except minor as blocking.

How to read a run without the field table: Reading the report.

AGPL-3.0-only