Build statuses and PR comments

ArchGuard reports in two places on every evaluated pull request: a build status named ArchGuard on the latest commit, and one comment on the pull request. The tables below use Bitbucket's wording. GitHub's is covered in On GitHub.

Build statuses

Status shown Description text Meaning
In progress ArchGuard: evaluating architecture The evaluation is running.
Successful ArchGuard: No architectural impact detected No file matched a structural trigger.
Successful ArchGuard: all guardrails passed The model found no violations.
Successful ArchGuard: passed with N warning(s) Only warning-level violations.
Failed ArchGuard: N blocking violation(s) At least one blocking rule was broken.
Successful ArchGuard: too large to evaluate (neutral) The pull request exceeded the size limits.
Successful ArchGuard: could not evaluate (neutral) The model did not return a valid answer.
Successful ArchGuard: evaluation failed (neutral). Merge is not blocked. An error persisted after all retries.
Successful ArchGuard: evaluation limit reached (neutral) A plan limit applied (not used in pilot mode).

Bitbucket only has successful, failed and in progress, so neutral results are shown as successful with an explanatory description. A new status always replaces the previous one for the same pull request.

On GitHub

GitHub shows the same results as a commit status with the context ArchGuard. The descriptions are identical (cut to GitHub's 140-character limit), and the states map like this:

ArchGuard GitHub
In progress pending
Successful (including neutral) success
Failed failure

The Details link opens the ArchGuard dashboard. The comment is a normal pull request comment and is updated in place. To make the check block merging, see Connect GitHub.

Merge blocking

A failed status blocks merging only if the repository requires it: in Bitbucket, Require successful builds to merge; in GitHub, a branch rule with Require status checks to pass that lists ArchGuard. Until you switch that on, ArchGuard advises but does not block. See Pilot and shadow mode.

Neutral results

A neutral result means ArchGuard could not give a verdict. It never blocks a merge. The comment says why:

  • Too large: the diff or its token size was over the limit, or GitHub refused to return the diff (over 300 files or 20,000 lines). Ask for a manual architecture review, or split the pull request.
  • Could not evaluate: the model's answer was not valid after one retry. Pushing a new commit starts a new evaluation.
  • Evaluation failed: a technical error survived all retries. An Admin can check the worker logs.

The pull request comment

The comment is updated in place on each run. It contains:

  • A headline: passed, failed or neutral.
  • A Blocking table and a Warnings table. Each row gives the rule, the file, the line, the issue and a suggested fix.
  • A link to the ADR draft, when one was created.

The comment never contains source code. If someone deletes the comment, the next run posts a new one.

Fixing a failed check

  1. Read the Suggested fix for each blocking row.
  2. Push a new commit. ArchGuard re-evaluates and the check updates.
  3. If the finding is wrong, ask an Architect to adjust the rule. See Guardrail rules.