Documentation menu

Reading the PR comment

Every part of the DiffCost pull request comment and the diffcost/cost-check commit status, and what each row and message means.

One comment per pull request#

DiffCost posts one comment on each pull request (PR) and updates it on every push, so the comment always describes the latest run. It finds its own comment by a hidden marker. The same report goes to the workflow run's job summary. DiffCost can't comment on PRs from forks, so read the job summary for those. Merge-queue runs post no comment either: their report is only in the job summary.

DiffCost updates a comment with its marker that a bot posted (github-actions[bot] first, then other apps' bots) or, when github_token is a personal access token, that the token's user posted. Otherwise a person's comment is never edited. If the token can't edit the earlier comment (for example after you switch tokens), DiffCost posts a new one and updates that one from then on.

A cost report, top to bottom#

πŸ’° DiffCost β€” BigQuery Cost Report (cost per run)

Query / Model Current This PR Scan Delta Cost Change
fct_daily_revenue $2.09 $9.66 +1.21 TiB +$7.57 (↑362%)
stg_orders $0.031 $0.031 0 B β€”
TOTAL $2.12 $9.69 +1.21 TiB +$7.57 (↑357%)

⚠️ This PR increases query costs by $7.57/run (+357%, β‰ˆ$227.10/mo at 30 runs/mo).

Largest single-query increase: fct_daily_revenue +$7.57/run (↑362%).

πŸ“Š Data Scanned per run: 347.51 GiB β†’ 1.55 TiB (+1.21 TiB)

⚠️ High Scan Warning: Query scans full table without partition filter.

πŸ” Root cause analysis (collapsed)

  • πŸ”΄ fct_daily_revenue: Partition filter removed: order_date >= '2024-01-01'

From the top, a report can contain:

  1. Verdict (only when the cost policy is exceeded): "🚫 Merge blocked β€” this PR exceeds a DiffCost cost policy", "⚠️ Cost policy exceeded β€” not enforced" or "⚠️ Cost policy exceeded β€” enforcement unconfirmed", with one bullet per reason. See The verdict.
  2. Status note (only when something went wrong): "No diffcost/cost-check status was posted", with the reason and the fix, or, when the DiffCost service didn't confirm it, "The diffcost/cost-check status could not be confirmed". See Troubleshooting.
  3. Slack failure (only when a Slack alert failed): "⚠️ Slack alert failed" naming the webhook (.diffcost.yml or dashboard threshold) and Slack's answer. See Slack alerts don't arrive.
  4. Cost policy warnings: one "⚠️ Cost policy: …" line per problem reading .diffcost.yml or its thresholds.
  5. ignored_paths hint (dbt only): "ℹ️ SQL outside the analyzed dbt project counts as unknown cost…", with the ignored_paths line to add to .diffcost.yml on the base branch.
  6. The cost table, with a TOTAL row when more than one query has a known cost.
  7. Summary lines: the net change per run with the monthly projection, the largest single-query increase, and the change in bytes scanned.
  8. Notes on queries with an unknown cost, and the High Scan Warning when a partition filter was removed (unless the dry run shows no extra bytes).
  9. Root cause analysis: a collapsed list of the structural risks found (πŸ”΄ high, 🟑 medium).
  10. Plan line: "⏳ DiffCost Team trial: N days remaining", "βŒ› Your Team trial has ended.", "πŸ›‘οΈ DiffCost Team β€” Merge blocking enabled." (with your plan's name: Starter, Team or Enterprise; the "Merge blocking enabled" part only when the status was posted), or, on the Free plan with an API key, "πŸ“Š Cost history Β· 🚫 Block expensive PRs β€” Starter plan". Without an API key, "πŸ”‘ No DiffCost API key, so this run can't block merges", with the likely cause (see No DiffCost API key). Without BigQuery credentials, a "πŸ’‘ Get per-run dollar estimates" line comes first. There's no plan line when the dashboard settings couldn't be fetched.

The verdict, the status note, the Slack failure, the cost policy warnings and the ignored_paths hint come before the table, so they survive when GitHub's comment size limit cuts a very large report. The comment then says it was truncated, and the job summary keeps a longer version.

The cost table#

The first column is the query: for plain SQL files, the file name without its directory or .sql, so marts/daily.sql and staging/daily.sql both show as daily; for dbt, the model name.

Row Current This PR Cost Change
Changed query cost per run cost per run +$d (↑N%) or -$d (↓N%); β€” for less than $0.0005
New β€” cost per run **New** (+$y)
Removed cost per run β€” **Removed** (was $x)
Cost unavailable β€” β€” ⚠️ Cost unavailable: {reason}
Dry run failed β€” β€” ⚠️ PR dry-run failed: {BigQuery error}, ⚠️ Production dry-run failed: …, or ⚠️ skipped: dry-run time budget exceeded
Not costed (dbt) cost per run β€” ⏳ Not costed: depends on objects this PR creates, plus " (structural cost risk, so it counts as unknown)" or " (structural analysis could not run on it, so it counts as unknown)" when it blocks

Two grouped rows can appear in dbt repositories, each as one Cost unavailable row: non-model dbt files (macros, seeds, config or packages changed, and their effect can't be costed) and SQL outside the analyzed dbt project (.sql files outside the project, not analyzed).

Scan Delta is the change in bytes scanned per run, in binary units (KiB, MiB, GiB, TiB).

Amounts: under $1, three decimals ($0.031); below $0.001, <$0.001.

BigQuery error text in a row is cut to 200 characters, and backtick-quoted names and any name with three dot-separated parts (table names such as project.dataset.table or project:dataset.table, but also permission names such as bigquery.jobs.create) are replaced with <redacted>. Other names in the error, such as a project ID ("Access Denied: Project …"), a project:dataset name or a column name, appear as BigQuery wrote them. The step log doesn't have more detail than the comment. To investigate an "Access Denied", check the service account's roles on the project and datasets, not your own. For dbt, the compiled SQL is in target/compiled/ after dbt compile.

Troubleshooting lists each reason a row can be unavailable, and what to do about it.

Summary lines#

Line When
⚠️ This PR increases query costs by $X/run (+N%, β‰ˆ$Y/mo at R runs/mo). The net change across costed queries is above +$0.005 per run.
⚠️ name costs +$X/run more; savings in other queries offset it (net ±$Z/run). The net change is $0.005 per run or less, but one query rose by more than $0.005. Thresholds still see the increase.
βœ… This PR reduces query costs by $X/run. The net change is a saving of more than $0.005 per run, and no query rose by more than $0.005.
βœ… No significant cost change detected. The net change is within Β±$0.005 per run, and no query rose by more than $0.005.
⚠️ No query in this PR could be costed. See the rows above. No changed query has a known cost (each row is Cost unavailable, dry-run failed or Not costed).
Largest single-query increase: name +$X/run (↑N%). The net change and the largest single-query increase are both above $0.005 per run, and more than one query has a known cost.
πŸ“Š Data Scanned per run: A β†’ B (Β±C) The total bytes scanned changed.
ℹ️ N queries could not be costed and are excluded from the totals above. Some queries have an unknown cost. Under a threshold they block.
ℹ️ 1 query reads tables or columns this PR creates, so it can be costed only after merge. A dbt model is Not costed and doesn't block: structural analysis found no risk on it, or it's a new model.

Structural risks#

πŸ”΄ High:

  • Partition filter removed (a removed WHERE condition on a date or partition column);
  • SELECT * added;
  • CROSS JOIN added;
  • All WHERE conditions removed (the base version had filters; the PR version has none).

🟑 Medium:

  • WHERE clause removed (one filter condition on another column was dropped);
  • New JOIN (for example "New LEFT JOIN");
  • N WHERE condition(s) removed (the PR version has fewer filter conditions than the base version);
  • New table reference.

🟒 Low (shown only in the structural-only report): new CTEs, subqueries or window functions; SELECT expanded by N columns; "Could not parse … SQL"; "Structural analysis failed".

A removed partition filter already explains a drop in WHERE conditions, so DiffCost doesn't report both. Structural analysis needs both versions of a query, so new and deleted files get none.

The verdict#

Heading Meaning
🚫 Merge blocked The policy was exceeded and the failing diffcost/cost-check status was posted.
⚠️ Cost policy exceeded β€” not enforced The policy was exceeded, but no failing status was posted, so GitHub doesn't block the merge (unless an earlier run already posted a failing status on this commit, which still applies). The note below it says why.
⚠️ Cost policy exceeded β€” enforcement unconfirmed The DiffCost service didn't answer in time, or answered without saying whether the status was posted, so DiffCost can't tell. Check the PR's checks, or re-run the job.

The reasons name the query and the limit, for example "fct_daily_revenue: +$7.57/run exceeds the $5.00/run per-query threshold", "fct_daily_revenue: +362% exceeds the 200% per-query threshold", or "2 queries could not be costed, so the cost policy cannot be confirmed". With Editions pricing, dollar reasons say "(on-demand equivalent)". The commit status and the dashboard word the same reasons differently: "Worst query +$7.57/run exceeds $5.00/run limit".

The diffcost/cost-check status#

With an API key, the DiffCost service posts this status on the PR's head commit (on a merge-queue run, on the queue's commit), as the DiffCost GitHub App. It is success or failure, never pending. Its Details link opens the dashboard overview. DiffCost cuts the description to 140 characters.

Description State When
Worst query +$7.57/run exceeds $5.00/run limit; … failure Over a threshold, or queries with an unknown cost under a threshold. The reasons are joined with "; ".
Worst query +$0.42/run (within threshold) success Every query is within the threshold.
Worst query +$0.42/run (no threshold set) success No threshold applies to the repository.
No cost regression detected success No query got more expensive.
No SQL changes in this commit success The PR changes no SQL.
Merge blocking is off on the Free plan (worst query +$0.42/run) success Free plan, including a trial that hasn't started or has ended.

"; N queries not costed" is added when some queries have an unknown cost and no threshold applies.

Other comments#

  • "πŸ’° DiffCost β€” No SQL cost changes detected in this PR." The PR changes no SQL, and comment_on_clean: true is set, or no status could be posted, so the PR says why a required check is waiting.
  • "πŸ’° DiffCost β€” No SQL changes in the latest push. Earlier cost results on this PR no longer apply." An earlier push changed SQL, the latest doesn't, and comment_on_clean isn't true: DiffCost replaces its old report (with comment_on_clean: true, the comment above replaces it).
  • "⚠️ DiffCost could not analyze this PR." DiffCost couldn't run; the message says why and what to fix. See Troubleshooting.
  • "πŸ’° DiffCost β€” SQL Change Analysis", with "No BigQuery credentials configured" or "BigQuery credentials were provided but could not be used": a structural-only report, with risk flags and no dollars. See BigQuery access.

With Editions pricing#

With the Editions pricing model (chosen in the dashboard, or with bq_pricing_model: editions without an API key), the table shows bytes instead of dollars (Current scan, This PR scan, Change), the change column shows scan multipliers such as 4.6x scan [Editions - Slot Based], and the summary reports the change in scan volume. Thresholds and outputs still use the on-demand equivalent at your Cost per TiB rate.

Something missing or wrong? Email support@diffcost.dev.