Troubleshooting
What each DiffCost error and warning means and how to fix it: failed runs, Cost unavailable rows, credentials, dbt manifests, API keys and missing commit statuses.
Start with the pull request (PR) comment or the job summary: DiffCost says what went wrong and what to do. The DiffCost step's log has the run-level messages as GitHub annotations (invalid configuration, a missing or rejected API key, a missing status, a failed Slack alert). Per-query problems (dry-run errors, Cost unavailable rows) are only in the comment and the job summary. Messages below are quoted with placeholders in {braces}.
If there's no comment and no job summary at all, see No comment at all.
DiffCost could not analyze this PR#
DiffCost stopped before producing a result. It evaluated no cost policy and posted no new diffcost/cost-check status, and analysis_status is error. A required check waits, unless an earlier run already posted a status on this commit, which still applies. The step still passes unless fail_on_error: true is set.
Git and checkout#
| Message | Fix |
|---|---|
The pull request's base commit {sha} is not in the checkout. / The merge queue's base commit {sha} is not in the checkout. |
Set fetch-depth: 0 on actions/checkout. |
Base branch {ref} was not found |
Set fetch-depth: 0, or fix the base_ref input. |
base_ref must be a branch name, a refs/heads/ or refs/remotes/ ref, or a full commit SHA |
Don't pass a tag or other ref. |
base_ref is not a valid branch name |
Remove spaces and special characters. |
| The checked-out commit does not contain this pull request's head commit | The workflow checked out the wrong code, usually with pull_request_target. Use on: pull_request, or check out ${{ github.event.pull_request.head.sha }} (trusted PRs only). |
The checked-out commit is not the merge queue commit {sha}. |
On a merge_group run, the checkout must be the queue's commit. Remove the ref: from actions/checkout (its default is the queue's commit), or set it to ${{ github.event.merge_group.head_sha }}. |
Could not find the pull request number in the merge queue branch {ref} (expected gh-readonly-queue/<branch>/pr-<number>-<sha>). |
DiffCost reads the PR number from the queue branch GitHub creates. Run DiffCost on merge_group events from GitHub's merge queue only. |
The merge queue event has no base commit (merge_group.base_sha). / The merge queue event has no head commit (merge_group.head_sha). |
The event payload is incomplete. Re-run the job; if it persists, email support. |
| Could not read the git repository: … the workspace is not a git checkout. | Add actions/checkout@v4 before DiffCost. |
| … the workspace is owned by a different user (git safe.directory). | Check out the repository in the same job, with actions/checkout. |
git {command} timed out |
Re-run the job. |
manifest_path (or config_path) must be a relative path without .. components |
Use a path relative to the repository root. |
dbt manifests#
| Message | Fix |
|---|---|
Manifest not found at {path} |
Run dbt compile --target prod before DiffCost, and point manifest_path at the file it writes (<project>/target/manifest.json for a project in a subdirectory). |
| The base branch has a dbt project (…) but no compiled manifest was found. | Run dbt compile before DiffCost, or set sql_paths to analyze plain SQL instead. On PRs from forks this is expected: they can't compile against BigQuery. |
| Found manifests for several dbt projects | Set manifest_path to the one project to analyze. DiffCost supports one run per PR. |
{path} is the copy committed in the repository, not one compiled in this run |
Run dbt compile in the job, and keep target/ out of git. |
{path} has N model(s) but no compiled SQL. |
Use dbt compile, not dbt parse. |
{path} contains no dbt models. |
Compile the right project. |
None of the models in {path} were found |
Point manifest_path at <project>/target/manifest.json, where <project> has the dbt_project.yml on the base branch. |
{path} is a symlink or inside a symlinked directory. |
Remove the symlink; the manifest must be the file dbt compile writes. |
The baseline manifest {file} is a symlink or inside a symlinked directory. |
Remove the symlink from the repository. The baseline must be a downloaded artifact. |
The baseline manifest {file} is committed in the repository |
Remove it from git (on the base branch too). The baseline must come from a downloaded artifact. |
The baseline manifest {file} is the same file as {manifest_path} |
Compile the PR to target/manifest.json and download the main-branch manifest to .diffcost-baseline/ at the repository root. |
The baseline manifest {file} has no compiled SQL |
Save it from dbt compile --target prod on your main branch. |
Could not use the baseline manifest {file}: … |
The downloaded file can't be read or isn't a manifest. Check the artifact download step and re-run the main-branch workflow. |
Could not read manifest.json / manifest.json is not a dbt manifest. |
Recompile, and check the artifact download. |
See dbt projects for the full setup.
DiffCost failed with an unexpected error#
If the log shows "DiffCost failed: {error}" and there's no comment or job summary, DiffCost hit an unexpected error. Re-run the job; if it happens again, email support@diffcost.dev with that log line.
Rows that show Cost unavailable#
The rest of the report is fine; DiffCost couldn't price these queries. Each one is an unknown cost: it's left out of the totals, and it blocks the PR when a threshold applies.
| Row text | Cause and fix |
|---|---|
| ⚠️ PR dry-run failed: Access Denied: … (or Production dry-run failed) | The service account can't read a table, or can't create jobs. Grant roles/bigquery.jobUser on the project and roles/bigquery.dataViewer on every dataset the query reads, including datasets in other projects. See BigQuery access. |
| ⚠️ … dry-run failed: {other BigQuery error} | Usually a syntax error, a template or query parameter in a plain SQL file, or a timeout. Re-run the job for a timeout. See Reproduce a dry-run error. |
| ⚠️ Production dry-run failed: … | The base version no longer runs (for example, a column it reads was dropped). The PR can't fix this row. Have an admin bypass the check, or add the file to ignored_paths in a separate PR to the base branch. |
| ⚠️ dry run failed ({ExceptionType}) | An unexpected error from the BigQuery client. Re-run the job. |
| skipped: dry-run time budget exceeded | The PR has so many changed queries that dry runs took over 10 minutes. Re-run, or split the PR. |
| Cost unavailable: dry run returned no byte estimate | BigQuery returned no estimate. Re-run the job; if it persists, email support. |
| Cost unavailable: a referenced table does not exist yet (unmaterialized upstream) | The base version reads a table that doesn't exist in BigQuery. For dbt, compile both manifests against the production target. Otherwise the PR can't fix this row: have an admin bypass the check, or add the file to ignored_paths in a separate PR. |
| Cost unavailable: the PR version reads a table or column that does not exist yet | Plain SQL: the PR's query reads an object that doesn't exist yet (for example one another file in the same PR creates). Under a threshold this blocks, and it can't clear before the merge. Create the object in an earlier PR, or have an admin bypass the check. |
| Cost unavailable: PR dry run failed, and nothing upstream changes in this PR: … | dbt: the PR's SQL fails, and no upstream model changed, so it's likely a typo. Fix the query. |
| Cost unavailable: the PR version reads an object that does not exist, and nothing upstream creates it in this PR | dbt: check the table name, and that the manifest was compiled against the production target. |
| Cost unavailable: the PR version reads an object that does not exist, and nothing upstream changes in this PR | dbt: BigQuery's error didn't name the object, and no upstream model changed. Check the table names in the model. |
| Cost unavailable: no production baseline (dbt manifest artifact missing) | The main-branch manifest artifact is missing or expired. See Save the main-branch manifest. |
| Cost unavailable: the model file changed but the PR manifest was not recompiled | Run dbt compile in the job before DiffCost. |
| Cost unavailable: the model is not compiled in the … manifest (selective compile?) | Compile the whole project, not with --select. |
non-model dbt files: … can change compiled SQL; their cost can't be determined without a production baseline manifest |
The PR changes macros, seeds, config or packages, and there's no baseline to compare against. Add the baseline artifact. |
SQL outside the analyzed dbt project: … were not analyzed |
The PR changes .sql files outside the dbt project. If they don't run on BigQuery, add them to ignored_paths in .diffcost.yml (the comment shows the line). Make that change in a separate PR to the base branch; the blocked PR picks it up on its next push. |
| Cost unavailable: the file is a symlink in this PR | DiffCost never follows symlinks. Replace it with a regular file. |
⏳ Not costed: depends on objects this PR creates isn't an error: a dbt model reads a column or table that an upstream model adds in the same PR. It counts as an unknown cost, and blocks, only when structural analysis also flags a risk on it, or can't run on a model that already exists. See dbt projects.
Reproduce a dry-run error#
DiffCost cuts error text to 200 characters and replaces backtick-quoted names and any name with three dot-separated parts (table names such as project.dataset.table or project:dataset.table, but also permission names such as bigquery.jobs.create) with <redacted>. Other names, such as a project ID in "Access Denied: Project …" or a column name, appear as BigQuery wrote them. The step log has no more detail than the comment.
- Access Denied: check the roles of the service account (
diffcost-ci), not your own: you probably have more access than it does. - Plain SQL: dry-run the file yourself in the BigQuery console or with
bq query --dry_run --nouse_legacy_sql. DiffCost runs it exactly as written, with no default dataset, so table names must be qualified. - dbt: run
dbt compile --target prod; the compiled SQL of each model is undertarget/compiled/.
No dollar amounts#
A "SQL Change Analysis" report is a structural-only report: risk flags, no costs.
- "No BigQuery credentials configured": set
gcp_credentials, or rungoogle-github-actions/authbefore DiffCost. DiffCost uses onlygcp_credentialsor the credentials file inGOOGLE_APPLICATION_CREDENTIALS; on a self-hosted runner, other Application Default Credentials (such as a VM's metadata server) aren't used. Withcreate_credentials_file: falseon the auth step, no file is written and you get this message. - "BigQuery credentials were provided but could not be used ({reason})":
- "the gcp_credentials input must be the raw JSON key …": store the decoded JSON key, starting with
{, not a base64 string; - "{ErrorType}: …", for example "DefaultCredentialsError: …" (the log says "Failed to create BigQuery client: …"): check the key, the credentials file and the
gcp_projectinput.
- "the gcp_credentials input must be the raw JSON key …": store the decoded JSON key, starting with
- Log: "GOOGLE_APPLICATION_CREDENTIALS points to a file that does not exist in the DiffCost container": the credentials file the auth step wrote is gone, usually because
actions/checkoutran aftergoogle-github-actions/auth(checkout cleans the workspace). Order the steps in one job: checkout, auth, DiffCost. - PRs from forks never get credentials. That's expected.
Workload Identity Federation#
The google-github-actions/auth step itself fails only without id-token: write (an error about ACTIONS_ID_TOKEN_REQUEST_URL): the job stops before DiffCost runs, with no comment and no job summary, and a required check waits. Add id-token: write to the job's permissions.
Google Cloud checks the provider, the attribute condition and the service account binding later, when DiffCost runs its first dry run (or when dbt compile connects). Every row then shows "⚠️ … dry-run failed: …", with parts of the error <redacted>. To see the full error, add token_format: access_token to the auth step for one run: the auth step then does the exchange itself and fails with one of these:
| Error in the auth step | Fix |
|---|---|
invalid_target: "The target service indicated by the "audience" parameters is invalid" |
The provider path must contain the project number and the right pool and provider IDs, and the provider must be enabled. Copy it from gcloud iam workload-identity-pools providers describe (step 2). |
| "Permission 'iam.serviceAccounts.getAccessToken' denied" | This repository has no roles/iam.workloadIdentityUser binding on the service account, or the repository name in the binding differs (capitalization counts). Add it (step 3); a new binding can take a few minutes to apply. |
| "The given credential is rejected by the attribute condition" | assertion.repository_owner in the provider's condition must equal your GitHub organization's login exactly, including capitalization. |
| "IAM Service Account Credentials API has not been used in project … or it is disabled" | Enable iamcredentials.googleapis.com (and the other APIs in the prerequisites). |
Remove token_format again once it works; DiffCost doesn't need it.
On PRs from forks the auth step must be skipped with its if:; see BigQuery access.
Merge blocking and the commit status#
No status was posted#
The comment says "No diffcost/cost-check status was posted ({reason})". If the check is required, the PR can't merge until a run posts one. A status an earlier run posted on the same commit still applies.
| Reason | Fix |
|---|---|
| the DiffCost GitHub App is not installed for this repository | Install the App on the repository (dashboard Settings), or add the repository to the installation's selected repositories. Then re-run the job. On the Free plan, Settings shows no install link: install the App from its GitHub page, or ignore the note (Free never blocks). |
| the DiffCost API key was rejected | Check the DIFFCOST_API_KEY secret and the key's repository scope in Settings → GitHub Action API keys. The repository must belong to your DiffCost organization. Then re-run. |
| the DiffCost service could not be reached | Re-run the job. |
| the DiffCost service rate-limited the request | Too many requests for this key, too many requests from the runner's IP address (shared by every key behind it), or too many failed key checks from that address. Wait a minute and re-run. If it keeps happening, email support. |
| DiffCost could not post the commit status | GitHub refused or didn't answer. Re-run the job. |
| the DiffCost service returned an error | Re-run the job. |
| the DiffCost service did not accept the results | Update to the latest diffcost/action@v1 and re-run. If it persists, email support with the warning from the log. |
| no commit SHA was available for the commit status | Re-run the job on the pull request. |
| merge blocking is not set up on the DiffCost server | A problem on DiffCost's side. Email support@diffcost.dev. |
"The diffcost/cost-check status could not be confirmed" means the DiffCost service didn't answer in time (12 seconds), or answered with something that isn't a DiffCost result (for example a proxy's page). The status may or may not have been posted: check the PR's checks, and re-run the job if there's none.
If the dashboard settings can't be fetched (key rejected, rate-limited or service unreachable), DiffCost checks the .diffcost.yml thresholds on the runner, on any plan, and the dashboard's pricing isn't applied in that run. If the DiffCost service then accepts the results, its verdict (dashboard thresholds and your plan) replaces that local check. Otherwise the comment and the policy_exceeded output show the local result, as "Cost policy exceeded — not enforced" when they're exceeded.
No DiffCost API key#
The comment (or, on a merge-queue run, the job summary) says "🔑 No DiffCost API key, so this run can't block merges", and the log has a notice. The api_key input was empty, so DiffCost ran as the free Action: it sent nothing to the DiffCost service and posted no diffcost/cost-check status. If that check is required, the PR waits for it.
GitHub passes an empty value for a secret that doesn't exist, so DiffCost can't tell a missing secret from a repository that doesn't use a paid plan. Every run without a key logs the notice; every report it writes (comment or job summary) has the line, with the likely cause. A "could not analyze" report doesn't have the line, so check the log there:
| Cause in the comment | Fix |
|---|---|
| pull requests from forks don't get repository secrets | Expected: GitHub gives fork PRs no secrets. An admin has to bypass the required check to merge one. See Merge blocking. |
| Dependabot pull requests only get Dependabot secrets | Add DIFFCOST_API_KEY as a Dependabot secret (Settings → Secrets and variables → Dependabot, in the repository or the organization). |
check that the DIFFCOST_API_KEY secret exists and the workflow passes it as api_key |
Check the secret's name, and that it reaches this repository (an organization secret's repository access; on GitHub Free, private repositories can't use organization secrets). Check that the DiffCost step has api_key: ${{ secrets.DIFFCOST_API_KEY }}. |
If you use DiffCost on the Free plan without an account, you can ignore the line.
The check is stuck on "Expected — Waiting for status to be reported"#
No run has posted a status on the PR's latest commit:
- The workflow didn't run: check for a
paths:filter or a branch filter inon:. - The DiffCost step was skipped: an earlier step failed (the auth step,
dbt compile, the artifact download). Fix it; see Workload Identity Federation. - The
api_keyinput is empty: the secret is missing or misnamed, or the PR comes from Dependabot without the Dependabot secret. DiffCost then posts no status, and the comment says "🔑 No DiffCost API key". See No DiffCost API key. - The PR is from a fork: forks get no secrets, so an admin must bypass the check.
- The PR is in a merge queue: the workflow must also run on
merge_group, and the credential steps'if:must let queue runs through. See Merge queues. - DiffCost couldn't analyze the PR, or crashed: see above.
- The comment has a "No status was posted" note: see the table above.
The PR wasn't blocked, but should have been#
- Free plan: the status says "Merge blocking is off on the Free plan". Check Billing: the trial may have ended or never started (a GitHub organization owner must sign in), or the Stripe subscription is unpaid or paused.
- No threshold: when a query got more expensive, the status says "Worst query +$X/run (no threshold set)". Add one in the dashboard, or in
.diffcost.ymlon the base branch. - The threshold was changed in the same PR:
.diffcost.ymlis read from the base branch, so the change applies after the merge. - The check isn't required, or its source isn't the DiffCost GitHub App. Check the branch protection rule or ruleset.
- Thresholds compare the worst single query, not the PR's total.
The PR is blocked, but shouldn't be#
- "N queries could not be costed": unknown costs block under a threshold. Fix the rows that show Cost unavailable (above).
- The threshold is lower than you think: a dashboard threshold for the repository, or the org-wide threshold (scope
*), replaces.diffcost.yml. The block reason gives the limit that was exceeded: compare it with Settings → Merge-blocking thresholds and the base branch's.diffcost.yml. - A stale status from an earlier run: re-run the DiffCost job.
API key and dashboard#
| Message | Fix |
|---|---|
| DiffCost rejected the API key for {repo} (HTTP 401|403). | The key was revoked or mistyped, or the repository is outside its scope or your organization. Check the secret and Settings → GitHub Action API keys. |
| DiffCost rejected results for {repo} (HTTP 403: API key not scoped for this repo) | Use a key whose scope includes the repository, or *. |
| … API key is not authorized for any repositories | The key's scope is empty. Create a key with a scope. |
| … Repository is not owned by this organization's GitHub account | Keys only work for repositories your organization owns. |
| Could not reach DiffCost backend | The runner couldn't reach the DiffCost service (network problem or outage). Re-run the job. Dashboard settings weren't applied in this run. |
| DiffCost rate-limited this runner (HTTP 429). | Wait a minute and re-run. |
Runs don't show in the dashboard:
- the workflow must pass
api_key; - runs that change no SQL aren't listed;
- runs DiffCost couldn't analyze, and runs whose results the DiffCost service didn't accept (key rejected, rate-limited, unreachable), aren't stored;
- the dashboard shows your plan's history window only (30 days on Free).
Sign-in problems: see Sign in. Messages you may see:
| Message | What to do |
|---|---|
| "Your seat is over the plan's limit" | You joined after the plan's seats ran out. Ask an admin to remove members or upgrade (Seats), or use Switch organization if you moved to another GitHub organization. |
| "{org} has used all {N} seats on its DiffCost plan…" | The organization is full, so new members can't join. Ask an admin to remove a member on the Team page or to upgrade. GitHub organization owners can always join. |
| "A primary email address is required…" | Set a primary email address in your GitHub profile, then sign in again. |
| "DiffCost can't see another GitHub organization of yours to switch to…" | The organization you want probably restricts OAuth App access. Ask one of its owners to approve DiffCost, then sign in again. |
.diffcost.yml warnings#
These warnings appear in the log and in the cost report as "⚠️ Cost policy: …". They make the cost policy weaker than configured, so with fail_on_error: true any of them fails the step.
| Message | Fix |
|---|---|
Could not read {path} from {base}: … Its thresholds were not applied. |
Check config_path, and that the file exists on the base branch. |
{path} on {base} is not valid YAML (…). Its thresholds were not applied. |
Fix the YAML syntax. |
{path} must be a YAML mapping. Its thresholds were not applied. / {path}: thresholds must be a mapping. Its thresholds were not applied. |
Use key: value pairs, as in the example. |
| .diffcost.yml thresholds.{key} must be a number (…). Ignoring it. / must be between 0 and {maximum}. Ignoring it. | Use a plain number: absolute from 0 to 1,000,000, relative from 0 to 100,000. |
These are only in the step's log, and never fail the step:
| Message | Fix |
|---|---|
{path}: default_runs_per_month must be an integer between 1 and 100,000. Using 30. |
Fix the value. |
| slack_webhook must be a https://hooks.slack.com/ URL. Ignoring. | Use a Slack incoming webhook URL. |
| slack_webhook resolved to an empty value. … | Write slack_webhook: ${{ SLACK_WEBHOOK }} and pass env: SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }} on the DiffCost step. Only SLACK_WEBHOOK, SLACK_WEBHOOK_URL and DIFFCOST_SLACK_WEBHOOK are expanded. |
| bq_pricing_model must be 'on_demand' or 'editions' (got '…'). Using on_demand. | Fix the value (input or .diffcost.yml). |
Input cost_per_tb must be a number (got '…'). Using 6.25. |
Fix the input. |
Missing or duplicate PR comments#
| Symptom | Fix |
|---|---|
| Log: "Failed to post PR comment (HTTP 403)" or "Failed to list PR comments (HTTP 403)" | Add permissions: pull-requests: write to the job. On PRs from forks the token is read-only: the report is in the job summary. |
| Two DiffCost comments on one PR | github_token changed (for example from the default token to a GitHub App's) and the new token can't edit the old comment, so DiffCost posted one with the new token. It updates that one from then on; delete the old one. |
| No comment on a PR that changes no SQL | Expected. Set comment_on_clean: true to get one. |
| No comment at all | Check the job summary and the DiffCost step's log. If the DiffCost step was skipped, an earlier step failed: fix it. "Not running in a pull request context" means the workflow isn't triggered by pull_request or merge_group. |
| No comment on a merge-queue run | Expected: queue runs write the report to the job summary only. |
Slack alerts don't arrive#
- Alerts need Starter, Team or the trial, and an API key.
- They're sent only when a run exceeds the cost policy.
- A webhook in
.diffcost.ymlis used only when no dashboard threshold applies to the repository, and only when DiffCost fetched the dashboard settings in that run (not when the key is rejected or the DiffCost service is unreachable). A dashboard threshold uses its own webhook (Settings → Merge-blocking thresholds, "Slack alert" column). - When an alert fails, the PR comment and the job summary say "⚠️ Slack alert failed", naming the webhook (
.diffcost.ymlor dashboard threshold) and Slack's answer, and the log has a warning. A dashboard threshold also shows its last failed alert on its row in Settings → Merge-blocking thresholds, until an alert or a test message succeeds, or you save a new webhook URL. A failed test message shows there too. - Slack's answer tells you what to fix.
no_servicemeans the webhook was revoked or disabled, or its Slack app was removed;invalid_token,channel_not_foundandchannel_is_archivedalso mean the webhook no longer works. Create a new incoming webhook and replace the URL: in the dashboard threshold, or in the secret that.diffcost.ymlreads.action_prohibitedmeans a Slack admin restricted the app. "timed out" means Slack didn't answer in time, and "request failed" that the request didn't reach Slack (a DNS or connection error): the next alert usually goes through. "blocked: …" (dashboard threshold only) means the DiffCost service refused to send to the webhook URL when it checked it again at send time; save the URL again or replace it. - To check a dashboard threshold's webhook without waiting for a blocked PR, use Send test message on the threshold (admins, on Starter, Team or the trial). For a
.diffcost.ymlwebhook, send a test message yourself:curl -X POST -H 'Content-type: application/json' --data '{"text":"DiffCost test"}' "$SLACK_WEBHOOK".
Still stuck?#
Email support@diffcost.dev with the repository, the PR link and the DiffCost step's log. Don't send your SQL or keys: DiffCost support doesn't need them.
Something missing or wrong? Email support@diffcost.dev.