Monitoring

Where to look#

Review cost changes in the dashboard#

The Overview shows four 30-day figures (PRs analyzed, cost added, cost removed, PRs blocked) next to the previous 30 days, the net cost change chart, and every analyzed run. Open a run to see why it passed or was blocked, and whether the commit status was posted. Filters are kept in the page URL, so you can bookmark a filtered view such as Over threshold and check it regularly.

The Details link of the diffcost/cost-check status on GitHub opens the dashboard overview.

The dashboard only lists runs the DiffCost service received: runs that DiffCost couldn't analyze, and runs whose results the service didn't accept, aren't stored. Merge-queue runs are listed with a Merge queue badge and aren't counted in the four numbers or the trend. See Using the dashboard for what each number means.

Slack alerts#

On Starter, Team and the trial, DiffCost can post to Slack when a run exceeds your cost policy: add a webhook to a dashboard threshold, or slack_webhook to .diffcost.yml. Runs within the policy send nothing, so the channel stays quiet otherwise. Each run that exceeds the policy sends one alert, so every push to an over-limit PR alerts again. See Thresholds and alerts.

If Slack refuses an alert (for example, the webhook was revoked), the PR comment and the job summary say "⚠️ Slack alert failed" with Slack's answer, and the log has a warning. A dashboard threshold also shows its last failed alert in Settings → Merge-blocking thresholds, where Send test message checks its webhook. See Slack alerts don't arrive.

Use the Action outputs#

The DiffCost step sets outputs that later steps can act on: fail the job, send an alert, label the PR, or send numbers to your own monitoring. Give the step an id:

      - uses: diffcost/action@v1
        id: diffcost
        with:
          api_key: ${{ secrets.DIFFCOST_API_KEY }}

      # Warn when the cost policy was exceeded but no failing status was confirmed
      - if: steps.diffcost.outputs.policy_exceeded == 'true' && steps.diffcost.outputs.blocked != 'true'
        run: |
          echo "::warning::DiffCost: cost policy exceeded, not enforced or unconfirmed (worst query +\$${{ steps.diffcost.outputs.max_query_cost_delta }}/run)"

      # Flag runs where DiffCost could not analyze the PR
      - if: steps.diffcost.outputs.analysis_status == 'error'
        run: |
          echo "::warning::DiffCost could not analyze this PR. See the DiffCost step log."

To fail the job instead, add exit 1 as the next line of a run: block. A failing job only blocks the merge if that job's check is also required in branch protection.

• policy_exceeded is true whenever the cost policy was exceeded, enforced or not. It is evaluated with an API key on a paid plan or the trial. It is also evaluated whenever an API key is set but DiffCost couldn't fetch the dashboard settings (key rejected, rate-limited or service unreachable): then the Action checks the .diffcost.yml thresholds on the runner, whatever your plan. Otherwise it is false.
• blocked is true only when the failing status was posted.
• analysis_status is error when DiffCost couldn't analyze the PR. The step still passes unless you set fail_on_error: true.

Check that enforcement is healthy#

A required check only protects you if DiffCost posts it on every PR, and only fails when a threshold applies. Check these when you set DiffCost up, whenever you add repositories, every month, and before the Team trial ends. DiffCost sends no email about the trial, your plan or payments, so these checks are how you find out.

1. Plan: Billing shows Starter or Team (or the trial, with days left), with no "Your Stripe subscription isn't active" message. On the Free plan, including after the trial ends or while the Stripe subscription is unpaid or paused, no threshold is evaluated and, with the DiffCost GitHub App installed, every status passes ("Merge blocking is off on the Free plan").
2. Settings badge: Settings → Merge-blocking thresholds shows "GitHub App installed for all repositories" or "for selected repositories" (or "Commit statuses enabled"). With selected repositories, make sure every repository that runs DiffCost is selected.
3. A threshold applies: open a recent passing run in Pull requests. Its details should name the threshold it was within (repository, org-wide or .diffcost.yml), not "No threshold applied to this repository when this ran." A deleted policy, a downgrade from Team to Starter (per-repository policies become inactive) or an invalid .diffcost.yml all leave a repository without a threshold.
4. .diffcost.yml warnings: recent PR comments have no "⚠️ Cost policy: …" line.
5. "Over threshold, not enforced" rows: in Pull requests, a row with this status was over your cost policy, but GitHub didn't block it. Open it: the GitHub commit status line says why, for example "GitHub App not installed for this repository".
6. API key activity: Settings → GitHub Action API keys → Last used shows when each key was last used. A key that should be in use but shows an old date points to a workflow that stopped running or uses another key. Last used is per key: to see it per repository, use one key per repository, or filter Pull requests by repository.
7. Required check: in each repository's branch protection or ruleset, diffcost/cost-check is required, with the DiffCost GitHub App as its source.
8. dbt baseline: the main-branch manifest artifact exists and is recent (the scheduled run keeps it from expiring). Without it, PRs that modify existing models show Cost unavailable.
9. PR comments: the comment says "not enforced", "enforcement unconfirmed" or "No diffcost/cost-check status was posted" whenever something is wrong, with the reason and what to do. See Troubleshooting.

The job summary and logs#

DiffCost writes its report to the workflow run's job summary on every PR and merge-queue run. This includes PRs from forks, where DiffCost can't comment, and runs it couldn't analyze. A crash leaves only the error in the log.

The DiffCost step's log has run-level messages as GitHub annotations: invalid configuration, a missing or rejected API key, a missing status, a failed Slack alert. Per-query problems (dry-run errors, Cost unavailable rows) are only in the comment and the job summary.

Audit log#

Enterprise plans include an audit log of admin and billing actions: sign-ins, API keys created and revoked, threshold and settings changes, Slack test messages and their result, role changes, members removed or switching organization, blocked runs and plan changes. Email support@diffcost.dev for access.