Security and data

What data leaves your runner#

The Action runs in your GitHub Actions runner, and analyzes your SQL there.

To Google BigQuery (your project): the full SQL text of each changed query, submitted as a dry-run job with your credentials.

To the DiffCost service (only with an API key):

• repository name, pull request (PR) number, and PR head commit SHA (on a merge-queue run, the queue's commit SHA, and that it's a queue run);
• per-run totals: cost before, after and delta, and bytes before and after;
• counts: files analyzed and queries that could not be costed;
• default_runs_per_month;
• thresholds from .diffcost.yml, if any;
• the PR author's GitHub login, only with send_author: true;
• for each costed query (up to 500: the largest increases first, plus always the largest percentage increase), its bytes and cost before and after, with no identifier, used to evaluate the worst-query threshold.

Every run also sends the repository name to fetch the dashboard settings.

Never sent to the DiffCost service: SQL text, table, dataset or column names, file paths, model names, query identifiers, row data, your BigQuery credentials, or the workflow's GITHUB_TOKEN.

To Slack (only if you configure it):

• For a dashboard threshold's webhook, the DiffCost service sends the alert. It contains the repository, the PR link, the worst query's increase per run and the block reasons, with no query names.
• For a .diffcost.yml webhook, the Action sends the alert from your runner straight to Slack. That alert also names queries: the one with the largest increase, and those in its block reasons. The names never reach the DiffCost service.

What the DiffCost service stores#

For each analyzed run:

• repository, PR number and commit SHA, and whether it was a merge-queue run;
• the PR author, only with send_author: true;
• the per-run totals and bytes listed above, the counts and the runs per month;
• the worst single query's increase per run (with no identifier). The service doesn't keep the per-query list;
• which threshold applied, its limits, the verdict and the block reasons (for example "Worst query +$7.57/run exceeds $5.00/run limit", or the worst percentage increase);
• whether the commit status was posted.

For your organization:

• its GitHub login and ID, and the DiffCost GitHub App's installation ID;
• your thresholds, including Slack webhook URLs (the dashboard never shows them again), and the reason and time of each threshold's last failed Slack alert;
• BigQuery pricing settings;
• API keys: a bcrypt hash, a 12-character prefix, a short SHA-256 check value, the key's name, scope and last use. The full key is shown only once, when it's created;
• billing identifiers: Stripe customer and subscription IDs, or Google Cloud Marketplace account and entitlement IDs;
• an audit log: sign-ins, API keys created and revoked, threshold and settings changes, Slack test messages and their result, role changes, members removed or switching organization, blocked PRs and plan changes.

For each person who signs in: their GitHub ID, login, email, avatar URL, role, join and last sign-in time. Each sign-in's audit entry includes a keyed hash (HMAC) of the client IP address. The key exists only on DiffCost's servers, so the stored value can't be reversed on its own. It's pseudonymous, not anonymous: DiffCost can match it to an IP address.

Outside the database: Google Cloud, which hosts DiffCost, logs every request to diffcost.dev (IP address, browser user agent, URL and time) and keeps those logs 30 days. Database backups are kept 7 days, so deleted data leaves the backups 7 days after it's deleted.

The plan's history window (30 days on Free, 12 months on Starter) limits what the dashboard shows; it doesn't delete anything. The DiffCost service keeps your organization's data until an admin asks support to delete it: email support@diffcost.dev from an admin's address. Anyone can also ask for their own account to be deleted; see the privacy policy.

Permissions DiffCost needs#

Google Cloud: a service account that can create jobs in one project and read the datasets your SQL uses (roles/bigquery.jobUser on the project, roles/bigquery.dataViewer on the datasets). DiffCost only submits dry runs, which return a byte estimate and no rows. Prefer keyless authentication, restricted to your repositories.

The workflow token: contents: read (to diff against the base branch) and pull-requests: write (to post the comment), plus id-token: write for Workload Identity Federation. The token stays in your runner.

The DiffCost GitHub App (merge blocking only): Commit statuses: read and write and Metadata: read. It can't read your code. The DiffCost service uses short-lived installation tokens scoped to the one repository it posts a status on, and only accepts an installation on your organization's own GitHub account.

Sign-in: the DiffCost OAuth App asks for read:org (to check your organization membership and whether you're an owner) and user:email.

Keep the required check trustworthy#

• Make the check's source the DiffCost GitHub App. Any workflow with statuses: write can post a status named diffcost/cost-check; pinning the source in branch protection makes GitHub ignore those.
• Protect the workflow. DiffCost reads .diffcost.yml from the PR's base commit, so a PR can't loosen its own thresholds by editing it. But a PR can edit .github/workflows/*.yml in the same PR, for example to skip DiffCost or to send different numbers with your API key. Protect .github/workflows/** with a CODEOWNERS entry and Require review from Code Owners, or run DiffCost as a required workflow in an organization ruleset.
• Don't use pull_request_target with a checkout of untrusted PR code: it runs that code with your secrets.
• Scope API keys to the repositories that use them, and revoke keys you no longer use. A key works only for repositories your organization's GitHub account owns.

Accounts and sessions#

• DiffCost admins are your GitHub organization owners. DiffCost re-checks roles from GitHub at every sign-in, so a change on GitHub applies at that person's next DiffCost sign-in:
  • a demoted owner keeps admin rights until then;
  • a person removed from the GitHub organization keeps their current session until it expires, and keeps their seat until an admin removes them.
• To cut someone off at once and free their seat, click Remove next to them on the dashboard's Team page.
• A dashboard session lasts 72 hours from sign-in. Signing out ends it immediately.
• Dashboard pages are protected against cross-site request forgery and framing, and use a strict content security policy.

Report a security issue#

Email support@diffcost.dev with "Security" in the subject. Please don't open a public GitHub issue for a vulnerability.