How I built an automated cloud warehouse pricing tracker

Every "BigQuery vs Snowflake cost" article I found while researching this was at least a year old, and cloud pricing changes often enough that the numbers were already wrong.

I wanted something that stayed correct without me touching it, so I built bigdataexplained.com: a static site (Astro, deployed on Cloudflare Pages) backed by a GitHub Actions job that runs daily, pulls current rates from each vendor's pricing API where one exists, and falls back to the published rate card where it doesn't.

A few things I had to solve:

• Detecting real price changes vs noise. The daily job diffs the incoming dataset against the last one, ignoring timestamp fields, so a commit only says "price changed" when a rate actually moved.

• Not silently going stale. If a vendor API starts failing, the site keeps the last confirmed value but flags it as "refresh failing" rather than pretending it's current.

• Making it useful, not just accurate. The site has a calculator for your own workload shape, and head-to-head pages comparing any two engines across data volumes and workload profiles (BI dashboard, ETL, ad hoc analytics).

It's fully static, no backend, no database beyond a committed JSON file that IS the database.

Live at https://bigdataexplained.com — happy to answer questions about the pipeline or the pricing model assumptions.