The BigQuery Safety Net

Last time, I said:

[BigQuery] doesn’t offer a “don’t bankrupt me without asking first” setting.

After further work, I find that’s not true! This setting is available in the UI, just a bit tricky to find. More importantly, there’s another set of controls elsewhere that you need to know about if you want to use BigQuery safely.

Thanks to Equal Experts for supporting this content.

Equal Experts logo [Read More]

$1,370 Gone in Sixty Seconds

Given a historical view of the Safety DB, I need to get a historical view of PyPI downloads. The source dataset contains 274TB at time of writing, and grows every day - one naive query against that table could cost $1,370. A query over a single day can scan hundreds of GB, so I need to do some work to make it usable.

Thanks to Equal Experts for supporting this content.

Equal Experts logo [Read More]