Pipelines and Python
Loading, cleaning and scheduling data with Python and plain SQL: what broke and how you fixed it.
Rules for this space
- A small example that fails beats a long script.
- Say what a correct result would look like.
Loading, cleaning and scheduling data with Python and plain SQL: what broke and how you fixed it.
My load job inserts the previous day's sales into a reporting table. One night it failed halfway, I re-ran it, and now that day has twice the rows and the totals are wrong. I do…
Question#idempotency#pipelinesby chloe_dubois3 replies