ACME Million: Stream Nested Sales Data and Audit Its Metadata

The challenge

A million transactions is plenty to keep a spreadsheet busy. Build a dataset you can query in small batches, try extracting a few PDF receipts, and see whether the summary tells the whole story.

Start here

Open the SharePoint folder linked above. mydataset.csv is about 407 MB, mydataset.json about 1,004 MB, mydataset.pdf about 35 MB with 102,642 pages, and mydataset.stats.json is a small summary. Inspect a few CSV rows, the JSON root structure, and selected PDF pages instead of opening all data in a spreadsheet.

Try it

1. Produce normalized transactions and transaction_items tables using limited memory.

2. Explain how to extract the printed receipt fields and which reference fields are absent from the PDF.

3. Recompute count, revenue, customer count, loyalty rate, units sold, and the observed date range.

4. Investigate discrepancies between item subtotals and transaction totals, and between metadata dates and actual records.

What to build

Build a schema and transformation walkthrough, a small PDF extraction sample, full-data aggregate answers, and a discrepancy report. A database, partitioned CSV, or JSON Lines output is suitable; include how many PDF receipts you tried.