Transaction Network 3: Detect a Duplicate Dataset Before Combining It

The challenge

A new folder arrives with familiar-looking transactions. Is it new activity or a duplicate delivery? Find out, then build an import that keeps the delivery history without counting the same transaction twice.

Start here

Open the SharePoint folder linked above. It contains transactions.pdf, transactions.csv, and transactions.json. Compare them with the identically named files in fraud_sim_2 when that folder is available. Preview this folder's PDF and inspect its own CSV/JSON so the ingestion procedure also works independently.

Try it

1. Establish whether the two folders contain independent data using file hashes and record keys.

2. Normalize this folder's records and document what its PDF can recover.

3. Calculate raw imported row count versus unique logical row count when both deliveries are ingested.

4. Explain the effect of using one folder for training and the other for testing.

What to build

Build a delivery manifest with filenames, byte lengths, and hashes; one main ledger; a record-to-delivery source details mapping; and a short duplicate and split-leakage report. An extracted seven-field PDF ledger is an optional second input route.