Skip to content

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

Amex Transaction Categorization

Money-laundering detection on 63M+ synthetic financial transactions, built as a 9-stage ML pipeline: data cleaning, EDA, graph feature engineering, model selection, training, evaluation, and a bias-mitigation ablation. Full stage-by-stage writeups, scripts, and rationale live in ML_Cycle/, starting at 1-Problem_Definition.

Results

The honesty check: subsampled vs. real-world evaluation

Model selection (stage 6) compares candidates on a fast, subsampled dataset (3.23% positive) for speed. That number is optimistic: the true laundering rate is roughly 1 in 1,200 transactions (0.082%). Stage 7 re-measures the selected model on 6.3M held-out transactions at the real rate to get production-honest numbers.

Subsampled vs real-world evaluation

Model selection: don't double-correct for class imbalance

Comparing a plain baseline against scale_pos_weight-weighted configs on top of already-subsampled data (stage 6), the weighted configs look fine on PR-AUC but collapse on F1 at a fixed threshold. Negative subsampling and scale_pos_weight are both imbalance corrections; stacking them over-corrects.

Model selection comparison

Bias-mitigation ablation: is the model just reading Payment Format?

The trained model's true positives are 99.9% Payment Format == ACH. Stage 9 tests whether that's a shortcut by retraining with the column removed entirely. The model still catches almost exclusively ACH transactions, even without ever seeing the label.

True positives by payment format

Removing the column costs overall PR-AUC and F1 but improves recall at a strict 90%-precision operating point. That points to the ACH concentration being a genuine property of how the underlying laundering patterns route through the data, not a shortcut a model could be trivially engineered out of.

Ablation metrics comparison

Summary table

Stage Metric Value
Stage 6: Model Selection (subsampled) PR-AUC / F1 @ 0.5 / Recall @ 90% precision 0.840 / 0.785 / 0.585
Stage 7: Model Training (real-world eval) PR-AUC / F1 @ 0.5 / Recall @ 90% precision 0.419 / 0.308 / 0.122
Stage 8: Model Evaluation Best-F1 threshold (vs. arbitrary 0.5) 0.822 (Precision 0.618, Recall 0.364, F1 0.458)
Stage 8: Model Evaluation Inference latency (p50 / p95) 2.70 ms / 8.90 ms
Stage 8: Model Evaluation Batch throughput @ 100K rows 2.02M transactions/sec
Stage 9: Bias Mitigation (ablated, no Payment Format) PR-AUC / F1 @ 0.5 / Recall @ 90% precision 0.360 / 0.280 / 0.164

Full methodology, rationale, and honest caveats for every number above are in each stage's own README under ML_Cycle/.

Dataset

Source: IBM Transactions for Anti Money Laundering (AML) (Kaggle).

Downloaded subset: Medium, both illicit-ratio variants (HI = higher illicit ratio, LI = lower illicit ratio), stored in dataset/.

HI-Medium_Trans.csv / LI-Medium_Trans.csv

Transaction records.

Column Description
Timestamp Date and time of the transaction
From Bank Bank ID of the sending account
Account (1st) Account number of the sender
To Bank Bank ID of the receiving account
Account (2nd) Account number of the receiver
Amount Received Transaction amount as received (in receiving currency)
Receiving Currency Currency of the amount received
Amount Paid Transaction amount as paid (in payment currency)
Payment Currency Currency of the amount paid
Payment Format Payment method/channel (e.g. ACH, Wire, Credit Card, Cheque)
Is Laundering Label: 1 if the transaction is part of a laundering pattern, else 0

Note: the raw CSV header repeats the column name Account for both the sender and receiver account fields.

HI-Medium_accounts.csv / LI-Medium_accounts.csv

Account-to-entity mapping.

Column Description
Bank Name Name of the bank holding the account
Bank ID Numeric ID of the bank (matches From Bank / To Bank in the transactions file)
Account Number Account number (matches Account in the transactions file)
Entity ID ID of the entity (individual/organization) that owns the account
Entity Name Name of the owning entity

HI-Medium_Patterns.txt / LI-Medium_Patterns.txt

Plain-text listing of the labeled laundering patterns embedded in the transaction data. Each block is delimited by BEGIN LAUNDERING ATTEMPT - <PATTERN TYPE> / END LAUNDERING ATTEMPT - <PATTERN TYPE> and contains the constituent transactions in the same column order as the Trans.csv files:

Timestamp, From Bank, From Account, To Bank, To Account, Amount Received, Receiving Currency, Amount Paid, Payment Currency, Payment Format, Is Laundering

Pattern types observed include STACK, CYCLE, and FAN-IN (fan-out and other topologies also appear in the full dataset).

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages