Whisk - #2

Open

February71st wants to merge 5 commits into

Commits on Apr 22, 2025

Edited preprocessing to fix an error where a mismatch in tokenisation caused punctuation and tokens attached to it would be removed regardless of settings
February71st
committed
953b91e
Cleaned up my previous code in preprocessing
February71st
committed
2c82501
Restored num_processes=10 to multiprocess test in preprocessing, after accidentally removing it while creating a test for my fix.
February71st
committed
01d6e22
Edited my test for readability, added docstring.
February71st
committed
04d7042
Removed unneccessary code from my test and fixed a typo in the docstring.
February71st
committed
35f4d9d