For future considerations we can add the ability to do distributed loading so you can use the dask-dashboard which is helpful for debugging these types of things. Is this a typical dataset size for you? Are you interested in taking larger datasets? If so then we can spend a bit of time streamlining/optimizing this workflow.
Originally posted by @CSSFrancis in #329 (comment)
It should be very similar to PRs #162 and #267 since the ripple uses memmap.
For future considerations we can add the ability to do distributed loading so you can use the dask-dashboard which is helpful for debugging these types of things. Is this a typical dataset size for you? Are you interested in taking larger datasets? If so then we can spend a bit of time streamlining/optimizing this workflow.
Originally posted by @CSSFrancis in #329 (comment)
It should be very similar to PRs #162 and #267 since the
rippleusesmemmap.