What steps will reproduce the problem?
Work is distributed in small chunks to workers, so the communication and
latency of distributing work and receiving status updates can significantly
affect overall runtime.
What is the expected output? What do you see instead?
Performance should scale more linearly with additional nodes. We should be
able to implement better I/O management. Our first pass will involve KLFM
bi-partitioning. We may implement other algorithms in the future, such as
those that allow replication or k-partitioning.
Original issue reported on code.google.com by
daniel2...@gmail.comon 18 Jan 2008 at 10:20