The current architecture of Gutenberg2zim suffers from from tight coupling and poor testability as described by @benoit74 here: #422 (comment).
From what I've realized that there is also a new architecture plan going on to introduce new ingestions/scrapers, example: openzim/zim-requests#1258 but this is another discussion which is going on here: #94.
Out of these two what are we leaning towards mostly? From what I think is that Gutenberg2zim has grown organically with time and now it probably requires an architecture refactor which would probably enhance testability and also with loosely coupled architecture it would significantly enhance performance and maintainability too.
Introducing new scrapers would require to extend this repo in which I probably think (Monorepo + Common Library) would be a good choice for this but again this requires a distinct discussion which is probably going on and also what I think is that building multi-source on top of the current architecture would compound existing problems.
We can keep this as a parent issue for other architecture related issues that will be opened in near future.
The current architecture of Gutenberg2zim suffers from from tight coupling and poor testability as described by @benoit74 here: #422 (comment).
From what I've realized that there is also a new architecture plan going on to introduce new ingestions/scrapers, example: openzim/zim-requests#1258 but this is another discussion which is going on here: #94.
Out of these two what are we leaning towards mostly? From what I think is that Gutenberg2zim has grown organically with time and now it probably requires an architecture refactor which would probably enhance testability and also with loosely coupled architecture it would significantly enhance performance and maintainability too.
Introducing new scrapers would require to extend this repo in which I probably think (Monorepo + Common Library) would be a good choice for this but again this requires a distinct discussion which is probably going on and also what I think is that building multi-source on top of the current architecture would compound existing problems.
We can keep this as a parent issue for other architecture related issues that will be opened in near future.