Refactor downloader and management commands for improved file handlin…#303
Refactor downloader and management commands for improved file handlin…#303marianoeramirez wants to merge 2 commits intomasterfrom
Conversation
…g and bulk insert functionality - Updated the Downloader class to use context managers for file and stream handling, enhancing resource management. - Enhanced the needs_downloading method to support local file checks and HTTP HEAD requests for better efficiency. - Introduced bulk_create functionality for country, region, and subregion imports in the management command, allowing for optimized database operations. - Added a BULK_BATCH_SIZE setting to control the number of records processed in bulk, improving performance during imports.
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ptions - Implemented deduplication logic in the management commands for country, region, and subregion imports to prevent duplicate entries during bulk creation. - Added a new configuration option, CITIES_LIGHT_BULK_BATCH_SIZE, in the README to control the batch size for bulk inserts, improving import performance.
…g and bulk insert functionality