Skip to content

Add MaxMind GeoLite Country crawler - #242

Merged
m-appel merged 3 commits into
InternetHealthReport:mainfrom
m-appel:add-maxmind-crawler
Jul 15, 2026
Merged

m-appel merged 3 commits into
InternetHealthReport:mainfrom
m-appel:add-maxmind-crawler

Conversation

@m-appel

@m-appel m-appel commented Jul 13, 2026

Copy link
Copy Markdown
Member

Adds support for the free MaxMind IP geolocation database. This data is already integrated via the ihr.rov crawler, but our modeling has changed and geolocation is not really the purpose of the ROV crawler.

Separating this into its own crawler makes things more logical, so the country information is not imported by ihr.rov anymore.

Description

The imported data has pretty much the same format as the existing IPinfo crawler. Map GeoPrefix to Country nodes. Relationship properties are a bit different, but only included for completeness anyways.

Also removes the Country part from ihr.rov.

Motivation and Context

Currently we already include MaxMind data (although not all of it) via the ihr.rov crawler. However, this is a bit unclean and also uses the old model where we mapped BGPPrefix nodes to Country nodes. Separating these parts makes everything cleaner. We can also use it to compare IPinfo and MaxMind discrepancies.

How Has This Been Tested?

Tested the new crawler and also checked that ihr.rov still works.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

m-appel added 2 commits July 13, 2026 11:41
Adds support for the free MaxMind IP geolocation database. This data is
already integrated via the ihr.rov crawler, but our modeling has changed
and geolocation is not really the purpose of the ROV crawler.

Separating this into its own crawler makes things more logical, so the
country information is not imported by ihr.rov anymore.
@m-appel
m-appel requested a review from romain-fontugne July 13, 2026 11:56

@romain-fontugne romain-fontugne left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just a couple typos in the doc.

Comment thread iyp/crawlers/maxmind/README.md Outdated
Comment thread ACKNOWLEDGMENTS.md Outdated
Co-authored-by: Romain Fontugne <romain-fontugne@users.noreply.github.com>
@m-appel
m-appel merged commit 9d1541b into InternetHealthReport:main Jul 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants