Hi, and thanks for maintaining this list! I consume blocklist.txt in a downstream
generator and hit a data-quality issue I think is worth flagging.
Summary
A subset of lines in blocklist.txt contain two or more domains concatenated with no
separator (no newline, no space). Because the concatenation is still a syntactically
valid FQDN, it passes validity checks — including, I believe, the "Valid 99.975%" badge —
and ends up in the list as a single bogus entry that resolves to nothing.
Evidence
The raw file uses pure \n line endings (no \r), yet some single physical lines hold
several domains run together, e.g.:
images-na.ssl-images-amazon.cominit.itunes.apple.comjs.moatads.com
which is three separate domains glued with zero delimiter:
images-na.ssl-images-amazon.cominit.itunes.apple.comjs.moatads.com
More examples (each is one physical line in the current file):
api-glb-ash.smoot.apple.comat1.listrakbi.com
xp.apple.com131788053.log.optimizely.comaccounts.google.comad.sxp.smartclip.net
0.0.0.0kryptonchain.org # a sink IP glued to the host, also no space
Longest single line I observed — 128 chars, several phishing domains concatenated:
paypal.de-signin-sicherheit-7295.paypal.de-signin-sicherheit-3339.amazon.de-signin-meinkundenservice-4630.amazon.de-signin-siche...
Rough line-length distribution over ~2.66M lines: ~25,800 lines are 51–80 chars and ~767
exceed 80 chars — well beyond a single domain, consistent with concatenation.
Why it can't be fixed downstream
Because apple.comjs is a valid DNS label and the whole string ends in a real TLD, a
consumer can't tell apple.com + js… from one legitimate domain without risking deletion
of real domains. I measured a Public-Suffix-List-based splitter at up to ~37% false
positives on legitimate domains, so the only robust fix is at the source.
Likely cause
A generation/merge step appears to drop line separators when joining records or chunks
(writing without a trailing \n, or concatenating lists without a separator). The
0.0.0.0host cases suggest the same when emitting hosts-format lines.
Suggested fix
- Ensure exactly one domain per line, with a trailing newline on every record.
- A cheap CI guard: after stripping any leading sink IP, reject a line that contains more
than one registrable domain (validate against the Public Suffix List), or flag lines whose
length / label-count is implausible for a single FQDN. (Pure FQDN-syntax validity won't
catch these, since the concatenations are syntactically valid.)
Happy to share the full list of affected lines if that helps. Thanks again!
Hi, and thanks for maintaining this list! I consume
blocklist.txtin a downstreamgenerator and hit a data-quality issue I think is worth flagging.
Summary
A subset of lines in
blocklist.txtcontain two or more domains concatenated with noseparator (no newline, no space). Because the concatenation is still a syntactically
valid FQDN, it passes validity checks — including, I believe, the "Valid 99.975%" badge —
and ends up in the list as a single bogus entry that resolves to nothing.
Evidence
The raw file uses pure
\nline endings (no\r), yet some single physical lines holdseveral domains run together, e.g.:
which is three separate domains glued with zero delimiter:
images-na.ssl-images-amazon.cominit.itunes.apple.comjs.moatads.comMore examples (each is one physical line in the current file):
Longest single line I observed — 128 chars, several phishing domains concatenated:
Rough line-length distribution over ~2.66M lines: ~25,800 lines are 51–80 chars and ~767
exceed 80 chars — well beyond a single domain, consistent with concatenation.
Why it can't be fixed downstream
Because
apple.comjsis a valid DNS label and the whole string ends in a real TLD, aconsumer can't tell
apple.com+js…from one legitimate domain without risking deletionof real domains. I measured a Public-Suffix-List-based splitter at up to ~37% false
positives on legitimate domains, so the only robust fix is at the source.
Likely cause
A generation/merge step appears to drop line separators when joining records or chunks
(writing without a trailing
\n, or concatenating lists without a separator). The0.0.0.0hostcases suggest the same when emitting hosts-format lines.Suggested fix
than one registrable domain (validate against the Public Suffix List), or flag lines whose
length / label-count is implausible for a single FQDN. (Pure FQDN-syntax validity won't
catch these, since the concatenations are syntactically valid.)
Happy to share the full list of affected lines if that helps. Thanks again!