fix: compare by equality, not just hash, in ExtendedSet - #40
Merged
toumorokoshi merged 2 commits intoJun 22, 2026
Merged
Conversation
ExtendedSet.__contains__ decided membership purely from the element's
hash, and _hash_element builds a lossy "key:value" string for dicts. So
distinct elements that collide on the hash were treated as equal, and
the append_unique list strategy silently dropped genuinely-distinct
items, e.g. {"a": "1", "b": "2"} vs {"a": "1,b:2"}, {"k": 1} vs
{"k": "1"}, or -1 vs -2 (hash(-1) == hash(-2)).
Group elements into hash-keyed buckets and verify equality against the
bucket members on insert and lookup. The hash is only a bucket key; the
== comparison is the arbiter. Equal dicts still dedupe regardless of key
order, and the public API is unchanged.
Owner
|
Thanks for the change, and great catch! if you can fix the lint errors, LGTM. |
Contributor
Author
|
Thanks. I pushed a Black formatting fix for Local checks run: uvx --python 3.12 black --check deepmerge
.venv/bin/validate-pyproject pyproject.toml
.venv/bin/mypy deepmerge
.venv/bin/pytest deepmergeThe new GitHub Actions run is currently waiting for maintainer approval ( |
Owner
|
great, thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
append_uniquelist strategy silently drops genuinely-distinct elements when they collide on a hash.ExtendedSet.__contains__decides membership purely from the element's hash and never checks equality, and_hash_elementbuilds a lossy"key:value"string for dicts:Three independent collision classes, all silently wrong:
{"a": "1", "b": "2"}vs{"a": "1,b:2"}"a:1,b:2"{"k": 1}vs{"k": "1"}"k:1"-1vs-2hash(-1) == hash(-2) == -2-2 in ExtendedSet([-1])isTrueFalseCause
ExtendedSetstored a single{hash: element}mapping and__contains__returnedself._hash_element(obj) in self._values_by_hash— a hash-only comparison. Equality was never consulted, so any hash collision (including the lossy dict-string hash) aliased distinct values together.Fix
Group elements into hash-keyed buckets (
dict[int, list]) and verify true==against the bucket members on both insert and lookup. The hash is only a bucket key; equality is the arbiter. No public API change.This preserves the existing intended behavior — equal dicts still dedupe regardless of key order (
test_strategy_append_similar_dict) and distinct dicts are still both kept (test_strategy_append_unique_nested_dict).Verification
test_strategy_append_unique_keeps_hash_colliding_dictsandtest_strategy_append_unique_hashable_hash_collision, covering all three collision classes. They fail before the fix and pass after.black --check,mypy, and the existingappend_unique/append_similartests all pass.This pull request was prepared with the assistance of AI, under my direction and review.