docs: expand README and prepare 0.4.0 changelog (#199)

Author: tisonkun

CHANGELOG.md

@@ -7,22 +7,29 @@ All significant changes to this project will be documented in this file.
 ### Breaking changes
 
 * Move the `hash_value` module to `hash::value`.
-* Remove `ThetaSketch::builder`, `ThetaUnion::builder`, and `TupleSketch::builder`. Construct `ThetaSketchBuilder`, `ThetaUnionBuilder`, and `TupleSketchBuilder` with `Default::default` instead.
-* Standardize Theta and Tuple set-operation constructors. Zero-configuration operators use
-  `Default::default()`, `TupleIntersection::new(policy)` uses the default seed, and `with_seed`
-  accepts a custom seed. This replaces the previous `new(seed)` and `new_with_default_seed`
-  methods.
+* Rename `Coupon::from_hash` to `Coupon::from_value` to reflect that the method hashes the supplied value itself.
+* Remove `ThetaSketch::builder`; construct `ThetaSketchBuilder` with `Default::default` instead.
+* Replace the sealed `ThetaSketchView` trait with a concrete borrowed `ThetaSketchView<'a>`. Call `as_view()` when a view value is needed; Theta set operations continue to accept references to mutable and compact sketches directly.
+* Change `ThetaSketch` and `CompactThetaSketch` iterators to yield `ThetaEntry` instead of raw `u64` hashes. Call `ThetaEntry::hash` to access a retained hash.
+* Replace `ThetaIntersection::new(seed)` and `new_with_default_seed()` with `with_seed(seed)` and `Default::default()`, respectively. Replace `result()` and `result_with_ordered(ordered)` with `to_sketch(ordered)`.
+* Make `CountMinValue` and `UnsignedCountMinValue` marker traits. Their previously exposed numeric constants and helper methods have been removed; use the corresponding primitive integer operations and conversions directly.
 
 ### New features
 
+* Add Tuple sketches behind the `tuple` feature, including custom summary update and combination policies, serialization, compact sketches, union, intersection, A-not-B, Jaccard similarity, and exact sketch-state equality checks that ignore summary values.
+* Add Theta union, A-not-B, Jaccard similarity, and exact sketch-state equality checks.
 * `FrequentItemsSketch` now supports borrowed-key updates via `update_ref` and `update_with_count_ref`, allowing sketches such as `FrequentItemsSketch<String>` to update from `&str` without allocating on existing-key hits. Frequency queries also accept borrowed key forms matching `Borrow<Q>`.
 * `FrequentItemsSketch` no longer requires item types to implement `Clone` for core updates, queries, and serialization. Custom `FrequentItemValue` implementations can now be non-`Clone`; APIs that return or merge owned items still require `Clone`.
-* `CountMinSketch` and `FrequentItemsSketch` now expose `estimated_size()`, reporting the in-memory footprint of the sketch in bytes, following the other sketches.
-* The stateful set operations `HllUnion`, `CpcUnion`, `ThetaUnion`, `ThetaIntersection`, `TupleUnion`, and `TupleIntersection` now expose `estimated_size()`, reporting the in-memory footprint of the operator's internal state in bytes.
+* Add `estimated_size()` to `BloomFilter`, `CountMinSketch`, `CpcSketch`, `FrequentItemsSketch`, `HllSketch`, `TDigestMut`, `TDigest`, mutable and compact Theta and Tuple sketches, and the stateful `HllUnion`, `CpcUnion`, `ThetaUnion`, `ThetaIntersection`, `TupleUnion`, and `TupleIntersection` operators.
 
 ### Bug fixes
 
+* HLL serialization now emits the compact auxiliary-map flag required for Java to read HLL4 images and matches Java and C++ coupon ordering for compact Set images.
 * `FrequentItemsSketch::serialize` now writes the full 8-byte preamble for an empty sketch, matching the Java and C++ encoding. Empty sketches previously serialized to 6 bytes, which `FrequentItemsSketch::deserialize` rejected with an insufficient-data error.
+* `FrequentItemsSketch` now preserves total weight and error state across serialization and merge when a purge removes every active item.
+* T-Digest interpolation now keeps quantiles finite for extreme finite inputs. Deserialization rejects non-finite extrema, invalid centroid weights, and total-weight overflow as invalid data instead of allowing invalid state or panicking.
+* Legacy Theta version 2 exact images with retained entries now deserialize as non-empty sketches and preserve their entries and exact estimates.
+* Bloom filter deserialization now rejects inconsistent cached bit counts, preventing malformed images from hiding populated bits and violating the no-false-negative guarantee.
 * `CpcSketch` and `CpcWrapper` now classify out-of-range fields in serialized images as `InvalidData` rather than `InvalidArgument`.
 
 ## v0.3.0 (2026-05-18)

README.md

@@ -35,15 +35,74 @@
 [actions-badge]: https://github.com/apache/datasketches-rust/actions/workflows/ci.yml/badge.svg
 [actions-url]: https://github.com/apache/datasketches-rust/actions/workflows/ci.yml
 
-This is the core Rust component of the DataSketches library.  It contains a subset of the sketching algorithms and can be accessed directly from user applications.
+Apache DataSketches Rust provides stochastic streaming algorithms for answering queries over large data sets with compact, mergeable summaries. It is the core Rust component of Apache DataSketches and currently implements a subset of the algorithms available in the other language components.
 
-Note that we have parallel core library components for Java, C++, Python, and Go implementations of many of the same sketch algorithms:
+## Getting started
 
-- [datasketches-java](https://github.com/apache/datasketches-java)
-- [datasketches-cpp](https://github.com/apache/datasketches-cpp)
-- [datasketches-python](https://github.com/apache/datasketches-python)
-- [datasketches-go](https://github.com/apache/datasketches-go)
+Sketch implementations are opt-in Cargo features; the crate enables none by default. For example, add the HyperLogLog implementation with:
 
-Please visit the main [DataSketches website](https://datasketches.apache.org) for more information.
+```shell
+cargo add datasketches --features hll
+```
 
-If you are interested in making contributions to this site, please see our [Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.
+Then build a sketch and query its distinct-count estimate:
+
+```rust
+use datasketches::hll::HllSketch;
+use datasketches::hll::HllType;
+
+let mut sketch = HllSketch::new(12, HllType::Hll8);
+for user in ["alice", "bob", "alice", "carol"] {
+    sketch.update(user);
+}
+
+assert!(sketch.estimate() >= 3.0);
+```
+
+Enable multiple algorithms by listing their features together, such as `features = ["hll", "theta"]` in `Cargo.toml`.
+
+## Available sketches
+
+| Feature | Main types | Use case |
+| --- | --- | --- |
+| `bloom` | `BloomFilter` | Space-efficient probabilistic set membership with a configurable false-positive rate. |
+| `countmin` | `CountMinSketch` | Approximate point-frequency queries over a stream. |
+| `cpc` | `CpcSketch`, `CpcUnion`, `CpcWrapper` | Highly compact distinct-count estimation and unions. |
+| `frequencies` | `FrequentItemsSketch` | Heavy-hitter discovery with upper and lower frequency bounds. |
+| `hll` | `HllSketch`, `HllUnion` | Fast distinct-count estimation and unions. |
+| `tdigest` | `TDigestMut`, `TDigest` | Quantile and rank estimation, with high accuracy near distribution tails. |
+| `theta` | `ThetaSketch` and set operations | Distinct counts, set expressions, and Jaccard similarity. |
+| `tuple` | `TupleSketch` and set operations | Theta-style keys with user-defined summaries attached to retained entries. |
+
+See the [API documentation](https://docs.rs/datasketches) for configuration, accuracy guarantees, serialization, and examples for each algorithm.
+
+## Compatibility
+
+The minimum supported Rust version is 1.86.0. The crate currently supports little-endian targets only.
+
+Supported serialization formats are tested with fixtures produced by Apache DataSketches Java, C++, and Go through the [DataSketches TCK](https://github.com/apache/datasketches-tck). When values must hash identically across language implementations, use the compatibility wrappers in `hash::value`.
+
+See the [changelog](CHANGELOG.md) for release notes and migration guidance.
+
+## Other language implementations
+
+Apache DataSketches also provides core library components for other languages:
+
+- [Java](https://github.com/apache/datasketches-java)
+- [C++](https://github.com/apache/datasketches-cpp)
+- [Python](https://github.com/apache/datasketches-python)
+- [Go](https://github.com/apache/datasketches-go)
+
+Visit the [Apache DataSketches website](https://datasketches.apache.org) for algorithm documentation, research background, and project-wide resources.
+
+## Community and contributing
+
+Questions, bug reports, and feature requests are welcome through [GitHub issues](https://github.com/apache/datasketches-rust/issues) and [GitHub discussions](https://github.com/apache/datasketches-rust/discussions). The [Apache DataSketches community page](https://datasketches.apache.org/docs/Community/) lists the public mailing lists and other ways to participate.
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) to build, test, and contribute to the Rust component. All project participation is governed by the [Apache Software Foundation Code of Conduct](https://www.apache.org/foundation/policies/conduct.html).
+
+To report a security vulnerability, follow the [ASF security reporting process](https://www.apache.org/security/) instead of opening a public issue.
+
+## License
+
+Licensed under the [Apache License, Version 2.0](LICENSE).