Skip to content

vertexfilter: Clamp exponential filter to finite range - #1079

Closed
carrerasdarren-cell wants to merge 1 commit into
zeux:masterfrom
carrerasdarren-cell:agent/clamp-exp-filter-exponent
Closed

carrerasdarren-cell wants to merge 1 commit into
zeux:masterfrom
carrerasdarren-cell:agent/clamp-exp-filter-exponent

Conversation

@carrerasdarren-cell

Copy link
Copy Markdown

Fixes #1077.

optlog2(FLT_MAX) returns 128, which does not fit the signed 8-bit exponent stored by the exponential filter. At low mantissa precision this made optexp2 construct an invalid scale and triggered undefined behavior during float-to-int conversion.

Clamp the stored exponent to 127, scale that boundary without relying on denormal handling, and constrain the mantissa to the largest range that still decodes to a finite float. The existing asymmetric -2^23 mantissa remains available at ordinary exponents.

The regression covers positive and negative FLT_MAX in shared-vector mode, checks the exact encoded words, and verifies finite decoded values.

Tests:

  • make -j8 config=sanitize test
  • make -j8 config=debug test
  • make -j8 config=release test
  • make -j8 config=coverage test
  • make -j8 config=coverage-scalar test

@zeux zeux added the slop label Aug 21, 2026
@zeux zeux closed this Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Float-cast-overflow in meshopt_encodeFilterExp via unclamped exponent producing -inf

2 participants