Repository navigation
Implement experimental meshlet codec - #1003
Merged
Merged
Conversation
This will be used to test various approaches to encoding meshlet data. As a baseline, we try to encode topology using index codec (and, for size estimation purposes, strip the "tail" which is a fixed codeaux table) and the vertex references using index sequence codec. This is not competitive on size or decoding performance but gives a baseline we can refine later.
For now no actual encoding or size estimation is taking place, but this allows us to add functionality here in the future. The initial version will be based on parts of indexcodec so this replicates some helper functions that we can use later.
This is a significantly simplified version of index codec, adapted for meshlets and to allow simpler decoding flow. Each triangle is encoded using a 4-bit code: 12 cases are used for edge reuse (6 edges + 1-bit next); the remaining 4 cases are used for 3-vertex triangles, with anywhere between 0 and 3 extra vertices per triangle. Triangles are rotated so that "next" is the third vertex for edge reuse, and so that the explicitly encoded triangles have a sequential pattern of "next". While this scheme *could* keep the 3/2 edges pushed based on the case, we are targeting a branchless decode that will make this more difficult; as such, restart codes also only push 2 triangle edges. That results in occasional issues in edge reuse but they can be fixed in the future with triangle pre-rotation. The size estimation currently assumes 6-bit extra references (maxv 64).
We now write the triangle codes as well as extra vertices to the output stream. No buffer size bounds checks are performed for simplicity; this can all be adjusted in the future, but the current encoding is very likely to see significant changes in the future to improve decoding performance potential. For simplicity, extra vertices are written using full 8 bits, which changes the estimation; this can be reworked later. Also, the previous estimates were accidentally "encoding" 8 edge references instead of the maximum 6 permitted, which this change fixes too.
Since encodeMeshlet now actually encodes the data, we can compute the sizes as-if the extras are using 6 bit or 8 bit encoding by adjusting the sizes returned. Also split IB/VB encoding for readability.
The decoding loop has a branch on restart codes; restart codes are relatively rare so this results in a fairly low rate of branch mispredictions. To manage the FIFO, we use a 3-element array with two edges encoded in a single uint. This seems to be faster than some other options like using a 64-bit integer to store 3 edges with shifts, and also means we can use a vertex order that is similar to final decoded triangle order (0xcba), so storing the final output is just an extra shift away. During decoding, we need to read 1 extra byte (unconditinally) for reuse codes and 1-3 extra bytes for restart codes. This means that for valid streams we can over-read by up to 1 byte, but there's currently no bounds checking performed because where and how these are done will depend on final framing. Compared to existing index codec, which decodes meshlets at ~0.25B tri/s on Zen 4, this code decodes meshlets at ~0.45B tri/s. Also properly mark encode/decode as experimental.
After encoding each meshlet we now immediately decode it and validate the output triangle sequence. It should match the original triangle order, but each triangle may be rotated, so we check all 3 rotation candidates.
Because each meshlet is fairly fast to decode, instead of measuring the original invocation of decodeMeshlet (which is difficult to time well due to timing overhead), we concatenate all encoded meshlets with a basic frame to allow us to decode the entire stream, and measure the full decoding process.
This change introduces the initial version of SIMD based decoding that decodes 4b triangle codes in pairs, using 8b extras. To achieve fast and branchless decoding, the entire decoding state is packed into a single 16 byte register; to decode a single triangle, we need the FIFO (3 triangles = 9 bytes), next (1 byte) and up to 3 extra values, for a total of 13 bytes. The first insight is that given a register with 13 bytes of data, we can use pshufb + add to produce the new state, where the new triangle is pushed to the FIFO and the old triangle is evicted. Each triangle code needs a unique pshufb/add vector, and can be loaded from a table. The second insight is that we can decode the second triangle from the same initial state; because the first triangle is decodable and the second triangle needs, at most, 3 extra bytes, a full state with 16 bytes can be converted to the full state after decoding both triangles, as long as we have a unique pshufb/add vector for each pair of control codes. This results in a 8K table (two 256*16 tables). The vectors have the same first 8 bytes so we should be able to shrink the tables in the future. After decoding the pair, we advance the extra pointer to be able to read the next 6 bytes and patch them into the 'extra' state. This process could be implemented as far back as SSSE3, but there isn't a good reason to use anything below SSE4.1, and SSE4.1 has some instructions that help make this a little faster. We might go back to SSSE3 in the future if the final implementation uses no 4.1 instructions. This is ~7x faster than the scalar decoding loop and runs at 3-3.5B tri/sec. Most of the complexity is just building the tables, which we do at initialization to avoid thread safety issues wrt lazy initialization during decoding.
We are now using code that's a little better aligned to the scalar decoder. Instead of manual expression repetition NEXT macro based on condition code is used. Instead of manually handling tri0 and tri1 we use a loop, and adjust some conditional and addressing math for tri1 to account for extra FIFO shifts. The resulting table are the same before and after this change.
Loading two 16 byte vectors from the tables works but it results in a larger cache footprint (8KB tables) and more load instructions. A critical observation is that the shuffle and next vectors are extremely conveniently redundant: the first 6 bytes of a shuffle vector only affect the state bytes that will be overwritten with extra anyway, and a next vector has *exactly* 6 non-zero integers at the end: last 7 bytes are the two decoded triangles and the updated next byte, but the first index of the first decoded triangle never needs an increment. So we can store the last 6 bytes of next vector in the first 6 bytes of shuffle vector, and it takes just a single vector shift to reconstruct the next vector. This also minimizes the latency of shuffle application as the shuffle vector is used as is. This cuts the table size in half (to 4KB) and makes it more cache coherent; the performance is broadly similar on Zen 4, with a <5% measured cost.
By default when tracing is enabled we only output summary stats for each meshlet; the per-triangle output is reserved for verbose trace builds. Also we now detect the cases where next sequence is broken by un-encodable restarts and flag them. These should be fixable by adjusting the triangle order during optimization.
Instead of assuming the extra vertices fit in 6 bits we assume they could be tightly packed based on the max_vertices number. Also, add post-deflate stats for the original 8 bit stream; all of this can be helpful for future work on compact extras.
We can now specify the meshlet configuration when testing meshlet encoding and test three configurations simultaneously; for now we assume tight packing of extra bytes in the packed configuration although this may change. Also clean up unused and irrelevant bits; we no longer need index codec or strip encoding as a baseline, and the decoding performance is now measured as minimum time over 10 runs.
meshopt_optimizeMeshlet has two passes: a pass that reorders triangles, and a pass that renumbers triangle indices according to the final triangle order. The index renumbering is sequential; this makes it difficult to rotate triangles after the renumbering is done, because that breaks the sequential index order (which is important for meshlet encodings like DGF). This change introduces an extra rotation pass in between the two existing passes that rotates triangles in winding-preserving fashion to improve compressibility: - If the first edge of a triangle is needed by subsequent triangles, but one of the other two edges isn't, we rotate the triangle such that the first edge is unused. This helps encodings that rely on strip connectivity. - If the first and third vertices of the triangle are new, we rotate the triangle such that the previously seen vertex is first. This helps encodings that can't represent the '101' "first" vertex bitcode.
Instead of encoding the triangles in meshopt_encodeMeshlet itself we now use a separate function encodeTriangles. This matches the decoding flow and makes it easier to separate the details of triangle encoding from the details of vertex encoding as well as framing. Also add verbose traces of vertex references; these are currently not encoded but for readability we print them relative to minimum value.
This is merely a function signature change for now; vertex encoding/decoding is not implemented yet and will be done in future commits, which should not affect callsites now. Also return a status after decoding similar to other decoding functions; we do not handle any actual errors for now.
meshopt_encodeMeshlet now will encode vertex references into the output stream. This may not be the final encoding; it works reasonably well but other approaches may be explored in the future. The encoding uses a variant of Stream VByte encoding: each value is encoded using up to 4 data bytes and a separate stream of 2-bit codes indicating the length of the value. Instead of encoding the values directly, we encode zigzagged deltas adjusted by 1; this ensures that in well optimized meshlets, we see a lot of 0 deltas. Accordingly, the encoding is adjusted so that the codes mean 0/1/2/4 lengths, so that 0 deltas can be encoded using only a 2-bit control code. The resulting encoding can be decoded quickly using SIMD pshufb lookups and provides a reasonable balance between pre-deflate and post-deflate sizes.
Even when compiling without SIMD, we can use a fairly efficient decoding method of grouped varint, by reading 4 bytes and masking the resulting read based on a table lookup. The delta adjustment and accumulation then proceeds as usual. The resulting code takes ~4.2 cycles per output integer to decode on Zen4.
Now that the focus is squarely on end-to-end encoding, we don't need index sequence encoding and can measure the entire meshlet encoding/decoding as is. This makes the estimate on bit-packed extra incorrect and not very useful, so that is removed as well. We measure meshlet sizes in two ways: individually (without external framing), and as a packed sequence (with 4-byte frames between meshlets). The details of internal vs external framing will change in the future, and regardless the delta isn't that significant. For now we're still assuming an "optimal" encoding pipeline that reorders the vertex references. This could be tested further in the future via 'reorder' flag.
When SSSE3 is available, we can use table-driven pshufb decoding as is standard for grouped varint approaches. This needs a pre-computed 256x16=4KB table, and a small table to compute the length to advance data stream by for each control byte. pshufb gives us encoded deltas directly (because it can patch in zero bytes for free), but we still have to unzigzag the deltas and accumulate them. Our deltas are serial, which requires extra instructions to compute the prefix sum within the vector. Between unzigzag and prefix sum the decoding doesn't run as fast as regular grouped varint does; we need ~4.75 cycles to decode each group of 4 integers on Zen 4, which is almost 4x faster than the scalar encoding and results in reasonable performance in-context. Meshlets often have more triangles than vertices, and since we decode triangles at ~1.6 cycles per triangle, decoding vertices at ~1.2 cycles/vertex is acceptable. We also can now return an error code when decoding didn't reach the end of the buffer.
Instead of writing the output directly to the provided buffer, we now encode meshlet components into stack buffers (given that the maximum number of vertices/triangles in a meshlet is limited, we need a comparatively small amount of space in the worst case), and then copy them to the output. In addition to making it easy to bounds check, this will also help reorder the encoded data in ways that is difficult to do by writing the meshlet start to finish.
Instead of the order codes -> extra -> control -> data, we now place the two fixed size blocks last, and place an extra gap between the blocks, yielding the following order: extra -> data -> gap -> codes -> control. This allows us to have a safe 16b overread buffer for SIMD decoding. We still have to decode triangles before vertices because we don't know the size of the individual blocks. The gap is up to 16 bytes but it's usually zero bytes because usually codes and control blocks together add up to 16 bytes or more; e.g. 24 triangles + 16 vertices require 12 + 4 = 16 bytes on their own.
Now that we have a gap buffer, we are guaranteed that we can read 16 bytes from any valid value of extra/data. This makes all existing reads (16 bytes for vertex decoding, 8 bytes for triangle decoding) safe for valid data; additionally, we can now bounds check these reads against a pre-computed 'bound' pointer which should guarantee memory safety when decoding invalid streams too. For now, we require that decoding stops exactly at the 'bound' pointer for valid data; this prevents accidental encoding of overlong gaps, but it might be something that we relax in the future.
We require SSE4.1 for SIMD decoding; when it's not available, we need to use scalar decoding as a fallback. We use cpuid and cache the results, similar to vertexcodec; for simplicity, instead of using a separate unsigned int cache, we can check cpuid while building SIMD tables and refuse to build them when SSE4.1 is not supported. To perform the dispatch, we check the flag and use scalar functions to decode. To make dispatch simpler, we now use meshopt::decodeMeshlet/Simd to decode both triangle and vertex data. This also has a side-effect of enabling inlining in Linux builds (where target attribute would previously prevent it); to fully leverage this, decodeMeshlet/Simd is now a tail call as it returns the decoding status. Also fix store intrinsic usage for 64-bit store; storel is the portable version that is supported by MSVC in SSE2 header.
The worst case is easy to compute: each triangle is 3 bytes and each vertex is 4 bytes, plus header space for fixed size portions and gap for smaller meshlets. The computation follows the logic in encode/decode. The function is trivially monotonic; it can be called on max_vertices/triangles once for a given meshlet configuration, or for each meshlet in isolation with a tighter bounds. In either case it's usually a significant overestimation.
Our current parameter order mirrored optimizeMeshlet; however, this function has an unfortunately odd order where the counts are specified separately from pointers, and the counts are *flipped* compared to the pointer order. Instead, put vertex_count after vertices in all functions. This also maintains the "vertices, triangles" order which matches meshlet building and meshlet configurations (V/T).
Decoding should be fully memory safe now, assuming the caller over-allocates output storage to be padded to a boundary of 4.
For meshlet roundtrip, we just test index-only meshlets for now. The encoding there is the most tricky and thus more subject to potential issues, and it's easier to plug in given the fuzzer's unstructured input. For meshlet decoding, we provide output arrays on stack; we could heap-allocate them but they would need to be padded to a boundary of 4, and the function implementation has structured fixed-size writes but complex variable size reads, so this should be sufficient in practice.
Vertices loop is a little more complex and has a bit higher latency; it seems like because of this it's beneficial to decode vertices first, followed by triangles, as that hides the latency more effectively. This is particularly true if the meshlets are vertex-limited, as the vertex loop has a very predictable iteration count from meshlet to meshlet. Overall this makes meshlet decoding ~2% faster. Note that to be able to switch the decoding order we need to also switch the order of variable sized blocks, as we do not store the sizes or offsets. For now we keep the order of fixed size blocks at the end the same, as there's not a strong reason to change it.
We currently measure the decoding bandwidth assuming "naive" 3b/triangle+4b/vertex storage format. This is an overestimate in practice, as it's probably more typical to store 2b/vertex instead, but for now it's a reasonable approximation and the relative changes here are more important than the absolute numbers anyhow. Also add release-avx config to make it easier to test VEX performance.
In an amalgamated build, we can't have symbol conflicts between different files; because topology encoding is similar to indexcodec, it uses the same functions. We remove kTriangleIndexOrder in favor of a rotations[] array declared inside the function, and change the EdgeFifo name to EdgeFifo8. The FIFO also now uses 8 entries (we only need 6, but 8 makes wraparound cheaper), which reduces the cost to find restart triangles during encoding. Also adjust decodeMeshlet to assert bound validity.
This follows a similar change done in meshletcodec and makes the code a little simpler. Note that rotations[] only needs 5 integers because there are only 3 valid rotations (0/1/2) even though getEdgeFifo uses 2 low bits to communicate the index.
Since meshlet encoding has two use cases, with vertex references and without, we need to benchmark both. The triangle-only use case estimates effective bandwidth as #tri*3, which makes this measure more intuitively connected to triangles/sec (9 GB/s = 3B tri/s). In both modes, it can be useful for decoding throughput to sort the meshlets to maximize configuration coherency; this reduces branch mispredictions in processing loops. For now this is not enabled to make tuning of various formats match real workflows, but it might make sense to enable in the future to get a sense of peak performance. Also fix integer truncation warnings on MSVC.
Code that's implicitly converting int/size_t to unsigned char now consistently uses explicit casts. Also, the repack shuffle constant was using 0x80 when calling an intrinsic that accepts signed bytes; use -1 instead as only the top bit matters.
zeux
changed the title
Also add encodeMeshlets to main processing to get test coverage.
Instead of explicitly enumerating all options, merge the branches a little further. This mostly just makes the code shorter, with no real impact on performance. Also fix case of hexadecimal constants, clarify precedence in a couple places and add an assertion to decodeMeshletSimd to ensure tables have been properly initialized even when it's called unconditionally.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
The data format as well as the function APIs is highly experimental. This PR will be merged in its current form, and the format and APIs will likely change over the course of the following months. Until a library point release, use in production applications is highly discouraged.
This PR implements initial version of meshlet codec that can compress and decompress meshlet data: triangle indices (8-bit) and, optionally, vertex references (32-bit).
On triangle indices, the compressed data reaches 5-7 bits per triangle depending on the meshlet configuration, and decodes at approximately 3B triangles/sec (~9 GB/s equivalent) on Zen 4. Accelerated decoding requires SSE4.1; scalar fallback is provided when necessary. The format is designed to work well on pre-optimized (via
meshopt_optimizeMeshlet) data; non-optimized meshlets won't compress as well.To facilitate better compression,
meshopt_optimizeMeshlethas been adjusted to rotate individual triangles so that the final order is more compressible. This is neutral on runtime performance so unconditional rotation simplifies the usage and avoids API churn; the resulting rotation does not affect triangle winding, and is similar to rotation used by existing index buffer codec.On vertex references, the results strongly depend on how well they are organized; some limited experiments have been done there but more work will hopefully happen in the future. Full meshlets with vertex references typically reach 10-14 bits/triangle depending on vertex reference entropy, and decode at approximately 2B triangles/sec.
The decoding currently unpacks triangles and vertices into 32-bit arrays; a triangle is encoded into
0xabcorder. This is intentional to maximize performance when the engine runtime format is a little different - it should be easy to write some minimal SIMD code in user space to repack this into common formats. That said, I plan to explore a separate "packed" decoder API that directly unpacks into tighter 3x8-bit + 16-bit refs format, as that seems to be a good default that some engines can use directly. For decoding, output triangle/vertex buffers should be over-allocated to have overhang space for triangle/vertex counts aligned to 4 (e.g. decoding a 5-triangle meshlet may write 8 uints into output, although presently only writes 6).For optimal size, similarly to vertex/index codec, use of fast general purpose compressors like Zstandard is recommended as a post-process. Zstandard is not as fast to decode but because it would decode pre-compressed data, the cumulative pipeline is much faster and results in much smaller compressed results compared to using Zstandard on original meshlet data directly.
The encoding format is currently "frameless": no header or version metadata is preserved. This is likely to change in the future as some allowances for future format changes should be present, but would ideally somehow minimize the size impact: even an extra byte per meshlet is a byte too many in some cases!
This PR doesn't have unit tests or NEON implementation; these will be added in future PRs once the formats are more final. Some limited performance tuning has been done but further improvements might be possible in some cases (e.g. initially topology decoding ran a little faster and is a little slower now due to bounds checks and per-meshlet function setup costs).
This contribution is sponsored by Valve.