Fix defragment_symbol_data dropping user metadata (#9889436827)#2991
Fix defragment_symbol_data dropping user metadata (#9889436827)#2991himsheda wants to merge 4 commits into
Conversation
himsheda
requested review from
IvoDD,
alexowens90 and
poodlewars
as code owners
himsheda
force-pushed
the
fix/defragment-symbol-data-drops-metadata
branch
from
765da9e to
b313aeb
Compare
| timestamp=result.timestamp, | ||
| ) | ||
| version_query = self._get_version_query(result.version) | ||
| _, udm = self.version_store.read_metadata(symbol, version_query) |
There was a problem hiding this comment.
The metadata should already available in the C++ layer at this point. Consider returning it directly to avoid extra API call and storage read.
phoebusm
left a comment
phoebusm
left a comment
There was a problem hiding this comment.
#2991 (comment)
Returning the metadata directly from C++ maybe too much scope for this PR. It would be sufficient to just update the docstring in Library to document that the returned metadata will be None
himsheda
force-pushed
the
fix/defragment-symbol-data-drops-metadata
branch
from
f6539a8 to
8ca9081
Compare
Handled. @phoebusm Please have a look |
himsheda
force-pushed
the
fix/defragment-symbol-data-drops-metadata
branch
from
8ca9081 to
2ff5427
Compare
Why is there still the |
Its because this is the bug this PR is solving. Current code not only returns none as metadata but also stores it. Causing meradata corruption. |
defragment_symbol_data_impl passed std::nullopt to collate_and_write for user_meta, silently discarding the metadata that had been correctly read into pipeline_context->user_meta_ from the previous version. Fix by passing the actual user_meta_ptr from the pipeline context. Also fix the Python wrapper which hardcoded metadata=None in the returned VersionedItem regardless of what was stored; it now reads the metadata back from the newly-written version using the existing read_metadata pattern. Add regression test test_defragment_preserves_metadata.
himsheda
force-pushed
the
fix/defragment-symbol-data-drops-metadata
branch
from
f4b924e to
fc712bf
Compare
3 participants
defragment_symbol_data_impl passed std::nullopt to collate_and_write for user_meta, silently discarding the metadata that had been correctly read into pipeline_context->user_meta_ from the previous version. Fix by passing the actual user_meta_ptr from the pipeline context.
Also fix the Python wrapper which hardcoded metadata=None in the returned VersionedItem regardless of what was stored; it now reads the metadata back from the newly-written version using the existing read_metadata pattern.
Add regression test test_defragment_preserves_metadata.
Reference Issues/PRs
#2611
What does this implement or fix?
Any other comments?
Checklist
Checklist for code changes...