Skip to content

Refactor validator return type to support warnings via ValidationResult wrapper (#1479) - #1480

Open
sejalpunwatkar wants to merge 15 commits into
hdmf-dev:devfrom
sejalpunwatkar:feature/1479-validation-result
Open

Refactor validator return type to support warnings via ValidationResult wrapper (#1479)#1480
sejalpunwatkar wants to merge 15 commits into
hdmf-dev:devfrom
sejalpunwatkar:feature/1479-validation-result

Conversation

@sejalpunwatkar

Copy link
Copy Markdown
Contributor

Motivation

Addresses #1479

This PR implements Phase 1 (the foundational plumbing) to introduce ValidationWarning infrastructure without breaking the existing error reporting model. This change ensures that downstream packages (like PyNWB and NWB Inspector) can eventually consume warning-level messages without modifying their immediate logic or incorrectly marking files as invalid.

How to test the behavior?

  1. Added Core Types: Introduced ValidationWarning class as a sibling to Error inside errors.py, preserving the matching internal shape (.name, .reason, .location).
  2. Built the Smart Container: Implemented the ValidationResult wrapper class in errors.py with custom Python overrides (__bool__, __len__, __iter__, __getitem__) pointing explicitly to the errors bucket to ensure backward compatibility.
  3. Wired the Plumbing: Updated ValidatorMap.validate() inside validator.py to capture the errors_list, wrap it into the new ValidationResult container with an empty warnings container (warnings=[]), and adjusted its strict @docval return type check to rtype=(list, ValidationResult).

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

@codecov

codecov Bot commented May 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.37037% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.10%. Comparing base (f1b2eb4) to head (d687dd3).

Files with missing lines Patch % Lines
src/hdmf/validate/errors.py 68.00% 16 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1480      +/-   ##
==========================================
- Coverage   93.22%   93.10%   -0.13%     
==========================================
  Files          41       41              
  Lines       10221    10272      +51     
  Branches     2115     2117       +2     
==========================================
+ Hits         9529     9564      +35     
- Misses        415      431      +16     
  Partials      277      277              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/hdmf/validate/errors.py Outdated
Comment thread src/hdmf/validate/errors.py
Comment thread src/hdmf/validate/validator.py Outdated
@rly

rly commented May 30, 2026

Copy link
Copy Markdown
Contributor

Thanks! Looks good overall. Could you please add some basic tests of ValidationResult:

  • validate() returns a ValidationResult with .warnings == [].
  • Backward compatibility: bool, len, iteration, indexing all reflect errors only.
  • Construct ValidationResult(warnings=[w]) and assert the warning survives.

Could you please also add a changelog entry that points to this PR?

Comment thread src/hdmf/validate/errors.py
@sejalpunwatkar
sejalpunwatkar force-pushed the feature/1479-validation-result branch 2 times, most recently from 3a072bb to 669261e Compare June 2, 2026 11:09
@sejalpunwatkar
sejalpunwatkar force-pushed the feature/1479-validation-result branch from 9d3ea64 to 88bb8e9 Compare June 2, 2026 11:13
@rly

rly commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

@sejalpunwatkar Thanks for your work on this. Let me know when it is ready for re-review.

@sejalpunwatkar

Copy link
Copy Markdown
Contributor Author

Hi @rly, the PR is now ready for your re-review!

I have fully addressed your feedback from the initial code review:

  1. Bug Fix: Fixed the initialization logic typo in ValidationResult.__init__ (warnings vs errors).
  2. Ecosystem Safety: Added the new classes to __all__ in errors.py.
  3. Type-Checking & Docs: Tightened @docval in validator.py to strictly enforce returning ValidationResult, and updated the corresponding docstrings and imports in src/hdmf/common/__init__.py.
  4. Enhanced Diagnostics: Added a human-readable __repr__ method to ValidationResult.
  5. Testing & Coverage: Added comprehensive unit tests at the bottom of test_validate.py covering the wrapper construction, magic methods, and empty state evaluations. 100% of the test suite is passing cleanly.
  6. Changelog: Added a release tracking entry under the ## HDMF 5.0.0 (Upcoming) section in CHANGELOG.md.

Comment thread src/hdmf/common/__init__.py
Comment thread src/hdmf/common/__init__.py Outdated
Comment thread CHANGELOG.md Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
return hash(self) == hash(other)


class ValidationWarning:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since ValidationWarning is almost entirely identical to Error, it would be better to create a shared base class ValidationIssue that both extend from and that contains all the shared code. Downstream code will still be able to differentiate between Error and ValidationWarning because they are different classes.

One minor thing is that because __eq__ is defined based on the hash and I believe the hash of a ValidationWarning and the hash of an identical Error object will be the same, then Error("name", "reason") == ValidationWarning("name", "reason") will be True. This might become an issue if/when errors and warnings are placed in a dict together.

To address this, I would change __eq__ to:

    def __eq__(self, other):
          return type(self) is type(other) and hash(self) == hash(other)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the below change to ValidationWarning.__eq__.

Since ValidationWarning and Error share significant code, could you also create a shared base class ValidationIssue that both extend from and that contains all the shared code? That would also make Error.__eq__ match ValidationWarning.__eq__, which it should.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With these changes, could you also add a quick test that an error and warning with the same name are not equal, so that these changed lines are tested?

@rly

rly commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Thanks @sejalpunwatkar . This is looking much better. I have a few more comments and suggestions above

@sejalpunwatkar
sejalpunwatkar force-pushed the feature/1479-validation-result branch from 5e33150 to 964acbf Compare June 17, 2026 11:12
@sejalpunwatkar

Copy link
Copy Markdown
Contributor Author

Hi @rly, I've addressed all your feedback: fixed the changelog section, updated the docstrings, corrected the test indentation, implemented the missing test, and updated the ValidationWarning.eq check. Ready for review!

Comment thread src/hdmf/common/__init__.py Outdated
rly and others added 2 commits July 23, 2026 17:34
Change "Validate an file" to "Validate a file" per code review suggestion.
@sejalpunwatkar

Copy link
Copy Markdown
Contributor Author

Thanks for catching that, @rly ! Fixed now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants