Add NDCG metric to rec_sys

[steaphenai]

Fixes #3581
Refs #3568 #3569 #3610

Description: Implements NDCG (Normalized Discounted Cumulative Gain) metric for recommendation systems and ranking evaluation.

Implementation Details

• Follows the same pattern as HitRate and MRR metrics
• Uses ranx library for verification (addressing [Feature]: Add Mean Reciprocal Rank (MRR) metric to rec_sys #3569 feedback)
• Supports both binary (0/1) and graded (0-4) labels
• relevance_threshold defaults to 1.0 for binary labels
• Uses stable=True for reproducible tie-breaking
• Handles k > num_items
• Will align with top_k API changes from [Feature] Implement TopK wrapper class for metrics #3568 when finalized

Mathematical Background

• DCG: Discounted Cumulative Gain - rewards relevant items, penalizes low positions
• IDCG: Ideal DCG - perfect ranking baseline
• NDCG: DCG / IDCG - normalized score from 0.0 to 1.0

Formula: NDCG@K = Σ(2^rel_i - 1) / log2(i + 1)

Testing

• 26 tests passing locally (verified against ranx)
• Parametrized tests across multiple configurations
• Edge cases covered (perfect prediction, zero relevance, graded labels)
• Distributed training testedcs

Check list:

• New tests are added (if a new feature is added)
• New doc strings: description and/or example code are in RST format
• Documentation is updated (if required)

[aaishwarymishra]

@steaphenai In docs/source/metrics.rst add rec_sys.NDCG in complete list of metrics to pass initial checks.

[steaphenai]

@steaphenai In docs/source/metrics.rst add rec_sys.NDCG in complete list of metrics to pass initial checks.

@aaishwarymishra Thanks!!

[vfdev-5]

@steaphenai I was checking your implementation compared to previous ones where contributors were blocked with vectorized computation of

discounted_gains = torch.tensor(
  [_tie_averaged_dcg(y_p, y_t, discount_cumsum, device) for y_p, y_t in zip(y_pred, y_true)], device=device
)

when ignore_ties is False. I can't find the same thing in your implementation, can you explain your implementation, what it based on?

[steaphenai]

@steaphenai I was checking your implementation compared to previous ones where contributors were blocked with vectorized computation of

discounted_gains = torch.tensor(
  [_tie_averaged_dcg(y_p, y_t, discount_cumsum, device) for y_p, y_t in zip(y_pred, y_true)], device=device
)

when ignore_ties is False. I can't find the same thing in your implementation, can you explain your implementation, what it based on?

@vfdev-5 My implementation is based on the standard NDCG formula and validated against the ranx library.
For tie handling, I use torch.argsort(..., stable=True) which breaks ties deterministically by index order (same as MRR). This ensures reproducible results and matches ranx's behavior.
I validated correctness by testing against ranx across various scenarios, all tests pass with matching results.

[steaphenai]

@vfdev-5 Just to clarify on the vectorization part, my implementation is fully vectorized using PyTorch operations (no Python loops).

Instead of looping through each example like:

[_tie_averaged_dcg(y_p, y_t, ...) for y_p, y_t in zip(y_pred, y_true)]

I use batched PyTorch operations:

ranked_indices = torch.argsort(y_pred, dim=-1, descending=True, stable=True)[:, :max_k]
ranked_relevance = torch.gather(y_for_dcg, dim=-1, index=ranked_indices)

This processes all examples in the batch simultaneously, which should be more efficient than the loop-based approach.

[vfdev-5]

To help with reviewing can you please run catalyst and ranx on some non-trivial examples involving ties etc and compare results.

• https://github.com/catalyst-team/catalyst/blob/master/catalyst/metrics/_ndcg.py
• test cases: https://github.com/catalyst-team/catalyst/blob/master/tests/catalyst/metrics/functional/test_ndcg.py

[steaphenai]

@vfdev-5 I added ignite.metrics.rec_sys.NDCG with support for top_k, ignore_zero_hits, and relevance_threshold. I also added a shared _get_ranked_items(...) helper for the ranking logic and added gain_function support (exp_rank and linear_rank). The NDCG docstring and docs were updated to clarify graded relevance and for validation, I added tests in tests/ignite/metrics/rec_sys/test_ndcg.py which parity checks against ranx and Catalyst, plus tie cases and edge cases (perfect ranking, zero relevance, threshold behavior, multi-batch accumulation, invalid inputs, and distributed integration).

[vfdev-5]

@steaphenai please check the failing CI job: https://github.com/pytorch/ignite/actions/runs/23947698404/job/70144090725?pr=3608

[steaphenai]

@steaphenai please check the failing CI job: https://github.com/pytorch/ignite/actions/runs/23947698404/job/70144090725?pr=3608

PS C:\ignite> pre-commit run --all-files nb-clean.................................................................Passed check toml...............................................................Passed check yaml...............................................................Passed fix end of files.........................................................Passed fix end of files.........................................................Passed fix end of files.........................................................Passed trim trailing whitespace.................................................Passed prettier.................................................................Passed ruff check...............................................................Passed ruff format..............................................................Passed

@vfdev-5 I verified with pre-commit run --all-files. All the tests were passed locally.

[vfdev-5]

Thanks for updates @steaphenai
We almost there, just few updates

[steaphenai]

Thanks for updates @steaphenai We almost there, just few updates

@vfdev-5 Done! I've set the imports to the top of the file.

[vfdev-5]

Thanks a lot for the PR @steaphenai , LGTM

[vfdev-5]

@steaphenai still failing CI code style check, please fix it

[steaphenai]

@vfdev-5 I moved the Catalyst import to the top-level as requested, but TPU CI was failing during test collection because importing catalyst triggers a torch_xla linkage error in that environment. I am thinking it to update it to a guarded top-level import (try/except) and marked only the Catalyst-dependent tests with skip-if when Catalyst is unavailable. This keeps import at top while avoiding unrelated CI breakage. Is this approach okay for this PR?
Something like this

from ranx import Qrels, Run, evaluate
try:
    from catalyst.metrics.functional import ndcg as catalyst_ndcg_fn

    HAS_CATALYST = True
except Exception:  
    catalyst_ndcg_fn = None
    HAS_CATALYST = False
    ```

[vfdev-5]

Catalyst is unmaintained now, so I was actually thinking if we can remove it from the tests and keep only ranx (which is also look a bit left aside).
We can simply remove all the tests involving catalyst. Other option is to do what you suggested.
What do you think?

[steaphenai]

@vfdev-5 Makes sense. I agree we should remove Catalyst-based parity tests and keep ranx-based validation. I’ll drop Catalyst imports/helpers/tests and keep the ranx checks as the reference path