Skip to content

Forgetting is not fixed in the latest release? #291

Description

@Newspaper8459

In metrics.py, there was a problem in the range, where it previously ranged from 0 to k-2. This had apparently been fixed in the past, and I can check that in this commit.

Below is the current code I can find in the repository:

def forgetting(all_preds, all_targets, all_tasks):
    ...

    for j in range(k - 1):
        # Accuracy on task j after learning current task k
        a_kj = _get_R_ij(k - 1, j, all_preds, all_targets, all_tasks)
        # Best previous accuracy on task j
        max_a_lj = max(
            _get_R_ij(l, j, all_preds, all_targets, all_tasks)
            for l in range(k - 1)
            if l >= j
        )
    ...

However, it hasn't been fixed in the latest release, which is 1.2.7.
Below is the code in the latest release 1.2.7.

def forgetting(all_preds, all_targets, all_tasks):
    ...
    for j in range(k - 2):
        # Accuracy on task j after learning current task k
        a_kj = _get_R_ij(k - 1, j, all_preds, all_targets, all_tasks)
        # Best previous accuracy on task j
        max_a_lj = max(_get_R_ij(l, j, all_preds, all_targets, all_tasks) for l in range(k - 2) if l >= j)
        f += max_a_lj - a_kj  # We want this results to be as low as possible
    ...

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions