Skip to content

Add test metadata - #373

Merged
larshp merged 2 commits into
exercism:mainfrom
mbtools:test-metadata
Aug 30, 2026
Merged

Add test metadata#373
larshp merged 2 commits into
exercism:mainfrom
mbtools:test-metadata

Conversation

@mbtools

@mbtools mbtools commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Closes #358

@IsaacG if we add tests to an exercise (change the zcl_...clas.testclasses.abap file), does it invalidate the solutions for all users? If it does I don't think we will make those changes since we have pretty good test coverage.

Missing tests

10 exercises have missing tests, totaling 50 tests (include = false, not superseded by another case).

Exercise Missing Notes
line-up 15 ordinal formatting cases
isogram 11 most of the spec
raindrops 9 extra Plang/Plong cases
clock 4 equality comparisons
reverse-string 3 Unicode/grapheme cases
anagram 2 Greek letters; same-byte characters
armstrong-numbers 2 seven zeroes; largest Armstrong number
state-of-tic-tac-toe 2 invalid boards after a win
flower-field 1 multiple adjacent flowers
resistor-color 1 Colors

47 exercises scanned, 616 total test cases.

@IsaacG

IsaacG commented Aug 18, 2026

Copy link
Copy Markdown
Member

When you merge a PR that touches a test file, you can control whether or not the tests are rerun with the commit message. They aren't always rerun. I suspect the number of solutions on this track is low enough that a rerun wouldn't be a big deal.

@BNAndras

Copy link
Copy Markdown
Member

These 10 are missing a tests.toml file, but adding them shouldn't require updating the test suite.

  1. atbash-cipher
  2. beer-song
  3. book-store
  4. etl
  5. grains
  6. hello-world
  7. knapsack
  8. matrix
  9. prime-factors
  10. run-length-encoding

These 14 are missing a tests.toml file, but adding them will require tests to be added

  1. affine-cipher
  2. anagram
  3. armstrong-numbers (might be skippable since the inputs are pretty large)
  4. clock
  5. crypto-square
  6. difference-of-squares
  7. hamming
  8. high-scores
  9. isogram
  10. phone-number
  11. raindrops
  12. reverse-string (might be skippable since new tests are Unicode)
  13. triangle
  14. word-count

These 3 have a tests.toml file and have new tests that do need to be added.

  1. flower-field
  2. line-up
  3. state-of-tic-tac-toe

Of these 27, I believe state-of-tic-tac-toe and word-count will also need their example solutions updated from experience.

@BNAndras

Copy link
Copy Markdown
Member

For what it's worth,

incorrectly tests the sum of squares method by calling the square of sum method. Given the expected return value is 1 either way, this passes the CI even though the test suite is incorrect.

@mbtools

mbtools commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

I don't want to add new test. That's why they are marked as "included = false".

I want this PR to have no impact to users on any existing solutions.

So no rerun of tests. Just added metadata. How do I make sure of this?

PS: if there are bugs in tests, separate issue or PR, please

@BNAndras

Copy link
Copy Markdown
Member

https://exercism.org/docs/building/tracks#h-avoiding-triggering-unnecessary-test-runs

At the moment, this PR won't rerun solutions since you're not touching any important files. The No important files changed workflow will trigger and alert you though if you're in a situation where solutions will be rerun by default. At that point, you can manually opt out of rerunning solutions for that PR.

@mbtools

mbtools commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. So we just need someone to approve and then merge 🙏

@IsaacG

IsaacG commented Aug 30, 2026

Copy link
Copy Markdown
Member

Thanks. So we just need someone to approve and then merge 🙏

"Someone" would have to be an abap maintainer which means @mbtools or @larshp :)

@larshp
larshp merged commit 927eef8 into exercism:main Aug 30, 2026
4 checks passed
@mbtools
mbtools deleted the test-metadata branch August 30, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🚨 configlet sync --test found unsynced tests

4 participants