Determine some more criteria for actions review

[potiuk]

Following the discussion in #674 (comment)

Possible criteria:

• Reproducibility
• Code review of differences vs. past approved version
• Passing cooldown (4 days now)
• Correctness of the action (for example errors in build pipelines)
• Hash-pinning of composite actions
• Compatible licencing

Maybe others?

I would love to hear what others think.

[potiuk]

I'd personally add [x] hash pinning and [x] Compatible licencing (even if optional and default is not-compatible). Possibly we should add some checks for the latter if the actions are used properly.

[potiuk]

Also I think good idea when we reject some action we should follow up with the 3rd party - to at least let them know they should fix things (or even create PR). That should be IMHO default behaviour when we reject things.

[dave2wave]

#683 is for discussion of what cooldown periods should be recommended.

[dave2wave]

What is the point of enforcing "compatible licensing"? Policy allows any build tool as long as it is unmodified.

[dave2wave]

• Reject actions with suspicious builds like those that use different dependencies than what the source implies.

[potiuk]

• Reject actions with suspicious builds like those that use different dependencies than what the source implies.

I don't think this was "suspicious" - there is clear explanation on why it happened (and why it could have happened for others). The problem was (see my PR SonarSource/sonarqube-scan-action#228 ) was that their dependabot PRs did not have a check if dist was rebuilt and release was made after that without rebuiliding it with the new tools.

This likely common pattern that might happen to others. Problem with rebuilding in dependabot PRs is that such dependabot PRs will always fail and will have to be manually updated - when such compiled javascript is committed to the repo. Or there must be another mechanism implemented to prevent the release if rebuild is not done (let's see what SonarQube will respond to my PR). So I think there is nothing "suspicious" about it - this is an innocent build/release pipeline bug.

[dave2wave]

One criteria would be to list which repositories are impacted by the approval. We have capability to survey periodically all repositories and provide an inventory. This inventory could be inverted to list out every use of an action along with its current pin. The users of the action should be able to help judge the correctness of any upgrade.