dra: support bumping releases minor/patch using the existing tools

[v1v]

Release notes

[rn:skip]

What does this PR do?

Honour #18765 so it can run the existing release automation already implemented using GitHub actions (see https://github.com/elastic/logstash/blob/main/.github/workflows/version_bumps.yml).

In this case, we use a shell script to trigger the existing minor or patch github workflow based on the WORKFLOW env variable.

Use https://github.com/elastic/gh-cli-buildkite-plugin

Why is it important/What is the impact to the user?

Align with the automation the Platform Productivity Release team automation.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• run exhaustive tests : Run the exhaustive tests Buildkite pipeline.

[mergify]

This pull request does not have a backport label. Could you fix it @v1v? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• If no backport is necessary, please add the backport-skip label

[v1v]

@ninalee12 , what's the value for BRANCH when running:

• minor release for main
• patch releases for 8.19 and 9.x

[ninalee12]

The values for minor flow would be the major.minor. So if 9.4.0 came out today, the branch would be 9.4. This is what I currently have implemented. Similar conversation here: https://elastic.slack.com/archives/C01DJ3DC0AX/p1773933668868239

As for the patch flow, if 9.3.3 release, the branch would be 9.3. It would be the same for 8.19.x, the branch would be 8.19

I hope this makes sense

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: ddc855f

Failed CI Steps

• :rspec: Ruby unit tests - FIPS mode

[fr4nc1sc0-r4m0n]

LGTM

[v1v]

We don't need this safeguard, it's gonna be triggered by the automation.

[donoghuc]

I think there may be some confusion about what version_bumps.yml does. That is a workflow for bumping the vendored gem components we ship with logstash artifacts. We do that typically around feature freeze (or any time we want to do a gem update).

I'm a bit confused about how this plays in to the larger architecture here. I thought the context here was preparing DRA builds with new stack versions. For that we would use the bump-logstash.yml https://github.com/elastic/logstash/blob/main/.github/workflows/bump-logstash.yml workflow.

Is the intent here to trigger a workflow to bump vendored gems or logstash version?

[v1v]

@donoghuc , what's the current process for the bump automation then? I asked since I recall I touched version_bumps.yml for updating mergify when FF:

• mergify: support backports automation with labels #16937

Happy to redo the work and find a better approach

[donoghuc]

The release related GHA are:

1. For feature freeze we run the version_bumps.yml which effectively updates all our gem dependencies: (example) which effectively just changes the single gemfile.lock that controls gem versions of deps. Then we kick off a DRA (staging/snapshot).
2. On release day we run bump_logstash.yml https://github.com/elastic/logstash/actions/workflows/bump-logstash.yml` which increments the stack version of logstash itself. Then we run DRA staging/snapshot to ensure that builds are ready for the next unified release.

I was under the impression that the automation this is working on was for 2, not for 1. But that might be a hole in my understanding?

[v1v]

I was under the impression that the automation this is working on was for 2, not for 1. But that might be a hole in my understanding?

This is for the feature freeze, not the release day.

However, I think the branch off from a new minor should be implemented somewhere.

This is about the release day! does bump_logstash.yml support a new minor branch?

Sorry for the misleading here

[donoghuc]

This is about the release day! does bump_logstash.yml support a new minor branch?

Thanks for the clarification! Yes, bump_logstash.yml is the cannonical way for adjusting logstash versions. The inputs are:

      logstash_version:
        description: 'Logstash version to update TO (example: 9.1.4)'
        required: true
        type: string
      logstash_branch:
        description: 'Logstash branch (example: 9.1)'
        required: true
        type: string
      logstash_release_track:
        description: 'Release track (example: 9.current, 9.previous, main)'
        required: true
        type: string

The new version and target branch are easy to comprehend/automate. The release track might be a bit more complex. It may be the case that we want to make that an optional parameter and just not adjust it if its not supplied.