Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,23 @@ steps:
key: "gha-ci"
plugins:
- github-actions#v0.9.3:
workflow: .github/workflows/ci.yml
workflows:
- .github/workflows/ci.yml

- label: ":rocket: Deploy"
depends_on: "gha-ci"
command: .buildkite/deploy.sh
```

The plugin is a thin wrapper around the hidden `buildkite-gha plugin` entrypoint. It uses mise to install and verify the selected CLI release, and defaults to the latest stable release. During the preview, leaving `version` unset means there is no CLI version to update as new stable releases ship.

Use `workflow: "*"` to select every tracked `.yml` and `.yaml` file directly under `.github/workflows`.
The plugin is a thin wrapper around the hidden `buildkite-gha plugin` entrypoint. It uses mise to install and verify the selected CLI release, and defaults to the latest stable release. During the preview, leaving `version` unset means there is no CLI version to update as new stable releases ship. List every workflow to import explicitly; plugin configuration does not accept directories or glob patterns.

To hold the CLI at a specific release instead, set `version` to an exact stable release from `0.9.0` onward:

```yaml
plugins:
- github-actions#v0.9.3:
workflow: .github/workflows/ci.yml
workflows:
- .github/workflows/ci.yml
version: "0.10.1"
Comment thread
lox marked this conversation as resolved.
```

Expand All @@ -60,7 +60,8 @@ labels with a native Darwin/arm64 queue:
```yaml
plugins:
- github-actions#v0.9.3:
workflow: .github/workflows/ci.yml
workflows:
- .github/workflows/ci.yml
runners:
- runs-on: ubuntu-latest
queue: hosted
Expand All @@ -72,7 +73,7 @@ Configured Linux profiles use the matching Noble or Jammy hosted-toolchains
image. A macOS label selects native Darwin/arm64, not a GitHub image or Xcode
inventory.

The imported workflow is a dynamic part of the Buildkite pipeline. Upload can take one tracked workflow glob or a list of explicit workflow paths and creates one aggregate group per directly runnable workflow in a single transaction. Workflows that do not declare the selected event become skipped groups. Each `:github: <workflow>` group depends on the importer; its GitHub check is named `Buildkite / <workflow> (<event>)`. This approach lets you keep existing workflows while moving jobs to native Buildkite steps over time.
The imported workflows are a dynamic part of the Buildkite pipeline. The plugin creates one aggregate group per explicitly listed, directly runnable workflow in a single transaction. Workflows that do not declare the selected event become skipped groups. Each `:github: <workflow>` group depends on the importer; its GitHub check is named `Buildkite / <workflow> (<event>)`. This approach lets you keep existing workflows while moving jobs to native Buildkite steps over time.

Buildkite owns build creation and schedule configuration. Within that build, `buildkite-gha` maps push, pull request, manual/API, and scheduled builds to `push`, `pull_request`, `workflow_dispatch`, and `schedule`, then applies the matching `on:` branch, tag, base-branch, and pull request activity filters. Cross-event workflows are excluded before event-dependent compilation and retained as skipped groups.

Expand Down
17 changes: 10 additions & 7 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,16 +102,19 @@ buildkite-gha upload .github/workflows/ci.yml
The importer must run on Linux/amd64 with `BUILDKITE=true` and `BUILDKITE_STEP_KEY`.

The hidden zero-argument `buildkite-gha plugin` entry point reads `workflows` and
`runners` from `BUILDKITE_PLUGIN_CONFIGURATION`; `workflows` accepts one selector
string or an array of explicit paths. It also accepts the plugin-owned `version`,
`source-ref`, and `minimum-release-age` fields. The legacy singular `workflow`
field remains supported for released plugin compatibility but cannot be combined
with `workflows`. The Linux/amd64 importer fetches the same release's Darwin
runtime only when a workflow requires it. Custom importers can use the public
flags below.
`runners` from `BUILDKITE_PLUGIN_CONFIGURATION`. Every `workflows` entry must be
an explicit path to a regular, tracked `.yml` or `.yaml` file inside the
repository; directories and glob patterns are rejected. It also accepts the
plugin-owned `version`, `source-ref`, and `minimum-release-age` fields. The legacy
singular `workflow` field remains supported as an explicit-path compatibility
alias for released plugins but cannot be combined with `workflows`. The
Linux/amd64 importer fetches the same release's Darwin runtime only when a
workflow requires it. Custom importers can use the public flags below.

### Select workflows

The public `upload` command retains selector expansion for custom importers.

Use `*` for every tracked `.yml` and `.yaml` file directly under `.github/workflows`:

```sh
Expand Down
2 changes: 1 addition & 1 deletion docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Steps remain inside one job because they share a workspace, environment files, a

### Aggregate workflow upload

`*` selects every tracked `.yml` and `.yaml` file directly under `.github/workflows`. Upload also accepts one literal, directory, or tracked glob, or two or more explicit workflow paths. Lists require regular, tracked `.yml` or `.yaml` files inside the repository; directories, missing or untracked files, outside paths, symlinks, and globs fail. Inputs are canonicalized, sorted, and deduplicated before workflow identities and job-key namespaces are assigned.
The plugin requires an explicit list of workflow paths. Each entry must identify a regular, tracked `.yml` or `.yaml` file inside the repository; directories, missing or untracked files, outside paths, symlinks, and globs fail. The public `upload` command for custom importers also accepts one literal, directory, or tracked glob; `*` selects every tracked `.yml` and `.yaml` file directly under `.github/workflows`. Inputs are canonicalized, sorted, and deduplicated before workflow identities and job-key namespaces are assigned.

All directly runnable workflows are represented in one artifact and pipeline transaction. Each becomes one aggregate group labeled `:github: <workflow-name>`, with its canonical path as the fallback for an unnamed workflow. A workflow that declares the effective event compiles into child jobs. A workflow that does not declare it becomes a skipped group with an ignored placeholder and no plan artifacts. The label is static across events. The group-level GitHub check is named `Buildkite / <workflow-name-or-path> (<effective-event>)`. Each group depends on the importer, while its child jobs omit that redundant dependency and their own check notifications.

Expand Down
11 changes: 4 additions & 7 deletions internal/cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ func plugin(args []string, stdout, stderr io.Writer, version string, runner tran
}
return uploadParsed(parsedUploadArgs{
workflowOperands: configuration.Workflows,
explicitWorkflowPaths: configuration.explicitWorkflowPaths,
explicitWorkflowPaths: true,
runnerTargets: configuration.runnerTargets,
pluginAcquisition: &pluginRuntimeAcquisition{version: version},
}, stdout, stderr, version, transport.Agent{Runner: runner})
Expand All @@ -192,9 +192,8 @@ func validateImporterPlatform(goos, goarch string) error {
}

type pluginConfiguration struct {
Workflows []string
explicitWorkflowPaths bool
runnerTargets map[string]compiler.RunnerTarget
Workflows []string
runnerTargets map[string]compiler.RunnerTarget
}

func parsePluginConfiguration(source string) (pluginConfiguration, error) {
Expand Down Expand Up @@ -229,7 +228,6 @@ func parsePluginConfiguration(source string) (pluginConfiguration, error) {
return pluginConfiguration{}, fmt.Errorf("%s workflow and workflows are mutually exclusive", pluginConfigurationEnvironment)
}
var workflows []string
explicitWorkflowPaths := false
if hasLegacyWorkflow {
workflow, ok := legacyWorkflow.(string)
if !ok || strings.TrimSpace(workflow) == "" {
Expand All @@ -244,7 +242,6 @@ func parsePluginConfiguration(source string) (pluginConfiguration, error) {
}
case []any:
if len(value) != 0 {
explicitWorkflowPaths = true
workflows = make([]string, len(value))
for index, entry := range value {
workflow, ok := entry.(string)
Expand Down Expand Up @@ -303,7 +300,7 @@ func parsePluginConfiguration(source string) (pluginConfiguration, error) {
targets[label] = target
}
}
return pluginConfiguration{Workflows: workflows, explicitWorkflowPaths: explicitWorkflowPaths, runnerTargets: targets}, nil
return pluginConfiguration{Workflows: workflows, runnerTargets: targets}, nil
}

func normalizePluginCommit(ctx context.Context, getenv func(string) string, setenv func(string, string) error, runner transport.Runner) error {
Expand Down
81 changes: 52 additions & 29 deletions internal/cli/cli_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -163,17 +163,14 @@ func TestParsePluginConfiguration(t *testing.T) {
if !slices.Equal(configuration.Workflows, []string{".github/workflows/ci.yml", ".github/workflows/release.yml"}) || len(configuration.runnerTargets) != 2 {
t.Fatalf("configuration = %#v", configuration)
}
if !configuration.explicitWorkflowPaths {
t.Fatal("workflows array did not retain explicit path semantics")
}
if got := configuration.runnerTargets["ubuntu-latest"]; got != (compiler.RunnerTarget{Queue: "hosted", Platform: compiler.PlatformLinuxAMD64, Image: image}) {
t.Fatalf("Linux target = %#v", got)
}
if got := configuration.runnerTargets["macos-14"]; got != (compiler.RunnerTarget{Queue: "macos-sonoma-arm64", Platform: compiler.PlatformDarwinARM64}) {
t.Fatalf("Darwin target = %#v", got)
}
minimal, err := parsePluginConfiguration(`{"workflows":"workflow.yml"}`)
if err != nil || !slices.Equal(minimal.Workflows, []string{"workflow.yml"}) || minimal.explicitWorkflowPaths || len(minimal.runnerTargets) != 0 {
if err != nil || !slices.Equal(minimal.Workflows, []string{"workflow.yml"}) || len(minimal.runnerTargets) != 0 {
t.Fatalf("minimal configuration = %#v, %v", minimal, err)
}
legacy, err := parsePluginConfiguration(`{"workflow":"legacy.yml"}`)
Expand Down Expand Up @@ -366,35 +363,48 @@ func TestPluginUploadsPluralWorkflowList(t *testing.T) {
}
}

func TestPluginRejectsGlobInSingleEntryWorkflowList(t *testing.T) {
func TestPluginRejectsNonExplicitWorkflowSelectors(t *testing.T) {
requireImporterHost(t)
configuration, err := json.Marshal(map[string]any{
"workflows": []string{filepath.Join("..", "..", "testdata", "smoke", ".github", "workflows", "*.yml")},
})
if err != nil {
t.Fatal(err)
}
t.Setenv(pluginConfigurationEnvironment, string(configuration))
setCLIPluginBuildkiteEnvironment(t, "plugin-workflows-glob")
runner := &cliCaptureRunner{}
var stdout, stderr bytes.Buffer
if code := run([]string{"plugin"}, &stdout, &stderr, "dev", runner); code != 1 || !strings.Contains(stderr.String(), "explicit paths") {
t.Fatalf("run() code/stderr = %d / %q", code, stderr.String())
}
if stdout.Len() != 0 || len(runner.commands) != 0 || len(runner.uploaded) != 0 {
t.Fatalf("invalid workflow list reached Buildkite: stdout %q, commands %#v, uploads %#v", stdout.String(), runner.commands, runner.uploaded)
workflowDirectory := filepath.Join("..", "..", "testdata", "smoke", ".github", "workflows")
for _, test := range []struct {
name string
field string
workflows any
want string
}{
{name: "all shorthand", field: "workflows", workflows: "*", want: "explicit paths"},
{name: "legacy all shorthand", field: "workflow", workflows: "*", want: "explicit paths"},
{name: "string glob", field: "workflows", workflows: filepath.Join(workflowDirectory, "*.yml"), want: "explicit paths"},
{name: "array glob", field: "workflows", workflows: []string{filepath.Join(workflowDirectory, "*.yml")}, want: "explicit paths"},
{name: "directory", field: "workflows", workflows: workflowDirectory, want: "does not name a regular tracked file"},
} {
t.Run(test.name, func(t *testing.T) {
configuration, err := json.Marshal(map[string]any{test.field: test.workflows})
if err != nil {
t.Fatal(err)
}
t.Setenv(pluginConfigurationEnvironment, string(configuration))
setCLIPluginBuildkiteEnvironment(t, "plugin-workflows-explicit")
runner := &cliCaptureRunner{}
var stdout, stderr bytes.Buffer
if code := run([]string{"plugin"}, &stdout, &stderr, "dev", runner); code != 1 || !strings.Contains(stderr.String(), test.want) {
t.Fatalf("run() code/stderr = %d / %q", code, stderr.String())
}
if stdout.Len() != 0 || len(runner.commands) != 0 || len(runner.uploaded) != 0 {
t.Fatalf("invalid workflow selection reached Buildkite: stdout %q, commands %#v, uploads %#v", stdout.String(), runner.commands, runner.uploaded)
}
})
}
}

func TestPluginPublishesMixedRuntimeDistributions(t *testing.T) {
requireImporterHost(t)
const fullCommit = "0123456789abcdef0123456789abcdef01234567"
root := t.TempDir()
workflowPath := filepath.Join(root, "mixed.yml")
workflow := []byte("on: push\njobs:\n linux:\n runs-on: ubuntu-latest\n steps:\n - run: echo linux\n macos:\n needs: linux\n runs-on: macos-15\n steps:\n - run: echo macos\n")
if err := os.WriteFile(workflowPath, workflow, 0o600); err != nil {
t.Fatal(err)
}
repository := writeUploadWorkflowRepository(t, map[string]string{
"mixed.yml": "on: push\njobs:\n linux:\n runs-on: ubuntu-latest\n steps:\n - run: echo linux\n macos:\n needs: linux\n runs-on: macos-15\n steps:\n - run: echo macos\n",
})
t.Chdir(repository)
workflowPath := filepath.Join(".github", "workflows", "mixed.yml")
linuxPath, err := os.Executable()
if err != nil {
t.Fatal(err)
Expand All @@ -409,7 +419,7 @@ func TestPluginPublishesMixedRuntimeDistributions(t *testing.T) {
0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00,
}
darwinPath := filepath.Join(root, "buildkite-gha-darwin")
darwinPath := filepath.Join(t.TempDir(), "buildkite-gha-darwin")
if err := os.WriteFile(darwinPath, darwinContents, 0o700); err != nil {
t.Fatal(err)
}
Expand Down Expand Up @@ -487,10 +497,23 @@ func TestPluginPublishesMixedRuntimeDistributions(t *testing.T) {
Command string
}{Image: step.Image, Queue: step.Agents["queue"], Command: step.Command}
}
if linux := steps["gha-linux"]; linux.Queue != "linux" || linux.Image != image || !strings.Contains(linux.Command, "--hosted-tool-cache") || !strings.Contains(linux.Command, strings.TrimPrefix(cliTestRuntimeDigest(), "sha256:")) {
var linux, macos struct {
Image string
Queue string
Command string
}
for key, step := range steps {
switch {
case strings.HasSuffix(key, "-linux"):
linux = step
case strings.HasSuffix(key, "-macos"):
macos = step
}
}
if linux.Queue != "linux" || linux.Image != image || !strings.Contains(linux.Command, "--hosted-tool-cache") || !strings.Contains(linux.Command, strings.TrimPrefix(cliTestRuntimeDigest(), "sha256:")) {
t.Fatalf("Linux pipeline step = %#v", linux)
}
if macos := steps["gha-macos"]; macos.Queue != "macos" || macos.Image != "" || strings.Contains(macos.Command, "--hosted-tool-cache") || !strings.Contains(macos.Command, strings.TrimPrefix(darwinDigest, "sha256:")) {
if macos.Queue != "macos" || macos.Image != "" || strings.Contains(macos.Command, "--hosted-tool-cache") || !strings.Contains(macos.Command, strings.TrimPrefix(darwinDigest, "sha256:")) {
t.Fatalf("Darwin pipeline step = %#v", macos)
}
}
Expand Down
Loading