Add marks parameter for subset test selection by dukelion · Pull Request #1054 · goss-org/goss

dukelion · 2026-04-17T17:07:29Z

Add `marks` parameter for subset test selection

Summary

This PR introduces a marks feature that allows users to tag resources and run only specific subsets of tests, similar to pytest's -m flag. This enables more flexible test workflows in CI/CD and local development.

Changes

Core Feature:

Added Marks []string field to all 16 resource types (addr, command, dns, file, gossfile, group, http, interface, kernel_param, matching, mount, package, port, process, service, user)
Added --marks and --exclude-marks CLI flags for validate and serve commands
Added ?marks= and ?exclude_marks= query parameters to HTTP endpoints
Added --marks flag to goss add and goss autoadd commands for tagging new resources

Infrastructure:

Thread-safe skip-state handling for concurrent HTTP requests
Cache isolation by marks (resources filtered by marks don't pollute cache)
Debug logging for mark filter summary (visible with -L DEBUG)

Usage

# goss.yaml
file:
  /etc/nginx/nginx.conf:
    exists: true
    marks:
      - nginx
      - config

port:
  tcp:80:
    listening: true
    marks:
      - nginx

command:
  echo test:
    exit-status: 0

# Run only nginx-related tests
goss validate --marks nginx

# Run all tests EXCEPT config tests
goss validate --exclude-marks config

# HTTP endpoint with marks
curl "http://localhost:8080/healthz?marks=nginx"

Testing

Unit tests added for mark filtering, config parsing, and HTTP endpoint behavior
All existing tests pass: go test -race -count=3 ./...
Tested serve-mode with concurrent requests using different marks

This change introduces a minimal util.Logger interface (Printf, Fatalf) as a first step toward decoupling goss from the standard library's global log package. It addresses some friction reported in goss-org#544 around using goss as a library, and provides a path toward resolving goss-org#991 where log level filtering doesn't catch all messages. What this improves: - Tests can now run with t.Parallel() without racing on log.SetOutput; TestLogger captures output per-test with goroutine-safe buffer - Library consumers can inject custom log sinks via util.WithLogger(), including filtered loggers that could address goss-org#991 - Data races are fixed in: color.NoColor (sync.Once), store.go globals (RWMutex), and format.UseStringerRepresentation (sync.Once) The approach keeps most API signatures stable - core functions return warnings as values and edge layers handle logging. WriteJSON now returns (string, error) to surface its warning message, which is the one public API change. New files: - util/logger.go: interface and DefaultLogger/TestLogger implementations - util/logger_test.go: unit tests including concurrent write coverage - util/color_init.go: shared InitNoColor for race-free color disable This doesn't fully resolve goss-org#544 (os.Exit remains in some paths) or goss-org#991 (a filtered logger would need to be provided), but it removes blockers and provides seams for both library embeddability and log control.

Adds a new top-level 'marks' field ([]string) on all resource types and CLI/HTTP filtering options. Addresses goss-org#544 (library usage improvements) and provides foundation for goss-org#991 (log level filtering). Usage: goss validate --marks critical,fast goss validate --exclude-marks slow,flaky goss serve --marks critical # overridable via ?marks= query param Backward Compatibility: - Resources without marks run by default (existing gossfiles unaffected) - Empty marks omitted from JSON output to avoid schema churn - When filters exclude all tests, goss exits 0 with "all skipped" output, consistent with DisabledResourceTypes behavior Why this matters: - Selective alerting: run only critical checks in production - Incident response: filter by category (network, storage, etc.) - Flaky test quarantine: exclude known-broken tests from CI gates - Resource constraints: skip memory/CPU-heavy checks under pressure Filter semantics: - --marks: include only resources with at least one matching mark - --exclude-marks: skip resources with any matching mark - Inclusion evaluated before exclusion when both specified Implementation highlights: - Cache keys encode mark filter to prevent cross-contamination - Query params override server-config without mutating shared state - Skip-state snapshot/restore under gossMu for serve-mode safety - Marks preserved across gossfile re-parses (with Title/Meta) Tests: 382 lines across marks_test.go, validate_marks_test.go, serve_marks_test.go, config_test.go covering round-trip serialization, filter logic, HTTP endpoint, cache isolation, and concurrency safety.

- Add SetMarks() to ResourceRead interface (all 16 resource types) - goss add/autoadd --marks flag for auto-tagging new resources - Debug logging for mark filter summary when -L DEBUG set - README section on marks usage with CLI and HTTP examples The --marks flag on add commands only applies to newly created resources; existing marks in the gossfile are preserved.

kgaughan · 2026-08-02T13:09:24Z

I managed to resolve all the merge conflicts, but it's now hitting some linter errors.

kgaughan · 2026-08-02T14:59:49Z

@dukelion I've fixed the two linter issues I introduced by accident when I was resolving the merge conflict, as well as a number of others that cropped up. While the tests are all passing, I'd appreciate if you'd take a look over things yourself (especially c773da4) to make sure I haven't missed anything.

dukelion · 2026-08-03T14:51:35Z

@kgaughan thanks for your fixes!
Ive noticed a couple issues with the 'marks' feature that needs fixing. I think the logger refactoring part of this MR has value regardless of that, do you mind if I split it into 2 separate MRs?

kgaughan · 2026-08-03T15:08:54Z

I think that would actually be a good thing! It'd nice to have the two changes decoupled, so go for it!

kgaughan · 2026-08-03T15:11:36Z

I would guess one of the things you're looking to do is to bundle up IncludeMarks and ExcludeMarksinGossConfig` into a single struct? That would help tidy things up, and you could have the methods for checking things hanging off of that instead.

kgaughan · 2026-08-03T15:12:42Z

Regarding logging, I've been mulling over the idea of switching to using log/slog. I dunno if you want to keep that in mind with the two separate pull requests you're planning.

dukelion · 2026-08-03T21:55:34Z

Extracted first part. Its somewhat helps with log/slog migration and closes #991 by coincidence.

kgaughan · 2026-08-04T18:34:35Z

#1092 is merged, so you should be good to go with the marks PR. Thanks!

kgaughan · 2026-08-04T18:53:37Z

Some bad news: after the merge, some tests started failing. Here's an excerpt:

        testing.go:1617: race detected during execution of test
==================
WARNING: DATA RACE
Write at 0x00c000268498 by goroutine 697:
  bytes.(*Buffer).grow()
      /opt/hostedtoolcache/go/1.25.12/x64/src/bytes/buffer.go:160 +0x3b1
  bytes.(*Buffer).Write()
      /opt/hostedtoolcache/go/1.25.12/x64/src/bytes/buffer.go:185 +0xc4
  log.(*Logger).output()
      /opt/hostedtoolcache/go/1.25.12/x64/src/log/log.go:244 +0x68b
  log.Printf()
      /opt/hostedtoolcache/go/1.25.12/x64/src/log/log.go:408 +0x1064
  github.com/goss-org/goss/outputs.Json.Output()
      /home/runner/work/goss/goss/outputs/json.go:93 +0xf3a
  github.com/goss-org/goss/outputs.(*Json).Output()
      <autogenerated>:1 +0x7d
  github.com/goss-org/goss.healthHandler.output()
      /home/runner/work/goss/goss/serve.go:126 +0x159
  github.com/goss-org/goss.healthHandler.processAndEnsureCached()
      /home/runner/work/goss/goss/serve.go:118 +0x4e7
  github.com/goss-org/goss.healthHandler.ServeHTTP()
      /home/runner/work/goss/goss/serve.go:93 +0x5f7
  github.com/goss-org/goss.TestServeHandlesConcurrentRequests.func1()
      /home/runner/work/goss/goss/serve_concurrency_test.go:40 +0x2ba

Previous read at 0x00c000268498 by goroutine 1089:
  bytes.(*Buffer).String()
      /opt/hostedtoolcache/go/1.25.12/x64/src/bytes/buffer.go:77 +0x585
  github.com/goss-org/goss.TestServeWithNoContentNegotiation.func1()
      /home/runner/work/goss/goss/serve_test.go:66 +0x57b
  testing.tRunner()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1934 +0x21c
  testing.(*T).Run.gowrap1()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x44

Goroutine 697 (running) created at:
  github.com/goss-org/goss.TestServeHandlesConcurrentRequests()
      /home/runner/work/goss/goss/serve_concurrency_test.go:37 +0x2f6
  testing.tRunner()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1934 +0x21c
  testing.(*T).Run.gowrap1()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x44

Goroutine 1089 (finished) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x9d2
  github.com/goss-org/goss.TestServeWithNoContentNegotiation()
      /home/runner/work/goss/goss/serve_test.go:46 +0x564
  testing.tRunner()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1934 +0x21c
  testing.(*T).Run.gowrap1()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x44
==================
--- FAIL: TestServeHandlesConcurrentRequests (0.19s)
    testing.go:1617: race detected during execution of test
--- FAIL: TestServeNegotiatingContent (0.22s)
    --- FAIL: TestServeNegotiatingContent/accept_application/json (0.05s)
        serve_test.go:182: testName "accept application/json" log output:
            [TRACE] : requesting health probe
            ...
    --- FAIL: TestServeNegotiatingContent/accept_text/json_translates_to_application/json (0.01s)
        serve_test.go:182: testName "accept text/json translates to application/json" log output:
            ...
            [DEBUG] : status 200
        testing.go:1617: race detected during execution of test
    --- FAIL: TestServeNegotiatingContent/when_accept_is_application/vnd.goss-json,_return_more_widely_known_application/json (0.02s)
        serve_test.go:182: testName "when accept is application/vnd.goss-json, return more widely known application/json" log output:
            ...
            [DEBUG] 192.0.2.1:1234: status 200
        testing.go:1617: race detected during execution of test

Here's the run: https://github.com/goss-org/goss/actions/runs/30939164511/job/92092838295

kgaughan · 2026-08-04T19:13:27Z

I've given the pipeline run another kick: it might fail in interesting ways that'll help diagnose the issue.

kgaughan · 2026-08-04T19:18:25Z

Here's the traceback from the second run:

WARNING: DATA RACE
Write at 0x00c00033ddd0 by goroutine 635:
  bytes.(*Buffer).tryGrowByReslice()
      /opt/hostedtoolcache/go/1.25.12/x64/src/bytes/buffer.go:123 +0xa5
  bytes.(*Buffer).Write()
      /opt/hostedtoolcache/go/1.25.12/x64/src/bytes/buffer.go:183 +0xb9
  log.(*Logger).output()
      /opt/hostedtoolcache/go/1.25.12/x64/src/log/log.go:244 +0x68b
  log.Printf()
      /opt/hostedtoolcache/go/1.25.12/x64/src/log/log.go:408 +0x1064
  github.com/goss-org/goss/outputs.Json.Output()
      /home/runner/work/goss/goss/outputs/json.go:93 +0xf3a
  github.com/goss-org/goss/outputs.(*Json).Output()
      <autogenerated>:1 +0x7d
  github.com/goss-org/goss.healthHandler.output()
      /home/runner/work/goss/goss/serve.go:126 +0x159
  github.com/goss-org/goss.healthHandler.processAndEnsureCached()
      /home/runner/work/goss/goss/serve.go:118 +0x4e7
  github.com/goss-org/goss.healthHandler.ServeHTTP()
      /home/runner/work/goss/goss/serve.go:93 +0x5f7
  github.com/goss-org/goss.TestServeHandlesConcurrentRequests.func1()
      /home/runner/work/goss/goss/serve_concurrency_test.go:40 +0x2ba

Previous read at 0x00c00033ddd0 by goroutine 1098:
  bytes.(*Buffer).String()
      /opt/hostedtoolcache/go/1.25.12/x64/src/bytes/buffer.go:77 +0x59b
  github.com/goss-org/goss.TestServeWithNoContentNegotiation.func1()
      /home/runner/work/goss/goss/serve_test.go:66 +0x57b
  testing.tRunner()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1934 +0x21c
  testing.(*T).Run.gowrap1()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x44

Goroutine 635 (running) created at:
  github.com/goss-org/goss.TestServeHandlesConcurrentRequests()
      /home/runner/work/goss/goss/serve_concurrency_test.go:37 +0x2f6
  testing.tRunner()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1934 +0x21c
  testing.(*T).Run.gowrap1()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x44

Goroutine 1098 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x9d2
  github.com/goss-org/goss.TestServeWithNoContentNegotiation()
      /home/runner/work/goss/goss/serve_test.go:46 +0x564
  testing.tRunner()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1934 +0x21c
  testing.(*T).Run.gowrap1()
      /opt/hostedtoolcache/go/1.25.12/x64/src/testing/testing.go:1997 +0x44

Link: https://github.com/goss-org/goss/actions/runs/30939164511/job/92103079669

dukelion added 3 commits April 17, 2026 17:12

dukelion requested a review from aelsabbahy as a code owner April 17, 2026 17:07

kgaughan reviewed May 18, 2026

View reviewed changes

Comment thread cmd/goss/goss.go Outdated

Merge branch 'master' into feature/marks-parameter

c773da4

kgaughan added 7 commits August 2, 2026 14:56

Fix linter issue in cmd/goss/goss.go

2bb32ff

Fix linter error in outputs/json.go

7a8e68a

Update testdata/marks.goss.yaml

e29bbf7

Fix linter issues in README.md

96de4c2

Fix linter issue in resource/marks_test.go

c6428b4

README.md: switch from console to sh to avoid linter errors

e5e9030

Same again...

db85418

dukelion mentioned this pull request Aug 3, 2026

Fix data races in goss serve and mute the cache-miss log by default #1092

Merged

dukelion mentioned this pull request Aug 3, 2026

Make log output injectable and structured, using log/slog #1093

Open

Navigation Menu

Uh oh!

Add marks parameter for subset test selection - #1054

Add marks parameter for subset test selection#1054
dukelion wants to merge 11 commits into
goss-org:masterfrom
dukelion:feature/marks-parameter

dukelion commented Apr 17, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Uh oh!

kgaughan commented Aug 2, 2026

Uh oh!

kgaughan commented Aug 2, 2026

Uh oh!

dukelion commented Aug 3, 2026

Uh oh!

kgaughan commented Aug 3, 2026

Uh oh!

kgaughan commented Aug 3, 2026

Uh oh!

kgaughan commented Aug 3, 2026

Uh oh!

dukelion commented Aug 3, 2026 •

edited

Loading

Uh oh!

kgaughan commented Aug 4, 2026

Uh oh!

kgaughan commented Aug 4, 2026

Uh oh!

kgaughan commented Aug 4, 2026

Uh oh!

kgaughan commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

dukelion commented Apr 17, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!