Skip to content

feat: Serde benchmark testing#1078

Merged
jbelkins merged 30 commits into
mainfrom
jbe/serde_benchmark
Jun 8, 2026
Merged

feat: Serde benchmark testing#1078
jbelkins merged 30 commits into
mainfrom
jbe/serde_benchmark

Conversation

@jbelkins

@jbelkins jbelkins commented May 22, 2026

Copy link
Copy Markdown
Contributor

Description of changes

Adds a specialization of the protocol test suite for testing serialization performance. "Serde benchmark" tests are defined as either HTTP request or HTTP response protocol tests, and are identified by the serde-benchmark tag applied to the test.

When a serde benchmark test is to be generated, the usual request / response protocol test body is modified to perform testing.

Specific changes:

  • Special models for testing various size payloads are used to generate tests (note that these are unchanged from the cross-platform spec.)
  • A special serde benchmark telemetry provider is installed to measure time taken during serialization.
  • Times to perform request serialization & response deserialization are measured repeatedly, and the results are transformed into measurements (i.e. mean, deviation, p50/90/99/etc.)
  • Generates a JSON report from collected measurements

Changes to protocol tests generally:

  • When Data is to be constructed as an input or expectation of a test, Swift "raw strings" are used, and the string is no longer rendered with any escaping.
  • Format and spacing changes

Note: companion PR awslabs/aws-sdk-swift#2172 has no code changes and is opened only to run SDK testing with this change applied.

Scope

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jbelkins jbelkins changed the title Jbe/serde benchmark feat: Serde benchmark testing May 23, 2026
{ "self.region" },
false,
false,
),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

region and signingRegion are added to non-AWS, smithy-swift-based clients because region & signing region are needed to configure sigv4 which is a Smithy-native auth scheme.

The properties are optional, so they may be omitted when configuring clients to use other auth schemes.

"Data(\"\"\"\n\$L\n\"\"\".utf8)",
test.body.get().replace("\\\"", "\\\\\""),
"Data(#\"\"\"\n\$L\n\"\"\"#.utf8)",
test.body.get(),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we use Swift "raw strings" (i.e. #""" ... """#) to include string literal content exactly as written, and we remove the backslash escaping.

url,
)
}
writer.write("telemetryProvider: telemetryProvider,")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The serde benchmark telemetry provider, which takes the performance measurements, is installed in serde benchmark tests and in protocol tests alike, though it has no effect when running protocol tests.

@jbelkins jbelkins marked this pull request as ready for review June 2, 2026 04:09
@jbelkins jbelkins requested a review from a team as a code owner June 2, 2026 04:09
@jbelkins jbelkins requested a review from dayaffe June 2, 2026 04:11
@jbelkins jbelkins merged commit 1ef1fc7 into main Jun 8, 2026
45 checks passed
@jbelkins jbelkins deleted the jbe/serde_benchmark branch June 8, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants