-
Notifications
You must be signed in to change notification settings - Fork 64
feat: add async-config-poll sample (async-egress engine demo) #146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| target/ | ||
| keploy/ | ||
| .local-logs/ | ||
| config-stub/config-stub | ||
| *.log |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| # async-config-poll | ||
|
|
||
| A Spring Boot 1.5 / Java 8 rule-engine sample that demonstrates Keploy's | ||
| **async-egress engine**. | ||
|
|
||
| The app has two HTTP endpoints backed by MySQL, and it depends on a central | ||
| config service in two different ways: | ||
|
|
||
| | Interaction | When | Keploy treats it as | | ||
| |-------------|------|---------------------| | ||
| | `GET /v1/buckets/app-common`, `app-features`, `app-config?watch=false` | once, at boot (blocking) | ordinary synchronous mocks — the app cannot boot without them | | ||
| | `GET /v1/buckets/app-config?watch=true&version=N` | forever, from a background daemon thread | **async egress** — fires on the app's own schedule, not tied to any ingress testcase | | ||
| | `SELECT ...` on MySQL | per request | ordinary synchronous mocks | | ||
|
|
||
| The background watch poll is the interesting part. Because it runs on a timer in | ||
| its own thread, it does not line up one-to-one with the recorded testcases. A | ||
| naive replay would fail: the app polls at replay time too, and the request | ||
| (`?version=17`, `?version=18`, …) never matches a recorded one exactly. | ||
|
|
||
| Keploy's async-egress engine handles this. The lane declared in `keploy.yml` | ||
| tells Keploy that this endpoint is async: | ||
|
|
||
| ```yaml | ||
| async: | ||
| lanes: | ||
| - name: config-watch | ||
| type: http | ||
| match: | ||
| pathRegex: "^/v1/buckets/app-config$" | ||
| matchQuery: | ||
| watch: "true" # only the background watch polls, not the boot call | ||
| volatileParams: ["version"] # the version query param varies every poll — treat as noise | ||
| ``` | ||
|
|
||
| At replay the engine serves the recorded watch responses back to the poller | ||
| independently of testcase ordering, treats the changing `version` param as | ||
| shape-noise, and keep-alives the poller when there is nothing left to serve — so | ||
| the app stays happy and the ingress tests still pass. At the end of replay Keploy | ||
| prints an `async egress verdict` line (served / shape_flags / not_exercised). | ||
|
|
||
| ## Endpoints | ||
|
|
||
| - `GET /health` — small health payload; runs `SELECT 1` against MySQL. | ||
| - `GET /rules/{useCase}` — ordered rules for `(useCase, tenant)` read from MySQL. | ||
| Requires headers `X-Tenant-Id` and `X-Agent-Id`. | ||
| Example: `GET /rules/ORDER_FLOW` with `X-Tenant-Id: ACME`, `X-Agent-Id: 957`. | ||
|
|
||
| ## Run it locally | ||
|
|
||
| Prerequisites: JDK 8, Maven, Docker, Go (for the config stub), and a Keploy | ||
| build that includes the async-egress engine. | ||
|
|
||
| ```bash | ||
| # 1. dependencies | ||
| docker compose up -d # MySQL 5.7 seeded from init.sql | ||
| go run ./config-stub & # config service stub on :9100 | ||
|
|
||
| # 2. build the app | ||
| mvn -B clean package -Dmaven.test.skip=true | ||
|
|
||
| # 3. record | ||
| sudo -E keploy record -c "java -jar target/async-config-poll.jar" | ||
| # drive traffic, then Ctrl-C keploy: | ||
| curl localhost:8080/health | ||
| curl -H "X-Tenant-Id: ACME" -H "X-Agent-Id: 957" localhost:8080/rules/ORDER_FLOW | ||
|
|
||
| # 4. replay (deps down — Keploy serves everything from mocks) | ||
| docker compose down | ||
| sudo keploy test -c "java -jar target/async-config-poll.jar" --delay 20 | ||
| ``` | ||
|
|
||
| To make a watch poll land in the *middle* of a testcase at replay (so the async | ||
| lane is actively exercised rather than drained between tests), lower the poll | ||
| interval and widen the request window: | ||
|
|
||
| ```bash | ||
| WATCH_INTERVAL_MS=150 RULES_DELAY_MS=800 sudo -E keploy record -c "java -jar target/async-config-poll.jar" | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| module config-stub | ||
|
|
||
| go 1.21 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| // Command config-stub is a stand-in for a central config service. It backs the | ||
| // app's boot-blocking config fetch and its background watch long-poll: | ||
| // | ||
| // - GET /v1/buckets/{name} -> current config (version 1) | ||
| // - GET /v1/buckets/app-config?watch=true&version=N -> long-poll: returns the | ||
| // NEXT version (N+1), simulating a config change on each watch poll. | ||
| // | ||
| // It is hit only during `keploy record`. At replay time Keploy serves the | ||
| // recorded responses instead, so this stub does not need to be running. | ||
| package main | ||
|
|
||
| import ( | ||
| "encoding/json" | ||
| "log" | ||
| "net/http" | ||
| "strconv" | ||
| "strings" | ||
| ) | ||
|
|
||
| func main() { | ||
| http.HandleFunc("/v1/buckets/", func(w http.ResponseWriter, r *http.Request) { | ||
| name := strings.TrimPrefix(r.URL.Path, "/v1/buckets/") | ||
| q := r.URL.Query() | ||
|
|
||
| version := 1 | ||
| if q.Get("watch") == "true" { | ||
| cur, _ := strconv.Atoi(q.Get("version")) | ||
| version = cur + 1 // deliver the next version -> a "change" per poll | ||
| } | ||
|
|
||
| w.Header().Set("Content-Type", "application/json") | ||
| w.WriteHeader(http.StatusOK) | ||
| _ = json.NewEncoder(w).Encode(map[string]interface{}{ | ||
| "name": name, | ||
| "version": version, | ||
| "keys": map[string]string{ | ||
| "feature.enabled": "true", | ||
| }, | ||
| }) | ||
| }) | ||
| log.Println("config-stub listening on :9100") | ||
| log.Fatal(http.ListenAndServe(":9100", nil)) | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| services: | ||
| mysql: | ||
| # MySQL 5.7 (not 8.x): Spring Boot 1.5 manages MySQL Connector/J to 5.1.x, | ||
| # which cannot speak MySQL 8's default caching_sha2_password auth plugin. | ||
| image: mysql:5.7 | ||
| command: --default-authentication-plugin=mysql_native_password | ||
| environment: | ||
| MYSQL_ROOT_PASSWORD: rootpass | ||
| MYSQL_DATABASE: ruledb | ||
| MYSQL_USER: app | ||
| MYSQL_PASSWORD: app | ||
| ports: | ||
| - "3306:3306" | ||
| volumes: | ||
| - ./init.sql:/docker-entrypoint-initdb.d/init.sql | ||
| # mysql:5.7 can trip an fd-limit config check on some Docker hosts; pin a | ||
| # sane nofile limit so mysqld starts and stays up under connection load. | ||
| ulimits: | ||
| nofile: | ||
| soft: 65535 | ||
| hard: 65535 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| -- Rule-engine schema + seed for the (ORDER_FLOW, ACME) use case. | ||
| -- The app reads these rows over the MySQL wire; Keploy captures that traffic | ||
| -- as mocks and serves it back on replay. | ||
| CREATE DATABASE IF NOT EXISTS ruledb; | ||
| USE ruledb; | ||
|
|
||
| CREATE TABLE IF NOT EXISTS rules ( | ||
| rule_id BIGINT PRIMARY KEY, | ||
| use_case VARCHAR(64) NOT NULL, | ||
| tenant VARCHAR(64) NOT NULL, | ||
| constraint_expr TEXT NOT NULL, | ||
| rule_type VARCHAR(16) NOT NULL, | ||
| INDEX idx_uc_tenant (use_case, tenant) | ||
| ); | ||
|
|
||
| CREATE TABLE IF NOT EXISTS rule_actions ( | ||
| id BIGINT AUTO_INCREMENT PRIMARY KEY, | ||
| rule_id BIGINT NOT NULL, | ||
| basic_action VARCHAR(255) NOT NULL, | ||
| action_details TEXT NOT NULL, | ||
| seq INT NOT NULL, | ||
| INDEX idx_rule (rule_id) | ||
| ); | ||
|
|
||
| INSERT INTO rules (rule_id, use_case, tenant, constraint_expr, rule_type) VALUES | ||
| (14,'ORDER_FLOW','ACME','status == "COMPLETED" && type.equals("CHECKOUT")','POST'), | ||
| (15,'ORDER_FLOW','ACME','status == "COMPLETED" && type.equals("PAYMENT")','POST'), | ||
| (16,'ORDER_FLOW','ACME','status == "COMPLETED" && type.equals("SHIPMENT")','POST'), | ||
| (17,'ORDER_FLOW','ACME','status == "IN_PROGRESS" && type.equals("REFUND")','POST'), | ||
| (18,'ORDER_FLOW','ACME','status == "COMPLETED" && type.equals("FULFILLMENT")','PRE'); | ||
|
|
||
| INSERT INTO rule_actions (rule_id, basic_action, action_details, seq) VALUES | ||
| (14,'com.example.rules.handlers.ForceSyncHandler','{}',1), | ||
| (15,'com.example.rules.handlers.PaymentTaskHandler','{}',1), | ||
| (15,'com.example.rules.handlers.NotifyHandler','{"optional":"true","channel":"email"}',2), | ||
| (16,'com.example.rules.handlers.ValidateHandler','{"optional":"true"}',1), | ||
| (16,'com.example.rules.handlers.ShipmentHandler','{}',2), | ||
| (17,'com.example.rules.handlers.RefundHandler','{}',1), | ||
| (18,'com.example.rules.handlers.ImageSyncHandler','{"optional":"true"}',1); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Keploy config for the async-config-poll sample. | ||
| # | ||
| # The only non-default section is `async.lanes`. It declares the config-service | ||
| # watch long-poll as async egress, so Keploy's async-egress engine records and | ||
| # replays it independently of the ingress testcase ordering. | ||
| # | ||
| # Lane "config-watch": | ||
| # - match.pathRegex : only the /v1/buckets/app-config endpoint | ||
| # - matchQuery.watch : "true" -> only the background watch polls (the | ||
| # one-time boot "get current version" call uses | ||
| # watch=false and stays an ordinary blocking mock) | ||
| # - volatileParams : ["version"] -> the version query param changes every | ||
| # poll, so it is treated as shape-noise, not a mismatch | ||
| async: | ||
| lanes: | ||
| - name: config-watch | ||
| type: http | ||
| match: | ||
| pathRegex: "^/v1/buckets/app-config$" | ||
| matchQuery: | ||
| watch: "true" | ||
| volatileParams: ["version"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" | ||
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <!-- async-config-poll: a Spring Boot 1.5 / Java 8 rule-engine sample that | ||
| demonstrates Keploy's async-egress engine. | ||
|
|
||
| The app serves two HTTP endpoints backed by MySQL (/health, | ||
| /rules/{useCase}) and, in the background, long-polls a central config | ||
| service for version changes (GET /v1/buckets/app-config?watch=true). | ||
| That watch poll fires from a daemon thread on the app's own schedule — | ||
| async relative to the ingress testcases — so Keploy records and | ||
| replays it through the async-egress engine (lane "config-watch"). --> | ||
|
|
||
| <groupId>com.example</groupId> | ||
| <artifactId>async-config-poll</artifactId> | ||
| <version>1.0.0</version> | ||
| <packaging>jar</packaging> | ||
|
|
||
| <parent> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-parent</artifactId> | ||
| <version>1.5.22.RELEASE</version> | ||
| <relativePath/> | ||
| </parent> | ||
|
|
||
| <properties> | ||
| <java.version>1.8</java.version> | ||
| <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> | ||
| <start-class>com.example.asyncconfig.Application</start-class> | ||
| </properties> | ||
|
|
||
| <dependencies> | ||
| <!-- Jersey / JAX-RS REST layer. The jersey starter pulls | ||
| spring-boot-starter-web (Tomcat); exclude Tomcat and run on | ||
| embedded Jetty instead. --> | ||
| <dependency> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-jersey</artifactId> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-tomcat</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-jetty</artifactId> | ||
| </dependency> | ||
|
|
||
| <!-- MySQL primary datastore (rules are read from here). --> | ||
| <dependency> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-jdbc</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>mysql</groupId> | ||
| <artifactId>mysql-connector-java</artifactId> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <finalName>async-config-poll</finalName> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-maven-plugin</artifactId> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </project> |
17 changes: 17 additions & 0 deletions
17
async-config-poll/src/main/java/com/example/asyncconfig/Application.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| package com.example.asyncconfig; | ||
|
|
||
| import org.springframework.boot.SpringApplication; | ||
| import org.springframework.boot.autoconfigure.SpringBootApplication; | ||
|
|
||
| /** | ||
| * async-config-poll — a small rule engine that serves /health and | ||
| * /rules/{useCase} on port 8080 (backed by MySQL), fetches boot-blocking | ||
| * config at startup, and then long-polls a config service in the background | ||
| * for version changes (see {@link com.example.asyncconfig.config.ConfigWatchService}). | ||
| */ | ||
| @SpringBootApplication | ||
| public class Application { | ||
| public static void main(String[] args) { | ||
| SpringApplication.run(Application.class, args); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in f5d608e — ConfigWatchService now reads
feature.enabledfrom the bucket's nestedkeysmap (via aboolFromKeyshelper), matching the config-stub's response shape. Previously the top-level read always returned null and silently fell back to the default.