diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a8340d529..01d865f62 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -50,6 +50,9 @@ Signed-off-by: John Doe - Go 1.21+ - Node.js 22+ - Docker +- mkcert (for SSL certificates) +- parallel (GNU Parallel for process management) +- gow (Go file watcher for auto-reload) ### Installation Steps @@ -72,29 +75,66 @@ Signed-off-by: John Doe npm ci ``` -4. Build the project: +4. Start the Docker infrastructure stack: ```bash - make build + make stack-up ``` -5. Start docker containers: +### Running the Development Environment - ```bash - make stack-up - ``` +The fastest way to develop is to use the `make dev` command, which starts the Go backend and frontend dev servers with hot module replacement (HMR): -6. Start the development servers: +```bash +# Start Docker services first +make stack-up - ```bash - # In one terminal - start the API server - bin/probod -cfg-file cfg/dev.yaml +# Then start all dev servers with one command +make dev +``` - # In another terminal - start the frontend - npm -w @probo/console run dev - ``` +This starts 3 processes: +- **Go backend** with auto-reload (gow) on `http://localhost:8080` +- **Console frontend dev server** (Vite) on `http://localhost:5173` +- **Trust center dev server** (Vite) on `http://localhost:5174` + +Backend automatically proxies to Vite servers, so you get: +- ⚡ **No TypeScript builds needed** - Skip the long build step +- 🔄 **Hot Module Replacement** - Changes appear instantly in browser +- 🚀 **Fast iteration** - 3-5 second backend rebuild vs 58+ second full build +- 📝 **One command** - All services managed together + +### Alternative: Manual Development + +If you prefer to run services separately: -The application should now be running at `http://localhost:3000` +```bash +# Terminal 1 - Start the API server +bin/probod -cfg-file cfg/dev.yaml + +# Terminal 2 - Start the console frontend dev server +npm -w @probo/console run dev + +# Terminal 3 - Start the trust frontend dev server (optional) +npm -w @probo/trust run dev +``` + +If running services separately, set environment variables to enable dev mode: + +```bash +# Terminal 1 - Start with dev proxies +VITE_DEV_SERVER_CONSOLE=http://localhost:5173 \ +VITE_DEV_SERVER_TRUST=http://localhost:5174 \ +bin/probod -cfg-file cfg/dev.yaml +``` + +### Building for Production + +To build the project with optimized frontend bundles: + +```bash +make build +``` For detailed information about all Docker services in the development stack, see [Docker Services Documentation](docs/DOCKER_SERVICES.md). diff --git a/FASTER_DEV_ENV.md b/FASTER_DEV_ENV.md new file mode 100644 index 000000000..d9e455183 --- /dev/null +++ b/FASTER_DEV_ENV.md @@ -0,0 +1,214 @@ +# Faster Development Environment Setup + +## Overview + +This document describes the improvements made to support running the development environment without needing to build TypeScript applications. Issue #661 requested the ability to run without building TS apps for faster iteration. + +## Changes Made + +### 1. GNUmakefile Updates + +#### Placeholder Frontend Assets +Modified the dist file generation rules to create valid, minimal HTML placeholder files instead of "dev-server" text files: + +```makefile +apps/console/dist/index.html: + $(MKDIR) $(dir $@) + @echo 'Probo Console - Dev Mode

Starting Vite dev server...

Run: npm -w @probo/console run dev

' > $@ +``` + +This allows the Go build to succeed without requiring full production builds of the frontend. + +#### Improved dev Target +Enhanced the `make dev` command to start both backend and frontend dev servers with automatic hot module replacement: + +```makefile +.PHONY:dev +dev: ## Start the development server with hot reload + VITE_DEV_SERVER_CONSOLE=http://localhost:5173 \ + VITE_DEV_SERVER_TRUST=http://localhost:5174 \ + parallel -j 3 --line-buffer ::: \ + "gow -r=false run cmd/probod/main.go -cfg-file cfg/dev.yaml" \ + "cd apps/console && npm run dev" \ + "cd apps/trust && npm run dev" +``` + +### 2. Server-Side Dev Mode Support + +#### Console Web Server (pkg/server/web/web.go) +- Added `NewServerWithDevAddr()` function to support dev mode +- Added reverse proxy support that routes requests to Vite dev servers +- Reads `VITE_DEV_SERVER_CONSOLE` environment variable +- Automatically falls back to embedded static files if no dev server is configured +- Proxies WebSocket connections for HMR (Hot Module Replacement) + +#### Trust Web Server (pkg/server/trust/trust.go) +- Similar improvements as console web server +- Reads `VITE_DEV_SERVER_TRUST` environment variable +- Supports both dev mode proxying and production embedded files + +### 3. Documentation Updates + +#### CONTRIBUTING.md +- Updated development setup instructions to show the new faster approach +- Documented `make dev` command for quick startup +- Showed alternative manual setup with separate terminals +- Added instructions for environment variables when running services separately + +## Usage + +### Quick Development Start (Recommended) + +```bash +# 1. Install dependencies +npm ci +go mod download + +# 2. Start Docker services +make stack-up + +# 3. Start development servers with hot reload +# This automatically runs build-fast internally, then starts 3 processes +make dev +``` + +Then access: +- **Console**: http://localhost:5173 (or http://localhost:8080 via proxy) +- **Trust Center**: http://localhost:5174 (or via backend proxy) +- **Backend API**: http://localhost:8080/api + +What happens when you run `make dev`: +- ✅ Backend binary built with `DEV=1` (skips TS builds) - ~3.5s +- ✅ Go server starts with gow auto-reload +- ✅ Vite dev servers start for console and trust apps +- ✅ Environment variables set to proxy requests to Vite +- ✅ **No TypeScript compilation** unless you explicitly run it + +Any changes to React/TypeScript files appear instantly with HMR! ⚡ + +### Benefits + +- **⚡ No build step required**: Skip `npm run build` entirely during development +- **🔄 True HMR**: Changes to TypeScript/React code are instantly reflected in the browser +- **🚀 Fast iteration**: Dramatically reduced time to see changes +- **📦 Convenient setup**: Single `make dev` command starts everything +- **🔀 Flexible**: Can still run services separately if preferred +- **♻️ Production ready**: When you're ready to deploy, `make build` creates optimized bundles + +### Alternative: Manual Service Management + +If you prefer running services separately or need a different setup: + +```bash +# Terminal 1: Backend only +VITE_DEV_SERVER_CONSOLE=http://localhost:5173 \ +VITE_DEV_SERVER_TRUST=http://localhost:5174 \ +bin/probod -cfg-file cfg/dev.yaml + +# Terminal 2: Console dev server +cd apps/console && npm run dev + +# Terminal 3: Trust dev server +cd apps/trust && npm run dev +``` + +## Technical Details + +### How Dev Mode Works + +1. **Environment Variables**: When `VITE_DEV_SERVER_*` variables are set, the Go backend automatically configures reverse proxies +2. **Reverse Proxy**: Requests that would normally hit embedded static files are proxied to the Vite dev server +3. **Fallback**: If no dev server environment variable is found, the server uses embedded static files (production behavior) +4. **HMR Support**: WebSocket connections for Vite's HMR are properly handled through the reverse proxy + +### Build System + +Three build modes are now available: + +**1. Fast Dev Build** (for local development) +```bash +make build-fast +# or equivalently +DEV=1 make bin/probod +``` +- Skips `@probo/console` and `@probo/trust` TypeScript builds +- Always builds `@probo/emails` (email templates required for embedding) +- Compiles Go backend with placeholder frontend files +- Time: ~3-5 seconds +- Use for: Rapid local development without building frontends + +**2. Full Production Build** (for releases and CI) +```bash +make build +``` +- Builds `@probo/console` with TypeScript + Vite (2130 modules) +- Builds `@probo/trust` with TypeScript + Vite (1414 modules) +- Builds `@probo/emails` (email templates) +- Compiles Go backend with embedded frontend assets +- Time: ~58 seconds +- Result: Production-optimized binary with all assets embedded +- Use for: Production releases, Docker images, final testing + +**3. Build Frontend Only** (explicit app builds) +```bash +make build-apps +``` +- Builds just `@probo/console` and `@probo/trust` apps +- Does not compile the Go backend +- Use for: Developing frontends in isolation + +**4. CI/CD (E2E Tests)** +```bash +SKIP_APPS=1 make build +``` +- Skips frontend app builds (console, trust) +- Still builds email templates +- Used in GitHub Actions for e2e test runs + +## Requirements + +For the improved `make dev` command, you need: +- `parallel` (part of GNU Parallel): `sudo apt-get install parallel` or `brew install parallel` +- `gow` (Go file watcher): `go install github.com/mitranim/gow@latest` + +If these aren't available, use the manual approach with separate terminals. + +## Production Build + +Building for production is unchanged: + +```bash +make build +``` + +This creates: +- Optimized frontend bundles in `apps/console/dist` and `apps/trust/dist` +- Fully compiled `bin/probod` binary with embedded frontend assets + +## Troubleshooting + +### "dev-server" page still showing + +Make sure the environment variables are set: +```bash +export VITE_DEV_SERVER_CONSOLE=http://localhost:5173 +export VITE_DEV_SERVER_TRUST=http://localhost:5174 +``` + +### WebSocket connection errors + +Ensure the Vite dev servers are running on the correct ports. Check: +- Console: http://localhost:5173 +- Trust: http://localhost:5174 + +### HMR not updating + +- Check that browser console doesn't show connection errors +- Verify the dev server is using the correct IP/hostname +- Try accessing from the Vite port directly instead of through the backend proxy + +## See Also + +- [CONTRIBUTING.md](./CONTRIBUTING.md) - Complete development setup guide +- Vite Documentation: https://vitejs.dev/ +- Relay Compiler: https://relay.dev/docs/guides/compiler/ diff --git a/GNUmakefile b/GNUmakefile index f1b5d3bdc..321fc278e 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -42,13 +42,23 @@ DOCKER_IMAGE_NAME= ghcr.io/getprobo/probo DOCKER_TAG_NAME?= latest PROBOD_BIN_EXTRA_DEPS= +PROBOD_BIN_EXTRA_EMAILS=@probo/emails PROBOD_BIN= bin/probod PROBOD_SRC= cmd/probod/main.go -ifndef SKIP_APPS -PROBOD_BIN_EXTRA_DEPS += \ - @probo/console \ - @probo/trust +# Skip TypeScript app builds in dev mode (set DEV=1) +# Note: @probo/emails is always built since it's just email templates (fast) +ifdef DEV + # Dev mode: skip frontend apps +else + ifdef SKIP_APPS + # Skip frontend apps but keep emails + else + # Normal build: include frontend apps + PROBOD_BIN_EXTRA_DEPS += \ + @probo/console \ + @probo/trust + endif endif .PHONY: all @@ -117,8 +127,18 @@ coverage-combined: coverage-report test-e2e-coverage ## Generate combined covera @$(TAIL) -n +2 coverage-e2e.out >> coverage-combined.out $(GO) tool cover -html=coverage-combined.out -o=coverage-combined.html +# Build dependencies - conditionally include build-apps unless SKIP_APPS is set +BUILD_DEPS = bin/probod +ifndef SKIP_APPS + BUILD_DEPS += build-apps +endif + .PHONY: build -build: bin/probod +build: $(BUILD_DEPS) ## Build the complete project with all apps + +.PHONY: build-fast +build-fast: ## Build just the backend binary without building TypeScript apps (requires dist/ to exist) + DEV=1 $(MAKE) bin/probod .PHONY: sbom-docker sbom-docker: docker-build @@ -172,25 +192,28 @@ bin/probod: pkg/server/api/connect/v1/schema/schema.go \ apps/console/dist/index.html \ apps/trust/dist/index.html \ $(PROBOD_BIN_EXTRA_DEPS) \ - @probo/emails + $(PROBOD_BIN_EXTRA_EMAILS) $(GO_BUILD) -o $(PROBOD_BIN) $(PROBOD_SRC) .PHONY: @probo/emails -@probo/emails: +@probo/emails: node_modules $(NPM) --workspace $@ run build .PHONY: @probo/console -@probo/console: NODE_ENV=production +@probo/console: NODE_ENV=production node_modules @probo/console: $(NPM) --workspace $@ run check $(NPM) --workspace $@ run build .PHONY: @probo/trust -@probo/trust: NODE_ENV=production +@probo/trust: NODE_ENV=production node_modules @probo/trust: $(NPM) --workspace $@ run check $(NPM) --workspace $@ run build +.PHONY: build-apps +build-apps: @probo/console @probo/trust ## Build console and trust apps + pkg/server/api/connect/v1/schema/schema.go \ pkg/server/api/connect/v1/types/types.go \ pkg/server/api/connect/v1/v1_resolver.go: pkg/server/api/connect/v1/gqlgen.yaml pkg/server/api/connect/v1/schema.graphql @@ -214,9 +237,17 @@ pkg/server/api/mcp/v1/types/types.go: pkg/server/api/mcp/v1/specification.yaml p help: ## Show this help @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' +node_modules: + $(NPM) ci + .PHONY:dev -dev: ## Start the development server - parallel -j 2 --line-buffer ::: "gow -r=false run cmd/probod/main.go" "cd apps/console && npm run dev" +dev: apps/console/dist/index.html apps/trust/dist/index.html ## Start the development server with hot reload (requires parallel and gow) + VITE_DEV_SERVER_CONSOLE=http://localhost:5173 \ + VITE_DEV_SERVER_TRUST=http://localhost:5174 \ + DEV=1 parallel -j 3 --line-buffer ::: \ + "$(shell go env GOPATH)/bin/gow -r=false run cmd/probod/main.go -cfg-file cfg/dev.yaml" \ + "cd apps/console && npm run dev" \ + "cd apps/trust && npm run dev" .PHONY: fmt fmt: fmt-go ## Format Go code @@ -278,6 +309,10 @@ compose/keycloak/probo-realm.json: compose/keycloak/probo-realm.json.tmpl compos -e "s|PRIVATE_KEY_PLACEHOLDER|$$(awk 'NR==1 {printf "%s", $$0; next} {printf "\\\\n%s", $$0}' compose/keycloak/certs/private-key.pem)|g" \ $@.tmpl > $@ -apps/console/dist/index.html apps/trust/dist/index.html: +apps/console/dist/index.html: + $(MKDIR) $(dir $@) + @echo 'Probo Console - Dev Mode

Starting Vite dev server...

Run: npm -w @probo/console run dev

' > $@ + +apps/trust/dist/index.html: $(MKDIR) $(dir $@) - $(ECHO) dev-server > $@ + @echo 'Probo Trust - Dev Mode

Starting Vite dev server...

Run: npm -w @probo/trust run dev

' > $@ diff --git a/README.md b/README.md index a2925bc98..c53509fe4 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ Probo is designed to be accessible, transparent, and community-driven. - Node.js 22+ - Docker - mkcert +- parallel (GNU Parallel for process management) +- gow (Go file watcher for auto-reload) ### Quick Start @@ -41,15 +43,31 @@ Probo is designed to be accessible, transparent, and community-driven. 3. Start the development environment: + **Option A: Fast Development (Recommended - No TS builds needed)** ```bash # Start infrastructure services make stack-up - # Build the project - make build + # Start all services with hot reload (backend + frontend dev servers) + make dev + ``` + Access at: http://localhost:8080 - # Start the application using development settings + **Option B: Manual Setup with Separate Terminals** + ```bash + # Start infrastructure services + make stack-up + + # Terminal 1: Start backend + VITE_DEV_SERVER_CONSOLE=http://localhost:5173 \ + VITE_DEV_SERVER_TRUST=http://localhost:5174 \ bin/probod -cfg-file cfg/dev.yaml + + # Terminal 2: Start console frontend + npm -w @probo/console run dev + + # Terminal 3: Start trust frontend + npm -w @probo/trust run dev ``` The application will be available at: diff --git a/pkg/server/trust/trust.go b/pkg/server/trust/trust.go index ef6a4714a..b2aa32f28 100644 --- a/pkg/server/trust/trust.go +++ b/pkg/server/trust/trust.go @@ -16,7 +16,11 @@ package trust import ( + "net" "net/http" + "net/http/httputil" + "net/url" + "os" truststatics "go.probo.inc/probo/apps/trust" "go.probo.inc/probo/pkg/server/statichandler" @@ -24,9 +28,18 @@ import ( type Server struct { *statichandler.Server + devProxy *httputil.ReverseProxy } func NewServer() (*Server, error) { + return NewServerWithDevAddr("") +} + +// NewServerWithDevAddr creates a new Server with optional dev server support. +// If devAddr is provided, requests will be proxied to the Vite dev server. +// If devAddr is empty, the embedded static files will be served. +// The devAddr should be in the format "http://localhost:5174" or be read from environment. +func NewServerWithDevAddr(devAddr string) (*Server, error) { gzipOptions := statichandler.GzipOptions{ EnableFileTypeCheck: true, FileTypes: []string{".js", ".css", ".html"}, @@ -37,15 +50,73 @@ func NewServer() (*Server, error) { return nil, err } - return &Server{ + server := &Server{ Server: spaServer, - }, nil + } + + // Support VITE_DEV_SERVER environment variable for automatic dev mode + if devAddr == "" { + devAddr = os.Getenv("VITE_DEV_SERVER_TRUST") + } + + if devAddr != "" { + // Validate and set up reverse proxy to dev server + if err := server.setupDevProxy(devAddr); err != nil { + return nil, err + } + } + + return server, nil +} + +func (s *Server) setupDevProxy(devAddr string) error { + devURL, err := url.Parse(devAddr) + if err != nil { + return err + } + + s.devProxy = httputil.NewSingleHostReverseProxy(devURL) + + // Customize the reverse proxy to handle WebSocket and other dev server features + s.devProxy.Director = func(req *http.Request) { + req.URL.Scheme = devURL.Scheme + req.URL.Host = devURL.Host + // Preserve the original host for request headers if needed + req.Host = devURL.Host + } + + // Handle upgrade requests for WebSocket (HMR) + originalTransport := s.devProxy.Transport + if originalTransport == nil { + originalTransport = http.DefaultTransport + } + + s.devProxy.Transport = &http.Transport{ + Dial: (&net.Dialer{}).Dial, + TLSHandshakeTimeout: originalTransport.(*http.Transport).TLSHandshakeTimeout, + } + + return nil } func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + // If dev proxy is configured, use it + if s.devProxy != nil { + s.devProxy.ServeHTTP(w, r) + return + } + + // Otherwise, use the static file handler s.Server.ServeHTTP(w, r) } func (s *Server) ServeSPA(w http.ResponseWriter, r *http.Request) { + // If dev proxy is configured, use it + if s.devProxy != nil { + s.devProxy.ServeHTTP(w, r) + return + } + + // Otherwise, use the static SPA handler s.Server.ServeSPA(w, r) } diff --git a/pkg/server/web/web.go b/pkg/server/web/web.go index 197ce1266..acf42572b 100644 --- a/pkg/server/web/web.go +++ b/pkg/server/web/web.go @@ -15,7 +15,11 @@ package web import ( + "net" "net/http" + "net/http/httputil" + "net/url" + "os" "go.probo.inc/probo/apps/console" "go.probo.inc/probo/pkg/server/statichandler" @@ -23,9 +27,18 @@ import ( type Server struct { *statichandler.Server + devProxy *httputil.ReverseProxy } func NewServer() (*Server, error) { + return NewServerWithDevAddr("") +} + +// NewServerWithDevAddr creates a new Server with optional dev server support. +// If devAddr is provided, requests will be proxied to the Vite dev server. +// If devAddr is empty, the embedded static files will be served. +// The devAddr should be in the format "http://localhost:5173" or be read from environment. +func NewServerWithDevAddr(devAddr string) (*Server, error) { gzipOptions := statichandler.GzipOptions{ EnableFileTypeCheck: false, } @@ -35,15 +48,73 @@ func NewServer() (*Server, error) { return nil, err } - return &Server{ + server := &Server{ Server: spaServer, - }, nil + } + + // Support VITE_DEV_SERVER environment variable for automatic dev mode + if devAddr == "" { + devAddr = os.Getenv("VITE_DEV_SERVER_CONSOLE") + } + + if devAddr != "" { + // Validate and set up reverse proxy to dev server + if err := server.setupDevProxy(devAddr); err != nil { + return nil, err + } + } + + return server, nil +} + +func (s *Server) setupDevProxy(devAddr string) error { + devURL, err := url.Parse(devAddr) + if err != nil { + return err + } + + s.devProxy = httputil.NewSingleHostReverseProxy(devURL) + + // Customize the reverse proxy to handle WebSocket and other dev server features + s.devProxy.Director = func(req *http.Request) { + req.URL.Scheme = devURL.Scheme + req.URL.Host = devURL.Host + // Preserve the original host for request headers if needed + req.Host = devURL.Host + } + + // Handle upgrade requests for WebSocket (HMR) + originalTransport := s.devProxy.Transport + if originalTransport == nil { + originalTransport = http.DefaultTransport + } + + s.devProxy.Transport = &http.Transport{ + Dial: (&net.Dialer{}).Dial, + TLSHandshakeTimeout: originalTransport.(*http.Transport).TLSHandshakeTimeout, + } + + return nil } func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + // If dev proxy is configured, use it + if s.devProxy != nil { + s.devProxy.ServeHTTP(w, r) + return + } + + // Otherwise, use the static file handler s.Server.ServeHTTP(w, r) } func (s *Server) ServeSPA(w http.ResponseWriter, r *http.Request) { + // If dev proxy is configured, use it + if s.devProxy != nil { + s.devProxy.ServeHTTP(w, r) + return + } + + // Otherwise, use the static SPA handler s.Server.ServeSPA(w, r) }