Add CLI binary tool for testing and development #3

Labels: enhancement, tooling, developer-experience, good-first-issue

📋 Summary

Add a command-line interface (CLI) binary to enable rapid testing, debugging, and validation of JSON Logic rules without requiring WASM compilation or Java integration.

🎯 Motivation

Current workflow limitations:

• Testing requires building WASM + Java integration OR writing Rust unit tests
• Slow iteration cycle for rule development
• High barrier for non-Rust developers to test rules
• No easy way to validate rules in CI/CD pipelines

Benefits of CLI tool:

• ⚡ Instant feedback - Test rules in seconds
• 🐛 Easy debugging - Quickly validate syntax and behavior
• 📚 Living documentation - Runnable examples
• 🔄 CI/CD integration - Automated rule validation
• 🎯 Lower barrier - Anyone can test rules without Rust/Java setup

💡 Proposed Solution

CLI Commands

# Evaluate a single rule
flagd-eval eval --rule <RULE> --data <DATA> [--pretty]

# Run test suite
flagd-eval test <TEST_FILE> [--verbose]

# List available operators
flagd-eval operators

Usage Examples

Inline evaluation:

flagd-eval eval \
  --rule '{"sem_ver": [{"var": "version"}, ">=", "2.0.0"]}' \
  --data '{"version": "2.1.0"}'

# Output:
# ✓ Evaluation succeeded
# Result: true

File-based evaluation:

flagd-eval eval \
  --rule @examples/rules/semver.json \
  --data @examples/data/app-v2.json \
  --pretty

Run test suite:

flagd-eval test examples/rules/test-suite.json

# Output:
# Running: Semantic Version Tests
# ✓ Version equals exact match (12ms)
# ✓ Caret range allows minor updates (8ms)
# ✓ Fractional is deterministic (15ms)
# 
# Results: 3 passed, 0 failed (35ms)

List operators:

flagd-eval operators

# Output:
# Available Custom Operators:
# 
# • fractional
#   Percentage-based bucket assignment
#   {"fractional": [{"var": "key"}, ["a", 50, "b", 50]]}
# 
# • sem_ver
#   Semantic version comparison
#   {"sem_ver": [{"var": "version"}, ">=", "1.0.0"]}

🏗️ Implementation Details

Project Structure

flagd-evaluator/
├── src/
│   ├── lib.rs              # WASM library
│   ├── bin/
│   │   └── flagd-eval.rs   # CLI binary (NEW)
│   ├── operators/
│   └── ...
├── examples/
│   ├── rules/              # Example rules (NEW)
│   │   ├── basic.json
│   │   ├── semver.json
│   │   ├── fractional.json
│   │   └── test-suite.json
│   ├── data/               # Example data (NEW)
│   │   ├── simple.json
│   │   └── app-v2.json
│   └── java/
└── tests/
    └── cli_tests.rs        # CLI integration tests (NEW)

Dependencies to Add

[lib]
crate-type = ["cdylib", "rlib"]  # Add rlib

[[bin]]
name = "flagd-eval"
path = "src/bin/flagd-eval.rs"

[dependencies]
# Existing deps...
clap = { version = "4.5", features = ["derive", "cargo"] }
colored = "2.1"

[dev-dependencies]
assert_cmd = "2.0"
predicates = "3.0"

Test Suite File Format

{
  "name": "Semantic Version Tests",
  "tests": [
    {
      "description": "Version equals exact match",
      "rule": {"sem_ver": [{"var": "version"}, "=", "1.2.3"]},
      "data": {"version": "1.2.3"},
      "expected": true
    },
    {
      "description": "Fractional is deterministic",
      "rule": {
        "fractional": [
          {"var": "userId"},
          ["control", 50, "treatment", 50]
        ]
      },
      "data": {"userId": "user-123"},
      "expected": "control"
    }
  ]
}

✅ Acceptance Criteria

Core Functionality

• CLI binary builds successfully alongside WASM module
• eval command supports inline JSON
• eval command supports file references with @ prefix
• test command executes test suites with clear output
• operators command lists all custom operators with descriptions
• Colored output (green for success, red for errors)

Testing

• Integration tests using assert_cmd cover all commands
• Tests for inline JSON evaluation
• Tests for file-based evaluation
• Tests for test suite execution
• Tests for error handling (invalid JSON, missing files)

Documentation & Examples

• Example rule files in examples/rules/
• Example data files in examples/data/
• Test suite examples with multiple operators
• README updated with CLI usage section
• Installation instructions (cargo install)

CI/CD

• GitHub Actions builds CLI on Linux, macOS, Windows
• CI runs CLI integration tests
• CI validates example files work
• Release workflow includes CLI binaries for all platforms

Quality

• CLI evaluation completes in <10ms for typical rules
• WASM module size remains <500KB (unaffected by CLI)
• All clippy warnings resolved
• Formatted with cargo fmt

📦 Deliverables

1. CLI Binary (src/bin/flagd-eval.rs)

   • Argument parsing with clap
   • Three subcommands: eval, test, operators
   • File and inline JSON support
   • Colored output

2. Example Files

   • examples/rules/basic.json
   • examples/rules/semver.json
   • examples/rules/fractional.json
   • examples/rules/test-suite.json
   • examples/data/simple.json
   • examples/data/app-v2.json

3. Integration Tests (tests/cli_tests.rs)

   • Test all subcommands
   • Test file loading
   • Test error cases
   • Test output formatting

4. Documentation Updates

   • README Quick Start section
   • CLI command reference
   • Example usage patterns
   • Installation guide

5. CI/CD Updates

   • Build CLI in CI pipeline
   • Run CLI tests
   • Cross-platform builds
   • Include binaries in releases

🗓️ Implementation Plan

Phase 1: Basic CLI (Week 1)

• Add clap and colored dependencies
• Implement basic CLI structure with clap derive
• Add eval command with inline JSON only
• Colored output for success/failure
• Basic integration tests

Phase 2: File Support (Week 1)

• Add @file.json prefix support
• Create example rule files
• Create example data files
• Update tests for file loading

Phase 3: Test Suite (Week 2)

• Implement test command
• Define test suite JSON format
• Create comprehensive test suite examples
• Add test result formatting

Phase 4: Operators Command (Week 2)

• Implement operators command
• Format operator descriptions
• Include syntax examples
• Test output

Phase 5: CI/CD & Documentation (Week 2)

• Update GitHub Actions workflows
• Add cross-platform builds
• Update README with CLI sections
• Add installation instructions
• Create release with binaries

🔗 Related Issues

• Create flagd-evaluator WebAssembly library for JSON Logic evaluation #1 - Implement flagd custom operators (CLI will help test these)

📚 Resources

• clap documentation
• assert_cmd documentation
• colored documentation
• Command Line Applications in Rust

🤔 Open Questions

1. Should we support JSON5 for more human-friendly rule files?
2. Should test command support glob patterns for multiple test files?
3. Should we add a --watch mode for continuous testing during development?
4. Should we support YAML format for rules/data as alternative to JSON?

💬 Additional Context

This CLI tool will not affect the WASM module in any way - it's a completely separate binary that shares the evaluation logic. The binary target is only for development/testing convenience and won't be included in the WASM output.

The CLI will be especially valuable for:

• Frontend developers testing feature flag rules
• DevOps engineers validating configurations
• QA teams testing rule logic
• Documentation with runnable examples

Priority: Medium
Estimated Effort: 2 weeks
Type: Enhancement

📝 Notes for Implementation

When implementing, ensure:

• CLI shares core evaluation logic with WASM (via rlib)
• No duplication of operator implementations
• Error messages are user-friendly and actionable
• File paths work cross-platform (Windows, Linux, macOS)
• Examples cover all custom operators
• Performance is suitable for CI/CD use (<10ms typical evaluation)

Ready to implement? Comment below if you'd like to take this on or have questions! 🚀