chore: add rate limiter and redis

[dan2k3k4]

Greptile Summary

This PR introduces Redis-backed rate limiting via fastapi-limiter on the /auth/validate-email endpoint (5 requests/60 s) and migrates the existing monitor_teams / hard_delete_teams background work from ad-hoc execution into an APScheduler with distributed locking. Redis is wired in through docker-compose.yml, Makefile test targets, and a new REDIS_URL config setting.

• Rate limiting on /auth/validate-email: RateLimiter(times=5, seconds=60) is added as a FastAPI dependency; tests are mocked via a new autouse fixture that emulates the Redis Lua-script return value and enforces limits in test names containing rate_limit.
• APScheduler jobs in lifespan: monitor_teams_job and hard_delete_teams_job are scheduled with environment-aware cron triggers and distributed DB locks to prevent duplicate execution across replicas.
• Infrastructure: Redis service added to docker-compose.yml with healthcheck; backend now declares depends_on: redis: condition: service_healthy.

Confidence Score: 3/5

The rate limiting on /auth/validate-email can be bypassed by any client that spoofs X-Forwarded-For, defeating the primary protection the feature is meant to provide.

The rate limit identifier trusts a client-controlled header, making it trivially bypassable. Redis is also now a hard startup dependency with no degraded-mode fallback.

app/api/auth.py and app/main.py need the most attention — the rate-limiter identifier and the hard Redis startup dependency.

Security Review

• X-Forwarded-For spoofing on rate-limited endpoint (app/api/auth.py): The fastapi-limiter default identifier reads X-Forwarded-For and uses split(\",\")[0]. If the upstream proxy appends rather than replaces this header, an attacker can pre-set an arbitrary IP value on each request and bypass the 5-request-per-minute limit on /auth/validate-email, enabling unlimited validation-code email delivery to arbitrary addresses or email enumeration.

Important Files Changed

Filename	Overview
app/main.py	Adds FastAPILimiter Redis init, APScheduler with monitor_teams and hard_delete_teams jobs in the lifespan; Redis is now a hard startup dependency with no fallback if unavailable
app/api/auth.py	Applies RateLimiter(5 req/60s) to /auth/validate-email; import reorder and walrus-operator refactor; rate limit bypassable via X-Forwarded-For spoofing with default identifier
app/core/config.py	Adds REDIS_URL (with REDIS_HOST/REDIS_PORT fallback) and RATE_LIMIT_VALIDATE_EMAIL settings; straightforward additions with correct Pydantic BaseSettings pattern
tests/conftest.py	Adds autouse mock_rate_limiting fixture that bypasses Redis for all tests and enforces limits in rate-limit-named tests; mock correctly patches both app.main and fastapi_limiter.depends namespaces
tests/test_auth.py	Adds test_validate_email_rate_limit covering the happy path (N requests succeed) and the 429 on N+1; test logic aligns correctly with mock_evalsha counting logic
docker-compose.yml	Adds redis service (uselagoon/redis-7) with healthcheck; backend gains REDIS_URL env var and depends_on redis:service_healthy — startup dependency properly guarded
requirements.txt	Adds fastapi-limiter==0.1.6, redis==4.6.0, and apscheduler==3.11.2 with pinned versions
Makefile	Adds test-redis target, includes REDIS_URL in all backend-test targets, and cleans up redis container in test-clean
tests/test_team_keys_policy.py	Minor cleanup: removes blank lines between test functions, no logic changes

Sequence Diagram

sequenceDiagram
    participant Client
    participant Proxy
    participant FastAPI
    participant RateLimiter
    participant Redis
    participant ValidateEmail

    Client->>Proxy: POST /auth/validate-email
    Proxy->>FastAPI: forward request (X-Forwarded-For: client_ip)
    FastAPI->>RateLimiter: resolve dependency
    RateLimiter->>Redis: EVALSHA lua_sha 1 ip:path times ms
    alt under limit
        Redis-->>RateLimiter: 0 (allowed)
        RateLimiter-->>FastAPI: continue
        FastAPI->>ValidateEmail: execute handler
        ValidateEmail-->>Client: 200 OK
    else limit exceeded
        Redis-->>RateLimiter: pexpire ms remaining
        RateLimiter-->>Client: 429 Too Many Requests
    end

    Note over FastAPI: lifespan startup
    FastAPI->>Redis: FastAPILimiter.init script_load
    FastAPI->>APScheduler: scheduler.start
    APScheduler-->>FastAPI: jobs registered monitor_teams hard_delete_teams

Loading

_{Reviews (1): Last reviewed commit: "fix: add apscheduler==3.11.2 to requirem..." | Re-trigger Greptile}

Greptile also left 3 inline comments on this PR.

[alagoa]

Looking good, just some suggestions particularly on cross container interaction.

Also, should we consider adding rate limiting to the sign-in/login routes as well?

[dan2k3k4]

Also, should we consider adding rate limiting to the sign-in/login routes as well?

We could extend it to sign-in/login and others but I want to get this one route working first (and merged in), then we can do that in a follow-up

[alagoa]

Looks good, so we test this out with the validation email endpoint, then roll it out to the other public routes?

[dan2k3k4]

We could test a bit on the PR env

• https://frontend.pr-283.amazeeai.us2.amazee.io
• https://backend.pr-283.amazeeai.us2.amazee.io/

[dspachos]

Hi @dan2k3k4 , if this PR is still valid, can you please resolve the merge conflicts and I can then review.

[Copilot]

Pull request overview

This PR introduces Redis-backed request rate limiting (via fastapi-limiter) and wires Redis into local/dev/test environments, plus adds scheduled background jobs in the FastAPI lifespan to run team monitoring and hard-delete maintenance tasks.

Changes:

• Add Redis service/configuration and wire REDIS_URL through settings, docker-compose, and test Makefile targets.
• Initialize FastAPILimiter on app startup and apply a RateLimiter dependency to /auth/validate-email.
• Add APScheduler-based recurring jobs in app.main for monitor_teams and hard_delete_expired_teams with DB-based locking.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
app/main.py	Initializes Redis rate limiter; adds APScheduler cron jobs with DB locks during lifespan.
app/api/auth.py	Applies rate limiting to the /auth/validate-email endpoint.
app/core/config.py	Adds REDIS_URL and RATE_LIMIT_VALIDATE_EMAIL settings.
docker-compose.yml	Adds a Redis service and makes backend depend on it; sets backend REDIS_URL.
Makefile	Adds a Redis container for backend test targets and exports REDIS_URL into test runs.
requirements.txt	Adds fastapi-limiter, redis, and pyrate-limiter dependencies.
tests/conftest.py	Adds an autouse mock to bypass Redis rate limiting in tests; adds an httpx.AsyncClient patch fixture.
tests/test_team_keys_policy.py	Minor test file import/whitespace changes.
tests/test_auth.py	Minor whitespace-only change.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[dan2k3k4]

@copilot apply changes based on this feedback

[Copilot]

The test test_validate_email_rate_limit was already added in commit 188c3ee. It makes settings.RATE_LIMIT_VALIDATE_EMAIL (default: 5) requests and asserts the next one returns HTTP 429. The conftest mock enforces the threshold dynamically via int(times) from the evalsha call, so it stays in sync with any configured value.

[smlx]

To confirm: is redis planned to be used for purposes other than rate limiting in future? The reason I ask is that the email validation is not expected to be a high-throughput endpoint, so maybe using database backing for the rate limiter and avoiding adding the complexity of redis just for this feature might be an option?

From what I can tell, if redis goes down this will currently fail closed which means that email validation will not be available for some period if redis restarts. Is that right?

[dspachos]

@dan2k3k4 please address failing tests

[dan2k3k4]

@copilot check and fix backend-tests

Digest: sha256:c5e375abb885e6b2021c0377879e4890bf76f9065b8922ffc113f2b226b9fc17
Status: Downloaded newer image for redis:alpine
f3f752191989baff6b278aeec50b1f8bd4aefcdd1485cad59883d81e33af1e2f
docker run --rm \
	--network amazeeai_default \
	-e DATABASE_URL="***amazee-test-postgres/postgres_service" \
	-e REDIS_URL="redis://amazee-test-redis:6379" \
	-e SECRET_KEY="test-secret-key" \
	-e POSTGRES_HOST="amazee-test-postgres" \
	-e POSTGRES_USER="postgres" \
	-e POSTGRES_PASSWORD="postgres" \
	-e POSTGRES_DB="postgres_service" \
	-e DYNAMODB_ROLE_NAME="test-role" \
	-e SES_ROLE_NAME="test-role" \
	-e TESTING="1" \
	-e ENV_SUFFIX="test" \
	-v /home/runner/work/amazee.ai/amazee.ai/app:/app/app \
	-v /home/runner/work/amazee.ai/amazee.ai/tests:/app/tests \
	amazee-backend-test pytest -v --cov=app tests/
ImportError while loading conftest '/app/tests/conftest.py'.
tests/conftest.py:12: in <module>
    from app.main import app
app/main.py:30: in <module>
    from apscheduler.schedulers.asyncio import AsyncIOScheduler
E   ModuleNotFoundError: No module named 'apscheduler'
make: *** [Makefile:79: backend-test-cov] Error 4

[Copilot]

@copilot check and fix backend-tests

...

Fixed in commit 45c0da6. The CI failure was ModuleNotFoundError: No module named 'apscheduler' — the package was imported in app/main.py (for AsyncIOScheduler / CronTrigger) but was missing from requirements.txt. Added apscheduler==3.11.2.

[greptile-apps]

Rate limit bypassable via X-Forwarded-For spoofing

The fastapi-limiter default identifier reads X-Forwarded-For first and takes split(",")[0]. If the reverse proxy appends to (rather than replaces) this header, an attacker can pre-set X-Forwarded-For: 1.2.3.4 on each request and appear as a new IP every time, effectively bypassing the rate limit and enabling unlimited validation-code emails to arbitrary addresses. A custom identifier should be used that either strips the header or uses a trusted proxy-only value.

[greptile-apps]

The comment # Get cookie domain from LAGOON_ROUTES is now orphaned — the get_cookie_domain() call it referenced was refactored into the walrus-operator expression below. The blank line and stale comment are misleading.

Suggested change

	# Get cookie domain from LAGOON_ROUTES
	cookie_domain = get_cookie_domain()
	# Set cookie expiration based on user role
	# Set cookie expiration based on user role

[greptile-apps]

Redis unavailability now prevents app startup

FastAPILimiter.init() calls redis.script_load(lua_script), which requires an active Redis connection. If Redis is unavailable when the app starts, the lifespan raises an exception and the app cannot serve any traffic — even endpoints with no rate limiting. The docker-compose.yml guards against this with condition: service_healthy, but this constraint doesn't exist in every deployment path (e.g., CI pipelines, bare-metal deployments, or Kubernetes rolling restarts where Redis may be transiently unreachable). Consider wrapping the init in a try/except with a warning log so the app can start degraded without rate limiting.