Phase 4: Circuit Breaker, DB Retry, Redis Graceful Degradation
- app/core/resilience.py: CircuitBreaker (CLOSED/OPEN/HALF_OPEN), retry_db, redis_call_with_fallback, InMemoryRateLimiter, CircuitBreakerMiddleware - app/core/auth.py: get_session_data now falls back to PostgreSQL sessions table when Redis is unavailable - app/core/permissions.py: get_cached_permissions falls back to direct DB resolution when Redis circuit is open - app/core/rate_limit.py: check_rate_limit falls back to in-memory limiter when Redis is down; reset_rate_limit clears both Redis and in-memory - app/core/middleware.py: CSRF validation uses get_session_data (Redis+DB fallback); sliding session TTL is best-effort during outage - app/core/db/__init__.py: get_db() wraps session creation with retry_db for transient connection errors; records circuit breaker success/failure - app/deps.py: refresh_session_ttl wrapped in try/except for Redis outage - app/main.py: CircuitBreakerMiddleware registered (returns 503 when DB circuit is OPEN, skips health/metrics endpoints) - app/config.py: Added resilience settings (thresholds, cooldown, retries) - tests/test_resilience.py: 30 tests covering all patterns 30/30 resilience tests pass. No regressions in plugin lifecycle tests.
This commit is contained in:
@@ -66,6 +66,13 @@ class Settings(BaseSettings):
|
||||
# Trusted proxy CIDRs (comma-separated) — only these proxies can set X-Forwarded-For
|
||||
trusted_proxy_cidrs: str = ""
|
||||
|
||||
# Resilience
|
||||
circuit_breaker_failure_threshold: int = 5
|
||||
circuit_breaker_window_seconds: int = 30
|
||||
circuit_breaker_cooldown_seconds: int = 60
|
||||
db_retry_max_attempts: int = 3
|
||||
db_retry_base_delay: float = 0.1
|
||||
|
||||
# Rate Limiting
|
||||
rate_limit_login_max: int = 5
|
||||
rate_limit_login_window: int = 900 # 15 min
|
||||
|
||||
Reference in New Issue
Block a user