T10: Monitoring, Performance, Documentation & Environment Config — 38 tests, ruff clean, docs OK
- Extended health endpoint: DB+Redis+Storage+Worker checks with degraded status - Prometheus metrics endpoint: admin-only, text/plain format - Metrics: http_requests_total, db_pool_connections, arq_jobs_total - Structured JSON logging (structlog): timestamp, level, method, path, status, duration_ms, tenant_id - Performance: page_size max 100 enforced (422), streaming CSV export (StreamingResponse) - Scripts: seed_perf_data.py, check_indexes.py - Docs: admin-guide.md (Deploy, Backup, Restore, Env-Vars, Troubleshooting), api-overview.md - README updated: prod setup, API section, env profiles, admin-guide link - .env.example: added SECRET_KEY, STORAGE_PATH, SMTP_* vars - 38 new tests, full regression 564/564 pass (0 failures) - Ruff: all checks passed
This commit is contained in:
@@ -0,0 +1,87 @@
|
||||
# T10: Monitoring, Performance, Documentation & Environment Config — Implementation Briefing
|
||||
|
||||
## Task
|
||||
Three modules in one task: (1) Monitoring & Alerting, (2) Performance, (3) Documentation.
|
||||
|
||||
## Acceptance Criteria (18 ACs)
|
||||
### Monitoring (AC1-6)
|
||||
1. GET /api/v1/health → 200 + JSON with status, checks.database, checks.redis, checks.storage, checks.worker
|
||||
2. GET /api/v1/health mit DB down → 200 + status=degraded, checks.database.status=down
|
||||
3. GET /api/v1/metrics → 200 + text/plain Prometheus format (admin only, 403 for non-admin)
|
||||
4. Prometheus metrics include leocrm_http_requests_total, leocrm_db_pool_connections, leocrm_arq_jobs_total
|
||||
5. Structured JSON log entry for API request: {timestamp, level, event, method, path, status, duration_ms, tenant_id}
|
||||
6. Error log includes stacktrace and request context
|
||||
|
||||
### Performance (AC7-12)
|
||||
7. scripts/seed_perf_data.py --count 200000 → creates 200k contacts in test DB
|
||||
8. GET /api/v1/contacts?page=1&page_size=25 with 200k records → response time <500ms
|
||||
9. GET /api/v1/contacts?search=Mueller with 200k records → response time <500ms
|
||||
10. page_size > 100 → 422 (max page_size enforced)
|
||||
11. CSV export >1000 records → ARQ background job started → notification on completion
|
||||
12. Streaming CSV export: GET /api/v1/contacts/export?format=csv → text/csv stream (not buffered)
|
||||
|
||||
### Documentation (AC13-18)
|
||||
13. README.md exists with Setup-Anleitung (dev + prod), API section, links to admin-guide
|
||||
14. Swagger UI available at /api/v1/docs (FastAPI auto-gen)
|
||||
15. docs/admin-guide.md exists with Deploy, Backup, Restore, Env-Vars, Troubleshooting sections
|
||||
16. docs/api-overview.md exists with endpoint summary table
|
||||
17. .env.example file exists with all required variables documented (database, redis, smtp, storage, secret_key)
|
||||
18. Environment-specific config: dev, test, prod profiles documented in docs/admin-guide.md
|
||||
|
||||
## Existing Code References
|
||||
- **Health endpoint:** app/routes/health.py (simple, needs extension)
|
||||
- **Health test:** tests/test_health.py (basic 200 check)
|
||||
- **Main app:** app/main.py (FastAPI app with CORS, CSRF middleware)
|
||||
- **Config:** app/config.py (settings with pydantic-settings)
|
||||
- **DB:** app/core/db.py (async engine)
|
||||
- **Routes:** app/routes/ (auth, companies, contacts, etc.)
|
||||
- **Contacts route:** app/routes/contacts.py (has search param, pagination)
|
||||
- **Companies route:** app/routes/companies.py (has search, pagination, export)
|
||||
- **README.md:** exists (basic, needs update with prod setup, API section, admin-guide link)
|
||||
- **.env.example:** exists (good coverage, may need SMTP/storage additions)
|
||||
- **docs/:** only requirements docs, needs admin-guide.md + api-overview.md
|
||||
- **Docker:** docker-compose.yml + Dockerfile exist
|
||||
- **Coolify:** COOLIFY_SETUP.md exists
|
||||
|
||||
## Files to Create
|
||||
- `app/core/monitoring.py` — Health check extensions, Prometheus metrics, structured logging
|
||||
- `app/routes/metrics.py` — Prometheus metrics endpoint (admin-only)
|
||||
- `scripts/seed_perf_data.py` — Performance test data seeding script
|
||||
- `scripts/check_indexes.py` — DB index verification script
|
||||
- `tests/test_monitoring.py` — Monitoring tests (health, metrics, logging)
|
||||
- `tests/test_performance.py` — Performance tests (pagination, export, page_size limit)
|
||||
- `docs/admin-guide.md` — Admin guide (Deploy, Backup, Restore, Env-Vars, Troubleshooting)
|
||||
- `docs/api-overview.md` — API endpoint summary
|
||||
|
||||
## Files to Modify
|
||||
- `app/routes/health.py` — Extend health check with DB+Redis+Storage+Worker status
|
||||
- `app/main.py` — Add metrics route, structured logging middleware, request timing
|
||||
- `app/routes/contacts.py` — Enforce page_size max 100, add streaming CSV export
|
||||
- `app/routes/companies.py` — Enforce page_size max 100, add streaming CSV export
|
||||
- `app/config.py` — Add SMTP/storage config if missing
|
||||
- `README.md` — Update with prod setup, API section, admin-guide link, env profiles
|
||||
- `.env.example` — Add SMTP/storage/secret_key vars if missing
|
||||
- `tests/test_health.py` — Update for extended health check
|
||||
- `requirements.txt` — Add prometheus-client, structlog if needed
|
||||
|
||||
## Dependencies to Add (if not present)
|
||||
- `prometheus-client>=0.20` (Prometheus metrics)
|
||||
- `structlog>=24.0` (structured JSON logging)
|
||||
|
||||
## Test Spec
|
||||
- Run: `cd /a0/usr/workdir/dev-projects/leocrm && python -m pytest tests/test_monitoring.py tests/test_performance.py tests/test_health.py -v --tb=short`
|
||||
- Coverage: `python -m pytest tests/test_monitoring.py --cov=app/core/monitoring --cov-report=term-missing`
|
||||
- Docs check: `test -f README.md && test -f docs/admin-guide.md && test -f docs/api-overview.md && echo 'Docs OK'`
|
||||
- Coverage target: 80%
|
||||
- Follow existing test pattern from tests/test_health.py or tests/test_companies.py
|
||||
|
||||
## Forbidden Patterns
|
||||
- No blocking I/O in async health check — use async DB ping
|
||||
- No credentials in logs or metrics
|
||||
- No unbounded pagination — max 100 per page enforced
|
||||
- No buffering large CSV exports — use StreamingResponse
|
||||
- No hardcoded config — use app/config.py settings
|
||||
|
||||
## Estimated Size
|
||||
- ~500 lines code (monitoring + scripts + docs)
|
||||
- ~300+ lines tests
|
||||
Reference in New Issue
Block a user