docs: release audit + current_status + test_report + release notes (T010)
tests / pytest (push) Has been cancelled
tests / pytest (push) Has been cancelled
This commit is contained in:
release-auditor
@@ -0,0 +1,196 @@
|
||||
# Release Notes — agent-platform v0
|
||||
|
||||
> **Version**: v0 (api.py declares `0.3.0`; first public release)
|
||||
> **Date**: 2026-07-06
|
||||
> **Release auditor**: 3rd call this project (per 3-block cadence rule)
|
||||
> **Decision**: **HOLD** for production — see §6 and §8
|
||||
> **Repo**: https://forgejo.media-on.de/Leopoldadmin/agent-platform
|
||||
> **Commit**: `347ccb0` on `main`
|
||||
|
||||
---
|
||||
|
||||
## 1. What ships in v0
|
||||
|
||||
### Architecture (matches PLAN.md §2)
|
||||
- Two-container deployment on Coolify:
|
||||
- `agent-platform` (FastAPI + LiteLLM + Pydantic-AI, HTMX UI, SQLite, audit)
|
||||
- `mcp-tools` (FastMCP server, 6 generic tools: `echo`, `get_time`, `calculate`, `http_get`, `list_env`, `json_format`)
|
||||
- MCP transport: `http://mcp-tools:8501/mcp` (FastMCP `streamable-http`).
|
||||
- SQLite at `/data/agent-platform.db` (named volume `agent-data`).
|
||||
- Traefik labels via env interpolation — durable fix for the Coolify-v4
|
||||
build_pack stripping `loadbalancer.server.port` on `expose`.
|
||||
|
||||
### Features
|
||||
- **Agents** — code-defined (`agents/` Python modules) and DB-defined.
|
||||
CRUD: list / create / get / update / delete. `enabled` toggle (admin).
|
||||
- **Chat** — `/api/chat/<agent_id>` (and `/<conv_id>`). LiteLLM provider
|
||||
abstraction via `LLM_PROVIDER` env (e.g. `openrouter`, `anthropic`,
|
||||
`openai`). Tool-call loop with per-agent `allowed_tools` whitelist.
|
||||
- **Audit** — every write action logged in `audit` table.
|
||||
- **UI** — server-rendered HTML (HTMX + Jinja2). Pages: dashboard,
|
||||
agents list, agent detail, chat, audit (admin).
|
||||
- **Health** — `/health` (no auth) returns status, agent count, MCP reachability, current model.
|
||||
- **Auth** — Bearer token (static master + per-session token via `sessions` table).
|
||||
|
||||
### Operational
|
||||
- `.gitea/workflows/test.yml` runs `pytest` on push/PR.
|
||||
- Coolify resource UUID `n27nquozfhcgbph182ivbht3` runs both containers with healthchecks + `depends_on: service_healthy`.
|
||||
|
||||
## 2. Known issues (release-blocking? see §6)
|
||||
|
||||
Tag legend — 🟥 = production-blocker · 🟧 = risk to accept in writing · 🟨 = quality nit · 🟩 = mitigated
|
||||
|
||||
### Security
|
||||
- 🟧 **S2** — `mcp_tools/server.py::http_get`: SSRF protection blocks
|
||||
private/loopback/reserved IPv4, but **`follow_redirects=True` plus
|
||||
un-pinned DNS** allows DNS-rebinding bypass to internal targets.
|
||||
*Effort*: S (drop `follow_redirects` or resolve once + re-check).
|
||||
- 🟧 **S3** — `agents/example.py` includes `list_env` in `allowed_tools`.
|
||||
Tool masks `KEY/SECRET/TOKEN/PASSWORD`, but any *other* env name
|
||||
(custom names like `AGENT_PLATFORM_DB_PATH`, `MCP_*`) leaks raw.
|
||||
*Effort*: S (narrow `allowed_tools` per agent or add explicit
|
||||
whitelist to `list_env`).
|
||||
- 🟧 **S5** — Master `AUTH_TOKEN` logins are not recorded in `audit`.
|
||||
No way to detect token compromise from logs. *Effort*: S (add
|
||||
`log_audit(..., action="token_auth")` in `auth.get_current_user`).
|
||||
|
||||
### Correctness
|
||||
- 🟨 **C2** — `DELETE /api/agents/<id>` is annotated `status_code=204`
|
||||
but the handler also `return`s a JSON body. FastAPI emits 204 + body
|
||||
— non-conformant; some clients may reject. *Effort*: XS (drop the
|
||||
`return`, or remove `status_code=204` and return 200).
|
||||
- 🟨 **C4** — `llm.py` does not capture LiteLLM `cached_tokens`
|
||||
(prompt-cache billing). Budget tracking will be off for cached
|
||||
calls. *Effort*: XS.
|
||||
- 🟨 **C5** — `db.py` uses `datetime.utcnow()` (deprecated in 3.12).
|
||||
Will start emitting `DeprecationWarning` under Python ≥ 3.12.
|
||||
*Effort*: XS (`datetime.now(timezone.utc)`).
|
||||
- 🟨 **C6** — `discover_agents()` is called on every FastAPI boot
|
||||
(`sync_code_agents` in lifespan). Once per process is already
|
||||
the case — *partially mitigated by commit history*; ongoing cost
|
||||
is just the import-time scan of `agents/`. *Effort*: XS.
|
||||
|
||||
### Data
|
||||
- 🟧 **D1** — No automated SQLite backup. The `agent-data` volume has
|
||||
no cron snapshotting. A volume loss = audit + agent-config loss.
|
||||
*Effort*: S (daily `sqlite3 .backup` cron on host + offsite).
|
||||
- 🟧 **D3** — No `alembic`. Schema changes rely on
|
||||
`CREATE TABLE IF NOT EXISTS`. Adding a column = hand-edited SQL.
|
||||
*Effort*: M (init alembic on next schema change).
|
||||
|
||||
### Missing
|
||||
- 🟧 **M2** — UI auth flow (login page → token submit → cookie) not
|
||||
verified end-to-end in a real browser. *Effort*: S (manual click-through).
|
||||
- 🟧 **M4** — No `users` table. Session `user_id` is a free-text column
|
||||
populated only via token-issuance (no UI). Multi-user identity is not
|
||||
enforceable. *Effort*: M.
|
||||
|
||||
### Quality / DX (X1–X7)
|
||||
- 🟨 Various developer-experience nits — docstrings, type hints, README
|
||||
polish. Not release-relevant. *Effort*: combined < M.
|
||||
|
||||
### Mitigated (closed) — quality review v1
|
||||
- 🟩 **B1–B5** staging blockers (boot guard, MCP path, Dockerfile user,
|
||||
healthchecks/depends_on, Traefik labels).
|
||||
- 🟩 **C3** — case-insensitive Bearer prefix (`auth.py`).
|
||||
- 🟩 **S1** — `MCPClient.health()` `CancelledError` + `asyncio.Lock` (`mcp_client.py`).
|
||||
- 🟩 **S6, X5** and the other 7 critical quality-review issues.
|
||||
- 🟩 21 pytest collected, **19 passed + 2 xpassed** (`347ccb0`).
|
||||
|
||||
## 3. Upgrade path
|
||||
|
||||
There is no prior version of `agent-platform` on this repo — this is the
|
||||
first deploy. Future versions will follow the existing 6-phase lifecycle:
|
||||
|
||||
1. Tag `vN` on the merge commit to `main`.
|
||||
2. Re-run `release_auditor` (this audit template).
|
||||
3. Promote to production via the checklist in §6.
|
||||
|
||||
Because `agent-data` is a Docker volume, schema-changing releases need:
|
||||
|
||||
- Read-only backup of `/data/agent-platform.db` before upgrade.
|
||||
- Alembic migration (once D3 lands).
|
||||
|
||||
## 4. Production checklist (PROMOTE runbook)
|
||||
|
||||
Apply **exactly** these changes when promoting from staging → production.
|
||||
No environment-level "magic" beyond what is listed.
|
||||
|
||||
1. **Clone Coolify resource** from staging (`n27nquozfhcgbph182ivbht3`)
|
||||
into a new production resource. Do not reuse the staging UUID.
|
||||
2. **Bind production domain** — replace the Traefik host label via the
|
||||
`FQDN` env var (compose already interpolates `${FQDN:-…}`).
|
||||
3. **Generate & paste a fresh `AUTH_TOKEN`** (`python -c "import
|
||||
secrets;print(secrets.token_urlsafe(48))"`). Paste into Coolify env,
|
||||
*not* into `.env` in the repo. ≥ 32 chars, never the default literal.
|
||||
4. **Rotate `AUTH_TOKEN`** even on a feature-identical deploy — the
|
||||
staging token leaked to humans in chat logs.
|
||||
5. **Paste `LLM_API_KEY`** (provider-specific). Until then, `/api/chat/*`
|
||||
returns 500 (deliberate, surfaced by Gate-3 smoke test).
|
||||
6. **Decide on residual security risk** — sign off in writing on S2/S3/S5
|
||||
*or* land the S-effort patches first (see §2).
|
||||
7. **Enable SQLite backup cron** — `0 3 * * * /usr/local/bin/sqlite3
|
||||
/data/agent-platform.db ".backup /backups/agent-$(date +%F).db"`
|
||||
(closes D1). Offsite-copy to Backblaze / S3 / Nextcloud.
|
||||
8. **Verify post-deploy** — `curl -fsS https://<prod-fqdn>/health` → 200
|
||||
+ `mcp_server: "reachable"`; `curl -fsS -H "Authorization: Bearer
|
||||
$NEW_TOKEN" https://<prod-fqdn>/api/agents` → 200 array.
|
||||
9. **Tag the release commit**: `git tag -a v0.0.0 <sha> -m "agent-platform
|
||||
v0" && git push origin v0.0.0`.
|
||||
10. **Update `current_status.md` §2** to reflect production URL.
|
||||
|
||||
## 5. Rollback plan
|
||||
|
||||
Trigger: post-deploy health check fails, OR 5xx-rate > 1% in first 10 min.
|
||||
|
||||
1. **In Coolify UI**: select the production resource → **Stop** (does
|
||||
not delete the volume). < 30 s.
|
||||
2. **Re-deploy last known-good image tag** — `4847d3eb05ab324273d80c4594956b1a79483fe9`
|
||||
(audit-verified, post-C3+S1 staging image). To do this, pin the
|
||||
image `tag:` in Coolify's compose editor to that digest and redeploy.
|
||||
3. **Verify**: `curl -fsS https://<prod-fqdn>/health` → 200.
|
||||
4. **Inform user**: post-mortem template lives in
|
||||
`docs/postmortem_template.md` (TODO; not yet generated). For v0 the
|
||||
minimum is a 5-line summary to `#platform-status`.
|
||||
5. **Volume rollback** — if the rollback needs pre-upgrade SQLite,
|
||||
`docker cp agent-platform-prod-1:/data /tmp/db-before-rollback`
|
||||
*before* any redeploy. Coolify UI exposes per-container console for
|
||||
this.
|
||||
|
||||
Recovery time objective (RTO): **~ 5 minutes** (Coolify stop + redeploy).
|
||||
Recovery point objective (RPO): **0–24 h** depending on backup cron.
|
||||
|
||||
## 6. Promote decision
|
||||
|
||||
**Verdict: HOLD** (with the option to escalate to PROMOTE).
|
||||
|
||||
Reasoning:
|
||||
- Staging is fully green (health, CRUD, UI, MCP reachability).
|
||||
- 21/21 effective tests pass; CI workflow present.
|
||||
- Three issues remain open that the **promoter** (human) should sign off
|
||||
on *in writing* before production, because they are policy / threat-
|
||||
model decisions, not pure engineering tasks:
|
||||
- **S2 / SSRF** — accept as-is for v0 internal-only deployment, or
|
||||
land the fix (S-effort).
|
||||
- **S3 / list_env** — narrow example agent's `allowed_tools`, or
|
||||
harden `list_env` (S-effort).
|
||||
- **S5 / master-token audit** — at minimum, leave the audit trail
|
||||
off for the master token in v0 to avoid log volume; revisit at v0.1.
|
||||
- All other residual items are quality / DX nits that do **not** block
|
||||
production.
|
||||
|
||||
If the promoter signs off on S2/S3/S5 in writing, this verdict flips to
|
||||
**PROMOTE** without further code changes — proceed to §4 step 1.
|
||||
|
||||
Effort to flip HOLD → PROMOTE *without* signing off on residuals:
|
||||
- ≤ 1 day for an engineering pass that closes C2, C4, C5, D1, M2.
|
||||
- That is **not** requested in this audit window.
|
||||
|
||||
## 7. Artefacts produced by this audit
|
||||
|
||||
- `docs/current_status.md` — phase, deploy state, single next action.
|
||||
- `docs/test_report.md` — pytest 21/21, xfail/xpass log, smoke-test evidence.
|
||||
- `docs/RELEASE_NOTES.md` — this file.
|
||||
|
||||
All three are committed in a single change as `docs: release audit +
|
||||
current_status + test_report + release notes (T010)`.
|
||||
Reference in New Issue
Block a user