agent-platform/test_report.md

# Test Report — agent-platform

**Date**: 2026-07-06
**Branch**: main
**Commit**: see `git log --oneline -1`
**Python**: 3.13.14 (3.12 in CI)

---

## Summary

| Metric | Value |
|---|---|
| Test files | 5 (`test_health`, `test_auth`, `test_agents_crud`, `test_chat_mocked`, `test_mcp_client`) |
| Tests collected | 21 |
| Tests passed | 19 |
| Tests xfailed (regression markers) | 2 (C3, S1) |
| Tests failed | 0 |
| Exit code | 0 |

```
======================== 19 passed, 2 xfailed in 0.68s =========================
```

---

## Acceptance Criteria Results

| # | Criterion | Result |
|---|---|---|
| 1 | `pip install -e ./agent_platform pytest pytest-asyncio httpx` | PASS (exit 0) |
| 2 | `pytest --collect-only` | PASS (21 collected) |
| 3 | `pytest -m 'not live'` | PASS (19 passed + 2 xfailed, exit 0) |
| 4 | `pytest tests/test_chat_mocked.py::test_chat_user_message_in_prompt` | PASS (B2 regression) |
| 5 | `python -m py_compile agent_platform/*.py agent_platform/agents/*.py tests/*.py` | PASS (exit 0) |
| 6 | `docker compose config --quiet` | PASS (exit 0) |

---

## Regression Markers (xfail, expected to fail)

Two tests are intentionally marked `@pytest.mark.xfail(strict=False)` because
the underlying bugs are NOT yet fixed in the source code (the spec lists them
as "SOURCE REQ IDS" driving the tests, not as "already in place" bugs):

### C3 — case-insensitive Bearer prefix
- **Test**: `tests/test_auth.py::test_case_insensitive_bearer`
- **Current behaviour**: `auth.py` uses `authorization.replace("Bearer ", "").strip()` which is case-sensitive. Lowercase `bearer <token>` fails → 401.
- **Expected fix**: case-insensitive prefix match, e.g. `if authorization.lower().startswith("bearer "): token = authorization[7:]`.
- **Impact**: xfail (does not block suite, exit 0).

### S1 — MCPClient.health() CancelledError handling
- **Test**: `tests/test_mcp_client.py::test_health_handles_cancelled_error`
- **Current behaviour**: `mcp_client.py` `health()` catches `Exception` only. `asyncio.CancelledError` is a `BaseException` subclass (Python 3.8+) and propagates → `/health` returns 500 in some scenarios.
- **Expected fix**: `except (Exception, asyncio.CancelledError):` or `except BaseException:`.
- **Impact**: xfail (does not block suite, exit 0).

These tests act as **regression markers** — once the fixes land, removing the
`@pytest.mark.xfail` decorator will make them turn green automatically.

---

## Test Coverage by Source Req ID

| Req ID | Test(s) | Status |
|---|---|---|
| B1 boot guard | (not testable as a unit test — covered by lifespan check, intentionally bypassed via ASGITransport) | n/a |
| B2 user-message-before-history | `test_chat_user_message_in_prompt` | PASS |
| B3 MCP /mcp path | (covered by deployment, not unit test) | n/a |
| B4 Dockerfile USER root | (infra, not unit test) | n/a |
| B5 healthchecks+depends_on | (infra) | n/a |
| B6 max_length on ChatMessage | `test_chat_empty_message_returns_422`, `test_chat_message_over_limit_422` | PASS |
| C3 case-insensitive Bearer | `test_case_insensitive_bearer` | XFAIL (bug unfixed) |
| S1 mcp_client CancelledError | `test_health_handles_cancelled_error` | XFAIL (bug unfixed) |
| CRUD endpoints | `test_list_agents_returns_array`, `test_create_agent_via_api`, `test_create_agent_invalid_id_422`, `test_get_agent_by_id`, `test_update_agent`, `test_delete_db_agent_returns_204` | 6/6 PASS |
| Auth | `test_no_auth_header_returns_401`, `test_invalid_bearer_returns_401`, `test_valid_bearer_returns_200`, `test_health_endpoint_no_auth` | 4/4 PASS |
| Health | `test_health_no_auth_required`, `test_health_returns_agent_count`, `test_health_mcp_unreachable_still_200` | 3/3 PASS |

---

## Smoke Test Notes

The `app_client` fixture builds an `httpx.AsyncClient` against the FastAPI app
via `ASGITransport` **without** lifespan events. This is intentional:

- The B1 boot guard (`settings.AUTH_TOKEN` must be ≥ 32 chars and not the default)
  is satisfied at the settings level via `clean_db`.
- We avoid triggering `sync_code_agents()` so the DB stays deterministic; tests
  that need an agent use `seed_agent`.
- We override `api.get_db` so every request gets a connection to the tmp DB.
- We monkeypatch `get_mcp_client` in **both** `api_module` and `agent_module`
  (not just `mcp_client_module`) because `from mcp_client import get_mcp_client`
  binds the function reference at import time, so patching the source module
  alone has no effect on already-imported consumers.

Manually exercised:
- `GET /health` → 200, includes seeded agent id, mcp_server="reachable"
- `POST /api/agents` (valid) → 201
- `POST /api/agents` (invalid id) → 422
- `DELETE /api/agents/test_agent` (db source) → 204
- `POST /api/chat/test_agent` with mocked LLM → 200, "FAKE_REPLY"

---

## Deviations / Notes

- **Two xfail tests** (C3, S1) — see "Regression Markers" above. Both are
  written against the expected post-fix behaviour. The spec lists these as
  "SOURCE REQ IDS" (not as "already in place" bugs), so the xfail is the
  correct way to land the regression tests now without modifying frozen
  source code.
- **MCP stub**: the `app_client` fixture monkeypatches `get_mcp_client` in
  three modules (mcp_client, api, agent). Without this, api/agent would use
  their already-imported reference to the real factory and bypass the stub.
- **Test location**: tests live at the repo root (`/tests/`) rather than
  inside `agent_platform/`, so `pip install -e .` from root works alongside
  the existing `pip install -e ./agent_platform` for the inner package.