agent-platform/test_report.md

Test Report — agent-platform

Date: 2026-07-06 Branch: main Commit: see git log --oneline -1 Python: 3.13.14 (3.12 in CI)

---

Summary

Metric	Value
Test files	5 (test_health, test_auth, test_agents_crud, test_chat_mocked, test_mcp_client)
Tests collected	21
Tests passed	19
Tests xfailed (regression markers)	2 (C3, S1)
Tests failed	0
Exit code	0

======================== 19 passed, 2 xfailed in 0.68s =========================

---

Acceptance Criteria Results

#	Criterion	Result
1	pip install -e ./agent_platform pytest pytest-asyncio httpx	PASS (exit 0)
2	pytest --collect-only	PASS (21 collected)
3	pytest -m 'not live'	PASS (19 passed + 2 xfailed, exit 0)
4	pytest tests/test_chat_mocked.py::test_chat_user_message_in_prompt	PASS (B2 regression)
5	python -m py_compile agent_platform/*.py agent_platform/agents/*.py tests/*.py	PASS (exit 0)
6	docker compose config --quiet	PASS (exit 0)

---

Regression Markers (xfail, expected to fail)

Two tests are intentionally marked @pytest.mark.xfail(strict=False) because the underlying bugs are NOT yet fixed in the source code (the spec lists them as "SOURCE REQ IDS" driving the tests, not as "already in place" bugs):

C3 — case-insensitive Bearer prefix

• Test: tests/test_auth.py::test_case_insensitive_bearer
• Current behaviour: auth.py uses authorization.replace("Bearer ", "").strip() which is case-sensitive. Lowercase bearer <token> fails → 401.
• Expected fix: case-insensitive prefix match, e.g. if authorization.lower().startswith("bearer "): token = authorization[7:].
• Impact: xfail (does not block suite, exit 0).

S1 — MCPClient.health() CancelledError handling

• Test: tests/test_mcp_client.py::test_health_handles_cancelled_error
• Current behaviour: mcp_client.py health() catches Exception only. asyncio.CancelledError is a BaseException subclass (Python 3.8+) and propagates → /health returns 500 in some scenarios.
• Expected fix: except (Exception, asyncio.CancelledError): or except BaseException:.
• Impact: xfail (does not block suite, exit 0).

These tests act as regression markers — once the fixes land, removing the @pytest.mark.xfail decorator will make them turn green automatically.

---

Test Coverage by Source Req ID

Req ID	Test(s)	Status
B1 boot guard	(not testable as a unit test — covered by lifespan check, intentionally bypassed via ASGITransport)	n/a
B2 user-message-before-history	test_chat_user_message_in_prompt	PASS
B3 MCP /mcp path	(covered by deployment, not unit test)	n/a
B4 Dockerfile USER root	(infra, not unit test)	n/a
B5 healthchecks+depends_on	(infra)	n/a
B6 max_length on ChatMessage	test_chat_empty_message_returns_422, test_chat_message_over_limit_422	PASS
C3 case-insensitive Bearer	test_case_insensitive_bearer	XFAIL (bug unfixed)
S1 mcp_client CancelledError	test_health_handles_cancelled_error	XFAIL (bug unfixed)
CRUD endpoints	test_list_agents_returns_array, test_create_agent_via_api, test_create_agent_invalid_id_422, test_get_agent_by_id, test_update_agent, test_delete_db_agent_returns_204	6/6 PASS
Auth	test_no_auth_header_returns_401, test_invalid_bearer_returns_401, test_valid_bearer_returns_200, test_health_endpoint_no_auth	4/4 PASS
Health	test_health_no_auth_required, test_health_returns_agent_count, test_health_mcp_unreachable_still_200	3/3 PASS

---

Smoke Test Notes

The app_client fixture builds an httpx.AsyncClient against the FastAPI app via ASGITransport without lifespan events. This is intentional:

• The B1 boot guard (settings.AUTH_TOKEN must be ≥ 32 chars and not the default) is satisfied at the settings level via clean_db.
• We avoid triggering sync_code_agents() so the DB stays deterministic; tests that need an agent use seed_agent.
• We override api.get_db so every request gets a connection to the tmp DB.
• We monkeypatch get_mcp_client in both api_module and agent_module (not just mcp_client_module) because from mcp_client import get_mcp_client binds the function reference at import time, so patching the source module alone has no effect on already-imported consumers.

Manually exercised:

• GET /health → 200, includes seeded agent id, mcp_server="reachable"
• POST /api/agents (valid) → 201
• POST /api/agents (invalid id) → 422
• DELETE /api/agents/test_agent (db source) → 204
• POST /api/chat/test_agent with mocked LLM → 200, "FAKE_REPLY"

---

Deviations / Notes

• Two xfail tests (C3, S1) — see "Regression Markers" above. Both are written against the expected post-fix behaviour. The spec lists these as "SOURCE REQ IDS" (not as "already in place" bugs), so the xfail is the correct way to land the regression tests now without modifying frozen source code.
• MCP stub: the app_client fixture monkeypatches get_mcp_client in three modules (mcp_client, api, agent). Without this, api/agent would use their already-imported reference to the real factory and bypass the stub.
• Test location: tests live at the repo root (/tests/) rather than inside agent_platform/, so pip install -e . from root works alongside the existing pip install -e ./agent_platform for the inner package.