feat(F): F-TEST + F-DOC + F-UI-TRIG — Phase F complete!
- F-TEST: tests/test_phase_f_agents.py (1425 lines, 45 tests, all pass) — ReAct Loop, Permissions, Approvals, Skills, Context Builder, Data Policy, Transparency, Workstream, Budget - F-DOC: docs/api-documentation.md (Phase F endpoints), docs/plugin-development-guide.md (Agent chapter 32), docs/test-strategy.md (Phase F test conventions) - F-UI-TRIG: trigger_dispatcher dispatches agents on ui.*/context.* events (already implemented in F-PROACTIVE) - Bug fix: approval.py metadata reserved attribute renamed to request_metadata - PROGRESS.md: Phase F marked done, ~155/223 tasks done (70%)
This commit is contained in:
Agent Zero
+6
-6
@@ -1,7 +1,7 @@
|
||||
# LeoPlatform — Fortschritts-Tracking
|
||||
|
||||
> **Letztes Update:** 2026-08-17
|
||||
> **Status:** Phase F — fast done (3 Tasks offen)
|
||||
> **Status:** Phase F — done
|
||||
|
||||
---
|
||||
|
||||
@@ -15,13 +15,13 @@
|
||||
| C.5 — Import/Export | `done` | 2026-08-13 | 2026-08-13 | 8 | 8 |
|
||||
| D — Undo/Restore | `done` | 2026-08-13 | 2026-08-13 | 13 | 13 |
|
||||
| E — Search | `done` | 2026-08-14 | 2026-08-14 | 24 | 24 |
|
||||
| F — Agents | `in_progress` | 2026-08-17 | — | ~38 | ~41 |
|
||||
| F — Agents | `done` | 2026-08-17 | 2026-08-17 | ~41 | ~41 |
|
||||
| G — Workflows | `not_started` | — | — | 0 | ~24 |
|
||||
| H — Knowledge | `not_started` | — | — | 0 | ~18 |
|
||||
| I — Integration & Workstream | `not_started` | — | — | 0 | ~25 |
|
||||
| J — Self-Improvement | `not_started` | — | — | 0 | ~12 |
|
||||
|
||||
**Gesamt:** ~152 / ~223 Tasks done
|
||||
**Gesamt:** ~155 / ~223 Tasks done
|
||||
|
||||
---
|
||||
|
||||
@@ -267,7 +267,7 @@
|
||||
| F-CONTACT | `done` | — | ✅ prebuilt/contact_enrichment_agent.py — Contact-Enrichment-Agent |
|
||||
| F-FOLLOW | `done` | — | ✅ prebuilt/follow_up_agent.py — Follow-up-Agent |
|
||||
| F-REPORT | `done` | — | ✅ prebuilt/report_agent.py — Report-Agent |
|
||||
| F-UI-TRIG | `not_started` | — | — |
|
||||
| F-UI-TRIG | `done` | — | ✅ trigger_dispatcher.py dispatcht agents auf ui.* und context.* Events via _dispatch_matching_agents(). ai_proactive handle_context_change bereits vorhanden |
|
||||
| F-TASK-MODEL | `done` | — | ✅ Extended Task model with polymorphic assignee/entity/creator, subtasks, dependencies, task_type, success_criteria, progress |
|
||||
| F-TASK-API | `done` | — | ✅ Extended task routes with polymorphic filters, subtasks, dependencies |
|
||||
| F-TASK-AGENT | `done` | — | ✅ ai_tools.py (191 lines) — create_task, assign_task, update_task_status, decompose_goal tools |
|
||||
@@ -276,8 +276,8 @@
|
||||
| F-TASK-MIG | `done` | — | ✅ Migration 0124 — new columns, data migration |
|
||||
| F-TASK-GOAL | `done` | — | ✅ Progress aggregation, success criteria evaluation, parent status propagation |
|
||||
| F-TASK-TEST | `done` | — | ✅ test_unified_tasks.py (414 lines) |
|
||||
| F-TEST | `not_started` | — | — |
|
||||
| F-DOC | `not_started` | — | — |
|
||||
| F-TEST | `done` | — | ✅ tests/test_phase_f_agents.py (1425 lines, 45 tests, all pass) — ReAct Loop, Permissions, Approvals, Skills, Context Builder, Data Policy, Transparency, Workstream, Budget Limits |
|
||||
| F-DOC | `done` | — | ✅ docs/api-documentation.md (Phase F endpoints), docs/plugin-development-guide.md (Agent chapter 32), docs/test-strategy.md (Phase F test conventions) |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -65,8 +65,8 @@ class ApprovalRequest(Base, TenantMixin):
|
||||
expires_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True
|
||||
)
|
||||
metadata: Mapped[dict[str, Any]] = mapped_column(
|
||||
JSONB, nullable=False, default=dict
|
||||
request_metadata: Mapped[dict[str, Any]] = mapped_column(
|
||||
"metadata", JSONB, nullable=False, default=dict
|
||||
)
|
||||
|
||||
|
||||
@@ -96,7 +96,7 @@ async def create_approval_request(
|
||||
approver_group=approver_group,
|
||||
status="pending",
|
||||
expires_at=expires_at,
|
||||
metadata=metadata or {},
|
||||
request_metadata=metadata or {},
|
||||
)
|
||||
db.add(req)
|
||||
await db.flush()
|
||||
|
||||
@@ -99,7 +99,7 @@ def _to_response(r: ApprovalRequest) -> ApprovalResponse:
|
||||
created_at=r.created_at.isoformat() if r.created_at else None,
|
||||
resolved_at=r.resolved_at.isoformat() if r.resolved_at else None,
|
||||
expires_at=r.expires_at.isoformat() if r.expires_at else None,
|
||||
metadata=r.metadata or {},
|
||||
metadata=r.request_metadata or {},
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -273,9 +273,55 @@ Agent Builder, Automation Builder, Cron-Scheduler, Agent Runner.
|
||||
|
||||
### agents (AI Agents)
|
||||
|
||||
Phase F — Agent system: CRUD, execution, streaming, runs, tools, skills, approvals, monitoring, AI use-case, unified tasks.
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET/POST/PUT/DELETE | `/api/v1/agents/*` | AI agent CRUD and runner endpoints. |
|
||||
| GET | `/api/v1/agents` | List agent definitions. |
|
||||
| POST | `/api/v1/agents` | Create an agent definition. |
|
||||
| GET | `/api/v1/agents/{id}` | Get an agent definition. |
|
||||
| PATCH | `/api/v1/agents/{id}` | Update an agent definition (optimistic lock via `version`). |
|
||||
| DELETE | `/api/v1/agents/{id}` | Delete an agent definition. |
|
||||
| POST | `/api/v1/agents/{id}/execute` | Execute an agent (manual trigger). |
|
||||
| GET | `/api/v1/agents/{id}/stream` | Stream agent run steps (SSE). |
|
||||
| GET | `/api/v1/agents/{id}/runs` | List runs for an agent. |
|
||||
| GET | `/api/v1/agents/{id}/runs/{run_id}/steps` | List steps for a run. |
|
||||
| GET | `/api/v1/agents/tools` | List available agent tools. |
|
||||
| GET | `/api/v1/agents/monitor/stats` | Agent monitor statistics. |
|
||||
| GET | `/api/v1/agents/{id}/ai-use-case` | Get AI use-case metadata. |
|
||||
| PATCH | `/api/v1/agents/{id}/ai-use-case` | Update AI use-case metadata. |
|
||||
|
||||
### skills (AI Skills)
|
||||
|
||||
Phase F — Skill registry for agent capabilities.
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/v1/skills` | List skills. |
|
||||
| POST | `/api/v1/skills` | Register a skill. |
|
||||
| PATCH | `/api/v1/skills/{id}` | Update a skill. |
|
||||
| DELETE | `/api/v1/skills/{id}` | Delete a skill. |
|
||||
|
||||
### approvals (Approval Requests)
|
||||
|
||||
Phase F — Human approval workflow for agent actions.
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/v1/approvals` | List approval requests (filters: status, entity_type, entity_id, requested_by). |
|
||||
| POST | `/api/v1/approvals` | Create an approval request. |
|
||||
| POST | `/api/v1/approvals/{id}/approve` | Approve a pending request. |
|
||||
| POST | `/api/v1/approvals/{id}/reject` | Reject a pending request. |
|
||||
|
||||
### unified-tasks (Unified Tasks)
|
||||
|
||||
Phase F — Polymorphic task assignment, subtasks, and goals.
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET/POST | `/api/v1/unified-tasks/*` | Unified task CRUD with polymorphic assignment. |
|
||||
| GET/POST | `/api/v1/unified-tasks/*/subtasks` | Subtask management. |
|
||||
| GET/POST | `/api/v1/unified-tasks/*/goals` | Goal management. |
|
||||
|
||||
### dms (Document Management System)
|
||||
|
||||
|
||||
@@ -2388,4 +2388,162 @@ Das Frontend nutzt die API-Client-Funktionen aus `importExport.ts`:
|
||||
|
||||
---
|
||||
|
||||
## 32. Agents (Phase F)
|
||||
|
||||
Phase F introduces a full agent system: agent definitions, a ReAct loop, a tool registry, a skill registry, a permission model, an approval workflow, and workstream integration. This chapter explains how plugins can contribute agents, tools, and skills.
|
||||
|
||||
### 32.1 Agent Definition
|
||||
|
||||
An agent is defined by an `AgentDefinition` record (automation plugin). Key fields:
|
||||
|
||||
- `name`, `description` — display and purpose.
|
||||
- `llm_model`, `provider`, `api_key`, `api_base` — LLM configuration (secrets never exposed via API).
|
||||
- `system_prompt` — the agent's base instructions.
|
||||
- `max_steps`, `max_tokens`, `max_duration_seconds` — execution limits.
|
||||
- `budget_limit_usd` — cumulative cost cap per agent.
|
||||
- `tool_ids`, `skill_ids` — which tools and skills the agent may use.
|
||||
- `mode` — `reactive` (manual/proactive trigger) or `proactive`.
|
||||
- `trace_mode` — `standard` or `extended` (extended posts ReAct steps to the workstream).
|
||||
- `ai_use_case_metadata` — allowed data categories for the data policy.
|
||||
|
||||
Create an agent via `POST /api/v1/agents` or directly in code:
|
||||
|
||||
```python
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
agent = AgentDefinition(
|
||||
tenant_id=tenant_id,
|
||||
name="Support Bot",
|
||||
description="Answers support questions",
|
||||
llm_model="gpt-4o",
|
||||
system_prompt="You are a helpful support assistant.",
|
||||
tool_ids=["mail_read", "contact_search"],
|
||||
skill_ids=["support_skill"],
|
||||
max_steps=10,
|
||||
budget_limit_usd=5.0,
|
||||
)
|
||||
```
|
||||
|
||||
### 32.2 Registering Tools in the ToolRegistry
|
||||
|
||||
Tools are registered in the central `ToolRegistry` (ai_assistant plugin). A tool exposes an OpenAI-style function schema and a handler.
|
||||
|
||||
```python
|
||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||
from app.ai.agent_tools import AITool
|
||||
|
||||
async def _search_contacts_handler(args: dict, ctx: dict) -> dict:
|
||||
# ... business logic ...
|
||||
return {"results": [...]}
|
||||
|
||||
registry = get_tool_registry()
|
||||
registry.register(AITool(
|
||||
name="contact_search",
|
||||
description="Search contacts by name or email",
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {"query": {"type": "string"}},
|
||||
"required": ["query"],
|
||||
},
|
||||
handler=_search_contacts_handler,
|
||||
required_permission="contacts:read",
|
||||
))
|
||||
```
|
||||
|
||||
- `required_permission` gates the tool: a user only gets the tool if they hold that permission.
|
||||
- The handler receives `(args, ctx)` where `ctx` contains `tenant_id`, `user_id`, `db`, and `agent_run_id`.
|
||||
|
||||
### 32.3 Registering Skills in the SkillRegistry
|
||||
|
||||
Skills bundle instructions and allowed tools. They are registered in the singleton `SkillRegistry`.
|
||||
|
||||
```python
|
||||
from app.ai.skill_registry import SkillDefinition, get_skill_registry
|
||||
|
||||
skill = SkillDefinition(
|
||||
name="support_skill",
|
||||
description="Support workflow instructions",
|
||||
instructions="Use contact_search then mail_read to answer support tickets.",
|
||||
allowed_tool_ids=["contact_search", "mail_read"],
|
||||
category="support",
|
||||
)
|
||||
get_skill_registry().register(skill)
|
||||
```
|
||||
|
||||
- `get_by_names([...])` resolves skills and skips unknown names.
|
||||
- A skill **never grants** a tool the user does not already have permission for — the effective tool set is the intersection of user, agent, skill, and tool permissions.
|
||||
|
||||
### 32.4 ReAct Loop
|
||||
|
||||
The ReAct loop (`app.ai.agent_loop.run_react_loop`) drives the agent:
|
||||
|
||||
1. Build context (system prompt + user message).
|
||||
2. Call the LLM with the available tool schemas.
|
||||
3. If the LLM returns a tool call, execute the tool handler and append the observation.
|
||||
4. Repeat until a final answer, `max_steps`, timeout, or budget is reached.
|
||||
|
||||
```python
|
||||
from app.ai.agent_loop import run_react_loop
|
||||
|
||||
result = await run_react_loop(
|
||||
agent_definition=agent,
|
||||
messages=[{"role": "user", "content": "Find the latest invoice"}],
|
||||
tools=tool_schemas,
|
||||
tool_registry=registry,
|
||||
db=db,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
agent_run_id=run_id,
|
||||
max_steps=20,
|
||||
timeout_seconds=300,
|
||||
)
|
||||
```
|
||||
|
||||
`result` is a `ReActResult` with `status`, `steps`, `final_content`, `total_cost_usd`, and `error`. Statuses: `completed`, `stopped_max_steps`, `stopped_timeout`, `stopped_error`, `budget_exceeded`.
|
||||
|
||||
### 32.5 Permission Model
|
||||
|
||||
Effective agent permissions are the **intersection** of four layers:
|
||||
|
||||
```
|
||||
User permissions ∩ Agent tool_ids ∩ Skill allowed_tool_ids ∩ Tool required_permission
|
||||
```
|
||||
|
||||
- `resolve_agent_permissions(db, tenant_id, user_id, agent)` returns an `AgentPermissionContext` with `effective_tool_ids` and `can_use_tool(name)`.
|
||||
- System admins bypass the permission check and get all tools configured on the agent.
|
||||
- `filter_visible_agents` respects `agents:read`; `check_agent_execute_permission` respects `agents:execute`.
|
||||
- Optimistic locking: PATCH/DELETE on agents require a matching `version`; a mismatch returns `409 conflict`.
|
||||
|
||||
### 32.6 Approval Workflow
|
||||
|
||||
Tools that require human approval pause the loop and create an `ApprovalRequest` (`app.core.approval`).
|
||||
|
||||
- Status lifecycle: `pending` → `approved` | `rejected` | `expired`.
|
||||
- Create: `create_approval_request(db, tenant_id, entity_type=..., entity_id=..., action=..., requested_by=..., metadata=...)`.
|
||||
- Resolve: `resolve_approval_request(db, tenant_id, request_id, decision="approved"|"rejected", approver_id=..., comment=...)`.
|
||||
- Expire: `expire_approval_request(db, tenant_id, request_id)`.
|
||||
- API: `POST /api/v1/approvals`, `POST /api/v1/approvals/{id}/approve|reject`.
|
||||
|
||||
### 32.7 Workstream Integration
|
||||
|
||||
Agents post messages, steps, and results to the communication system (`app.ai.agent_workstream`):
|
||||
|
||||
- `post_agent_message` — text or block message, marked AI-generated.
|
||||
- `post_agent_step` — ReAct step as an `action_card` (only in `extended` trace mode).
|
||||
- `post_agent_result` — final result with `status`, `steps_taken`, `total_cost_usd`, `run_id`.
|
||||
- `post_approval_request` — approval card.
|
||||
|
||||
All messages are marked with AI-generated transparency metadata.
|
||||
|
||||
### 32.8 Data Policy & Transparency
|
||||
|
||||
- `enforce_data_policy(db, tenant_id, messages, agent)` strips sensitive fields, enforces allowed data categories from `ai_use_case_metadata`, and checks provider compliance before content reaches the LLM.
|
||||
- `mark_as_ai_generated(content, metadata)` adds `ai_generated: true` and `ai_metadata` to any outbound message.
|
||||
|
||||
### 32.9 Pre-Built Agents
|
||||
|
||||
LeoCRM ships pre-built agents in the automation plugin. Plugins can register additional agents at activation time by creating `AgentDefinition` records and registering their tools/skills in the registries.
|
||||
|
||||
---
|
||||
|
||||
*This document is authoritative for all plugin development at LeoCRM.*
|
||||
|
||||
@@ -343,6 +343,59 @@ with patch("app.plugins.builtins.unified_search.embedding.llm_embed", new_callab
|
||||
|
||||
**Regeln:**
|
||||
- `llm_complete` liefert ein Dict mit `normalized_query`, `facets`, `summary` (und optional `suggestions`).
|
||||
|
||||
## Phase F — Agent-System Test-Konventionen
|
||||
|
||||
### Neue Test-Datei: `tests/test_phase_f_agents.py` (45 Tests)
|
||||
|
||||
| Test-Gruppe | Tests | Status |
|
||||
|-------------|-------|--------|
|
||||
| ReAct Loop (Multi-Step, max_steps, Timeout, Error-Recovery, Cost, Dry-Run, Audit) | 8 | ✅ |
|
||||
| Agent-Permissions (Intersection, System-Admin, Visibility, Execute, Optimistic Lock) | 9 | ✅ |
|
||||
| Approval Requests (Create, Approve/Reject/Expire, List-Filter) | 6 | ✅ |
|
||||
| Skill Registry (Registration, get_by_names, keine Permission-Grants) | 3 | ✅ |
|
||||
| Context Builder (System-Prompt, ReAct-Format, Sensitive-Fields, Tool-Descriptions) | 5 | ✅ |
|
||||
| Data Policy (Sensitive-Fields, Provider-Compliance, Allowed-Categories) | 4 | ✅ |
|
||||
| Transparency (AI-Generated-Marking, AI-Participant-Erkennung) | 2 | ✅ |
|
||||
| Workstream (Message, Step, Result) | 3 | ✅ |
|
||||
| Budget Limits (Run-Stopp bei Budget, Cost-Akkumulation) | 2 | ✅ |
|
||||
|
||||
### Konventionen für Agent-Tests
|
||||
|
||||
1. **Keine echte DB / kein echtes LLM / kein Redis:** Alle externen Abhängigkeiten werden mit `AsyncMock` / `MagicMock` gemockt. Die Tests überschreiben die `conftest`-Fixtures `db_setup` und `clean_tables` mit No-Op-Fixtures, damit kein PostgreSQL/Redis benötigt wird.
|
||||
2. **Patch-Targets am Ursprungsmodul:** Funktionen, die innerhalb einer Funktion importiert werden, müssen am Ursprungsmodul gepatcht werden. Beispiel: `get_provider_compliance` wird in `enforce_data_policy` aus `app.ai.llm_client` importiert → Patch auf `app.ai.llm_client.get_provider_compliance`, nicht `app.ai.data_policy.get_provider_compliance`.
|
||||
3. **Mock-LLM-Responses:** `llm_complete` wird mit `AsyncMock` gemockt und liefert Dicts mit `content`, `usage`, `cost_usd`, `model`, `raw_response` (mit `choices[0].message.content` und `message.tool_calls`).
|
||||
4. **Tool-Calls:** Mock-Tool-Calls haben `id`, `function.name`, `function.arguments` (JSON-String). Tool-Handler werden als `AsyncMock` registriert.
|
||||
5. **Keine zufälligen UUIDs in Assertions:** Echte Entity-IDs aus Mocks verwenden; UUIDs nur als generierte Test-IDs.
|
||||
6. **SQLAlchemy-Modelle in Mocks:** Für `select(model.version)` in Optimistic-Lock-Tests `sqlalchemy.column()` verwenden, nicht Plain-Strings (sonst `ArgumentError`).
|
||||
7. **Reservierte Attributnamen:** SQLAlchemy-Modelle dürfen kein `metadata`-Attribut haben (reserviert in der Declarative API). `ApprovalRequest` nutzt `request_metadata` mit DB-Spaltenname `metadata` via `mapped_column("metadata", ...)`.
|
||||
|
||||
### Mock-Patterns für Agent-Tests
|
||||
|
||||
```python
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
# LLM-Call
|
||||
with patch("app.ai.agent_loop.llm_complete", new_callable=AsyncMock) as mock_llm:
|
||||
mock_llm.return_value = {
|
||||
"content": "Final answer",
|
||||
"usage": {"total_tokens": 100},
|
||||
"cost_usd": 0.001,
|
||||
"model": "gpt-4o",
|
||||
"raw_response": raw_response,
|
||||
}
|
||||
# ... Test
|
||||
|
||||
# Provider-Compliance (in data_policy importiert aus llm_client)
|
||||
with patch("app.ai.llm_client.get_provider_compliance", new_callable=AsyncMock) as mock_compliance:
|
||||
mock_compliance.return_value = {"allowed_data_classes": ["internal"]}
|
||||
# ... Test
|
||||
|
||||
# Tool-Handler
|
||||
handler = AsyncMock(return_value="result")
|
||||
registry = MagicMock()
|
||||
registry.get = lambda name: MagicMock(handler=handler) if name == "search" else None
|
||||
```
|
||||
- `generate_embedding` / `llm_embed` liefern eine Liste von Floats (Embedding-Vektor).
|
||||
- Bei Fehlerpfaden: `mock_llm.side_effect = Exception("...")` oder `return_value = None` für Fallback-Verhalten testen.
|
||||
- DB-Session-Factory in AI-Tool-Handler-Tests: `patch("app.core.db.get_session_factory", return_value=sf)` mit `async_sessionmaker(bind=db_session.bind, ...)`.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user