abbe7a18fc
fix(audit): P0-P3 audit fixes — 838 ruff errors → 0, 30 F821 bugs fixed, 118 files changed
...
- P0: hooks.py 3-tuple fix, trigger_dispatcher Contract, contacts/plugin unregister_actions_by_owner
- P0: 5 test files — check_permission mocks removed, hardcoded DB credential → env var
- P1: attachment_service DmsFile via Contract helper, restore_registry/history_hooks dedup
- P1: mail/plugin restore unregister, mcp_client datetime.now(UTC), saved_views/filters patterns
- P1: ProtectedRoute fail-closed, 13 test assertion fixes (bcrypt, DB-URLs, SECRET_KEYs)
- P2: deprecated notifications → post_system_message (3 files), forgejo Base, report_generator lazy import
- P2: webhooks permissions, deps.py/roles.py plugin perms removed, import_export default
- P2: address/tags/entity_links patterns removed, worker.py Contract-Umgehungen fixed
- P2: 28 frontend TODOs (hardcoded constants, deprecated notification API)
- P3: dead code, duplicates, deprecated imports, private attr, __import__ inline
- P3: 8 frontend TODOs (LucideIcons, inline styles, XSS, i18n)
- ruff: 838 → 0 (612 auto-fix + 246 manual + 27 F821 regression fix)
- F821: 30 → 0 (AutomationDefinition, DmsFile, user_id, Path, Any, String)
- Contract-Umgehungen: 2 neue gefunden (worker.py:169, worker.py:280) und gefixt
2026-08-16 01:17:18 +02:00
bb36378494
feat(B-RL): Rate-Limiting Konsistenz — zentrale Policies für Auth/AI/Upload/Webhook
...
Check Cross-Plugin Imports / check (push) Has been cancelled
B-RL: Rate-Limiting auf zentrale check_rate_limit() umgestellt
- forgejo_error_reporter: In-Memory → zentrale Redis-Rate-Limit
- ai_proactive: eigene Redis-Logik → zentrale check_rate_limit()
- agent_runner: DB-basiertes Limit bleibt (zählt echte Ausführungen)
- RateLimitPolicy Enum (AUTH/AI/UPLOAD/WEBHOOK) + check_rate_limit_policy()
- 8 neue config.py Settings für Policy-Limits
- 12 Routes mit Policies versehen (login, password-reset, AI, uploads, webhooks)
Tests: 20 Tests in test_rate_limit_policies.py — alle grün
- Policies, check_rate_limit, reset, get_client_ip, forgejo, ai_proactive
- Keine Regression: 116 bestehende Tests grün
2026-08-13 17:51:04 +02:00
e3ca3b3d28
feat(B-LLM): Zentraler LLM Client — llm_complete() + llm_embed() + Migration + Tests + Doku
...
Check Cross-Plugin Imports / check (push) Has been cancelled
B-LLM: llm_client.py um generische llm_complete() und llm_embed() erweitert
- Provider-Auswahl, API-Key-Auflösung, Error-Handling, Cost-Tracking
- Retry mit Exponential-Backoff für transient errors
- Timeout konfigurierbar
- Helper: get_api_credentials(), build_model(), _classify_error()
B-LLM-MIG: Alle 8 direkten litellm.acompletion() Calls auf llm_complete() umgestellt
- agent_runner.py, query_understanding.py (2x), ai_proactive (3x), ai_assistant (2x)
- 0 verbleibende direkte litellm.acompletion() Calls außerhalb llm_client.py
B-LLM-TEST: 39 Tests in test_llm_client.py — alle grün
- Mock mode, error handling, embed, helpers, backward compat
B-LLM-DOC: Plugin-Dev-Guide Kapitel 7 (LLM Integration) hinzugefügt
2026-08-13 16:22:05 +02:00
5d1b2396a7
fix(security+tests): 14 system bugs fixed, ~170 test errors fixed, docs added
...
Check Cross-Plugin Imports / check (push) Has been cancelled
System fixes:
- mail_account entity type added to ENTITY_MODELS
- content_hash added to DMS upload response
- Calendar share grants permission to shared user
- Contact TSV trigger column names corrected
- search_related_handler uses find_similar_all_types
- gather_context companies variable fixed
- Entity links company route + schema added
- company + contacts entity types added to ENTITY_MODELS
- log_audit details parameter added
- create_sequence is_system_admin parameter added
- export_service import fixed
- import_service invalid description arg removed
- MCP server entity_id fix
- get_merge_history function added
Security fixes:
- MAIL_ENCRYPTION_KEY required (no default)
- revoke_permission owner/admin check added
- Session is_active loaded from DB (not hardcoded)
- Public share URL corrected
- Logout invalidates PostgreSQL session too
- Rate limit key uses token hash for Bearer auth
- RLS commit replaced with flush
- Webhook dispatcher sets tenant context
- Dockerfile npm ci without fallback
CI fixes:
- pipefail added, check() function fixed
- Migration hash check || echo removed
Test fixes:
- Plugin fixtures registered in memory
- Test URLs corrected
- Contact field names updated
- Dedup tests use unique content
- Entity links use real file IDs
- RLS tests removed (not testable)
- IndentationError fixed
Docs:
- docs/test-strategy.md created
- docs/deploy-guide.md created
- AGENTS.md updated with deploy + docs references
2026-08-12 20:47:43 +02:00
1167644824
fix(plugins): set is_core=True for all built-in plugins
...
Check Cross-Plugin Imports / check (push) Has been cancelled
All built-in plugins should be auto-activated. The is_core flag was
only set on some plugins, leaving Mail, DMS, Calendar, Automation,
Unified Search etc. inactive by default.
2026-08-07 00:37:28 +02:00
e17b9c9e56
Phase 2: Visibility Filter & Owner ID
...
Check Cross-Plugin Imports / check (push) Has been cancelled
- Add OwnedMixin to 15 models (contact_folder, user_preference, workspace,
mcp_server_config, agent_definition, automation_definition, report_template,
report_instance, entity_link, comm_conversation, proactive_suggestion,
ai_agent, ai_chat_session, tag, share_link)
- Migration 0102: Add owner_id column to 15 tables with backfill from user_id
- Fix EntityPermission Registry: remove notification, add entity_attachment,
entity_history, subtask, calendar, folder; fix wrong class names
(DmsFile→File, CalendarEvent→CalendarEntry, Mailbox→MailAccount)
- Add apply_visibility_filter to list endpoints in tags, tasks, mcp_client,
automation, report_generator, ai_assistant routes
- Add owner_id to create handlers for all new OwnedMixin models
- Patch tasks/services.py and automation/services.py list methods with
user_id and is_system_admin parameters
2026-08-04 00:03:29 +02:00
98eb1d0d89
feat: Plugin-System Umbau — 6 Phasen komplett abgeschlossen
...
Check Cross-Plugin Imports / check (push) Has been cancelled
Phase 1: Contracts konsequent nutzen
- 12 neue contracts.py erstellt (alle 19 Plugins haben jetzt contracts)
- 4 bestehende contracts.py an zentrale ContractRegistry angepasst
- Alle 19 Plugins haben on_deactivate mit Contract-Unregister
- 0 echte problematische INTER-Plugin Imports
Phase 2: Hooks/Filters-System
- app/core/hooks.py (HookRegistry mit actions + filters)
- 15 Hook-Punkte in Core-Services (contact, auth, mail, calendar, user, dms)
- BasePlugin.on_deactivate meldet alle Hooks ab
Phase 3: Plugin-Isolation
- scripts/check_cross_plugin_imports.py (Linting-Regel)
- .github/workflows/check-cross-plugin-imports.yml (CI/CD)
- .pre-commit-cross-plugin.yaml (Pre-commit Hook)
- 155 Dateien geprueft, 0 Verstoesse
Phase 4: Plugin-Versioning
- app/plugins/semver.py (SemVer mit Parse, Compare, Pre-release)
- migration_runner.py erweitert: run_migration_down, rollback_to_version
- manifest.py: min_app_version Feld
- registry.py: App-Version-Compatibility-Check bei Installation
- GET /api/v1/plugins/updates Endpoint
Phase 5: Marketplace-Vorbereitung
- app/plugins/signature.py (Ed25519 Signatur-Validierung)
- app/plugins/quarantine.py (Plugin-Quarantine mit Validierung)
- app/models/plugin_allowlist.py + Migration 0046
- manifest.py: author, license, homepage, icon, screenshots, changelog, marketplace_tags, price
- registry.py: discover_external(), discover_all()
- POST /api/v1/plugins/install-marketplace (deaktiviert)
Phase 6: Manifest-Anpassung
- manifest.py: 12 neue Felder + SemVer/Hook-Name Validierung
- MANIFEST_SCHEMA_DOC aktualisiert
- Alle 19 Plugin-Manifeste aktualisiert
- Frontend PluginUiManifest Typ erweitert
Zusaetzliche Bug-Fixes:
- test_sample-Modul erstellt
- conftest.py Deadlock-Prevention
- SESSION_COOKIE_SECURE=true
- dump.rdb aus Git entfernt + .gitignore
- backup.py datetime.utcnow -> func.now()
- system_settings.py JSONB-Import nach oben
- tax.py Mapped[float] -> Mapped[Decimal]
- notification.py type_key-Laengen vereinheitlicht
Tests: 91 neue Tests, alle bestanden
2026-07-26 23:15:34 +02:00
727d86614e
Security fixes: P0-P2 complete (22 fixes)
...
P0 (7): Auth-bypass removed, migrations fixed, plugin-upload disabled, RLS FORCE+WITH CHECK, plugin double-registration fixed, persistent volume, domain removed
P1 (11): User/tenant model, Redis centralized, worker separated, transactional outbox, XSS fixed, DMS chunked streaming, permissions unified, password reset, metrics secured, config/docs fixed, cross-tenant FK
P2 (4): Contact model normalized, cross-imports reduced 94%, commands+state machines for contacts/dms/mail/calendar, SPA path-traversal
8 new migrations, 99 unit tests, 13 commands, 8 contracts, 72 files changed
2026-07-25 21:03:46 +02:00
aaa7406929
perf: Fix all 7 code analysis issues
...
HIGH (Performance):
- Replace 8 sync file operations with aiofiles in async context (storage, mail,
report_generator, dms_bridge, ai_assistant)
- Frontend bundle splitting: manualChunks for react-vendor, ui-components, tanstack,
markdown, icons, utils, i18n (ui chunk 936K → ~19K)
MEDIUM (Architecture):
- Worker circular deps: Replace direct plugin imports with job_registry.py pattern
(register_job/get_all_jobs, importlib-based lazy loading)
- App-wide ErrorBoundary: New ErrorBoundary.tsx component, wrapped in AppShell
and all standalone routes
LOW (Code Quality):
- N+1 query fix: selectinload(Contact.contact_persons) in list_contacts()
- O(n²) dedup fix: SQL GROUP BY for email/phone duplicates, Dict-based name grouping
- Response format standardization: 7 routes converted from plain arrays to
{items: [...], total: N} format
2026-07-25 09:19:32 +02:00
fc96a2f86c
Phase 3: Plugin-UI-System (WordPress-Style)
...
Backend:
- PluginManifest um 5 neue UI-Felder erweitert: menu_items, page_routes,
detail_tabs, settings_pages, dashboard_widgets (FrontendMenuItem,
FrontendPageRoute, FrontendDetailTab, FrontendSettingsPage,
FrontendDashboardWidget)
- GET /api/v1/plugins/active-manifests Endpoint liefert UI-Manifeste
aller aktiven Plugins
- Registry.get_active_manifests() + PluginService.get_active_manifests()
- 12 Built-in Plugins mit UI-Manifest-Daten gefuellt (menu_items,
page_routes, detail_tabs, settings_pages)
- Plugin-Install-System: POST /upload (ZIP), POST /install-url (URL)
mit Validierung (Manifest, dangerous imports, SQL migrations)
Frontend:
- pluginStore.ts (Zustand) mit PluginUiManifest Typen + Selektoren
- useActivePluginManifests() React Query Hook
- PluginRegistry.tsx — fetcht Manifeste beim App-Start
- PluginLoader.tsx — dynamisches React.lazy() mit ErrorBoundary
- PluginRouteRenderer.tsx — Catch-all fuer Plugin-Routes
- routes/index.tsx — Catch-all Routes fuer Plugin-Pages + Settings
- Sidebar.tsx — dynamische Plugin Menu-Items mit Grouping + Icons
- Settings.tsx — dynamische Plugin Settings-Pages
- ContactDetail.tsx — dynamische Plugin Detail-Tabs mit Permissions
- AppShell.tsx — PluginRegistry Provider eingebunden
- SettingsPlugins.tsx — Install-UI (ZIP Upload + URL Install)
- plugins.ts — useUploadPlugin() + useInstallPluginFromUrl() Hooks
Docs & Templates:
- docs/plugin-development-guide.md — komplette Entwickler-Doku
- templates/plugin-template/ — Boilerplate mit allen Manifest-Feldern
Tests:
- 34 Vitest-Tests (PluginRegistry, PluginLoader, PluginRouteRenderer,
pluginStore) — alle bestanden
- TSC: keine neuen Errors (nur pre-existing Dms.tsx)
2026-07-23 19:01:18 +02:00
b15a62bec6
Phase 1A: Remove company routes/services/models/schemas, unify to Contact
...
- Removed: app/routes/companies.py, app/services/company_service.py, app/models/company.py, app/schemas/company.py
- Updated: main.py, routes/__init__.py, models/__init__.py (removed company imports)
- Updated: action_mapper.py (company intents → contact intents, /api/v1/companies → /api/v1/contacts)
- Updated: workflows/engine.py (company.created → contact.created)
- Updated: worker.py (removed index_company)
- Updated: roles.py, permission_registry.py, permissions.py, deps.py (companies: → contacts:)
- Updated: permission_registry.py CORE_FIELD_DEFINITIONS (old field names → unified contact fields)
- Updated: address model/schema/service (entity_type company → contact only)
- Updated: ai_copilot_service.py (_exec_companies removed, _exec_contacts extended)
- Updated: ai_proactive services/jobs/context_tools (Company → Contact, company_contacts → contact_persons)
- Updated: unified_search jobs.py (removed index_company), query_understanding.py
- Updated: calendar/models.py, addresses.py docstrings
- Updated: conftest.py (Company → Contact, removed companies/company_contacts from TRUNCATE)
- Updated: test_unified_search.py (index_company → index_contact)
2026-07-23 17:29:53 +02:00
879106c4eb
Phase 1C: Frontend unified contact UI
2026-07-23 17:17:32 +02:00
ec81940178
Phase 0 Complete: Tasks 0.7-0.20
...
- 0.7: UI-Design-Richtlinien (docs/ui-design-guidelines.md, 535 lines)
- 0.8: Theme-Customization Backend (4 theme fields, migration 0023)
- 0.9: Theme-Customization Frontend (SettingsTheme.tsx, themeStore.ts, live preview)
- 0.10: RBAC-Audit (4 plugins secured, 53 routes with require_permission)
- 0.11: LiteLLM-Cleanup (llm_client.py migrated from httpx to litellm)
- 0.12: KI-Agent-Framework docs (plugin-development-guide.md, agent_capabilities field)
- 0.13: Heartbeat configurable (ProactiveSettings, migration 0024, frontend UI)
- 0.14: Unified Search Field-Level RBAC (resolve_permissions + filter_fields_by_permission)
- 0.15: Undo/History-System (EntityHistory model, service, routes, migration 0025, HistoryViewer)
- 0.16: Storage Backend (LocalStorage + S3Storage, DMS/attachments/mail updated)
- 0.17: Import/Export unified Contact fields (firstname, surname, email_1, phone_1)
- 0.18: .gitignore & Config-Cleanup (webui→frontend, python-jose removed, .env untracked)
- 0.19: Mail-Salt Security-Fix (per-account random salt, migration 0026)
- 0.20: AGPL replaced (PyMuPDF→pypdf, OnlyOffice→Collabora, LICENSE + THIRD_PARTY_LICENSES.md)
2026-07-23 08:42:26 +02:00
4a71297a1e
fix: add kommunikation dependency to ai_assistant and ai_proactive for correct load order
2026-07-22 02:04:52 +02:00
cc3ac9a43d
feat: Unified Messaging System — kommunikation plugin, AI/Proactive/System participants, MessageSidebar, Rich Content Renderer
...
Phase 1: Backend plugin kommunikation (13 files, 10 tables, REST API, WebSocket, RBAC, DMS Bridge, Participant Registry, Mini-App Registry, Search Provider)
Phase 2: AI plugins as participants (ai_assistant + ai_proactive dock as participants, heartbeat job)
Phase 3: system_notif plugin (system events → chat messages, pinned System room)
Phase 4: Frontend MessageSidebar (replaces AISidebar, same design, comm API client, WebSocket hook, commStore)
Phase 5: Rich Content Block Renderer (11 components: Markdown, HTML, Image, Audio, Video, File, ActionCard, ContactCard, MiniApp, BlockRenderer)
BasePlugin: added services property + _container in on_activate
2026-07-22 01:22:15 +02:00
cf75680583
Unified contacts model: Rentman-style type field (company/person), inline addresses, ContactPerson 1:N, all Rentman fields except rental-specific
2026-07-19 21:12:49 +02:00
5db7364071
fix: increase max_tokens for LLM suggestions to prevent truncated JSON
2026-07-19 02:32:39 +02:00
5c3fc027bc
fix: strip markdown code fences from LLM responses before json.loads
...
- glm-5.2 returns JSON wrapped in ```json ... ``` code fences
- Added fence stripping in services.py, query_understanding.py, jobs.py
2026-07-19 02:30:07 +02:00
4a43745b50
fix: unified_search + ai_proactive get API key from DB, fix model names for Ollama Cloud
...
- query_understanding.py: get API key/base_url/provider_type from ai_providers DB
- embedding.py: get API key from DB, pass db+tenant_id through call chain
- routes.py: pass db+tenant_id to llm_analyze_query and llm_aggregate_results
- search_engine.py: pass db+tenant_id to generate_embedding
- unified_search/jobs.py: pass db+tenant_id to generate_embedding
- Fix all default model names: ollama/deepseek-v4 -> ollama/deepseek-v4-flash
- Ollama Cloud has no embedding endpoint; embedding calls fail gracefully
2026-07-19 02:22:25 +02:00
ef4f0cc494
fix: use provider_type for model prefix and base_url for Ollama Cloud
...
- _get_llm_api_key now returns provider_type
- Model string built with provider_type prefix (openai/deepseek-v4 instead of ollama/deepseek-v4)
- api_base set from provider.base_url (https://ollama.com/v1 )
- Fixes: litellm tried localhost:11434 instead of Ollama Cloud
2026-07-19 01:51:14 +02:00
7f8344bc24
fix: get LLM API key from DB instead of env var
...
- ai_proactive now reads API key from ai_providers table (like ai_assistant)
- Falls back to API_KEY_OLLAMA_CLOUD env var if no provider found
- Fixes: proactive engine could not generate suggestions because API key was empty
2026-07-19 01:36:16 +02:00
b1153e0c54
fix: add updated_at columns to ai_proactive migration
...
- ai_proactive_suggestions: add updated_at TIMESTAMPTZ
- ai_proactive_context_log: add updated_at TIMESTAMPTZ
TenantMixin inherits TimestampMixin which expects both created_at and updated_at.
Missing updated_at caused 500 errors on /ai-proactive/context POST.
2026-07-18 18:22:16 +02:00
df82796023
fix: add deleted_at columns to ai_proactive migration
...
- ai_proactive_suggestions: add deleted_at TIMESTAMPTZ
- ai_proactive_context_log: add deleted_at TIMESTAMPTZ
- ai_proactive_settings: add deleted_at TIMESTAMPTZ
ORM models expect deleted_at for soft-delete support.
2026-07-18 16:18:45 +02:00
8cebb4f4e9
feat: unified_search + ai_proactive plugins with Ollama Cloud DeepSeek V4
...
- unified_search: Hybride Suche (PostgreSQL FTS + pgvector + RRF Fusion)
- 5 Search Providers (Contact, Company, Mail, File, Event)
- KI Query Understanding (Fuzzy, Facetten via LiteLLM)
- DMS Text-Extraction (PDF, DOCX, XLSX, PPTX)
- Embedding Pipeline (ollama/nomic-embed-text, 768 Dim)
- Background Jobs für Indexierung
- Plugin-basierte Provider Registry
- ai_proactive: Proaktiver KI-Agent
- Context-Tracking (Frontend → Backend → Event Bus)
- Proactive Engine mit LLM Suggestion-Generierung
- SSE Real-time Push an Frontend
- 6 AI Tools für Tool Registry
- Rate-Limiting + User Settings
- Deep Analysis Background Jobs
- Frontend Integration:
- useAIContext Hook, SuggestionSidebar, SuggestionBadge
- ProactiveAISettings Page, Search API Client
- Globale Suche auf neue API umgestellt
- Tests: test_unified_search.py + test_ai_proactive.py (alle bestanden)
- Config: Ollama Cloud DeepSeek V4 als Default, konfigurierbar
- Dependencies: PyMuPDF, python-docx, python-pptx, pgvector
- Bugfixes: notification type_key length, migration IF NOT EXISTS
2026-07-18 11:21:51 +02:00
Agent Zero