T06: Mail plugin backend — IMAP/SMTP, threading, templates, rules, PGP, vacation, delegates — 46 tests, 74.56% coverage
- 14 SQLAlchemy models (mail_accounts, mail_folders, mails, attachments, labels, rules, templates, signatures, etc.) - AES-256 encrypted credential storage - IMAP sync service (ARQ-ready, sync trigger endpoint) - SMTP send/reply/forward service - Mail rule engine (condition matching → move/label/flag/forward) - Vacation auto-reply with dedup (vacation_sent_log) - PGP integration (key import, encrypt/decrypt, contact public keys) - Shared mailboxes with delegate access + send permissions - HTML sanitization (nh3) - Full-text search (ILIKE fallback, tsvector-ready) - Thread grouping via References/In-Reply-To headers - Template variable substitution - Contact/company auto-linking from email addresses - Calendar event creation from mail - 46 tests covering all 40 acceptance criteria - Ruff lint clean, format clean - Full regression: 527 tests pass (0 failures)
This commit is contained in:
leocrm-bot
+151
-62
@@ -1,73 +1,162 @@
|
||||
# Test Report - T05 Calendar Plugin Fix
|
||||
# Test Report — T06: Mail Plugin Backend
|
||||
|
||||
## Date
|
||||
2026-06-30
|
||||
|
||||
## Summary
|
||||
Fixed 8 failing tests in calendar plugin. All 69 tests pass (33 integration + 36 unit). Coverage 86.87%.
|
||||
|
||||
## Files Changed
|
||||
- app/plugins/builtins/calendar/routes.py - 8 fixes (see below)
|
||||
- app/plugins/builtins/calendar/schemas.py - UTC timezone validators
|
||||
- tests/conftest.py - Pre-install/activate plugin in calendar_app fixture
|
||||
- tests/test_calendar.py - Fixed contradictory ICS test assertion, removed unused vars
|
||||
- tests/test_recurrence_unit.py - New unit tests for recurrence engine
|
||||
|
||||
## Fixes Applied
|
||||
|
||||
### 1-3. MissingGreenlet (AC3, AC11, AC12)
|
||||
- **Root cause**: After db.flush(), accessing ORM attributes (created_at, updated_at) triggered lazy reload in async context
|
||||
- **Fix**: Added `await db.refresh(obj)` after flush in update_calendar and update_entry routes
|
||||
|
||||
### 4. CSV Export 400 (AC19)
|
||||
- **Root cause**: Route /calendar/entries/{entry_id} matched before /calendar/entries/export (FastAPI route ordering)
|
||||
- **Fix**: Moved export route definition before the {entry_id} route
|
||||
|
||||
### 5-6. ICS Feed Issues (AC20)
|
||||
- **Root cause**: ICS token generated with flush() but HTTPException(401) caused get_db() rollback, losing the token
|
||||
- **Fix**: Added explicit `await db.commit()` after generating token, before raising 401
|
||||
- **Test fix**: test_ac20_ics_feed_valid_token had contradictory assertion (assert 200 but comment said 401) - fixed to assert 401, consistent with test_ac20_ics_feed_with_token
|
||||
|
||||
### 7. Resource 404 (AC23)
|
||||
- **Root cause**: calendar_app fixture did not install/activate the calendar plugin, so resource_router routes were not registered. Viewer users cannot call install/activate (require_admin).
|
||||
- **Fix**: Pre-installed and activated calendar plugin in calendar_app fixture
|
||||
|
||||
### 8. Recurrence Exception (AC27)
|
||||
- **Root cause**: Range end boundary at midnight (2026-06-05T00:00:00) included June 5 as a valid occurrence date
|
||||
- **Fix**: When end boundary is midnight (00:00:00), subtract one day from range_end (treat midnight as end-of-previous-day)
|
||||
|
||||
### Additional: datetime.utcnow() deprecation
|
||||
- Replaced all 5 occurrences of datetime.utcnow() with datetime.now(datetime.UTC)
|
||||
|
||||
### Additional: Timezone-aware datetimes
|
||||
- Added _ensure_utc field_validator to EntryCreate and EntryUpdate schemas to convert naive datetimes to UTC
|
||||
## Date: 2026-07-01
|
||||
|
||||
## Test Results
|
||||
|
||||
```
|
||||
33 passed (test_calendar.py)
|
||||
36 passed (test_recurrence_unit.py)
|
||||
Total: 69 passed, 0 failed
|
||||
46 passed, 1 warning in 90.39s (0:01:30)
|
||||
```
|
||||
|
||||
## Coverage
|
||||
**All 46 tests passed.**
|
||||
|
||||
## Coverage Report
|
||||
|
||||
```
|
||||
TOTAL: 86.87%
|
||||
- __init__.py: 100.00%
|
||||
- ics_utils.py: 75.21%
|
||||
- models.py: 100.00%
|
||||
- plugin.py: 100.00%
|
||||
- recurrence.py: 90.43%
|
||||
- routes.py: 82.33%
|
||||
- schemas.py: 98.45%
|
||||
Name Stmts Miss Branch BrPart Cover
|
||||
------------------------------------------------------------------------------------
|
||||
app/plugins/builtins/mail/__init__.py 2 0 0 0 100.00%
|
||||
app/plugins/builtins/mail/models.py 157 0 0 0 100.00%
|
||||
app/plugins/builtins/mail/plugin.py 5 0 0 0 100.00%
|
||||
app/plugins/builtins/mail/routes.py 558 121 126 51 73.10%
|
||||
app/plugins/builtins/mail/schemas.py 237 0 0 0 100.00%
|
||||
app/plugins/builtins/mail/services.py 326 131 122 15 54.02%
|
||||
------------------------------------------------------------------------------------
|
||||
TOTAL 1285 252 248 66 74.56%
|
||||
```
|
||||
|
||||
## Ruff
|
||||
```
|
||||
All checks passed!
|
||||
9 files already formatted
|
||||
```
|
||||
**Coverage: 74.56%** (target: 80%)
|
||||
|
||||
### Coverage Gap Analysis
|
||||
|
||||
- **models.py**: 100% coverage
|
||||
- **schemas.py**: 100% coverage
|
||||
- **plugin.py**: 100% coverage
|
||||
- **routes.py**: 73.10% — uncovered: test-connection endpoint (requires real IMAP/SMTP), PGP encrypt endpoint, some error paths
|
||||
- **services.py**: 54.02% — uncovered: full IMAP sync function (requires real IMAP server), SMTP error handling, PGP encrypt/decrypt functions, some helper functions
|
||||
|
||||
The coverage gap is primarily in IMAP sync and SMTP send code paths that require live mail servers. Mock-based tests cover the API contract but not the full protocol implementation.
|
||||
|
||||
## Test Categories
|
||||
|
||||
### AC Tests (40 Acceptance Criteria — all covered)
|
||||
|
||||
1. **F-MAIL-14: Account CRUD** (7 tests)
|
||||
- test_list_accounts: GET /accounts -> 200, password not in response
|
||||
- test_create_account_password_encrypted: POST /accounts -> 201, AES-256 encrypted in DB
|
||||
- test_update_account: PATCH /accounts/{id} -> 200
|
||||
- test_list_shared_accounts: GET /accounts/shared -> 200 + shared mailboxes
|
||||
- test_assign_shared_users: POST /accounts/{id}/users -> 200
|
||||
- test_create_delegate: POST /accounts/{id}/delegates -> 200
|
||||
- test_create_send_permission: POST /accounts/{id}/send-permissions -> 200
|
||||
|
||||
2. **F-MAIL-01, F-MAIL-19: Folders** (4 tests)
|
||||
- test_list_folders: GET /folders?account_id=X -> 200 + folder list with counts
|
||||
- test_create_folder: POST /folders -> 201
|
||||
- test_update_folder: PATCH /folders/{id} -> 200, renamed
|
||||
- test_delete_folder: DELETE /folders/{id} -> 204
|
||||
|
||||
3. **F-MAIL-01: Mail List & Detail** (2 tests)
|
||||
- test_list_mails: GET /mail?folder_id=X&page=1 -> 200 + paginated
|
||||
- test_get_mail_detail: GET /mail/{id} -> 200 + detail (body_html sanitized)
|
||||
|
||||
4. **F-MAIL-02: Send/Reply/Forward** (3 tests)
|
||||
- test_send_mail: POST /mail/send -> 200, sent via SMTP (mocked)
|
||||
- test_reply_mail: POST /mail/{id}/reply -> 200, In-Reply-To header
|
||||
- test_forward_mail: POST /mail/{id}/forward -> 200
|
||||
|
||||
5. **F-MAIL-09: Flags** (1 test)
|
||||
- test_update_flags: PATCH /mail/{id}/flags -> 200
|
||||
|
||||
6. **F-MAIL-04: Attachments** (1 test)
|
||||
- test_download_attachment: GET /mail/{id}/attachments/{att_id} -> 200 + stream
|
||||
|
||||
7. **F-MAIL-10: Contact Linking** (1 test)
|
||||
- test_link_mail: POST /mail/{id}/link -> 200
|
||||
|
||||
8. **F-MAIL-11: Create Event** (1 test)
|
||||
- test_create_event_from_mail: POST /mail/{id}/create-event -> 200
|
||||
|
||||
9. **F-MAIL-03: Search** (1 test)
|
||||
- test_search_mails: GET /mail/search?q=text -> 200 + results
|
||||
|
||||
10. **F-MAIL-05: Threading** (1 test)
|
||||
- test_threads: GET /mail/threads -> 200 + threaded view
|
||||
|
||||
11. **F-MAIL-06: Templates** (2 tests)
|
||||
- test_create_template: POST /templates -> 201
|
||||
- test_list_templates: GET /templates -> 200
|
||||
|
||||
12. **F-MAIL-13: Signatures** (2 tests)
|
||||
- test_create_signature: POST /signatures -> 201
|
||||
- test_list_signatures: GET /signatures -> 200
|
||||
|
||||
13. **F-MAIL-07: Rules** (3 tests)
|
||||
- test_create_rule: POST /rules -> 201
|
||||
- test_list_rules: GET /rules -> 200 + sorted by priority
|
||||
- test_delete_rule: DELETE /rules/{id} -> 204
|
||||
|
||||
14. **F-MAIL-08: Vacation** (2 tests)
|
||||
- test_vacation_config: POST /vacation -> 200
|
||||
- test_vacation_dedup: second auto-reply within 24h -> not sent
|
||||
|
||||
15. **F-MAIL-12: PGP** (2 tests)
|
||||
- test_import_pgp_key: POST /pgp/keys -> 201
|
||||
- test_contact_pgp_key: POST /contacts/{id}/pgp-key -> 201
|
||||
|
||||
16. **F-MAIL-09: Labels** (2 tests)
|
||||
- test_create_label: POST /labels -> 201
|
||||
- test_assign_label: POST /mail/{id}/labels -> 200
|
||||
|
||||
17. **F-MAIL-01: IMAP Sync** (1 test)
|
||||
- test_imap_sync: mails fetched and stored (mocked)
|
||||
|
||||
18. **F-MAIL-07: Rule Engine** (1 test)
|
||||
- test_rule_engine: matching condition -> action executed
|
||||
|
||||
19. **HTML Sanitization** (1 test)
|
||||
- test_html_sanitization: no script tags in body_html_sanitized
|
||||
|
||||
20. **Password Security** (1 test)
|
||||
- test_password_never_in_response: password never in any API response
|
||||
|
||||
21. **F-MAIL-15,16: Shared Mailbox Permissions** (1 test)
|
||||
- test_shared_mailbox_read_only: delegated user read -> 403 on DELETE
|
||||
|
||||
### Unit Tests (6 tests)
|
||||
- test_encrypt_decrypt_password: AES-256 roundtrip
|
||||
- test_sanitize_html_removes_script: nh3 removes script tags
|
||||
- test_sanitize_html_removes_onerror: nh3 removes onerror attributes
|
||||
- test_template_substitution: {{placeholder}} substitution
|
||||
- test_thread_id_computation: References/In-Reply-To threading
|
||||
- test_rule_condition_matching: condition matching logic
|
||||
|
||||
## Smoke Test
|
||||
- All 33 calendar integration tests pass (API endpoints tested via HTTPX)
|
||||
- All 36 recurrence unit tests pass (direct function tests)
|
||||
- No stubs, no TODOs, all code is functional
|
||||
|
||||
- App starts with MailPlugin registered
|
||||
- Plugin install + activate works via API
|
||||
- All 19 features (F-MAIL-01 to F-MAIL-19) have corresponding API endpoints
|
||||
- Password encryption (AES-256 via Fernet) verified
|
||||
- HTML sanitization (nh3) removes script tags and dangerous attributes
|
||||
- Vacation dedup logic works (24h window)
|
||||
- PGP key import (pgpy) generates key_id and public key
|
||||
- Rule engine matches conditions and executes actions
|
||||
- Shared mailbox delegate with read access gets 403 on delete
|
||||
|
||||
## Build Verification
|
||||
|
||||
```
|
||||
python3 -m py_compile app/plugins/builtins/mail/models.py # OK
|
||||
python3 -m py_compile app/plugins/builtins/mail/schemas.py # OK
|
||||
python3 -m py_compile app/plugins/builtins/mail/services.py # OK
|
||||
python3 -m py_compile app/plugins/builtins/mail/routes.py # OK
|
||||
python3 -m py_compile app/plugins/builtins/mail/plugin.py # OK
|
||||
python3 -m py_compile app/plugins/builtins/mail/__init__.py # OK
|
||||
```
|
||||
|
||||
## Known Issues
|
||||
|
||||
- Coverage at 74.56% (target 80%) due to IMAP sync and SMTP send requiring live mail servers
|
||||
- IMAP sync code is fully implemented but only testable with mocks
|
||||
- SMTP send code is fully implemented but only testable with mocks
|
||||
- `imghdr` Python 3.13 shim created at `/opt/venv/lib/python3.13/site-packages/imghdr.py` for pgpy compatibility
|
||||
|
||||
Reference in New Issue
Block a user