Commit Graph

13 Commits

Author SHA1 Message Date
080a7275d9 Phase 3.3: Harden error handling and recovery patterns
- Add optional Odoo circuit-breaker for transient failures
- Unify timeout handling in Odoo and MQTT clients
- Improve transient error classification (timeout/connection/5xx/429)
- Add focused unit tests for recovery and circuit-breaker behavior
- Mark Phase 3.3 tasks as completed in optimization plan
2026-02-19 19:07:13 +01:00
ff4ef2f563 Phase 3: Complete type safety & logging unification (3.1-3.2)
Phase 3.1: Type Safety
- Add bridge_types.py for shared type aliases (EventDict, PowerWatts, Timestamp, DeviceID)
- Define protocols for callbacks and message parsers
- Strict type annotations on all core modules (session_detector, event_queue, device_manager)
- Fix Optional handling and type guards throughout codebase
- Achieve full mypy compliance: 0 errors across 47 source files

Phase 3.2: Logging Unification
- Migrate from stdlib logging to pure structlog across all runtime modules
- Convert all logs to structured event+fields format (snake_case event names)
- Remove f-string and printf-style logger calls
- Add contextvars support for per-request correlation
- Implement FastAPI middleware to bind request_id, http_method, http_path
- Propagate X-Request-ID header in responses
- Remove stdlib logging imports except setup layer (utils/logging.py)
- Ensure log-level consistency across all modules

Files Modified:
- iot_bridge/bridge_types.py (new) - Central type definitions
- iot_bridge/core/* - Type safety and logging unification
- iot_bridge/clients/* - Structured logging with request context
- iot_bridge/parsers/* - Type-safe parsing with structured logs
- iot_bridge/utils/logging.py - Pure structlog setup with contextvars
- iot_bridge/api/server.py - Added request correlation middleware
- iot_bridge/tests/* - Test fixtures updated for type safety
- iot_bridge/OPTIMIZATION_PLAN.md - Phase 3 status updated

Validation:
- mypy . → 0 errors (47 files)
- All unit tests pass
- Runtime behavior unchanged
- API response headers include X-Request-ID
2026-02-18 23:54:27 +01:00
4214a61b19 Complete phase 2 rest points: env overrides and unit tests 2026-02-18 23:30:33 +01:00
a4ea77e6b3 feat(phase2.4): add dependency injection container and factory wiring
Implemented Phase 2.4 (Dependency Injection Pattern):

- Added new dependencies module with DI container and runtime context
  - RuntimeContainer for injectable factories
  - RuntimeContext for resolved runtime objects
  - create_service_manager() factory
  - build_runtime_context() composition root
- Refactored main.py to use dependency container wiring
  - Main orchestration now resolves runtime via DI factories
  - Reduced direct constructor coupling in entrypoint
- Added unit tests for DI behavior with mocked dependencies
  - Verifies factory injection for service manager creation
  - Verifies runtime composition uses injected callables
- Updated optimization plan checkboxes for Phase 2.4

Validation:
- py_compile passed for new/changed files
- tests/unit/test_dependencies.py passed
- regression test test_event_queue::test_enqueue passed

Notes:
- Keeps existing runtime behavior unchanged
- Establishes clear composition root for future testability improvements
2026-02-18 23:17:22 +01:00
e675256e0e chore: Apply code quality tools (black, isort, ruff)
Automated code quality improvements:

Black (Code Formatter):
- Reformatted 30 files to match PEP 8 style
- Consistent line length (100 chars)
- Consistent quote style and spacing

isort (Import Sorter):
- Fixed import order in 30+ files
- Grouped imports: stdlib → third-party → first-party → local
- Consistent multi-line import formatting

Ruff (Linter):
- Auto-fixed 161 issues:
  * Modernized type hints (List → list, Optional[X] → X | None)
  * Removed unused imports (json, os, typing utilities)
  * Updated deprecated typing imports (typing.List → list)
  * Simplified type annotations with | operator (Python 3.10+)
- Remaining 20 issues (acceptable):
  * 9x unused-method-argument (callback signatures, cannot be changed)
  * 2x bare-except (intentional for resilience)
  * 2x unused-variable (will be addressed in Phase 2)
  * Minor simplification opportunities (not critical)

Code Quality Stats:
- 32 files modified
- +1749/-1589 lines (net cleanup)
- Code is now consistent and formatted
- Ready for gradual type safety improvements (Phase 3)

All changes are non-functional - pure formatting and import organization.
2026-02-18 22:20:25 +01:00
fdddba7469 Phase 1: Foundation & Quality Tools
Implemented comprehensive development infrastructure:

1.1 Code Quality Tools:
- Created pyproject.toml with configurations for:
  * Black (code formatter, line-length=100)
  * isort (import sorter, black profile)
  * mypy (type checker, lenient settings for gradual adoption)
  * pytest (test runner with coverage)
  * ruff (modern linter replacing flake8/pylint)
  * bandit (security scanner)
- Added .pre-commit-config.yaml with:
  * File hygiene hooks (trailing whitespace, EOF, YAML/JSON checks)
  * Code formatting (black, isort, ruff)
  * Security checks (bandit)
- Created requirements-dev.txt for development dependencies
- Added .gitignore for Python projects

1.2 Test Infrastructure:
- Created tests/conftest.py with shared fixtures:
  * Config fixtures (mqtt_config, odoo_config, bridge_config, etc.)
  * MQTT client mocks (mock_mqtt_client, mock_paho_mqtt_client)
  * Odoo client mocks (mock_odoo_client, mock_requests)
  * Device/parser fixtures (shelly_pm_message, mock_parser)
  * Session detector & event queue fixtures
- Created tests/helpers.py with test utilities:
  * assert_event_valid, assert_mqtt_topic_valid
  * create_mock_mqtt_message
  * wait_for_condition helper
- Created tests/fixtures/ with specialized fixtures:
  * mqtt_fixtures.py: MQTT message factory, simulated broker
  * config_fixtures.py: Config file factory, minimal/full configs
  * device_fixtures.py: Device events, session events, Shelly payloads

1.3 CI/CD Pipeline:
- Created .github/workflows/iot-bridge-test.yml:
  * Lint job: ruff, black, isort, bandit
  * Type check job: mypy
  * Test job: pytest on Python 3.10, 3.11, 3.12 with coverage
  * Integration test job: with Mosquitto MQTT broker service
  * Docker build test job
  * Quality gate job: ensures all checks pass

Documentation:
- Updated README.md with Development section:
  * Local setup instructions
  * Code quality tools usage
  * Testing commands
  * Updated project structure diagram

Benefits:
- Automated code quality checks via pre-commit hooks
- Comprehensive test fixtures reduce test boilerplate
- CI/CD pipeline catches issues early
- Matrix testing ensures Python 3.10-3.12 compatibility
- Foundation for gradual type safety improvements (Phase 3)

See OPTIMIZATION_PLAN.md Phase 1 for details.
2026-02-18 22:11:25 +01:00
a5929ed290 Phase 0: Reorganize directory structure for better code organization
- Create modular package structure (core/, clients/, parsers/, api/, config/, utils/)
- Move core business logic to core/ (session_detector, event_queue, device_manager)
- Move external clients to clients/ (mqtt_client, odoo_client)
- Split config.py into config/schema.py (dataclasses) and config/loader.py (I/O)
- Split config_server.py into api/server.py (FastAPI) and api/models.py (Pydantic)
- Create parsers/base.py with MessageParser protocol for extensible parser architecture
- Move utilities to utils/ (logging, status_monitor)
- Update all imports across project (main.py, tests)
- Add __init__.py to all packages with proper exports
- Update README.md with new project structure
- All git mv commands preserve file history

This reorganization improves:
- Code discoverability (clear package responsibilities)
- Maintainability (separation of concerns)
- Extensibility (protocols for parsers, clean API separation)
- Testing (isolated packages easier to mock/test)

See OPTIMIZATION_PLAN.md for full roadmap (Phase 0-5)
2026-02-18 21:48:14 +01:00
3eaf2e933d feat: IoT Bridge Health Monitoring & Dynamic MQTT Reconnection
Features:
- Added ows.mqtt.bridge model with health status monitoring
- Bridge list/form views with status indicators (Online/Offline/Unknown)
- Scheduled action for periodic health checks (every 2 minutes)
- Health metrics: devices count, subscriptions, last seen timestamp
- Manual health check button in UI
- Smart button linking brokers to bridge instance

Infrastructure:
- Added ows.mqtt.broker model for MQTT broker configurations
- Bridge-Broker relation via Many2one field
- Dynamic MQTT reconnection without container restart
- Robust startup: Bridge starts even when MQTT fails
- TLS reconfiguration limitation documented (paho-mqtt)

Technical Changes:
- Updated models to use @api.model_create_multi for Odoo 18
- Fixed view definitions: tree → list for Odoo 18 compatibility
- Removed mail.thread dependency (unnecessary chatter)
- Added ir.cron for automated health monitoring
- Security/ACL rules for bridge and broker models
- Menu integration under IoT/MQTT section

Documentation:
- Updated IMPLEMENTATION_PLAN.md with new features
- Updated Odoo module README with bridge/broker models
- iot_bridge/README.md already documents dynamic reconnection

Testing:
- All changes tested via Odoo module upgrade
- Health check endpoint verified: GET /health
- Bridge reconnection tested with broker config changes
2026-02-18 20:36:11 +01:00
d3a28fddb6 feat: benutzerfreundliche GUI für Device-Konfiguration mit Auto-Config-Push
Hauptänderungen:
─────────────
 Benutzerfreundliche GUI statt JSON-Eingabe
  • Abrechnungs-Intervall (Minuten) - klar verständlich statt heartbeat_interval_s
  • Einschalt-/Arbeits-Schwellen (Watt) - statt technischer Begriffe
  • Anlauf-/Abschalt-Verzögerung (Sekunden) - erklärt was es tut
  • Icons, Hilfe-Texte, Beispiele direkt am Feld

🔄 Auto-Config-Push an Bridge
  • write() Hook: Automatischer Push bei Feldänderungen
  • create() Hook: Push bei neuem Device
  • unlink() Hook: Push bei Device-Löschung
  • Retry-Logic: 3 Versuche mit exponential backoff
  • Manueller Button: 🔄 Push Config to Bridge

📊 Transparente Abrechnung
  • Duration-Breakdown: Total = Working + Standby + Idle
  • Alle Zeiten in Minuten (billing-friendly)
  • Idle-Zeit zeigt Overhead (Debounce/Timeout) transparent

🧹 Code-Cleanup
  • Deprecated write() Hook entfernt
  • Überflüssige _onchange_session_strategy() entfernt
  • Computed Field: strategy_config automatisch generiert
  • Validation mit verständlichen deutschen Fehlermeldungen

Technische Details:
──────────────────
Models (mqtt_device.py):
  • billing_interval_min, power_on_threshold_w, active_work_threshold_w,
    startup_delay_s, shutdown_delay_s, message_timeout_s
  • Computed: strategy_config = f(alle benutzerfreundlichen Felder)
  • relevant_fields erweitert für Auto-Push Trigger

Views (mqtt_device_views.xml):
  •  Leistungs-Schwellenwerte
  • ⏱️ Zeitsteuerung
  • 💰 Abrechnung mit Info-Box
  • Generierte Config (readonly) für Debugging

Sessions (mqtt_session.py):
  • idle_duration_s = max(0, total - working - standby)
  • *_duration_min Felder für alle Zeiten
  • Displays konvertiert: hours → minutes

Bridge (session_detector.py):
  • Payload standardisiert: power_w, state (konsistent in allen Events)
  • Heartbeat-Intervall aus Odoo billing_interval_min

Feature Request:
  • Implementierungsstatus aktualisiert (Phase 1+2 komplett)
  • Lessons Learned dokumentiert
  • Nächste Schritte definiert

 Tests bestätigt:
  • Events kommen im konfigurierten Abrechnungs-Intervall (1 Min)
  • Config-Push funktioniert automatisch
  • Duration-Breakdown ist transparent
  • GUI ist verständlich und hilfreich
2026-02-17 00:09:51 +01:00
ba588842ad feat: Add automatic API documentation generation and device status monitoring
- Implement build script (build_docs.py) with AST parser to auto-generate HTML docs from docstrings
- Add comprehensive Google-style docstrings to all controllers and models
- Create static/description/index.html for Odoo Apps UI with module overview
- Generate api_reference.html (20.5 KB) from source code, linked from Odoo UI
- Add DOCUMENTATION_STRATEGY.md with comparison of 5 documentation approaches
- Create API.md with complete REST API documentation

Device Status Monitoring:
- Implement device_status_monitor.py with health checks and offline detection
- Add /status endpoint for device health overview
- Automatic offline detection after message_timeout_s

Config Push Architecture:
- Add POST /config endpoint to IoT Bridge for dynamic device management
- Auto-push device config from Odoo on create/write/unlink
- Implement device_manager.py for runtime device updates

E2E Tests:
- All 6 E2E tests passing (Create, Update, Push, Delete, Restart, Status Monitor)
- Test coverage for device lifecycle and config synchronization

Documentation is auto-generated via: ./build_docs.sh
View in Odoo: Settings → Apps → Open Workshop MQTT → API Reference
2026-02-15 11:03:22 +01:00
f1af17fd39 docs: complete phase 4 documentation 2026-02-12 21:12:02 +01:00
19be0903d2 feat: Implement HTTP Config API for Bridge (Phase 3.1)
Implements PUSH architecture - Bridge receives config from Odoo via HTTP

NEW FEATURES:
- HTTP Config Server (FastAPI on port 8080)
  - POST /config: Receive device configuration from Odoo
  - GET /config: Query currently active configuration
  - GET /health: Health check endpoint with device statistics
  - Optional Bearer token authentication (BRIDGE_API_TOKEN)

- Config Validation (Pydantic models)
  - BridgeConfig, DeviceConfig, SessionConfig schemas
  - Validation: unique device_id, unique mqtt_topic
  - Threshold validation: working_threshold_w > standby_threshold_w

- Dynamic Device Management (DeviceManager)
  - Device diff detection (add/update/remove)
  - Automatic MQTT subscribe/unsubscribe for topic changes
  - SessionDetector lifecycle management
  - Active session closing on device removal

- Config Persistence
  - Saves received config to /data/config-active.yaml
  - Bridge startup: loads config-active.yaml as fallback
  - Docker volume iot-bridge-data:/data for persistence

NEW FILES:
- iot_bridge/config_server.py          - FastAPI HTTP server
- iot_bridge/device_manager.py         - Device lifecycle management
- iot_bridge/tests/test-config-push.sh - Integration test script
- iot_bridge/tests/test-config-push.json - Test configuration

MODIFIED FILES:
- iot_bridge/main.py                   - Integrated HTTP server in thread
- iot_bridge/mqtt_client.py            - Added subscribe()/unsubscribe()
- iot_bridge/requirements.txt          - Added fastapi, uvicorn, pydantic
- iot_bridge/Dockerfile                - EXPOSE 8080, HTTP healthcheck
- odoo/docker-compose.dev.yaml         - Port 8080, volume /data

TESTING:
Verified via iot_bridge/tests/test-config-push.sh:
 Config push via HTTP works (200 OK)
 Old devices removed, new devices added
 MQTT topics dynamically updated (subscribe/unsubscribe)
 Config persisted to /data/config-active.yaml
 Health endpoint shows last_config_update timestamp
 Bridge restarts load config-active.yaml automatically

NEXT STEP:
Phase 3.2 - Implement Odoo-side config push (Model hooks)
2026-02-12 20:05:49 +01:00
807436df58 initial version 2026-02-10 20:00:27 +01:00