odoo_mqtt/DOCUMENTATION_STRATEGY.md
matthias.lotz ba588842ad feat: Add automatic API documentation generation and device status monitoring
- Implement build script (build_docs.py) with AST parser to auto-generate HTML docs from docstrings
- Add comprehensive Google-style docstrings to all controllers and models
- Create static/description/index.html for Odoo Apps UI with module overview
- Generate api_reference.html (20.5 KB) from source code, linked from Odoo UI
- Add DOCUMENTATION_STRATEGY.md with comparison of 5 documentation approaches
- Create API.md with complete REST API documentation

Device Status Monitoring:
- Implement device_status_monitor.py with health checks and offline detection
- Add /status endpoint for device health overview
- Automatic offline detection after message_timeout_s

Config Push Architecture:
- Add POST /config endpoint to IoT Bridge for dynamic device management
- Auto-push device config from Odoo on create/write/unlink
- Implement device_manager.py for runtime device updates

E2E Tests:
- All 6 E2E tests passing (Create, Update, Push, Delete, Restart, Status Monitor)
- Test coverage for device lifecycle and config synchronization

Documentation is auto-generated via: ./build_docs.sh
View in Odoo: Settings → Apps → Open Workshop MQTT → API Reference
2026-02-15 11:03:22 +01:00

820 lines
24 KiB
Markdown

# API Documentation Strategy
## Best Practices für IoT Bridge ↔ Odoo Schnittstellen
**Stand:** 2026-02-15
**Status:** ✅ Beide APIs funktionstüchtig, Dokumentation verbesserungswürdig
---
## 📋 Übersicht der Schnittstellen
```
┌──────────────┐ ┌──────────────────┐
│ Odoo │ POST /config │ IoT Bridge │
│ │ ────────────────────────> │ │
│ │ (Device Config Push) │ │
│ │ │ │
│ │ GET /health │ │
│ │ ────────────────────────> │ │
│ │ (Health Check) │ │
└──────────────┘ └──────────────────┘
┌──────────────┐ ┌──────────────────┐
│ IoT Bridge │ POST /ows/iot/event │ Odoo │
│ │ ────────────────────────> │ │
│ │ (Event Submission) │ │
│ │ JSON-RPC 2.0 │ │
└──────────────┘ └──────────────────┘
```
### API 1: **IoT Bridge Config API** (Odoo → Bridge)
- **Technologie:** FastAPI (Python)
- **Dokumentation:** ✅ Automatisch via OpenAPI/Swagger
- **Endpunkte:** `POST /config`, `GET /config`, `GET /health`
- **Zugriff:** http://localhost:8080/docs
### API 2: **Odoo IoT Event API** (Bridge → Odoo)
- **Technologie:** Odoo HTTP Controller (JSON-RPC)
- **Dokumentation:** ⚠️ Manuell zu erstellen
- **Endpunkt:** `POST /ows/iot/event`
- **Format:** JSON-RPC 2.0
---
## 🎯 Empfohlene Dokumentationsstrategie
### Prinzipien
1. **Single Source of Truth** - Code ist die Quelle, Dokumentation wird daraus generiert
2. **Living Documentation** - Tests als ausführbare Beispiele
3. **Developer-First** - Fokus auf Nutzbarkeit für Entwickler
4. **Automatisierung** - Swagger/OpenAPI wo möglich
5. **Versionierung** - API-Versionen klar kennzeichnen
### Werkzeuge
| Aspekt | Bridge API | Odoo API | Tool |
|--------|-----------|----------|------|
| Schema Definition | ✅ Automatisch | ⚠️ Manuell | Pydantic / Python Docstrings |
| Interactive Docs | ✅ Swagger UI | ❌ Fehlt | FastAPI built-in / Postman Collection |
| Code Examples | ✅ Tests | ✅ Tests | pytest / test_e2e_push_architecture.py |
| Changelog | ⚠️ Git only | ⚠️ Git only | CHANGELOG.md |
| Type Safety | ✅ Pydantic | ⚠️ Basic | -/- |
---
## 🔧 Implementierungsplan
### 1. IoT Bridge API (FastAPI) - ✅ Gut, kleinere Verbesserungen
**Bereits vorhanden:**
- ✅ OpenAPI Schema automatisch generiert
- ✅ Swagger UI: http://localhost:8080/docs
- ✅ ReDoc: http://localhost:8080/redoc
- ✅ Pydantic Models für Validation
- ✅ API.md mit Beispielen
**Verbesserungen:**
#### a) Erweitere FastAPI Endpoint-Dokumentation mit response_model und Examples
```python
# In config_server.py
@self.app.post(
"/config",
response_model=ConfigResponse,
summary="Receive device configuration from Odoo",
description="""
Receives full device configuration push from Odoo.
**Triggered automatically by Odoo when:**
- Device is created
- Device is updated (relevant fields changed)
- Device is deleted
- Manual push button clicked
**Side effects:**
- Validates config schema
- Computes device diff (added/updated/removed)
- Updates MQTT subscriptions
- Persists config to /data/config-active.yaml
""",
responses={
200: {
"description": "Configuration successfully applied",
"content": {
"application/json": {
"example": {
"status": "success",
"message": "Configuration applied",
"devices_configured": 2,
"added": ["device-123"],
"updated": ["device-456"],
"removed": [],
"timestamp": "2026-02-15T10:30:45.123456"
}
}
}
},
422: {
"description": "Validation error",
"content": {
"application/json": {
"example": {
"detail": [
{
"loc": ["body", "devices", 0, "session_config", "working_threshold_w"],
"msg": "working_threshold_w must be greater than standby_threshold_w",
"type": "value_error"
}
]
}
}
}
}
},
tags=["Configuration"]
)
async def receive_config(...):
...
```
#### b) Füge OpenAPI Metadata hinzu
```python
# In config_server.py __init__
self.app = FastAPI(
title="IoT Bridge Config API",
description="""
**Autonomous MQTT-to-Odoo Integration Service**
This API receives device configuration from Odoo and manages MQTT subscriptions.
## Features
- Push-based configuration (no polling)
- Dynamic MQTT subscription management
- Config persistence to /data/config-active.yaml
- Pydantic validation for type safety
## Workflow
1. Odoo pushes config via POST /config
2. Bridge validates and applies config
3. Bridge subscribes to MQTT topics
4. Bridge persists config for restart survival
## Authentication
Optional Bearer token via `BRIDGE_API_TOKEN` environment variable.
""",
version="1.0.0",
contact={
"name": "Open Workshop Team",
"email": "lotz.matthias@gmail.com"
},
license_info={
"name": "MIT",
"url": "https://opensource.org/licenses/MIT"
},
openapi_tags=[
{
"name": "Configuration",
"description": "Device configuration management"
},
{
"name": "Health",
"description": "Service health and monitoring"
}
]
)
```
#### c) Exportiere OpenAPI Schema in CI/CD
```bash
# In iot_bridge/Makefile oder CI-Pipeline
docs-export:
python3 -c "from config_server import ConfigServer; import json; \
server = ConfigServer(lambda x: None); \
print(json.dumps(server.app.openapi(), indent=2))" \
> docs/openapi.json
docs-serve:
cd docs && python3 -m http.server 8888
```
---
### 2. Odoo IoT Event API (JSON-RPC) - ⚠️ Braucht Dokumentation
**Status:** Funktioniert, aber keine strukturierte Dokumentation außer Docstrings
**Empfohlener Ansatz:**
#### a) Erstelle OpenAPI-ähnliches Dokument
```markdown
# Odoo IoT Event API Reference
**Base URL:** `http://localhost:9018`
**Endpoint:** `POST /ows/iot/event`
**Protocol:** JSON-RPC 2.0
**Authentication:** Odoo Session (Cookie) oder API Key
## Request Format (JSON-RPC 2.0)
All requests must use JSON-RPC 2.0 format:
{
"jsonrpc": "2.0",
"params": {
"event_uid": "550e8400-e29b-41d4-a716-446655440000",
"event_type": "session_started",
"device_id": "shellypmminig3-48f6eeb73a1c",
"session_id": "optional-session-uuid",
"timestamp": "2026-02-15T10:30:45.123456Z",
"payload": {
"power_w": 125.5,
"state": "working"
}
}
}
## Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| event_uid | String (UUID) | ✅ Yes | Unique event identifier (prevents duplicates) |
| event_type | String (Enum) | ✅ Yes | Event type (see below) |
| device_id | String | ✅ Yes | Device identifier (must exist in mqtt.device) |
| session_id | String | ❌ Optional | Session identifier (for session events) |
| timestamp | String (ISO 8601) | ✅ Yes | Event timestamp (UTC) |
| payload | Object | ❌ Optional | Device-specific data |
## Event Types
### device_online
Device came online (MQTT connection established).
**Effect:** Sets device.state = 'idle'
**Example:**
{
"jsonrpc": "2.0",
"params": {
"event_uid": "a1b2c3d4-...",
"event_type": "device_online",
"device_id": "shelly-pm-001",
"timestamp": "2026-02-15T10:00:00Z",
"payload": {}
}
}
### device_offline
Device went offline (MQTT timeout or disconnect).
**Effect:** Sets device.state = 'offline'
**Example:**
{
"jsonrpc": "2.0",
"params": {
"event_uid": "e5f6g7h8-...",
"event_type": "device_offline",
"device_id": "shelly-pm-001",
"timestamp": "2026-02-15T10:05:00Z",
"payload": {}
}
}
### session_started
Work session started (power crossed working threshold).
**Effect:**
- Sets device.state = 'active'
- Creates mqtt.session record
**Example:**
{
"jsonrpc": "2.0",
"params": {
"event_uid": "i9j0k1l2-...",
"event_type": "session_started",
"device_id": "shelly-pm-001",
"session_id": "session-abc123",
"timestamp": "2026-02-15T10:15:00Z",
"payload": {
"power_w": 125.5,
"state": "working"
}
}
}
### session_stopped
Work session ended (power dropped below standby threshold).
**Effect:**
- Sets device.state = 'idle'
- Updates mqtt.session.end_time
- Calculates session.duration
**Example:**
{
"jsonrpc": "2.0",
"params": {
"event_uid": "m3n4o5p6-...",
"event_type": "session_stopped",
"device_id": "shelly-pm-001",
"session_id": "session-abc123",
"timestamp": "2026-02-15T11:30:00Z",
"payload": {
"power_w": 3.2,
"total_duration_s": 4500
}
}
}
### session_heartbeat
Periodic update during active session.
**Effect:** Updates session statistics (avg_power, peak_power, etc.)
**Example:**
{
"jsonrpc": "2.0",
"params": {
"event_uid": "q7r8s9t0-...",
"event_type": "session_heartbeat",
"device_id": "shelly-pm-001",
"session_id": "session-abc123",
"timestamp": "2026-02-15T10:20:00Z",
"payload": {
"power_w": 110.3,
"avg_power_w": 115.2,
"peak_power_w": 130.1,
"state": "working"
}
}
}
## Response Format
### Success (201 Created)
{
"status": "success",
"event_id": 123,
"event_uid": "550e8400-e29b-41d4-a716-446655440000",
"timestamp": "2026-02-15T10:30:45.234567"
}
### Duplicate Event (409 Conflict)
{
"status": "duplicate",
"event_id": 122,
"event_uid": "550e8400-e29b-41d4-a716-446655440000",
"message": "Event already processed"
}
### Validation Error (400 Bad Request)
{
"status": "error",
"error": "Missing required fields: event_uid, device_id",
"code": 400
}
### Device Not Found (404 Not Found)
{
"status": "error",
"error": "Device not found: unknown-device-123",
"code": 404
}
## Idempotency
The API is **idempotent** via `event_uid`:
- Same `event_uid` submitted twice → 409 Conflict (second request ignored)
- Prevents duplicate event processing
- Bridge can safely retry failed submissions
## Rate Limiting
Currently **no rate limiting** implemented.
## Testing
### Manual Test (curl)
```bash
curl -X POST http://localhost:9018/ows/iot/event \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"params": {
"event_uid": "550e8400-e29b-41d4-a716-446655440000",
"event_type": "device_online",
"device_id": "testshelly-simulator",
"timestamp": "2026-02-15T10:30:45Z",
"payload": {}
}
}'
```
### Automated Tests
```bash
cd iot_bridge/tests
python3 test_e2e_push_architecture.py
```
## Changelog
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0 | 2026-02-15 | Initial implementation with JSON-RPC 2.0, event_uid for idempotency |
```
#### b) Füge Docstrings für IDEs hinzu
```python
# In extra-addons/open_workshop/open_workshop_mqtt/controllers/iot_api.py
@route('/ows/iot/event', type='json', auth='none', methods=['POST'], csrf=False, save_session=False)
def receive_iot_event(self, event_uid=None, event_type=None, device_id=None,
session_id=None, timestamp=None, payload=None, **kw):
"""
Receive IoT event from Bridge (JSON-RPC 2.0).
Args:
event_uid (str): Unique event identifier (UUID, required for idempotency)
event_type (str): Event type - one of:
- 'device_online': Device came online → state='idle'
- 'device_offline': Device went offline → state='offline'
- 'session_started': Session started → state='active', creates mqtt.session
- 'session_stopped': Session ended → state='idle', finalizes mqtt.session
- 'session_heartbeat': Periodic update during session
device_id (str): Device identifier (must exist in mqtt.device, required)
session_id (str, optional): Session identifier (required for session events)
timestamp (str): ISO 8601 timestamp (required)
payload (dict, optional): Device-specific data
Returns:
dict: Response with status, event_id, timestamp
Example Request (JSON-RPC 2.0):
{
"jsonrpc": "2.0",
"params": {
"event_uid": "550e8400-e29b-41d4-a716-446655440000",
"event_type": "session_started",
"device_id": "shellypmminig3-48f6eeb73a1c",
"session_id": "abc-123",
"timestamp": "2026-02-15T10:30:45.123456Z",
"payload": {"power_w": 125.5}
}
}
Example Response (Success):
{
"status": "success",
"event_id": 123,
"event_uid": "550e8400-e29b-41d4-a716-446655440000",
"timestamp": "2026-02-15T10:30:45.234567"
}
Example Response (Duplicate):
{
"status": "duplicate",
"event_id": 122,
"event_uid": "550e8400-e29b-41d4-a716-446655440000",
"message": "Event already processed"
}
HTTP Status Codes:
201: Event created successfully
409: Duplicate event_uid (idempotent retry)
400: Validation error (missing fields, invalid data)
404: Device not found
500: Internal server error
Side Effects:
- Creates iot.event record
- May update mqtt.device.state (online/offline/idle/active)
- May create/update mqtt.session record (for session events)
- Idempotent via event_uid (duplicate submissions return 409)
"""
# ... implementation
```
---
### 3. Postman Collection - Für beide APIs
**Vorteil:** Interaktive Dokumentation + Testing Tool
Erstelle Postman Collection mit:
- ✅ Alle Endpoints (Bridge + Odoo)
- ✅ Beispiel-Requests mit Variablen
- ✅ Pre-request Scripts (UUID Generation für event_uid)
- ✅ Tests (Assertions für Status Codes)
```json
{
"info": {
"name": "IoT Bridge Odoo Integration",
"description": "Complete API collection for IoT Bridge ↔ Odoo communication",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"variable": [
{
"key": "bridge_url",
"value": "http://localhost:8080"
},
{
"key": "odoo_url",
"value": "http://localhost:9018"
}
],
"item": [
{
"name": "Bridge API",
"item": [
{
"name": "Health Check",
"request": {
"method": "GET",
"url": "{{bridge_url}}/health"
}
},
{
"name": "Get Config",
"request": {
"method": "GET",
"url": "{{bridge_url}}/config"
}
},
{
"name": "Push Config (from Odoo)",
"request": {
"method": "POST",
"url": "{{bridge_url}}/config",
"header": [
{
"key": "Content-Type",
"value": "application/json"
}
],
"body": {
"mode": "raw",
"raw": "{\n \"devices\": [\n {\n \"device_id\": \"testshelly-simulator\",\n \"machine_name\": \"Test Machine\",\n \"mqtt_topic\": \"testshelly-simulator/status/pm1:0\",\n \"parser_type\": \"shelly_pm\",\n \"session_config\": {\n \"standby_threshold_w\": 5.0,\n \"working_threshold_w\": 50.0,\n \"start_debounce_s\": 3.0,\n \"stop_debounce_s\": 15.0,\n \"message_timeout_s\": 20.0,\n \"heartbeat_interval_s\": 300.0\n }\n }\n ]\n}"
}
}
}
]
},
{
"name": "Odoo API",
"item": [
{
"name": "Submit Device Online Event",
"request": {
"method": "POST",
"url": "{{odoo_url}}/ows/iot/event",
"header": [
{
"key": "Content-Type",
"value": "application/json"
}
],
"body": {
"mode": "raw",
"raw": "{\n \"jsonrpc\": \"2.0\",\n \"params\": {\n \"event_uid\": \"{{$guid}}\",\n \"event_type\": \"device_online\",\n \"device_id\": \"testshelly-simulator\",\n \"timestamp\": \"{{$isoTimestamp}}\",\n \"payload\": {}\n }\n}"
}
}
},
{
"name": "Submit Session Started Event",
"request": {
"method": "POST",
"url": "{{odoo_url}}/ows/iot/event",
"header": [
{
"key": "Content-Type",
"value": "application/json"
}
],
"body": {
"mode": "raw",
"raw": "{\n \"jsonrpc\": \"2.0\",\n \"params\": {\n \"event_uid\": \"{{$guid}}\",\n \"event_type\": \"session_started\",\n \"device_id\": \"testshelly-simulator\",\n \"session_id\": \"session-{{$timestamp}}\",\n \"timestamp\": \"{{$isoTimestamp}}\",\n \"payload\": {\n \"power_w\": 125.5,\n \"state\": \"working\"\n }\n }\n}"
}
}
}
]
}
]
}
```
**Nutzung:**
1. Importiere in Postman
2. Setze Environment Variables (bridge_url, odoo_url)
3. Führe Requests aus
4. Teile Collection mit Team (Export als JSON)
---
### 4. Tests als Living Documentation
**Bereits vorhanden:** `test_e2e_push_architecture.py`
**Verbesserungen:**
#### a) Füge docstrings zu Test-Funktionen hinzu
```python
def test_device_status_monitoring(odoo: OdooAPI, bridge: BridgeAPI):
"""
Test: Device Status Lifecycle (offline → idle → active → idle → offline)
Scenario:
1. Send device_online event → Device state = 'idle'
2. Send session_started event → Device state = 'active'
3. Send session_stopped event → Device state = 'idle'
4. Send device_offline event → Device state = 'offline'
Validates:
- JSON-RPC 2.0 format for Odoo event API
- event_uid requirement for idempotency
- State transitions via device.write() in controllers/iot_api.py
- Proper timestamp handling (ISO 8601)
Related:
- controllers/iot_api.py::receive_iot_event()
- models/mqtt_device.py::state field
- DOCUMENTATION_STRATEGY.md::Odoo IoT Event API
"""
# ... test implementation
```
#### b) Generiere HTML Test Report mit Beispielen
```bash
# In iot_bridge/tests/
pytest test_e2e_push_architecture.py \
--html=report.html \
--self-contained-html \
--capture=no
```
Zeigt:
- ✅ Welche Tests liefen
- ✅ Request/Response Examples aus den Tests
- ✅ Failure Details mit Stacktraces
---
## 📦 Deliverables (Was konkret erstellen?)
### Sofort (heute):
1. **✅ DOCUMENTATION_STRATEGY.md** (dieses Dokument)
- Übersicht der APIs
- Werkzeuge & Best Practices
- Implementierungsplan
2. **🔧 Erweitere config_server.py** (15 min)
- Füge OpenAPI Metadata hinzu (contact, license, tags)
- Erweitere endpoint docstrings mit examples
- Siehe Code-Snippet oben
3. **📝 ODOO_EVENT_API.md** erstellen (30 min)
- Vollständige Referenz für /ows/iot/event
- Alle Event Types dokumentiert
- Beispiele für jeden Event-Typ
- Siehe Template oben
### Kurzfristig (diese Woche):
4. **🧪 Test Docstrings erweitern** (20 min)
- Alle test_*() Funktionen dokumentieren
- Verlinke zu relevanten Code-Stellen
- Siehe Beispiel oben
5. **📮 Postman Collection erstellen** (45 min)
- Alle Endpoints beider APIs
- Environment Variables
- Pre-request Scripts für UUID/Timestamp
- Export als JSON im Repo
6. **📊 OpenAPI Export im CI/CD** (15 min)
- Script zum Exportieren von openapi.json
- Commit in Repo für Versionskontrolle
- Siehe Makefile-Snippet oben
### Mittelfristig (nächster Sprint):
7. **🔄 API Changelog einführen**
- CHANGELOG.md für Breaking Changes
- Semantic Versioning (1.0.0 → 1.1.0 → 2.0.0)
- Git Tags für API-Versionen
8. **🏷️ API Versioning Strategy**
- Entscheide: URL-basiert (/v1/event) oder Header-basiert?
- Implementiere Version Header: `X-API-Version: 1.0`
- Plane Deprecation Policy (wie lange Support für alte Version?)
9. **📈 API Monitoring Dashboard**
- Prometheus Metrics für beide APIs
- Grafana Dashboard mit:
- Request Rate (req/s)
- Error Rate (4xx/5xx)
- Latency (p50, p95, p99)
- Active Devices
---
## 🎨 Empfohlene Verzeichnisstruktur
```
iot_bridge/
├── API.md # ✅ Bereits vorhanden (Bridge API)
├── docs/
│ ├── openapi.json # 🆕 Exportiertes OpenAPI Schema
│ └── postman/
│ └── collection.json # 🆕 Postman Collection
└── tests/
└── test_e2e_push_architecture.py # ✅ Mit erweiterten Docstrings
extra-addons/open_workshop/open_workshop_mqtt/
├── API.md # 🆕 Odoo Event API Referenz
├── controllers/
│ └── iot_api.py # 🔧 Erweiterte Docstrings
└── docs/
└── postman/
└── collection.json # 🆕 Odoo API Beispiele
```
---
## ✅ Vorteile dieser Strategie
### Für Entwickler:
-**Automatische Swagger Docs** - kein manuelles Update nötig
-**Type Safety** - Pydantic Models verhindern Fehler
-**Executable Examples** - Tests zeigen echte Nutzung
-**IDE Support** - Docstrings → Autocomplete
### Für DevOps:
-**OpenAPI Schema** - kann in API Gateway integriert werden
-**Health Endpoint** - für Load Balancer / Monitoring
-**Postman Collection** - für Smoke Tests nach Deployment
### Für QA:
-**E2E Tests** - validieren komplette Workflows
-**Postman Collection** - für manuelle Explorative Tests
-**Test Reports** - HTML Report mit Beispielen
### Für neue Teammitglieder:
-**Swagger UI** - interaktiv API ausprobieren
-**Markdown Docs** - leicht zu lesen
-**Tests** - zeigen "How to use" Patterns
---
## 🚀 Quick Wins (Start hier!)
### 30-Minuten-Sprint:
```bash
# 1. Erstelle Odoo API Dokumentation
cd extra-addons/open_workshop/open_workshop_mqtt
cat > API.md << 'EOF'
[Inhalt von oben: "# Odoo IoT Event API Reference" ...]
EOF
# 2. Teste Swagger UI
curl http://localhost:8080/docs
# 3. Exportiere OpenAPI Schema
curl http://localhost:8080/openapi.json > iot_bridge/docs/openapi.json
# 4. Commit
git add .
git commit -m "docs: Add comprehensive API documentation for Bridge and Odoo APIs"
```
---
## 📚 Weiterführende Ressourcen
- [FastAPI Docs - Metadata and Doc URLs](https://fastapi.tiangolo.com/tutorial/metadata/)
- [OpenAPI Specification](https://swagger.io/specification/)
- [Postman Collections](https://learning.postman.com/docs/getting-started/creating-your-first-collection/)
- [Semantic Versioning](https://semver.org/)
- [API Versioning Best Practices](https://www.freecodecamp.org/news/rest-api-best-practices-rest-endpoint-design-examples/)
---
**Autor:** Open Workshop Team
**Letzte Aktualisierung:** 2026-02-15
**Status:** 🟢 Aktiv implementiert