Observability
Logging
- Format: JSON structured
- Fields:
timestamp,level,service,trace_id,message,context - Aggregation: Loki или ELK
Пример:
{
"level": "info",
"service": "game-runtime",
"trace_id": "abc123",
"message": "action applied",
"game_id": "...",
"action_type": "choose_cell",
"duration_ms": 42
}
Metrics (Prometheus)
| Metric | Type | Labels |
|---|---|---|
http_request_duration_seconds |
histogram | service, path, status |
ws_connections_active |
gauge | instance |
sessions_active |
gauge | — |
game_actions_total |
counter | plugin_id, action_type |
nats_publish_errors |
counter | topic |
Dashboards (Grafana)
- Overview — RPS, error rate, active sessions
- Game — actions/min, apply latency p95
- Realtime — WS connections, event lag
- Bot — callback latency, edit_message errors
Tracing (OpenTelemetry)
- Propagate
traceparentHTTP header - Span:
action.apply,media.render,ws.push - Export: OTLP → Jaeger/Tempo
Alerting
| Alert | Condition | Severity |
|---|---|---|
| HighErrorRate | 5xx > 1% 5m | critical |
| WSDown | health fail 2m | critical |
| SlowActions | p95 > 500ms 10m | warning |
| RedisDown | ping fail | critical |
Audit log
- Table
game_actions— immutable player moves (payload,result_state,channel) - Table
game_events— lifecycle and rejected actions (game.started,game.finished,action.rejected, etc.) - Retention 90 days (NFR-027)
- Export for dispute resolution via admin panel (
/admin/games/{id}/log)
Admin log viewer
Platform operator accesses /admin in the web client after Telegram OTP login. See admin-panel.md.
NFR
NFR-024, NFR-025, NFR-026, NFR-027