johanleroy/ENI-projet-piscine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(monitoring): supervise l'API, la base et l'hôte avec Prometheus et Grafana

Browse Source 
L'API exposait /metrics, mais aucun collecteur ne le lisait : monitoring/ ne contenait que des
.gitkeep.

Sous le profil Compose `monitoring` : prometheus, alertmanager, grafana, postgres-exporter,
node-exporter et cadvisor. Tous ont un mem_limit, pour environ 700 Mo au total sur la VM de 8 Go,
et leurs interfaces n'écoutent que sur 127.0.0.1. Le profil est actif en prod via
COMPOSE_PROFILES, donc à chaque déploiement, et se lance à la demande ailleurs
(make monitoring-up).

- Neuf règles d'alerte (API, base, hôte, cibles). Chacune a un cas dans les tests joués par
  `promtool test rules`, en CI comme par make monitoring-check.
- Alertmanager route les alertes par courriel vers Mailpit ; un critical masque le warning de la
  même cible.
- Grafana est provisionné : sources Prometheus et TimescaleDB, et trois tableaux de bord (API,
  données et dérive du modèle, infrastructure).
- Le rôle PostgreSQL `supervision` est en lecture seule sur les seules tables métier
  (db/roles/supervision.sql), posé par make db-ensure-supervision et par stack-up quand le
  profil est actif.
- Le jeton de /metrics passe à Prometheus en secret Compose (APP_METRICS_TOKEN) ;
  provision-host.sh génère ce secret et les deux autres.

Backend :
- un APP_METRICS_TOKEN vide vaut absent ;
- les sondes de santé ne comptent plus dans les métriques ;
- seaux de latence fins autour de 500 ms ;
- un registre Prometheus par application, sans quoi toute application créée après la première
  (dans les tests) ne mesurait rien.

Réf : #26
This commit is contained in:
Johan LEROY
2026-09-23 09:41:12 +02:00
parent 2ed9e1cee4
commit dc952d13aa
28 changed files with 2364 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
monitoring/README.md
+79 -7
View File
@@ -1,10 +1,82 @@
# Monitoring
# Supervision
Prometheus, Grafana et Alertmanager. Non initialise, voir le ticket dedie.
Prometheus, Alertmanager, Grafana et trois exporteurs, sous le profil Compose `monitoring`.
Issue #26, décisions dans l'ADR 0016, vue d'architecture dans
`docs/architecture/60-observabilite.md`.
- `prometheus` : configuration de collecte et regles d'alerte.
- `grafana/provisioning` : sources de donnees et fournisseurs de dashboards.
- `grafana/dashboards` : dashboards versionnes au format JSON.
- `alertmanager` : routage et inhibition des alertes.
| Service | Image | Rôle | Accès |
|---|---|---|---|
| `prometheus` | `prom/prometheus` | Collecte toutes les 15 s, évalue les règles, garde 15 jours (1 Go au plus) | `127.0.0.1:${PROMETHEUS_PORT:-9090}` |
| `alertmanager` | `prom/alertmanager` | Groupe les alertes et les envoie par courriel à Mailpit | `127.0.0.1:${ALERTMANAGER_PORT:-9093}` |
| `grafana` | `grafana/grafana` | Trois tableaux de bord provisionnés, dossier « EnerVision » | `127.0.0.1:${GRAFANA_PORT:-3001}` |
| `postgres-exporter` | `prometheuscommunity/postgres-exporter` | Connexions, transactions, taille des bases | réseau interne |
| `node-exporter` | `prom/node-exporter` | Processeur, mémoire et disque de l'hôte | réseau interne |
| `cadvisor` | `gcr.io/cadvisor/cadvisor` | Mémoire et processeur par conteneur | réseau interne |
Le backend expose deja ses metriques sur `/metrics` au format Prometheus.
Les interfaces n'écoutent que sur `127.0.0.1`. Depuis un poste, on passe par un tunnel SSH,
comme pour Airflow :
```bash
ssh -L 3001:127.0.0.1:3001 -L 9090:127.0.0.1:9090 enervision@10.101.200.37
```
## Démarrer
- **Prod.** `COMPOSE_PROFILES=monitoring` dans le `.env` : `make stack-up`, donc chaque
déploiement, démarre la supervision et pose le rôle `supervision` après les migrations.
- **Recette et poste.** À la demande, sur une stack déjà démarrée : `make monitoring-up`. Les
services partent en `--no-deps`, sans toucher aux autres.
Trois secrets sont requis, et `make stack-up` comme `make monitoring-up` refusent de démarrer
s'il en manque un. `scripts/provision-host.sh` les génère pour un nouvel environnement.
| Variable | Rôle |
|---|---|
| `APP_METRICS_TOKEN` | Jeton que Prometheus présente sur `/metrics`, et que l'API exige dès qu'il est posé |
| `GRAFANA_ADMIN_PASSWORD` | Compte `admin` de Grafana. Sans lui, le conteneur refuse de démarrer |
| `SUPERVISION_DB_PASSWORD` | Rôle PostgreSQL `supervision`, en lecture seule (`db/roles/supervision.sql`) |
L'API doit tourner en conteneur (`make stack-up`, ou `docker compose up -d backend`) :
Prometheus la joint en `backend:8000`, sur le réseau du projet. Une API lancée par `make dev`
sur l'hôte reste hors de sa portée, et l'alerte `ApiIndisponible` le signale.
## Tableaux de bord
| Tableau | Source | Contenu |
|---|---|---|
| EnerVision · API | Prometheus | Débit, erreurs 5xx, latences p50/p95/p99, globales et par route. C'est lui qu'on regarde pendant un tir k6 |
| EnerVision · Données et modèle | TimescaleDB | Fraîcheur des relevés par site, relevés ingérés, alertes par sévérité, dérive du modèle (`drift_report`) |
| EnerVision · Infrastructure | Prometheus | Hôte, mémoire et processeur par conteneur (recette et prod comprises), PostgreSQL |
Les fichiers JSON de `grafana/dashboards` sont la source : Grafana les recharge et refuse de
les modifier depuis l'interface. Pour changer un tableau, l'exporter en JSON depuis Grafana et
remplacer le fichier.
## Alertes
`prometheus/rules/enervision.yml` définit les règles, et `prometheus/tests/enervision.test.yml`
porte un cas par règle, joué par `promtool test rules`.
| Alerte | Condition | Sévérité |
|---|---|---|
| `ApiIndisponible` | `/metrics` injoignable pendant 2 min | critical |
| `ApiErreursServeur` | Plus de 5 % de 5xx sur 5 min | critical |
| `ApiLatenceElevee` | p95 au-delà d'une seconde pendant 10 min | warning |
| `BaseIndisponible` | Exportateur sans connexion pendant 2 min | critical |
| `BaseConnexionsSaturees` | Plus de 80 % de `max_connections` | warning |
| `HoteMemoireSaturee` | Mémoire au-delà de 90 % pendant 10 min | warning |
| `HoteDisquePlein` | Moins de 10 % libres sur `/` | critical |
| `HoteCpuSature` | Processeur au-delà de 90 % pendant 15 min | warning |
| `CibleInjoignable` | Un exporteur muet pendant 5 min | warning |
Alertmanager envoie les courriels à `supervision@enervision.fr` via Mailpit, qui les capture :
ils se lisent dans son interface. Un `critical` masque le `warning` de la même cible.
## Vérifier la configuration
```bash
make monitoring-check # promtool check config et test rules, amtool check-config, JSON des tableaux
```
La CI joue les mêmes commandes (job « Validation des fichiers Compose et de la supervision »)
dès que `monitoring/` ou un fichier Compose change.
monitoring/alertmanager/.gitkeep
View File
monitoring/alertmanager/alertmanager.yml
+31
View File
@@ -0,0 +1,31 @@
# Contrainte : Mailpit est le seul serveur SMTP de la stack, et il ne relaie rien vers l'extérieur
# - alertmanager.yml. Les alertes se lisent dans son interface (MAILPIT_UI_PORT), comme les
# courriels de réinitialisation de l'API.
global:
smtp_smarthost: mailpit:1025
smtp_from: supervision@enervision.fr
smtp_require_tls: false
route:
receiver: equipe
group_by: [alertname, severity]
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
routes:
- matchers: ['severity="critical"']
receiver: equipe
group_wait: 10s
repeat_interval: 1h
receivers:
- name: equipe
email_configs:
- to: supervision@enervision.fr
send_resolved: true
inhibit_rules:
- source_matchers: ['severity="critical"']
target_matchers: ['severity="warning"']
equal: [instance]
monitoring/grafana/dashboards/.gitkeep
View File
monitoring/grafana/dashboards/api.json
+578
View File
@@ -0,0 +1,578 @@
{
"uid": "enervision-api",
"title": "EnerVision · API",
"description": "Débit, erreurs et latences vus par l'API elle-même (prometheus-fastapi-instrumentator).",
"tags": [
"enervision"
],
"timezone": "browser",
"editable": false,
"graphTooltip": 1,
"refresh": "10s",
"schemaVersion": 39,
"version": 1,
"time": {
"from": "now-1h",
"to": "now"
},
"templating": {
"list": []
},
"annotations": {
"list": []
},
"links": [],
"panels": [
{
"id": 1,
"type": "stat",
"title": "API",
"gridPos": {
"x": 0,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "up{job=\"backend\"}",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "red",
"value": null
},
{
"color": "green",
"value": 1
}
]
},
"color": {
"mode": "thresholds"
},
"mappings": [
{
"type": "value",
"options": {
"0": {
"text": "Injoignable",
"color": "red"
},
"1": {
"text": "Joignable",
"color": "green"
}
}
}
]
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
},
"description": "Prometheus joint-il /metrics ?"
},
{
"id": 2,
"type": "stat",
"title": "Débit",
"gridPos": {
"x": 6,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum(rate(http_requests_total{job=\"backend\"}[5m]))",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "reqps",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "blue",
"value": null
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
}
},
{
"id": 3,
"type": "stat",
"title": "Erreurs 5xx",
"gridPos": {
"x": 12,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum(rate(http_requests_total{job=\"backend\", status=\"5xx\"}[5m])) / sum(rate(http_requests_total{job=\"backend\"}[5m]))",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "percentunit",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "orange",
"value": 0.01
},
{
"color": "red",
"value": 0.05
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
},
"description": "Seuil d'alerte : 5 % pendant 5 minutes."
},
{
"id": 4,
"type": "stat",
"title": "Latence p95",
"gridPos": {
"x": 18,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "histogram_quantile(0.95, sum by (le) (rate(http_request_duration_highr_seconds_bucket{job=\"backend\"}[5m])))",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "s",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "orange",
"value": 0.5
},
{
"color": "red",
"value": 1
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
},
"description": "Seuil de charge : 500 ms (ADR 0015). Alerte au-delà d'une seconde pendant 10 minutes."
},
{
"id": 5,
"type": "timeseries",
"title": "Débit par route",
"gridPos": {
"x": 0,
"y": 4,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum by (handler) (rate(http_requests_total{job=\"backend\"}[1m]))",
"legendFormat": "{{handler}}",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "reqps"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 6,
"type": "timeseries",
"title": "Latence de l'API",
"gridPos": {
"x": 12,
"y": 4,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "histogram_quantile(0.50, sum by (le) (rate(http_request_duration_highr_seconds_bucket{job=\"backend\"}[1m])))",
"legendFormat": "p50",
"range": true
},
{
"refId": "B",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "histogram_quantile(0.95, sum by (le) (rate(http_request_duration_highr_seconds_bucket{job=\"backend\"}[1m])))",
"legendFormat": "p95",
"range": true
},
{
"refId": "C",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "histogram_quantile(0.99, sum by (le) (rate(http_request_duration_highr_seconds_bucket{job=\"backend\"}[1m])))",
"legendFormat": "p99",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "s"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 7,
"type": "timeseries",
"title": "Latence p95 par route",
"gridPos": {
"x": 0,
"y": 12,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "histogram_quantile(0.95, sum by (le, handler) (rate(http_request_duration_seconds_bucket{job=\"backend\"}[5m])))",
"legendFormat": "{{handler}}",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "s"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"description": "Estimée sur les seaux 50 ms à 2,5 s de http_request_duration_seconds."
},
{
"id": 8,
"type": "timeseries",
"title": "Réponses par classe de statut",
"gridPos": {
"x": 12,
"y": 12,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum by (status) (rate(http_requests_total{job=\"backend\"}[1m]))",
"legendFormat": "{{status}}",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "reqps"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 9,
"type": "timeseries",
"title": "Mémoire du processus API",
"gridPos": {
"x": 0,
"y": 20,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "process_resident_memory_bytes{job=\"backend\"}",
"legendFormat": "résidente",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "bytes"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 10,
"type": "timeseries",
"title": "Requêtes en cours de traitement",
"gridPos": {
"x": 12,
"y": 20,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum(rate(http_request_duration_highr_seconds_sum{job=\"backend\"}[1m]))",
"legendFormat": "secondes de traitement par seconde",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "short"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"description": "Loi de Little : durée moyenne multipliée par le débit, soit le nombre moyen de requêtes simultanées."
}
]
}
monitoring/grafana/dashboards/donnees.json
+528
View File
@@ -0,0 +1,528 @@
{
"uid": "enervision-donnees",
"title": "EnerVision · Données et modèle",
"description": "Fraîcheur des relevés, alertes métier et dérive du modèle, lus dans TimescaleDB par le rôle supervision.",
"tags": [
"enervision"
],
"timezone": "browser",
"editable": false,
"graphTooltip": 1,
"refresh": "1m",
"schemaVersion": 39,
"version": 1,
"time": {
"from": "now-7d",
"to": "now"
},
"templating": {
"list": []
},
"annotations": {
"list": []
},
"links": [],
"panels": [
{
"id": 1,
"type": "stat",
"title": "Sites suivis",
"gridPos": {
"x": 0,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT count(*) AS \"Sites\" FROM site",
"format": "table",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "blue",
"value": null
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
}
},
{
"id": 2,
"type": "stat",
"title": "Relevés sur 24 h",
"gridPos": {
"x": 6,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT count(*) AS \"Relevés\" FROM reading WHERE timestamp > now() - interval '24 hours'",
"format": "table",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "red",
"value": null
},
{
"color": "green",
"value": 1
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
},
"description": "Zéro : l'import de la Mock API ne tourne plus."
},
{
"id": 3,
"type": "stat",
"title": "Alertes sur 24 h",
"gridPos": {
"x": 12,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT count(*) AS \"Alertes\" FROM alert WHERE timestamp > now() - interval '24 hours'",
"format": "table",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "orange",
"value": 1
},
{
"color": "red",
"value": 10
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
}
},
{
"id": 4,
"type": "stat",
"title": "Sites en dérive",
"gridPos": {
"x": 18,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT count(*) AS \"Sites\" FROM (SELECT DISTINCT ON (site_id) status FROM drift_report WHERE site_id IS NOT NULL ORDER BY site_id, computed_at DESC) AS derniers WHERE status = 'derive'",
"format": "table",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 1
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
},
"description": "Dernier rapport de dérive de chaque site (ADR 0013)."
},
{
"id": 5,
"type": "table",
"title": "Fraîcheur des relevés par site",
"gridPos": {
"x": 0,
"y": 4,
"w": 12,
"h": 8
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT s.site_name AS \"Site\", max(r.timestamp) AS \"Dernier relevé\", round(extract(epoch FROM now() - max(r.timestamp)) / 60) AS \"Âge (min)\"\nFROM site AS s\nLEFT JOIN reading AS r ON r.site_id = s.site_id AND r.timestamp > now() - interval '7 days'\nGROUP BY s.site_name\nORDER BY 3 DESC NULLS FIRST",
"format": "table",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {},
"overrides": []
},
"options": {
"showHeader": true
},
"description": "Un site sans relevé depuis sept jours apparaît vide, en tête de liste."
},
{
"id": 6,
"type": "table",
"title": "Dérive du modèle, dernier rapport",
"gridPos": {
"x": 12,
"y": 4,
"w": 12,
"h": 8
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT DISTINCT ON (d.site_id) coalesce(s.site_name, 'Tout le parc') AS \"Site\", d.status AS \"Statut\", round(d.mae::numeric, 2) AS \"MAE\", round(d.reference_mae::numeric, 2) AS \"MAE de référence\", round(d.bias::numeric, 2) AS \"Biais\", d.computed_at AS \"Calculé le\"\nFROM drift_report AS d\nLEFT JOIN site AS s ON s.site_id = d.site_id\nORDER BY d.site_id, d.computed_at DESC",
"format": "table",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {},
"overrides": []
},
"options": {
"showHeader": true
},
"description": "Une ligne par site, plus la ligne globale (ADR 0013)."
},
{
"id": 7,
"type": "timeseries",
"title": "Consommation par site",
"gridPos": {
"x": 0,
"y": 12,
"w": 12,
"h": 8
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT $__timeGroupAlias(r.timestamp, $__interval), s.site_name AS metric, avg(r.consumption_kw) AS value\nFROM reading AS r\nJOIN site AS s ON s.site_id = r.site_id\nWHERE $__timeFilter(r.timestamp)\nGROUP BY 1, 2\nORDER BY 1",
"format": "time_series",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"unit": "kwatt"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 8,
"type": "timeseries",
"title": "Relevés ingérés par heure",
"gridPos": {
"x": 12,
"y": 12,
"w": 12,
"h": 8
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT time_bucket('1 hour', timestamp) AS time, count(*) AS \"Relevés\"\nFROM reading\nWHERE $__timeFilter(timestamp)\nGROUP BY 1\nORDER BY 1",
"format": "time_series",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"unit": "short"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 9,
"type": "timeseries",
"title": "Alertes par sévérité",
"gridPos": {
"x": 0,
"y": 20,
"w": 12,
"h": 8
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT time_bucket('1 hour', timestamp) AS time, severity AS metric, count(*) AS value\nFROM alert\nWHERE $__timeFilter(timestamp)\nGROUP BY 1, 2\nORDER BY 1",
"format": "time_series",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"unit": "short"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 10,
"type": "timeseries",
"title": "Erreur absolue moyenne du modèle",
"gridPos": {
"x": 12,
"y": 20,
"w": 12,
"h": 8
},
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "grafana-postgresql-datasource",
"uid": "timescaledb"
},
"rawSql": "SELECT d.computed_at AS time, coalesce(s.site_name, 'Tout le parc') AS metric, d.mae AS value\nFROM drift_report AS d\nLEFT JOIN site AS s ON s.site_id = d.site_id\nWHERE $__timeFilter(d.computed_at)\nORDER BY 1",
"format": "time_series",
"rawQuery": true,
"editorMode": "code"
}
],
"fieldConfig": {
"defaults": {
"unit": "kwatt"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
}
]
}
monitoring/grafana/dashboards/infra.json
+583
View File
@@ -0,0 +1,583 @@
{
"uid": "enervision-infra",
"title": "EnerVision · Infrastructure",
"description": "Hôte (node-exporter), conteneurs (cAdvisor) et PostgreSQL (postgres-exporter). L'hôte porte la recette et la prod.",
"tags": [
"enervision"
],
"timezone": "browser",
"editable": false,
"graphTooltip": 1,
"refresh": "30s",
"schemaVersion": 39,
"version": 1,
"time": {
"from": "now-6h",
"to": "now"
},
"templating": {
"list": []
},
"annotations": {
"list": []
},
"links": [],
"panels": [
{
"id": 1,
"type": "stat",
"title": "Mémoire de l'hôte",
"gridPos": {
"x": 0,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "percentunit",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "orange",
"value": 0.8
},
{
"color": "red",
"value": 0.9
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
},
"description": "Alerte au-delà de 90 % pendant 10 minutes."
},
{
"id": 2,
"type": "stat",
"title": "Processeur de l'hôte",
"gridPos": {
"x": 6,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "1 - avg(rate(node_cpu_seconds_total{mode=\"idle\"}[5m]))",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "percentunit",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "orange",
"value": 0.7
},
{
"color": "red",
"value": 0.9
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
}
},
{
"id": 3,
"type": "stat",
"title": "Disque libre",
"gridPos": {
"x": 12,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "node_filesystem_avail_bytes{mountpoint=\"/\", fstype!~\"tmpfs|overlay\"} / node_filesystem_size_bytes{mountpoint=\"/\", fstype!~\"tmpfs|overlay\"}",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "percentunit",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "red",
"value": null
},
{
"color": "orange",
"value": 0.1
},
{
"color": "green",
"value": 0.2
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
}
},
{
"id": 4,
"type": "stat",
"title": "PostgreSQL",
"gridPos": {
"x": 18,
"y": 0,
"w": 6,
"h": 4
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "pg_up",
"legendFormat": "",
"range": true
}
],
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "red",
"value": null
},
{
"color": "green",
"value": 1
}
]
},
"color": {
"mode": "thresholds"
},
"mappings": [
{
"type": "value",
"options": {
"0": {
"text": "Injoignable",
"color": "red"
},
"1": {
"text": "Joignable",
"color": "green"
}
}
}
]
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "area",
"textMode": "auto"
}
},
{
"id": 5,
"type": "timeseries",
"title": "Mémoire par conteneur",
"gridPos": {
"x": 0,
"y": 4,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum by (name) (container_memory_working_set_bytes{name!=\"\"})",
"legendFormat": "{{name}}",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "bytes"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"description": "Recette et prod partagent l'hôte : leurs conteneurs se distinguent par le préfixe du projet Compose."
},
{
"id": 6,
"type": "timeseries",
"title": "Processeur par conteneur",
"gridPos": {
"x": 12,
"y": 4,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum by (name) (rate(container_cpu_usage_seconds_total{name!=\"\"}[5m]))",
"legendFormat": "{{name}}",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "short"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 7,
"type": "timeseries",
"title": "Mémoire de l'hôte",
"gridPos": {
"x": 0,
"y": 12,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes",
"legendFormat": "utilisée",
"range": true
},
{
"refId": "B",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "node_memory_MemTotal_bytes",
"legendFormat": "totale",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "bytes"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 8,
"type": "timeseries",
"title": "Connexions PostgreSQL par état",
"gridPos": {
"x": 12,
"y": 12,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum by (state) (pg_stat_activity_count)",
"legendFormat": "{{state}}",
"range": true
},
{
"refId": "B",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "max(pg_settings_max_connections)",
"legendFormat": "maximum",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "short"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 9,
"type": "timeseries",
"title": "Transactions validées par base",
"gridPos": {
"x": 0,
"y": 20,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "sum by (datname) (rate(pg_stat_database_xact_commit{datname!~\"template.*|postgres\"}[5m]))",
"legendFormat": "{{datname}}",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "ops"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
},
{
"id": 10,
"type": "timeseries",
"title": "Taille des bases",
"gridPos": {
"x": 12,
"y": 20,
"w": 12,
"h": 8
},
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"targets": [
{
"refId": "A",
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"expr": "pg_database_size_bytes{datname!~\"template.*|postgres\"}",
"legendFormat": "{{datname}}",
"range": true
}
],
"fieldConfig": {
"defaults": {
"unit": "bytes"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
}
}
]
}
monitoring/grafana/provisioning/dashboards/.gitkeep
View File
monitoring/grafana/provisioning/dashboards/dashboards.yml
+10
View File
@@ -0,0 +1,10 @@
apiVersion: 1
providers:
- name: enervision
folder: EnerVision
type: file
disableDeletion: true
allowUiUpdates: false
options:
path: /etc/grafana/dashboards
monitoring/grafana/provisioning/datasources/.gitkeep
View File
monitoring/grafana/provisioning/datasources/datasources.yml
+28
View File
@@ -0,0 +1,28 @@
# Pourquoi : Grafana lit la base avec le rôle `supervision`, en lecture seule sur les seules tables
# métier (db/roles/supervision.sql) - datasources.yml. Jamais le compte applicatif.
apiVersion: 1
datasources:
- name: Prometheus
uid: prometheus
type: prometheus
access: proxy
url: http://prometheus:9090
isDefault: true
editable: false
- name: TimescaleDB
uid: timescaledb
type: grafana-postgresql-datasource
access: proxy
url: db:5432
user: supervision
jsonData:
database: ${POSTGRES_DB}
sslmode: disable
postgresVersion: 1700
timescaledb: true
secureJsonData:
password: ${SUPERVISION_DB_PASSWORD}
editable: false
monitoring/prometheus/prometheus.yml
+43
View File
@@ -0,0 +1,43 @@
# Contrainte : Prometheus ne lit pas les variables d'environnement dans ce fichier - prometheus.yml.
# Le jeton de `/metrics` lui parvient en secret Compose (`metrics_token`, tiré d'APP_METRICS_TOKEN) ;
# vide, l'API n'en exige aucun (app/api/security.py).
global:
scrape_interval: 15s
evaluation_interval: 15s
rule_files:
- /etc/prometheus/rules/*.yml
alerting:
alertmanagers:
- static_configs:
- targets: ["alertmanager:9093"]
scrape_configs:
- job_name: backend
authorization:
type: Bearer
credentials_file: /run/secrets/metrics_token
static_configs:
- targets: ["backend:8000"]
- job_name: prometheus
static_configs:
- targets: ["localhost:9090"]
- job_name: alertmanager
static_configs:
- targets: ["alertmanager:9093"]
- job_name: postgres
static_configs:
- targets: ["postgres-exporter:9187"]
- job_name: node
static_configs:
- targets: ["node-exporter:9100"]
- job_name: cadvisor
static_configs:
- targets: ["cadvisor:8080"]
monitoring/prometheus/rules/.gitkeep
View File
monitoring/prometheus/rules/enervision.yml
+103
View File
@@ -0,0 +1,103 @@
# Contrainte : chaque règle a son cas dans tests/enervision.test.yml, joué par `promtool test
# rules` en CI - enervision.yml. Une règle modifiée sans son test fait échouer le job Infra.
# Pourquoi : `critical` réveille (mail immédiat, rappel toutes les heures), `warning` se lit le
# lendemain ; alertmanager.yml masque le warning d'une cible déjà en critical.
groups:
- name: api
rules:
- alert: ApiIndisponible
expr: up{job="backend"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: "L'API ne répond plus"
description: "Prometheus ne joint plus {{ $labels.instance }} depuis 2 minutes."
- alert: ApiErreursServeur
expr: |
sum(rate(http_requests_total{job="backend", status="5xx"}[5m]))
/ sum(rate(http_requests_total{job="backend"}[5m])) > 0.05
for: 5m
labels:
severity: critical
annotations:
summary: "Plus de 5 % de réponses 5xx"
description: "{{ $value | humanizePercentage }} des réponses de l'API sont des erreurs serveur."
- alert: ApiLatenceElevee
expr: |
histogram_quantile(0.95,
sum by (le) (rate(http_request_duration_highr_seconds_bucket{job="backend"}[5m]))
) > 1
for: 10m
labels:
severity: warning
annotations:
summary: "p95 de l'API au-dessus d'une seconde"
description: "Le p95 des réponses vaut {{ $value | humanizeDuration }} depuis 10 minutes."
- name: base
rules:
- alert: BaseIndisponible
expr: pg_up == 0
for: 2m
labels:
severity: critical
annotations:
summary: "PostgreSQL ne répond plus"
description: "L'exportateur ne se connecte plus à la base depuis 2 minutes."
- alert: BaseConnexionsSaturees
expr: |
sum by (instance) (pg_stat_activity_count)
/ max by (instance) (pg_settings_max_connections) > 0.8
for: 5m
labels:
severity: warning
annotations:
summary: "Connexions PostgreSQL au-delà de 80 %"
description: "{{ $value | humanizePercentage }} des connexions autorisées sont ouvertes."
- name: hote
rules:
- alert: HoteMemoireSaturee
expr: 1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes > 0.9
for: 10m
labels:
severity: warning
annotations:
summary: "Mémoire de l'hôte au-delà de 90 %"
description: "{{ $value | humanizePercentage }} de la mémoire est utilisée, recette et prod comprises."
- alert: HoteDisquePlein
expr: |
node_filesystem_avail_bytes{mountpoint="/", fstype!~"tmpfs|overlay"}
/ node_filesystem_size_bytes{mountpoint="/", fstype!~"tmpfs|overlay"} < 0.1
for: 10m
labels:
severity: critical
annotations:
summary: "Moins de 10 % de disque libre"
description: "Il reste {{ $value | humanizePercentage }} d'espace sur la racine de l'hôte."
- alert: HoteCpuSature
expr: 1 - avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) > 0.9
for: 15m
labels:
severity: warning
annotations:
summary: "Processeur de l'hôte au-delà de 90 %"
description: "Le processeur est occupé à {{ $value | humanizePercentage }} depuis 15 minutes."
- name: supervision
rules:
- alert: CibleInjoignable
expr: up{job!="backend"} == 0
for: 5m
labels:
severity: warning
annotations:
summary: "Cible de supervision injoignable"
description: "Prometheus ne joint plus {{ $labels.job }} ({{ $labels.instance }}) depuis 5 minutes."
monitoring/prometheus/tests/enervision.test.yml
+112
View File
@@ -0,0 +1,112 @@
rule_files:
- ../rules/enervision.yml
evaluation_interval: 1m
tests:
- name: l'API injoignable déclenche une alerte critique au bout de 2 minutes
interval: 1m
input_series:
- series: 'up{job="backend", instance="backend:8000"}'
values: "1 0 0 0 0"
alert_rule_test:
- eval_time: 2m
alertname: ApiIndisponible
exp_alerts: []
- eval_time: 3m
alertname: ApiIndisponible
exp_alerts:
- exp_labels:
severity: critical
job: backend
instance: backend:8000
exp_annotations:
summary: "L'API ne répond plus"
description: "Prometheus ne joint plus backend:8000 depuis 2 minutes."
- name: une API qui répond ne déclenche rien
interval: 1m
input_series:
- series: 'up{job="backend", instance="backend:8000"}'
values: "1x10"
alert_rule_test:
- eval_time: 10m
alertname: ApiIndisponible
exp_alerts: []
- name: dix pour cent de 5xx déclenchent l'alerte d'erreurs serveur
interval: 1m
input_series:
- series: 'http_requests_total{job="backend", handler="/api/v1/sites", method="GET", status="5xx"}'
values: "0+10x20"
- series: 'http_requests_total{job="backend", handler="/api/v1/sites", method="GET", status="2xx"}'
values: "0+90x20"
alert_rule_test:
- eval_time: 12m
alertname: ApiErreursServeur
exp_alerts:
- exp_labels:
severity: critical
exp_annotations:
summary: "Plus de 5 % de réponses 5xx"
description: "10% des réponses de l'API sont des erreurs serveur."
- name: un p95 au-delà d'une seconde déclenche l'alerte de latence
interval: 1m
input_series:
- series: 'http_request_duration_highr_seconds_bucket{job="backend", le="0.5"}'
values: "0x30"
- series: 'http_request_duration_highr_seconds_bucket{job="backend", le="1"}'
values: "0x30"
- series: 'http_request_duration_highr_seconds_bucket{job="backend", le="2"}'
values: "0+10x30"
- series: 'http_request_duration_highr_seconds_bucket{job="backend", le="+Inf"}'
values: "0+10x30"
alert_rule_test:
- eval_time: 20m
alertname: ApiLatenceElevee
exp_alerts:
- exp_labels:
severity: warning
exp_annotations:
summary: "p95 de l'API au-dessus d'une seconde"
description: "Le p95 des réponses vaut 1.95s depuis 10 minutes."
- name: la mémoire de l'hôte au-delà de 90 % déclenche un avertissement
interval: 1m
input_series:
- series: 'node_memory_MemAvailable_bytes{job="node", instance="node-exporter:9100"}'
values: "500000000x15"
- series: 'node_memory_MemTotal_bytes{job="node", instance="node-exporter:9100"}'
values: "10000000000x15"
alert_rule_test:
- eval_time: 12m
alertname: HoteMemoireSaturee
exp_alerts:
- exp_labels:
severity: warning
job: node
instance: node-exporter:9100
exp_annotations:
summary: "Mémoire de l'hôte au-delà de 90 %"
description: "95% de la mémoire est utilisée, recette et prod comprises."
- name: un exportateur muet déclenche l'alerte de cible, pas celle de l'API
interval: 1m
input_series:
- series: 'up{job="postgres", instance="postgres-exporter:9187"}'
values: "0x10"
alert_rule_test:
- eval_time: 6m
alertname: CibleInjoignable
exp_alerts:
- exp_labels:
severity: warning
job: postgres
instance: postgres-exporter:9187
exp_annotations:
summary: "Cible de supervision injoignable"
description: "Prometheus ne joint plus postgres (postgres-exporter:9187) depuis 5 minutes."
- eval_time: 6m
alertname: ApiIndisponible
exp_alerts: []