Compare commits

..
Author SHA1 Message Date
Meryemel-gham fe100653d0 fix(etl): traite les retours de revue du DAG historique
Airflow / Lint et intégrité des DAGs (push) Successful in 57s
Airflow / Construction de l'image (push) Successful in 2m52s
SonarQube / build-back (push) Successful in 1m17s
SonarQube / test-ml (push) Failing after 1m50s
SonarQube / build-front (push) Successful in 9m44s
SonarQube / test-back (push) Failing after 55s
SonarQube / test-front (push) Failing after 5m3s
SonarQube / SonarQube (push) Skipped
2026-09-21 16:05:55 +02:00
Meryemel-gham cb4df846cb docs(etl): documente l'orchestration de l'import historique 2026-09-21 14:41:38 +02:00
Meryemel-gham ae4082c584 ci(etl): valide l'import historique dans l'image Airflow 2026-09-21 14:41:26 +02:00
Meryemel-gham f18d4f9ef9 test(etl): couvre le DAG d'import historique 2026-09-21 14:41:08 +02:00
Meryemel-gham 308769b325 feat(etl): orchestre l'import historique avec Airflow 2026-09-21 14:40:21 +02:00
Dorian 3c378c177f ci(ml,etl): analyse ml/ et etl/airflow dans SonarCloud avec un rapport de couverture ML 2026-09-21 14:12:38 +02:00
Dorian 2adfdf0eb0 fix(backend): supprime les vulnerabilites Sonar du Dockerfile et allege les tests d'exception 2026-09-21 14:08:04 +02:00
PhyriosandGitHub 44f3416ffe Merge branch 'main' into dev 2026-09-21 13:48:50 +02:00
Johan LEROYandGitHub 342128ccff Merge pull request #121 from ineszang/docs/livrables-ec03-ec06
docs(architecture,ml): vue CI/CD, ML-START.md et SAST Bandit
2026-09-21 13:45:46 +02:00
PhyriosandGitHub c3fd9327ea Definition des jalons 2026-09-14 11:30:01 +02:00
24 changed files with 538 additions and 309 deletions
+5 -4
View File
@@ -91,11 +91,12 @@ jobs:
bash -c "cd /opt/ml && env -u VIRTUAL_ENV uv run --no-sync python -m enervision_ml.train --help" bash -c "cd /opt/ml && env -u VIRTUAL_ENV uv run --no-sync python -m enervision_ml.train --help"
# `--help` sort par argparse avant `get_settings()` : ni base ni secret requis, et # `--help` sort par argparse avant `get_settings()` : ni base ni secret requis, et
# l'import du module prouve que l'environnement /opt/backend est complet. Les deux # l'import des modules prouve que l'environnement /opt/backend est complet.
# commandes du DAG `alertes` sont couvertes, `app.cli` tirant tout FastAPI derrière lui. # Les deux commandes du DAG `alertes` et la commande du DAG historique sont couvertes.
- name: Vérifie que les deux commandes du DAG alertes s'importent sans réseau - name: Vérifie que les trois commandes backend s'importent sans réseau
run: > run: >
docker run --rm --network none enervision-airflow:ci docker run --rm --network none enervision-airflow:ci
bash -c "cd /opt/backend bash -c "cd /opt/backend
&& env -u VIRTUAL_ENV uv run --no-sync python -m app.detection.internal_alerts --help && env -u VIRTUAL_ENV uv run --no-sync python -m app.detection.internal_alerts --help
&& env -u VIRTUAL_ENV uv run --no-sync python -m app.cli generate-recommendations --help" && env -u VIRTUAL_ENV uv run --no-sync python -m app.cli generate-recommendations --help
&& env -u VIRTUAL_ENV uv run --no-sync python -m app.etl.historical_import --help"
+38 -1
View File
@@ -5,11 +5,15 @@ on:
paths: paths:
- "apps/frontend/**" - "apps/frontend/**"
- "apps/backend/**" - "apps/backend/**"
- "ml/**"
- "etl/airflow/**"
- ".github/workflows/sonarqube.yml" - ".github/workflows/sonarqube.yml"
pull_request: pull_request:
paths: paths:
- "apps/frontend/**" - "apps/frontend/**"
- "apps/backend/**" - "apps/backend/**"
- "ml/**"
- "etl/airflow/**"
- ".github/workflows/sonarqube.yml" - ".github/workflows/sonarqube.yml"
@@ -108,8 +112,36 @@ jobs:
name: backend-coverage name: backend-coverage
path: apps/backend/coverage.xml path: apps/backend/coverage.xml
test-ml:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Installe uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
cache-dependency-glob: ml/uv.lock
- name: Installe l'interpréteur déclaré par .python-version
run: uv python install
working-directory: ml
- name: Synchronise les dépendances sans dévier du verrou
run: uv sync --all-groups --frozen
working-directory: ml
- name: Lancement des tests et génération du rapport de couverture (ML)
run: uv run pytest --cov-report=xml
working-directory: ml
- name: Upload coverage
uses: actions/upload-artifact@v4
with:
name: ml-coverage
path: ml/coverage.xml
sonarqube: sonarqube:
needs: [build-front, build-back, test-front, test-back] needs: [build-front, build-back, test-front, test-back, test-ml]
name: SonarQube name: SonarQube
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
@@ -126,6 +158,11 @@ jobs:
with: with:
name: backend-coverage name: backend-coverage
path: apps/backend path: apps/backend
- name: Téléchargement du rapport de couverture (ML)
uses: actions/download-artifact@v4
with:
name: ml-coverage
path: ml
- name: SonarQube Scan - name: SonarQube Scan
uses: SonarSource/sonarqube-scan-action@v8 uses: SonarSource/sonarqube-scan-action@v8
env: env:
+2 -2
View File
@@ -21,7 +21,7 @@ Ce que la documentation apporte à chacun : [docs/architecture/00-vue-ensemble.m
| Backend | FastAPI, Python 3.14 | `apps/backend` | Initialise | | Backend | FastAPI, Python 3.14 | `apps/backend` | Initialise |
| Frontend | Angular 22, Node 24 LTS | `apps/frontend` | Tableau de bord | | Frontend | Angular 22, Node 24 LTS | `apps/frontend` | Tableau de bord |
| Base | PostgreSQL 17 + TimescaleDB | `db` | Initialise | | Base | PostgreSQL 17 + TimescaleDB | `db` | Initialise |
| ETL | Apache Airflow | `etl/airflow` | Trois DAGs | | ETL | Apache Airflow | `etl/airflow` | Quatre DAGs |
| Infra | Terraform (k3s single-node) | `infra/terraform` | Initialise | | Infra | Terraform (k3s single-node) | `infra/terraform` | Initialise |
| Reverse proxy | Nginx, TLS | `infra/proxy` | En place | | Reverse proxy | Nginx, TLS | `infra/proxy` | En place |
| CI/CD | GitHub Actions | `.github/workflows` | Backend en place | | CI/CD | GitHub Actions | `.github/workflows` | Backend en place |
@@ -48,7 +48,7 @@ L'etat detaille de chaque brique et les vues d'architecture sont dans
│ ├── migrations/ Migrations SQL versionnees │ ├── migrations/ Migrations SQL versionnees
│ └── seeds/ Jeux de donnees de reference │ └── seeds/ Jeux de donnees de reference
├── etl/airflow/ ├── etl/airflow/
│ ├── dags/ DAGs d'orchestration (pipeline ML, alertes) │ ├── dags/ DAGs d'orchestration (pipeline ML, alertes, import historique)
│ ├── plugins/ Operateurs et hooks maison │ ├── plugins/ Operateurs et hooks maison
│ ├── include/ Requetes SQL et ressources des DAGs │ ├── include/ Requetes SQL et ressources des DAGs
│ └── tests/ Tests d'integrite des DAGs │ └── tests/ Tests d'integrite des DAGs
+5 -4
View File
@@ -11,13 +11,14 @@ WORKDIR /app
RUN --mount=type=cache,target=/root/.cache/uv \ RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=uv.lock,target=uv.lock \ --mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \ --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --locked --no-install-project --no-dev uv sync --locked --no-install-project --no-dev --no-build
# Le projet lui-meme n'est pas installe (pas de second `uv sync`) : il tourne depuis /app, le
# repertoire de travail, et rien ne lit ses metadonnees. L'installer imposerait de le construire
# (backend hatchling), donc de retirer `--no-build` de l'etape ci-dessus, qui garantit que
# l'installation des dependances n'execute aucun script de build (regle Sonar docker:S8541).
COPY . /app COPY . /app
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev
FROM python:3.14-slim AS runtime FROM python:3.14-slim AS runtime
+29 -19
View File
@@ -112,8 +112,10 @@ async def test_duplicate_reading_is_rejected_when_key_matches(
) )
await data_connection.execute(statement) await data_connection.execute(statement)
savepoint = data_connection.begin_nested()
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
async with data_connection.begin_nested(): async with savepoint:
await data_connection.execute(statement) await data_connection.execute(statement)
@@ -147,9 +149,12 @@ async def test_invalid_reading_is_rejected_when_constraints_fail(
} }
values.update(changes) values.update(changes)
statement = insert(Reading).values(**values)
savepoint = data_connection.begin_nested()
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
async with data_connection.begin_nested(): async with savepoint:
await data_connection.execute(insert(Reading).values(**values)) await data_connection.execute(statement)
async def test_prediction_requires_period_when_energy_is_predicted( async def test_prediction_requires_period_when_energy_is_predicted(
@@ -164,8 +169,10 @@ async def test_prediction_requires_period_when_energy_is_predicted(
model_reference="test-model/1", model_reference="test-model/1",
) )
savepoint = data_connection.begin_nested()
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
async with data_connection.begin_nested(): async with savepoint:
await data_connection.execute(statement) await data_connection.execute(statement)
@@ -212,21 +219,22 @@ async def test_alert_rejects_prediction_when_site_differs(
) )
).scalar_one() ).scalar_one()
statement = insert(Alert).values(
source_alert_id=str(uuid4()),
site_id=other_site,
source="enervision",
timestamp=MOMENT,
type="spike",
severity="high",
message="Test",
prediction_id=prediction_id,
raw_data={},
)
savepoint = data_connection.begin_nested()
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
async with data_connection.begin_nested(): async with savepoint:
await data_connection.execute( await data_connection.execute(statement)
insert(Alert).values(
source_alert_id=str(uuid4()),
site_id=other_site,
source="enervision",
timestamp=MOMENT,
type="spike",
severity="high",
message="Test",
prediction_id=prediction_id,
raw_data={},
)
)
async def test_recommendation_is_unique_when_alert_and_rule_match( async def test_recommendation_is_unique_when_alert_and_rule_match(
@@ -256,6 +264,8 @@ async def test_recommendation_is_unique_when_alert_and_rule_match(
) )
await data_connection.execute(statement) await data_connection.execute(statement)
savepoint = data_connection.begin_nested()
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
async with data_connection.begin_nested(): async with savepoint:
await data_connection.execute(statement) await data_connection.execute(statement)
@@ -106,13 +106,15 @@ def test_validate_source_accepts_valid_dataset():
def test_validate_source_rejects_missing_column(): def test_validate_source_rejects_missing_column():
frame = make_dataframe().drop(columns=["consumption_kwh"]) frame = make_dataframe().drop(columns=["consumption_kwh"])
metadata = make_metadata()
with pytest.raises( with pytest.raises(
ValueError, ValueError,
match="Colonnes obligatoires absentes", match="Colonnes obligatoires absentes",
): ):
validate_source( validate_source(
frame, frame,
make_metadata(), metadata,
) )
@@ -124,13 +126,15 @@ def test_validate_source_rejects_duplicates():
"timestamp", "timestamp",
] ]
metadata = make_metadata()
with pytest.raises( with pytest.raises(
ValueError, ValueError,
match="doublons", match="doublons",
): ):
validate_source( validate_source(
frame, frame,
make_metadata(), metadata,
) )
@@ -139,13 +143,15 @@ def test_validate_source_rejects_unknown_site():
frame.loc[1, "site_id"] = "SITE999" frame.loc[1, "site_id"] = "SITE999"
metadata = make_metadata()
with pytest.raises( with pytest.raises(
ValueError, ValueError,
match="Sites incohérents", match="Sites incohérents",
): ):
validate_source( validate_source(
frame, frame,
make_metadata(), metadata,
) )
@@ -49,8 +49,10 @@ async def test_the_database_refuses_to_mutate_the_audit_log(
) -> None: ) -> None:
await une_ligne(session) await une_ligne(session)
requete = text(instruction)
with pytest.raises(DBAPIError, match="ajout seul"): with pytest.raises(DBAPIError, match="ajout seul"):
await session.execute(text(instruction)) await session.execute(requete)
await session.rollback() await session.rollback()
@@ -131,11 +131,14 @@ async def test_the_database_refuses_two_tokens_sharing_a_fingerprint(
user_agent=None, user_agent=None,
) )
empreinte = fingerprint_refresh(secret)
expiration = datetime.now(UTC) + DUREE
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
await depot.create( await depot.create(
user_id=compte, user_id=compte,
token_hash=fingerprint_refresh(secret), token_hash=empreinte,
expires_at=datetime.now(UTC) + DUREE, expires_at=expiration,
client_ip=None, client_ip=None,
user_agent=None, user_agent=None,
) )
@@ -178,12 +178,16 @@ async def test_the_database_refuses_two_tokens_sharing_a_fingerprint(
user_agent=None, user_agent=None,
) )
famille = uuid.uuid4()
empreinte = fingerprint_refresh(secret)
expiration = datetime.now(UTC) + DUREE
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
await depot.create( await depot.create(
user_id=compte, user_id=compte,
family_id=uuid.uuid4(), family_id=famille,
token_hash=fingerprint_refresh(secret), token_hash=empreinte,
expires_at=datetime.now(UTC) + DUREE, expires_at=expiration,
client_ip=None, client_ip=None,
user_agent=None, user_agent=None,
) )
+5 -7
View File
@@ -31,14 +31,12 @@ async def test_the_database_refuses_an_email_written_in_upper_case(
) -> None: ) -> None:
saisie = adresse().upper() saisie = adresse().upper()
requete = text(
"insert into app_user (email, password_hash, role) values (:e, '$argon2id$x', 'lecteur')"
)
with pytest.raises(IntegrityError): with pytest.raises(IntegrityError):
await session.execute( await session.execute(requete, {"e": saisie})
text(
"insert into app_user (email, password_hash, role) "
"values (:e, '$argon2id$x', 'lecteur')"
),
{"e": saisie},
)
await session.rollback() await session.rollback()
+4 -6
View File
@@ -116,13 +116,11 @@ async def test_list_history_normalizes_naive_datetimes_to_utc() -> None:
async def test_list_history_raises_when_start_is_after_end() -> None: async def test_list_history_raises_when_start_is_after_end() -> None:
service = ReadingService(readings=FakeRepository([])) service = ReadingService(readings=FakeRepository([]))
debut = datetime(2026, 9, 2, tzinfo=UTC)
fin = datetime(2026, 9, 1, tzinfo=UTC)
with pytest.raises(FenetreInverseeError): with pytest.raises(FenetreInverseeError):
await service.list_history( await service.list_history(start=debut, end=fin, limit=500, offset=0)
start=datetime(2026, 9, 2, tzinfo=UTC),
end=datetime(2026, 9, 1, tzinfo=UTC),
limit=500,
offset=0,
)
async def test_list_history_raises_when_start_equals_end() -> None: async def test_list_history_raises_when_start_equals_end() -> None:
+3 -1
View File
@@ -235,5 +235,7 @@ async def test_every_operation_refuses_an_unknown_account(action: str) -> None:
if action == "set_active": if action == "set_active":
arguments["is_active"] = False arguments["is_active"] = False
methode = getattr(attirail.service, action)
with pytest.raises(UserNotFoundError): with pytest.raises(UserNotFoundError):
await getattr(attirail.service, action)(**arguments) await methode(**arguments)
+6 -2
View File
@@ -19,13 +19,17 @@ def test_build_parser_reads_the_create_admin_arguments() -> None:
def test_build_parser_requires_a_subcommand() -> None: def test_build_parser_requires_a_subcommand() -> None:
parser = cli.build_parser()
with pytest.raises(SystemExit): with pytest.raises(SystemExit):
cli.build_parser().parse_args([]) parser.parse_args([])
def test_build_parser_requires_an_email() -> None: def test_build_parser_requires_an_email() -> None:
parser = cli.build_parser()
with pytest.raises(SystemExit): with pytest.raises(SystemExit):
cli.build_parser().parse_args(["create-admin"]) parser.parse_args(["create-admin"])
def test_read_password_generates_a_long_secret_when_asked( def test_read_password_generates_a_long_secret_when_asked(
+1
View File
@@ -36,6 +36,7 @@ x-airflow-common: &airflow-common
volumes: volumes:
- ./etl/airflow/dags:/opt/airflow/dags - ./etl/airflow/dags:/opt/airflow/dags
- ./etl/airflow/plugins:/opt/airflow/plugins - ./etl/airflow/plugins:/opt/airflow/plugins
- ./data/raw:/opt/data/raw:ro
- airflow_logs:/opt/airflow/logs - airflow_logs:/opt/airflow/logs
- airflow_ml_state:/opt/ml/state - airflow_ml_state:/opt/ml/state
restart: unless-stopped restart: unless-stopped
+5 -4
View File
@@ -70,10 +70,11 @@ Le lien `front -.-> api` reste en pointillé : le frontend appelle bien une API,
intercepteur répond à sa place tant que les endpoints n'existent pas. Voir intercepteur répond à sa place tant que les endpoints n'existent pas. Voir
[30-frontend.md](30-frontend.md). [30-frontend.md](30-frontend.md).
Le lien `airflow --> db` est maintenant en trait plein : trois DAGs tournent, deux pour Le lien `airflow --> db` est maintenant en trait plein : quatre DAGs tournent, deux pour
l'entraînement et le scoring du modèle ML (issue #115), un pour la détection d'alertes et la l'entraînement et le scoring du modèle ML (issue #115), un pour la détection d'alertes et la
génération des recommandations (issue #116), cf. plus bas et [20-backend.md](20-backend.md). Le génération des recommandations (issue #116), et `historical_import` pour l'ingestion du dataset
reste du périmètre Airflow envisagé (ingestion, issues #15/#16) reste en pointillé, non construit. historique (issue #119). L'orchestration de l'import API Mock et la réconciliation globale des
deux sources restent à compléter dans l'issue #15.
Le lien `prom -.-> api` de même : l'API expose bien `/metrics` au format Prometheus, mais aucun Le lien `prom -.-> api` de même : l'API expose bien `/metrics` au format Prometheus, mais aucun
collecteur ne vient le lire. collecteur ne vient le lire.
@@ -88,7 +89,7 @@ collecteur ne vient le lire.
| ML | LightGBM, MLflow | `ml` | `En cours` | Pipeline d'entraînement et de scoring (`enervision_ml.train`/`.score`, features par lags/moyennes glissantes partagées entre les deux, baseline de persistance saisonnière, suivi MLflow local), exposé en lecture via `GET /predictions`, orchestré par Airflow (`ml_train`/`ml_score`). Voir [ADR 0005](../adr/0005-modele-prediction-lightgbm.md) et [ML-START.md](../ML-START.md). Surveillance de dérive (EC06, #44/#45) pas encore construite | | ML | LightGBM, MLflow | `ml` | `En cours` | Pipeline d'entraînement et de scoring (`enervision_ml.train`/`.score`, features par lags/moyennes glissantes partagées entre les deux, baseline de persistance saisonnière, suivi MLflow local), exposé en lecture via `GET /predictions`, orchestré par Airflow (`ml_train`/`ml_score`). Voir [ADR 0005](../adr/0005-modele-prediction-lightgbm.md) et [ML-START.md](../ML-START.md). Surveillance de dérive (EC06, #44/#45) pas encore construite |
| Infra | Docker Compose, Nginx, Terraform, k3s single-node | `infra`, `docker-compose.prod.yml` | `En cours` | Reverse proxy et overlay de déploiement écrits et validés, jamais lancés sur le serveur ([ADR 0007](../adr/0007-terminaison-tls-et-reverse-proxy-nginx.md)). Module d'installation k3s jamais appliqué, aucune ressource Kubernetes déclarée | | Infra | Docker Compose, Nginx, Terraform, k3s single-node | `infra`, `docker-compose.prod.yml` | `En cours` | Reverse proxy et overlay de déploiement écrits et validés, jamais lancés sur le serveur ([ADR 0007](../adr/0007-terminaison-tls-et-reverse-proxy-nginx.md)). Module d'installation k3s jamais appliqué, aucune ressource Kubernetes déclarée |
| Monitoring | Prometheus, Grafana, Alertmanager | `monitoring` | `Cible` | Rien, hors le `/metrics` exposé par l'API | | Monitoring | Prometheus, Grafana, Alertmanager | `monitoring` | `Cible` | Rien, hors le `/metrics` exposé par l'API |
| ETL | Apache Airflow | `etl/airflow` | `En cours` | Webserver + scheduler (LocalExecutor) tournent via docker-compose, base de métadonnées Postgres dédiée. Trois DAGs en sous-processus `uv run` : `ml_train` manuel et `ml_score` `@hourly` pour le pipeline ML (issue #115), `alertes` à `15 * * * *` pour la détection et les recommandations (issue #116, [ADR 0008](../adr/0008-airflow-execute-le-code-du-backend.md)). L'ingestion (issues #15/#16) n'a pas encore de DAG | | ETL | Apache Airflow | `etl/airflow` | `En cours` | Webserver + scheduler (LocalExecutor) tournent via docker-compose, base de métadonnées Postgres dédiée. Quatre DAGs en sous-processus `uv run` : `ml_train`, `ml_score`, `alertes` et `historical_import`. Le DAG historique orchestre `app.etl.historical_import` et charge `dataset`, `site` et `reading`. L'orchestration API Mock reste à compléter dans #15 |
| CI/CD | GitHub Actions | `.github/workflows` | `En cours` | 5 workflows, 16 jobs : lint, typage, tests avec seuil de couverture bloquant, tests d'intégration sur TimescaleDB réel, audit de dépendances, SAST Bandit, quality gate SonarCloud, intégrité des DAGs Airflow. Détail dans [50-cicd.md](50-cicd.md). **Aucun job de déploiement** (#21) | | CI/CD | GitHub Actions | `.github/workflows` | `En cours` | 5 workflows, 16 jobs : lint, typage, tests avec seuil de couverture bloquant, tests d'intégration sur TimescaleDB réel, audit de dépendances, SAST Bandit, quality gate SonarCloud, intégrité des DAGs Airflow. Détail dans [50-cicd.md](50-cicd.md). **Aucun job de déploiement** (#21) |
## Flux bout en bout ## Flux bout en bout
+7 -1
View File
@@ -51,7 +51,7 @@ Trois pièges sont documentés en tête du `docker-compose.yml`, ils ne se devin
- `LocalExecutor` exécute les tâches comme sous-processus du **scheduler**, jamais du webserver : - `LocalExecutor` exécute les tâches comme sous-processus du **scheduler**, jamais du webserver :
c'est le scheduler qui a besoin du volume `airflow_ml_state` (modèle, magasin MLflow). c'est le scheduler qui a besoin du volume `airflow_ml_state` (modèle, magasin MLflow).
### Airflow (issues #115 et #116) ### Airflow (issues #115, #116 et #119)
Trois services, `docker compose profiles` non utilisés (démarrage explicite via `make Trois services, `docker compose profiles` non utilisés (démarrage explicite via `make
airflow-up`, pas dans `make dev`) : airflow-up`, pas dans `make dev`) :
@@ -75,6 +75,12 @@ l'[ADR 0008](../adr/0008-airflow-execute-le-code-du-backend.md).
| `ml_train` | manuelle | `enervision_ml.train`, dans `/opt/ml/.venv` | | `ml_train` | manuelle | `enervision_ml.train`, dans `/opt/ml/.venv` |
| `ml_score` | `0 * * * *` | `enervision_ml.score`, dans `/opt/ml/.venv` | | `ml_score` | `0 * * * *` | `enervision_ml.score`, dans `/opt/ml/.venv` |
| `alertes` | `15 * * * *` | `app.detection.internal_alerts` puis `app.cli generate-recommendations`, dans `/opt/backend/.venv` | | `alertes` | `15 * * * *` | `app.detection.internal_alerts` puis `app.cli generate-recommendations`, dans `/opt/backend/.venv` |
| `historical_import` | manuelle | `app.etl.historical_import`, dans `/opt/backend/.venv` ; les fichiers de `data/raw` sont montés en lecture seule dans `/opt/data/raw` |
Le DAG `historical_import` réutilise le pipeline historique existant sans dupliquer sa logique.
Il reste manuel, car le dataset sert à initialiser l'environnement. Le montage
`./data/raw:/opt/data/raw:ro` permet au scheduler de lire les fichiers CSV/JSON sans pouvoir les
modifier.
**Pourquoi `alertes` tourne à la quinzième minute.** Sa règle `anomaly` compare une lecture à la **Pourquoi `alertes` tourne à la quinzième minute.** Sa règle `anomaly` compare une lecture à la
`prediction` du même instant, que `ml_score` écrit à l'heure pile. Le décalage laisse le scoring `prediction` du même instant, que `ml_score` écrit à l'heure pile. Le décalage laisse le scoring
+13 -4
View File
@@ -44,6 +44,7 @@ flowchart TB
subgraph sq["SonarQube · sonarqube.yml"] subgraph sq["SonarQube · sonarqube.yml"]
sb1["build-front / test-front"] sb1["build-front / test-front"]
sb2["build-back / test-back"] sb2["build-back / test-back"]
sb3["test-ml"]
sscan["sonarqube<br/>quality gate SonarCloud"] sscan["sonarqube<br/>quality gate SonarCloud"]
end end
@@ -132,10 +133,18 @@ partie de la suite, et son taux n'aurait aucun sens face au seuil de 85 %.
## SonarCloud, et l'incident qui a immobilisé trois PR ## SonarCloud, et l'incident qui a immobilisé trois PR
Le workflow `sonarqube.yml` exécute quatre jobs de préparation (`build-front`, `test-front`, Le workflow `sonarqube.yml` exécute cinq jobs de préparation (`build-front`, `test-front`,
`build-back`, `test-back`) qui produisent chacun un rapport de couverture en artefact, puis un `build-back`, `test-back`, `test-ml`) dont les tests produisent chacun un rapport de couverture en
cinquième job qui les télécharge et lance `SonarSource/sonarqube-scan-action@v8` avec le secret artefact, puis un dernier job qui les télécharge et lance `SonarSource/sonarqube-scan-action@v8`
`SONAR_TOKEN`. Le périmètre est décrit par `sonar-project.properties` à la racine. avec le secret `SONAR_TOKEN`. Le périmètre est décrit par `sonar-project.properties` à la racine.
Le périmètre couvre `apps/frontend`, `apps/backend`, `ml/` et `etl/airflow` (les deux derniers
ajoutés après coup : ils n'étaient pas analysés, une PR qui ne touchait qu'eux ne lançait pas
Sonar). `ml/` publie `ml/coverage.xml` (`pytest-cov`, même mécanisme que le backend, sans seuil
propre : la gate porte sur le code neuf). `etl/airflow` est exclu de la **couverture**
(`sonar.coverage.exclusions`) : ses tests ne font que charger les DAGs, ils ne mesurent rien.
Piège : tout nouveau dossier de tests doit être déclaré dans `sonar.tests`, faute de quoi il est
compté comme code de production non couvert (cf. l'incident ci-dessous).
**L'incident, à raconter tel quel.** Les 18 et 19 septembre, trois PR (#103, #105, #107) sont **L'incident, à raconter tel quel.** Les 18 et 19 septembre, trois PR (#103, #105, #107) sont
restées bloquées sur une quality gate rouge annonçant une couverture du code neuf à 0 %, alors que restées bloquées sur une quality gate rouge annonçant une couverture du code neuf à 0 %, alors que
+10 -2
View File
@@ -663,8 +663,16 @@ mock_api_import.py
La logique d'extraction, de transformation et de chargement est donc disponible pour les deux sources de données du MVP. La logique d'extraction, de transformation et de chargement est donc disponible pour les deux sources de données du MVP.
Airflow tourne désormais réellement (`etl/airflow/`, `make airflow-up`) et orchestre le pipeline ML (`ml_train`/`ml_score`, issue #115) ainsi que la détection d'alertes et la génération des recommandations (`alertes`, issue #116). Il n'orchestre pas encore ces deux imports : `historical_import.py` et `mock_api_import.py` (normalisation et chargement micro-batch, issues #15/#16) restent à faire. Airflow tourne désormais réellement (`etl/airflow/`, `make airflow-up`) et orchestre le pipeline
ML (`ml_train`/`ml_score`, issue #115), la détection d'alertes et la génération des
recommandations (`alertes`, issue #116), ainsi que l'import historique
(`historical_import`, issue #119).
Airflow permet de planifier les traitements, gérer leur ordre d'exécution, suivre leur état et remonter les erreurs. Il ne remplace pas la logique ETL Python existante : les scripts actuels restent responsables de l'extraction, de la validation, de la transformation et du chargement. `etl/airflow/dags/ml_train.py`, `ml_score.py` et `alertes.py` montrent le patron retenu (des `BashOperator` qui invoquent le script tel quel, dans l'environnement `uv` que l'image embarque pour lui). Le DAG `historical_import` est déclenché manuellement. Il exécute
`app.etl.historical_import` avec les fichiers montés en lecture seule depuis `data/raw` vers
`/opt/data/raw`. L'orchestration de l'import API Mock et la réconciliation globale des deux
sources restent couvertes par l'issue #15.
Airflow permet de planifier les traitements, gérer leur ordre d'exécution, suivre leur état et remonter les erreurs. Il ne remplace pas la logique ETL Python existante : les scripts actuels restent responsables de l'extraction, de la validation, de la transformation et du chargement. `etl/airflow/dags/ml_train.py`, `ml_score.py` et `alertes.py` et `historical_import.py` montrent le patron retenu (des `BashOperator` qui invoquent le script tel quel, dans l'environnement `uv` que l'image embarque pour lui).
Le pipeline Data servira ensuite à préparer les données nécessaires au modèle de Machine Learning. Le pipeline Data servira ensuite à préparer les données nécessaires au modèle de Machine Learning.
+45
View File
@@ -0,0 +1,45 @@
"""DAG d'import du dataset historique EnerVision (issue #119).
Orchestre le pipeline existant `app.etl.historical_import` sans dupliquer sa logique ETL.
Le dataset historique sert à initialiser l'environnement : le DAG reste donc manuel.
Le backend est exécuté dans l'environnement `/opt/backend` embarqué dans l'image Airflow,
sur le même patron que le DAG `alertes` (ADR 0008).
"""
from __future__ import annotations
from datetime import datetime, timedelta
from airflow.models.dag import DAG
from airflow.operators.bash import BashOperator
COMMANDE_BACKEND = "cd /opt/backend && env -u VIRTUAL_ENV uv run --no-sync python -m"
CSV_PATH = "/opt/data/raw/all_sites_combined.csv"
METADATA_PATH = "/opt/data/raw/dataset_metadata.json"
SOURCE_TIMEZONE = "UTC"
BATCH_SIZE = 1000
with DAG(
dag_id="historical_import",
description="Importe le dataset historique CSV/JSON dans dataset, site et reading.",
schedule=None,
start_date=datetime(2026, 1, 1),
catchup=False,
max_active_runs=1,
tags=["etl", "historical"],
) as dag:
BashOperator(
task_id="import_historical",
bash_command=(
f"{COMMANDE_BACKEND} app.etl.historical_import "
f"--csv {CSV_PATH} "
f"--metadata {METADATA_PATH} "
"--source-timezone UTC "
"--batch-size 1000"
),
retries=1,
retry_delay=timedelta(minutes=2),
execution_timeout=timedelta(minutes=30),
)
+27 -1
View File
@@ -10,12 +10,13 @@ from airflow.models.dagbag import DagBag
DAGS_FOLDER = Path(__file__).resolve().parent.parent / "dags" DAGS_FOLDER = Path(__file__).resolve().parent.parent / "dags"
DAG_IDS = ["ml_train", "ml_score", "alertes"] DAG_IDS = ["ml_train", "ml_score", "alertes", "historical_import"]
TACHES = [ TACHES = [
("ml_train", "train"), ("ml_train", "train"),
("ml_score", "score"), ("ml_score", "score"),
("alertes", "detection"), ("alertes", "detection"),
("alertes", "recommandations"), ("alertes", "recommandations"),
("historical_import", "import_historical"),
] ]
@@ -47,6 +48,10 @@ def test_alertes_runs_after_the_hourly_scoring(dagbag: DagBag) -> None:
assert dagbag.dags["alertes"].timetable.summary == "15 * * * *" assert dagbag.dags["alertes"].timetable.summary == "15 * * * *"
def test_historical_import_has_no_schedule(dagbag: DagBag) -> None:
assert dagbag.dags["historical_import"].timetable.summary == "None"
def test_ml_train_task_calls_the_training_module(dagbag: DagBag) -> None: def test_ml_train_task_calls_the_training_module(dagbag: DagBag) -> None:
tache = dagbag.dags["ml_train"].get_task("train") tache = dagbag.dags["ml_train"].get_task("train")
assert "enervision_ml.train" in tache.bash_command assert "enervision_ml.train" in tache.bash_command
@@ -67,12 +72,29 @@ def test_alertes_recommendation_task_calls_the_backend_cli(dagbag: DagBag) -> No
assert "app.cli generate-recommendations" in tache.bash_command assert "app.cli generate-recommendations" in tache.bash_command
def test_historical_import_calls_the_existing_backend_module(dagbag: DagBag) -> None:
tache = dagbag.dags["historical_import"].get_task("import_historical")
assert "app.etl.historical_import" in tache.bash_command
def test_historical_import_uses_the_expected_source_files(dagbag: DagBag) -> None:
commande = dagbag.dags["historical_import"].get_task("import_historical").bash_command
assert "--csv /opt/data/raw/all_sites_combined.csv" in commande
assert "--metadata /opt/data/raw/dataset_metadata.json" in commande
@pytest.mark.parametrize("task_id", ["detection", "recommandations"]) @pytest.mark.parametrize("task_id", ["detection", "recommandations"])
def test_alertes_tasks_run_in_the_backend_environment(dagbag: DagBag, task_id: str) -> None: def test_alertes_tasks_run_in_the_backend_environment(dagbag: DagBag, task_id: str) -> None:
# Le backend a son propre venv dans l'image, distinct de celui de ml/ (ADR 0008). # Le backend a son propre venv dans l'image, distinct de celui de ml/ (ADR 0008).
assert "/opt/backend" in dagbag.dags["alertes"].get_task(task_id).bash_command assert "/opt/backend" in dagbag.dags["alertes"].get_task(task_id).bash_command
def test_historical_import_runs_in_the_backend_environment(dagbag: DagBag) -> None:
commande = dagbag.dags["historical_import"].get_task("import_historical").bash_command
assert "/opt/backend" in commande
def test_alertes_generates_recommendations_after_detecting(dagbag: DagBag) -> None: def test_alertes_generates_recommendations_after_detecting(dagbag: DagBag) -> None:
# `recommendation.alert_id` est une cle etrangere `NOT NULL` : la generation n'a rien a lire # `recommendation.alert_id` est une cle etrangere `NOT NULL` : la generation n'a rien a lire
# tant que la detection n'a pas ecrit. # tant que la detection n'a pas ecrit.
@@ -133,6 +155,10 @@ def test_alertes_retries_after_a_transient_failure(dagbag: DagBag, task_id: str)
assert dagbag.dags["alertes"].get_task(task_id).retries >= 1 assert dagbag.dags["alertes"].get_task(task_id).retries >= 1
def test_historical_import_retries_after_a_transient_failure(dagbag: DagBag) -> None:
assert dagbag.dags["historical_import"].get_task("import_historical").retries >= 1
@pytest.mark.parametrize(("dag_id", "task_id"), TACHES) @pytest.mark.parametrize(("dag_id", "task_id"), TACHES)
def test_tasks_never_resync_the_baked_environment( def test_tasks_never_resync_the_baked_environment(
dagbag: DagBag, dag_id: str, task_id: str dagbag: DagBag, dag_id: str, task_id: str
+1 -1
View File
@@ -103,7 +103,7 @@ prevision (utile plus tard pour comparer prevision et realise, surveillance de d
uv run ruff check . # lint uv run ruff check . # lint
uv run ruff format . # format uv run ruff format . # format
uv run mypy enervision_ml tests # typage strict uv run mypy enervision_ml tests # typage strict
uv run pytest # tests uv run pytest # tests + couverture (ml/coverage.xml avec --cov-report=xml, lu par Sonar)
``` ```
Depuis la racine du monorepo, via le `Makefile` : `make install-ml`, `make ml-lint`, Depuis la racine du monorepo, via le `Makefile` : `make install-ml`, `make ml-lint`,
+11 -1
View File
@@ -17,6 +17,7 @@ dev = [
"ruff>=0.16.7", "ruff>=0.16.7",
"mypy>=2.3.1", "mypy>=2.3.1",
"pytest>=9.1.1", "pytest>=9.1.1",
"pytest-cov>=7.1.0",
"pandas-stubs>=3.0.5.260914", "pandas-stubs>=3.0.5.260914",
] ]
@@ -75,5 +76,14 @@ ignore_missing_imports = true
[tool.pytest.ini_options] [tool.pytest.ini_options]
testpaths = ["tests"] testpaths = ["tests"]
addopts = "-q --strict-markers -m 'not integration'" addopts = "-q --strict-markers -m 'not integration' --cov=enervision_ml --cov-report=term-missing"
markers = ["integration: requiert une base PostgreSQL joignable"] markers = ["integration: requiert une base PostgreSQL joignable"]
# Rapport lu par SonarCloud (`ml/coverage.xml`, cf. sonar-project.properties), meme mecanisme que
# apps/backend. Pas de seuil ici : celui de la quality gate porte sur le code nouveau.
[tool.coverage.run]
source = ["enervision_ml"]
branch = true
[tool.coverage.report]
show_missing = true
Generated
+55
View File
@@ -388,6 +388,45 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/19/37/c9aa45e47819dc15a38fc5c81a2fb987fde55e9d3b991fbde514e3b6b5f5/contourpy-1.4.0-cp314-cp314t-win_arm64.whl", hash = "sha256:fc9feef8f1f001c5b87decadc67c4a5d1eebb62ca39c4763d1237ff62cf2b707", size = 587071, upload-time = "2026-09-11T19:04:09.898Z" }, { url = "https://files.pythonhosted.org/packages/19/37/c9aa45e47819dc15a38fc5c81a2fb987fde55e9d3b991fbde514e3b6b5f5/contourpy-1.4.0-cp314-cp314t-win_arm64.whl", hash = "sha256:fc9feef8f1f001c5b87decadc67c4a5d1eebb62ca39c4763d1237ff62cf2b707", size = 587071, upload-time = "2026-09-11T19:04:09.898Z" },
] ]
[[package]]
name = "coverage"
version = "7.16.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/65/2d/c738872f477f5687152acae68635790387425d407ae37dd3d3a8a6692307/coverage-7.16.1.tar.gz", hash = "sha256:f83981779bcf9dfa06fa0a8d4cb43e0faec1706328ce07aa3e7b665b4ac0f210", size = 969651, upload-time = "2026-09-13T19:12:21.422Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/8e/b4/2a7c793965bae9f067aabab793a44d7a2f3ee7fb16b01ce1976bbd4a0218/coverage-7.16.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:cc0b37fe6f5ce5f1ccc62ad4fa9b1ad201d8e9b6027fd5e0170877beee4b2d15", size = 223546, upload-time = "2026-09-13T19:10:06.019Z" },
{ url = "https://files.pythonhosted.org/packages/ef/e2/633469076a2dbbea036cc15a268a3a5d6b2c7dd5d9a9567b2553dfc5ad61/coverage-7.16.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:6618f481053b63fc6121faf8fc676bd9b7163c2a19d9e984a2e850002c28ab57", size = 223881, upload-time = "2026-09-13T19:10:08.246Z" },
{ url = "https://files.pythonhosted.org/packages/de/c3/f06150c13284569d53273b909f31222874276a595637b7852571dfeb2c18/coverage-7.16.1-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:fa02d561eb1d8d2f8ba43ba6e3cef4c6c402a3b632a9460fa329fcadcd5df6a3", size = 254919, upload-time = "2026-09-13T19:10:10.254Z" },
{ url = "https://files.pythonhosted.org/packages/d5/40/47e25b215ae18a29010c8e29be8782a6e04d18ba6224be2bf6cebfce6427/coverage-7.16.1-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:bc5354a124799f1f87b7637bbe6f18cd4bc66a1f37f6aa2b5db40f9adad531dc", size = 257428, upload-time = "2026-09-13T19:10:12.124Z" },
{ url = "https://files.pythonhosted.org/packages/27/4b/1e2a4267d14cbd12a8489364a9d40020233e6be836d929b363f0e77209e2/coverage-7.16.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:34bafe9f4094315248573e6223e11af0ec1b25f9cbca43bf0e9a26a189ba2751", size = 258771, upload-time = "2026-09-13T19:10:14.031Z" },
{ url = "https://files.pythonhosted.org/packages/be/2e/9aa6146cea929fab9185bb2642ffef7f47520a6e5efe407f75f9b12f4cf0/coverage-7.16.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:29c4d3e32a3b5efa420a3dc627c7e570deb80ef997def52c7686a474f5edc7ab", size = 261086, upload-time = "2026-09-13T19:10:16.213Z" },
{ url = "https://files.pythonhosted.org/packages/13/3c/f9ad8bcd4fb3d21c9d20a16d6d6c6f999eee8f4498ed7659a3dbd2f4b74a/coverage-7.16.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f2066c447fdd0bca39a9633a082d8ce67bf9a539a203b85059a364a405dc9fe9", size = 254895, upload-time = "2026-09-13T19:10:18.602Z" },
{ url = "https://files.pythonhosted.org/packages/b7/d1/47eda9fd1eaeea39fa7b5b13a63b2bed92ab901841fb120b3f9f5e1dc30c/coverage-7.16.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fd8ac10cd2458b3c6343aac082fb9bd0e3fa806cb2c4975f2280153474b88412", size = 256783, upload-time = "2026-09-13T19:10:20.778Z" },
{ url = "https://files.pythonhosted.org/packages/38/c3/565edf044877cb8cd3373c56885347ffc38f0edfd1f1679a487b208c19a8/coverage-7.16.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:9d8c54ec32e5c102b9241f75d88ae26538b53662868ca491736611db448d9c7a", size = 254742, upload-time = "2026-09-13T19:10:22.733Z" },
{ url = "https://files.pythonhosted.org/packages/fd/88/87d2b2aeaba719192b2089ff1c2cf89a06cf73a6d2e9f1f145626617700c/coverage-7.16.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:6dd8dda3402a01a1a8fe8b753a282466f615128574a5590a9108acd07b1f8540", size = 259016, upload-time = "2026-09-13T19:10:24.769Z" },
{ url = "https://files.pythonhosted.org/packages/fc/1b/70813185b125768abdcf7899fec4d37edc2e5fc9b60c7045c8f4271ec757/coverage-7.16.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:79afa9726438912e5cddd1fe541815cea9763c92935f594835e4c432565b68a9", size = 254559, upload-time = "2026-09-13T19:10:26.781Z" },
{ url = "https://files.pythonhosted.org/packages/d8/fa/e7aa5af279aafda633a1ede8bfd7d6916b0c8b2082be86759e0b52e73a61/coverage-7.16.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:3db3978211c3cead5437a80136ca0556bab8bc7828de15a762884b0598c41361", size = 256215, upload-time = "2026-09-13T19:10:28.714Z" },
{ url = "https://files.pythonhosted.org/packages/38/87/7a894fa4f8c6662d2b6a87a3436950e15b1fa56e01765c9d6634fb2cbeb8/coverage-7.16.1-cp314-cp314-win32.whl", hash = "sha256:49c39c7068a494f8eb427155f5682f44feee43f9b3107fd54b1e52465379c54b", size = 225719, upload-time = "2026-09-13T19:10:30.743Z" },
{ url = "https://files.pythonhosted.org/packages/8b/01/fa7193c8005fb85488f02b0e1cc3c05a233cf2640206dd978af447aeecbf/coverage-7.16.1-cp314-cp314-win_amd64.whl", hash = "sha256:c510dad19552d912058e4c3e3cbec3fb155dbe8d0ce0ceb7e7dbf5c5822bae0b", size = 226208, upload-time = "2026-09-13T19:10:32.698Z" },
{ url = "https://files.pythonhosted.org/packages/da/5c/a08634c714924c3eaef811bb3576c044128aa5e7dfa86c75e52f0761849e/coverage-7.16.1-cp314-cp314-win_arm64.whl", hash = "sha256:b7d4d7e6dcaf33e85f1919f03346403bdcc27437c420a78835f3805bca0ab71f", size = 225633, upload-time = "2026-09-13T19:10:34.79Z" },
{ url = "https://files.pythonhosted.org/packages/43/df/ddb8a4c664046b1a0ee29c9c2d25b993e5dbc8fbde715df3694a64532781/coverage-7.16.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:3d0a3681c12d3e0bcdea3d9414b04087828d6c1a482802d6f7f42c37ed530152", size = 224281, upload-time = "2026-09-13T19:10:36.853Z" },
{ url = "https://files.pythonhosted.org/packages/e2/d0/9076e0c762d8afd91182e60a520fa5c92c4a334785eeb9fd6b8ef8fe7e3c/coverage-7.16.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3f3b4469d3da3ecced775d1a8c9c5d9fc80f259e30b7b89f9fed0700d6035ecb", size = 224547, upload-time = "2026-09-13T19:10:39.359Z" },
{ url = "https://files.pythonhosted.org/packages/03/e5/9c59e64b6161704f35fe91549bb19b2bb355e95caf596c26a2065564807c/coverage-7.16.1-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:c08ae35c1be2fe1ce4b4c628df5c6fc0dc9a87f8e5fe8e20238d249678984741", size = 265906, upload-time = "2026-09-13T19:10:41.434Z" },
{ url = "https://files.pythonhosted.org/packages/57/5a/13ccaffb77f766101bf6f38be9dba9e468b02cc92da4552a57877dbf1c1f/coverage-7.16.1-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:8ee71a38c54bb2676bbe762b8b0943a79ccb1c2fd6a52054f66e63eda392f8c1", size = 268023, upload-time = "2026-09-13T19:10:43.533Z" },
{ url = "https://files.pythonhosted.org/packages/ad/a1/05cfcf01d3c7c922832698ad46e51d3441d820ce87a943014bb5cf5710dd/coverage-7.16.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:76491917771f179f9772efe218c5ccc65950dbdb35f4439298d8a8dfc6ec1f72", size = 270442, upload-time = "2026-09-13T19:10:45.895Z" },
{ url = "https://files.pythonhosted.org/packages/72/15/a2f1544b8e3835d7b769f7dabcc9ac0283e0b646ef3344703ff8f18d83e6/coverage-7.16.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f4aa0b0a6f81fa3deb211e643f6954e78b4376b62b9c218271236cfa757664e8", size = 271565, upload-time = "2026-09-13T19:10:48.123Z" },
{ url = "https://files.pythonhosted.org/packages/df/5b/963c2993a82bd313f298d663afe03e164b96ace4d9d4c7561740a559e13d/coverage-7.16.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:756ba2d96d073c5a2a55d67fa22784763710fadbe22c41adde2d9cfa4dd78a8c", size = 264959, upload-time = "2026-09-13T19:10:50.195Z" },
{ url = "https://files.pythonhosted.org/packages/12/59/5eba06d1943735d7cd61d46d8c8a20ffe8ddd2da06b3c94366078dadeb9b/coverage-7.16.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:99bf9ea435cefcefd220f8687c3ddbbf78dc2de0bd11b57c3ae9fbbdf8d5561a", size = 267897, upload-time = "2026-09-13T19:10:52.252Z" },
{ url = "https://files.pythonhosted.org/packages/bd/48/af6c30f6ea431bb9b83f9070d268a9cc4fc97490abd32080164177ea999f/coverage-7.16.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:35cbc81f937fc402971df45c897d2df2bfb2014efcd990360032aa0a651635da", size = 265504, upload-time = "2026-09-13T19:10:54.432Z" },
{ url = "https://files.pythonhosted.org/packages/80/f2/6e13852a8656d05fa83284567dd5a5b1e6d89bef79fe3effca2787159eab/coverage-7.16.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:8fae08e85b334ac6ac886002b5041396a31bcf805225bbe19847627203da99e2", size = 269235, upload-time = "2026-09-13T19:10:56.563Z" },
{ url = "https://files.pythonhosted.org/packages/c2/32/b4fe465daa64ece674f83a750dfa4ba0fa3c5c74d6ef5dbb8dfce892cf0d/coverage-7.16.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:83362b64e215ef00b0ba33fcf13655ace6c9fdd144d5ad2ab59ac86c2daf166e", size = 264347, upload-time = "2026-09-13T19:10:58.634Z" },
{ url = "https://files.pythonhosted.org/packages/54/f3/88b5c0e4ca3994c6d5feb7b1bf4c9a62cee205553159184968426930a7b1/coverage-7.16.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:33300f2e140ccf26af3d8152e62bff71993f9310cfc63ba7a20940b0d246a0ae", size = 266660, upload-time = "2026-09-13T19:11:00.746Z" },
{ url = "https://files.pythonhosted.org/packages/97/72/6eff5456d7ba7f1c4678af531c33f9d957cae3201bd229b056fd13a204a3/coverage-7.16.1-cp314-cp314t-win32.whl", hash = "sha256:5539304fdbb2cc144df684d35a33b81145334d23e1c2367b5a923d25107f70b2", size = 226026, upload-time = "2026-09-13T19:11:02.846Z" },
{ url = "https://files.pythonhosted.org/packages/8e/c8/6e5ae3d8d4d0f2c0078985bf4db55fafd90e8107b1bf91ee3547a13f5694/coverage-7.16.1-cp314-cp314t-win_amd64.whl", hash = "sha256:715dcb72c3280c428c3a20134b87e42c29acec9669136e899ab2de69ca86218d", size = 226862, upload-time = "2026-09-13T19:11:04.921Z" },
{ url = "https://files.pythonhosted.org/packages/be/c7/68f9f0734afc904a92b974b489545b6a15700f3b1c4bd36eae764561e661/coverage-7.16.1-cp314-cp314t-win_arm64.whl", hash = "sha256:dac8b84c03e6029d272b8249c77018db83de59ca009a9adef7c144b4a62ee5e6", size = 226171, upload-time = "2026-09-13T19:11:06.969Z" },
{ url = "https://files.pythonhosted.org/packages/96/1a/d6d16babd0a5fe4c3fae40702158c570351694e74516d8d81b86c5637448/coverage-7.16.1-py3-none-any.whl", hash = "sha256:3d8bd4e58b6a5c2018d808f297905393c6c61da466a48c3f0596a76a4900ebe4", size = 215264, upload-time = "2026-09-13T19:12:18.895Z" },
]
[[package]] [[package]]
name = "cryptography" name = "cryptography"
version = "50.0.1" version = "50.0.1"
@@ -494,6 +533,7 @@ dev = [
{ name = "mypy" }, { name = "mypy" },
{ name = "pandas-stubs" }, { name = "pandas-stubs" },
{ name = "pytest" }, { name = "pytest" },
{ name = "pytest-cov" },
{ name = "ruff" }, { name = "ruff" },
] ]
@@ -512,6 +552,7 @@ dev = [
{ name = "mypy", specifier = ">=2.3.1" }, { name = "mypy", specifier = ">=2.3.1" },
{ name = "pandas-stubs", specifier = ">=3.0.5.260914" }, { name = "pandas-stubs", specifier = ">=3.0.5.260914" },
{ name = "pytest", specifier = ">=9.1.1" }, { name = "pytest", specifier = ">=9.1.1" },
{ name = "pytest-cov", specifier = ">=7.1.0" },
{ name = "ruff", specifier = ">=0.16.7" }, { name = "ruff", specifier = ">=0.16.7" },
] ]
@@ -1591,6 +1632,20 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" }, { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
] ]
[[package]]
name = "pytest-cov"
version = "7.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "coverage" },
{ name = "pluggy" },
{ name = "pytest" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b1/51/a849f96e117386044471c8ec2bd6cfebacda285da9525c9106aeb28da671/pytest_cov-7.1.0.tar.gz", hash = "sha256:30674f2b5f6351aa09702a9c8c364f6a01c27aae0c1366ae8016160d1efc56b2", size = 55592, upload-time = "2026-03-21T20:11:16.284Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/9d/7a/d968e294073affff457b041c2be9868a40c1c71f4a35fcc1e45e5493067b/pytest_cov-7.1.0-py3-none-any.whl", hash = "sha256:a0461110b7865f9a271aa1b51e516c9a95de9d696734a2f71e3e78f46e1d4678", size = 22876, upload-time = "2026-03-21T20:11:14.438Z" },
]
[[package]] [[package]]
name = "python-dateutil" name = "python-dateutil"
version = "2.9.0.post0" version = "2.9.0.post0"
+6 -4
View File
@@ -3,15 +3,17 @@ sonar.organization=groupe3-ener-vision
sonar.sourceEncoding=UTF-8 sonar.sourceEncoding=UTF-8
# Dossier contenant le code source # Dossier contenant le code source
sonar.sources=apps/frontend/src,apps/backend sonar.sources=apps/frontend/src,apps/backend,ml,etl/airflow
# Dossier contenant les tests # Dossier contenant les tests
sonar.tests=apps/frontend/src,apps/backend/tests sonar.tests=apps/frontend/src,apps/backend/tests,ml/tests,etl/airflow/tests
sonar.test.inclusions=**/*.spec.ts,**/*.test.ts,**/*test_*.py,**/*test.py sonar.test.inclusions=**/*.spec.ts,**/*.test.ts,**/*test_*.py,**/*test.py
# Liste des fichiers et dossiers à exclure de l'analyse # Liste des fichiers et dossiers à exclure de l'analyse
sonar.exclusions=.pytest_cache,.venv,alembic,tests,**/*/node_modules/**,**/*/dist/**,**/*/build/**,**/*.spec.ts,**/*.test.ts,**/*test_*.py,**/*test.py,**/*.spec.ts sonar.exclusions=.pytest_cache,.venv,.airflow_home,alembic,tests,ml/data/**,ml/models/**,ml/mlruns/**,ml/mlartifacts/**,**/*/node_modules/**,**/*/dist/**,**/*/build/**,**/*.spec.ts,**/*.test.ts,**/*test_*.py,**/*test.py,**/*.spec.ts
# Chemin vers le rapport de couverture de code # Chemin vers le rapport de couverture de code
# Fichier généré par Pytest # Fichier généré par Pytest
sonar.python.coverage.reportPaths=apps/backend/coverage.xml sonar.python.coverage.reportPaths=apps/backend/coverage.xml,ml/coverage.xml
# Les DAGs n'ont pas de couverture mesurable : leurs tests ne font que les charger (DagBag)
sonar.coverage.exclusions=etl/airflow/**
sonar.javascript.lcov.reportPaths=apps/frontend/coverage/frontend/lcov.info sonar.javascript.lcov.reportPaths=apps/frontend/coverage/frontend/lcov.info