Implemented the next 4h-plan phase: dual-run support + explicit cutover gate.
This commit is contained in:
@@ -39,8 +39,8 @@ scripts/rainml_py.sh scripts/train_rain_model.py \
|
||||
--threshold-policy "walk_forward" \
|
||||
--walk-forward-folds 4 \
|
||||
--model-version "rain-auto-v2-extended-4h" \
|
||||
--out "models/rain_model.pkl" \
|
||||
--report-out "models/rain_model_report.json" \
|
||||
--out "models/rain_model_4h.pkl" \
|
||||
--report-out "models/rain_model_report_4h.json" \
|
||||
--model-card-out "models/model_card_{model_version}.md" \
|
||||
--dataset-out "models/datasets/rain_dataset_{model_version}_{feature_set}.csv"
|
||||
```
|
||||
@@ -61,7 +61,7 @@ Review in report:
|
||||
```sh
|
||||
scripts/rainml_py.sh scripts/predict_rain_model.py \
|
||||
--site home \
|
||||
--model-path "models/rain_model.pkl" \
|
||||
--model-path "models/rain_model_4h.pkl" \
|
||||
--model-name "rain_next_4h" \
|
||||
--horizon-hours 4 \
|
||||
--dry-run
|
||||
@@ -72,7 +72,7 @@ scripts/rainml_py.sh scripts/predict_rain_model.py \
|
||||
```sh
|
||||
scripts/rainml_py.sh scripts/predict_rain_model.py \
|
||||
--site home \
|
||||
--model-path "models/rain_model.pkl" \
|
||||
--model-path "models/rain_model_4h.pkl" \
|
||||
--model-name "rain_next_4h" \
|
||||
--horizon-hours 4
|
||||
```
|
||||
@@ -155,6 +155,14 @@ The script exits non-zero on failure, so it can directly drive alerting.
|
||||
- `RAIN_MODEL_BACKUP_PATH`
|
||||
- `RAIN_MODEL_CARD_PATH`
|
||||
|
||||
Dual-run note:
|
||||
- `rainml` is configured as 4-hour model training/inference with dedicated artifact paths.
|
||||
- `rainml_1h` is available as an optional shadow/baseline service via profile `shadow`.
|
||||
- Start both (4h + 1h shadow):
|
||||
`docker compose --profile shadow up -d rainml rainml_1h`
|
||||
- Run one-off script against the 1h service:
|
||||
`RAINML_PY_SERVICE=rainml_1h scripts/rainml_py.sh scripts/train_rain_model.py ...`
|
||||
|
||||
Recommended production defaults:
|
||||
- Enable tuning daily or weekly (`RAIN_TUNE_HYPERPARAMETERS=true`)
|
||||
- Set `RAIN_THRESHOLD_POLICY=walk_forward` with `RAIN_WALK_FORWARD_FOLDS=4` for temporally robust threshold selection
|
||||
@@ -217,6 +225,19 @@ scripts/rainml_py.sh scripts/compare_rain_reports.py \
|
||||
--baseline "models/rain_model_report_1h.json" \
|
||||
--candidate "models/rain_model_report_4h.json"
|
||||
```
|
||||
|
||||
4b. Apply an explicit cutover gate (exit code 0 = pass):
|
||||
|
||||
```sh
|
||||
scripts/rainml_py.sh scripts/check_rain_cutover_gate.py \
|
||||
--baseline "models/rain_model_report_1h.json" \
|
||||
--candidate "models/rain_model_report_4h.json" \
|
||||
--min-candidate-precision 0.60 \
|
||||
--max-precision-drop 0.05 \
|
||||
--max-pr-auc-drop 0.05 \
|
||||
--max-roc-auc-drop 0.05 \
|
||||
--max-brier-increase 0.03
|
||||
```
|
||||
5. Run dry-run inference, then live inference with 4h model name/horizon:
|
||||
|
||||
```sh
|
||||
|
||||
Reference in New Issue
Block a user