Abdoul Ahad Binizi
1242 lines
59 KiB
Plaintext
1242 lines
59 KiB
Plaintext
{
|
||
"nbformat": 4,
|
||
"nbformat_minor": 0,
|
||
"metadata": {
|
||
"colab": {
|
||
"provenance": [],
|
||
"gpuType": "T4",
|
||
"toc_visible": true
|
||
},
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"name": "python"
|
||
},
|
||
"accelerator": "GPU"
|
||
},
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "BsjhisC9CtK8"
|
||
},
|
||
"source": [
|
||
"# 🧠 AHAD QUANT V4 — Entraînement Complet sur Google Colab\n",
|
||
"\n",
|
||
"> **GPU recommandé** : Runtime → Modifier le type de Runtime → T4 GPU (gratuit)\n",
|
||
"\n",
|
||
"## 📋 Étapes\n",
|
||
"| # | Étape | Durée estimée |\n",
|
||
"|---|-------|---------------|\n",
|
||
"| 1 | Upload du projet + installation | ~5 min |\n",
|
||
"| 2 | Téléchargement données Forex | ~5 min |\n",
|
||
"| 3 | Entraînement ML (LGB + XGB + RF + TFT + TGRU) | ~45 min |\n",
|
||
"| 4 | Entraînement RL (PPO) | ~60 min |\n",
|
||
"| 5 | Export modèle unifié + téléchargement | ~2 min |\n",
|
||
"\n",
|
||
"---\n",
|
||
"⚠️ **Garde cet onglet actif** — Colab déconnecte après 90 min d'inactivité.\n",
|
||
"En cas de déconnexion : relancer depuis la **cellule 4.2** avec `--resume`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "wyUAORUuCtK_"
|
||
},
|
||
"source": [
|
||
"## 📦 ÉTAPE 1 — Upload & Installation"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 160
|
||
},
|
||
"id": "rTjwFHDUCtLA",
|
||
"outputId": "532d5b73-5d63-4fdb-dd91-1b68c7903853"
|
||
},
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"📁 Sélectionne le fichier ahad_quant_v32_FINAL_updated_v4.zip ...\n"
|
||
]
|
||
},
|
||
{
|
||
"output_type": "display_data",
|
||
"data": {
|
||
"text/plain": [
|
||
"<IPython.core.display.HTML object>"
|
||
],
|
||
"text/html": [
|
||
"\n",
|
||
" <input type=\"file\" id=\"files-f3b4b8ed-6ba0-4237-8bf7-ee586407edae\" name=\"files[]\" multiple disabled\n",
|
||
" style=\"border:none\" />\n",
|
||
" <output id=\"result-f3b4b8ed-6ba0-4237-8bf7-ee586407edae\">\n",
|
||
" Upload widget is only available when the cell has been executed in the\n",
|
||
" current browser session. Please rerun this cell to enable.\n",
|
||
" </output>\n",
|
||
" <script>// Copyright 2017 Google LLC\n",
|
||
"//\n",
|
||
"// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||
"// you may not use this file except in compliance with the License.\n",
|
||
"// You may obtain a copy of the License at\n",
|
||
"//\n",
|
||
"// http://www.apache.org/licenses/LICENSE-2.0\n",
|
||
"//\n",
|
||
"// Unless required by applicable law or agreed to in writing, software\n",
|
||
"// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||
"// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||
"// See the License for the specific language governing permissions and\n",
|
||
"// limitations under the License.\n",
|
||
"\n",
|
||
"/**\n",
|
||
" * @fileoverview Helpers for google.colab Python module.\n",
|
||
" */\n",
|
||
"(function(scope) {\n",
|
||
"function span(text, styleAttributes = {}) {\n",
|
||
" const element = document.createElement('span');\n",
|
||
" element.textContent = text;\n",
|
||
" for (const key of Object.keys(styleAttributes)) {\n",
|
||
" element.style[key] = styleAttributes[key];\n",
|
||
" }\n",
|
||
" return element;\n",
|
||
"}\n",
|
||
"\n",
|
||
"// Max number of bytes which will be uploaded at a time.\n",
|
||
"const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
|
||
"\n",
|
||
"function _uploadFiles(inputId, outputId) {\n",
|
||
" const steps = uploadFilesStep(inputId, outputId);\n",
|
||
" const outputElement = document.getElementById(outputId);\n",
|
||
" // Cache steps on the outputElement to make it available for the next call\n",
|
||
" // to uploadFilesContinue from Python.\n",
|
||
" outputElement.steps = steps;\n",
|
||
"\n",
|
||
" return _uploadFilesContinue(outputId);\n",
|
||
"}\n",
|
||
"\n",
|
||
"// This is roughly an async generator (not supported in the browser yet),\n",
|
||
"// where there are multiple asynchronous steps and the Python side is going\n",
|
||
"// to poll for completion of each step.\n",
|
||
"// This uses a Promise to block the python side on completion of each step,\n",
|
||
"// then passes the result of the previous step as the input to the next step.\n",
|
||
"function _uploadFilesContinue(outputId) {\n",
|
||
" const outputElement = document.getElementById(outputId);\n",
|
||
" const steps = outputElement.steps;\n",
|
||
"\n",
|
||
" const next = steps.next(outputElement.lastPromiseValue);\n",
|
||
" return Promise.resolve(next.value.promise).then((value) => {\n",
|
||
" // Cache the last promise value to make it available to the next\n",
|
||
" // step of the generator.\n",
|
||
" outputElement.lastPromiseValue = value;\n",
|
||
" return next.value.response;\n",
|
||
" });\n",
|
||
"}\n",
|
||
"\n",
|
||
"/**\n",
|
||
" * Generator function which is called between each async step of the upload\n",
|
||
" * process.\n",
|
||
" * @param {string} inputId Element ID of the input file picker element.\n",
|
||
" * @param {string} outputId Element ID of the output display.\n",
|
||
" * @return {!Iterable<!Object>} Iterable of next steps.\n",
|
||
" */\n",
|
||
"function* uploadFilesStep(inputId, outputId) {\n",
|
||
" const inputElement = document.getElementById(inputId);\n",
|
||
" inputElement.disabled = false;\n",
|
||
"\n",
|
||
" const outputElement = document.getElementById(outputId);\n",
|
||
" outputElement.innerHTML = '';\n",
|
||
"\n",
|
||
" const pickedPromise = new Promise((resolve) => {\n",
|
||
" inputElement.addEventListener('change', (e) => {\n",
|
||
" resolve(e.target.files);\n",
|
||
" });\n",
|
||
" });\n",
|
||
"\n",
|
||
" const cancel = document.createElement('button');\n",
|
||
" inputElement.parentElement.appendChild(cancel);\n",
|
||
" cancel.textContent = 'Cancel upload';\n",
|
||
" const cancelPromise = new Promise((resolve) => {\n",
|
||
" cancel.onclick = () => {\n",
|
||
" resolve(null);\n",
|
||
" };\n",
|
||
" });\n",
|
||
"\n",
|
||
" // Wait for the user to pick the files.\n",
|
||
" const files = yield {\n",
|
||
" promise: Promise.race([pickedPromise, cancelPromise]),\n",
|
||
" response: {\n",
|
||
" action: 'starting',\n",
|
||
" }\n",
|
||
" };\n",
|
||
"\n",
|
||
" cancel.remove();\n",
|
||
"\n",
|
||
" // Disable the input element since further picks are not allowed.\n",
|
||
" inputElement.disabled = true;\n",
|
||
"\n",
|
||
" if (!files) {\n",
|
||
" return {\n",
|
||
" response: {\n",
|
||
" action: 'complete',\n",
|
||
" }\n",
|
||
" };\n",
|
||
" }\n",
|
||
"\n",
|
||
" for (const file of files) {\n",
|
||
" const li = document.createElement('li');\n",
|
||
" li.append(span(file.name, {fontWeight: 'bold'}));\n",
|
||
" li.append(span(\n",
|
||
" `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
|
||
" `last modified: ${\n",
|
||
" file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
|
||
" 'n/a'} - `));\n",
|
||
" const percent = span('0% done');\n",
|
||
" li.appendChild(percent);\n",
|
||
"\n",
|
||
" outputElement.appendChild(li);\n",
|
||
"\n",
|
||
" const fileDataPromise = new Promise((resolve) => {\n",
|
||
" const reader = new FileReader();\n",
|
||
" reader.onload = (e) => {\n",
|
||
" resolve(e.target.result);\n",
|
||
" };\n",
|
||
" reader.readAsArrayBuffer(file);\n",
|
||
" });\n",
|
||
" // Wait for the data to be ready.\n",
|
||
" let fileData = yield {\n",
|
||
" promise: fileDataPromise,\n",
|
||
" response: {\n",
|
||
" action: 'continue',\n",
|
||
" }\n",
|
||
" };\n",
|
||
"\n",
|
||
" // Use a chunked sending to avoid message size limits. See b/62115660.\n",
|
||
" let position = 0;\n",
|
||
" do {\n",
|
||
" const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
|
||
" const chunk = new Uint8Array(fileData, position, length);\n",
|
||
" position += length;\n",
|
||
"\n",
|
||
" const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
|
||
" yield {\n",
|
||
" response: {\n",
|
||
" action: 'append',\n",
|
||
" file: file.name,\n",
|
||
" data: base64,\n",
|
||
" },\n",
|
||
" };\n",
|
||
"\n",
|
||
" let percentDone = fileData.byteLength === 0 ?\n",
|
||
" 100 :\n",
|
||
" Math.round((position / fileData.byteLength) * 100);\n",
|
||
" percent.textContent = `${percentDone}% done`;\n",
|
||
"\n",
|
||
" } while (position < fileData.byteLength);\n",
|
||
" }\n",
|
||
"\n",
|
||
" // All done.\n",
|
||
" yield {\n",
|
||
" response: {\n",
|
||
" action: 'complete',\n",
|
||
" }\n",
|
||
" };\n",
|
||
"}\n",
|
||
"\n",
|
||
"scope.google = scope.google || {};\n",
|
||
"scope.google.colab = scope.google.colab || {};\n",
|
||
"scope.google.colab._files = {\n",
|
||
" _uploadFiles,\n",
|
||
" _uploadFilesContinue,\n",
|
||
"};\n",
|
||
"})(self);\n",
|
||
"</script> "
|
||
]
|
||
},
|
||
"metadata": {}
|
||
},
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"Saving ahad_quant_v32_CLEAN.zip to ahad_quant_v32_CLEAN.zip\n",
|
||
"\n",
|
||
"✅ Reçu : ahad_quant_v32_CLEAN.zip (0.2 MB)\n",
|
||
"📂 Projet extrait dans : /content/ahad_quant/ahad_quant_v32_fixed\n",
|
||
"✅ Répertoire de travail configuré\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# ── 1.1 Upload du ZIP AHAD QUANT ──────────────────────────────────────────────\n",
|
||
"from google.colab import files\n",
|
||
"import os, zipfile, shutil\n",
|
||
"\n",
|
||
"print('📁 Sélectionne le fichier ahad_quant_v32_FINAL_updated_v4.zip ...')\n",
|
||
"uploaded = files.upload()\n",
|
||
"\n",
|
||
"zip_name = list(uploaded.keys())[0]\n",
|
||
"print(f'\\n✅ Reçu : {zip_name} ({os.path.getsize(zip_name)/1024/1024:.1f} MB)')\n",
|
||
"\n",
|
||
"# Extraire dans /content/ahad_quant\n",
|
||
"os.makedirs('/content/ahad_quant', exist_ok=True)\n",
|
||
"with zipfile.ZipFile(zip_name, 'r') as z:\n",
|
||
" z.extractall('/content/ahad_quant')\n",
|
||
"\n",
|
||
"# Trouver le dossier extrait\n",
|
||
"subdirs = [d for d in os.listdir('/content/ahad_quant') if os.path.isdir(f'/content/ahad_quant/{d}')]\n",
|
||
"PROJECT_DIR = f'/content/ahad_quant/{subdirs[0]}'\n",
|
||
"print(f'📂 Projet extrait dans : {PROJECT_DIR}')\n",
|
||
"os.chdir(PROJECT_DIR)\n",
|
||
"print('✅ Répertoire de travail configuré')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "HAPFJvJkCtLC",
|
||
"outputId": "2356f3dd-ff8b-4d15-c261-6d2e2437889d"
|
||
},
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"📦 Installation des packages...\n",
|
||
"\n",
|
||
"✅ Packages installés\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# ── 1.2 Installation des dépendances ─────────────────────────────────────────\n",
|
||
"print('📦 Installation des packages...')\n",
|
||
"\n",
|
||
"# Packages standards depuis PyPI\n",
|
||
"!pip install -q lightgbm xgboost scikit-learn optuna yfinance stable-baselines3[extra] gymnasium shimmy python-dotenv\n",
|
||
"\n",
|
||
"# Torch GPU (CUDA 11.8) depuis l'index PyTorch dédié\n",
|
||
"!pip install -q torch torchvision --index-url https://download.pytorch.org/whl/cu118\n",
|
||
"\n",
|
||
"print('\\n✅ Packages installés')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 16,
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "ScKxJIYmCtLD",
|
||
"outputId": "3896a169-43f0-4d7d-8566-2dd1f3c4afc0"
|
||
},
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"✅ GPU détecté : Tesla T4 (14.6 GB)\n",
|
||
"\n",
|
||
"📋 Config active :\n",
|
||
" DATA_SOURCE = yfinance\n",
|
||
" PAPER_MODE = true\n",
|
||
" MAX_SEQ_PER_PAIR = 4000\n",
|
||
" AHAD_QUANT_BATCH_SIZE = 512\n",
|
||
" AHAD_QUANT_MAX_EPOCHS = 20\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# ── 1.3 Configuration GPU + variables d'environnement ────────────────────────\n",
|
||
"import torch, os\n",
|
||
"\n",
|
||
"GPU_OK = torch.cuda.is_available()\n",
|
||
"if GPU_OK:\n",
|
||
" gpu_name = torch.cuda.get_device_name(0)\n",
|
||
" gpu_mem = torch.cuda.get_device_properties(0).total_memory / 1024**3\n",
|
||
" print(f'✅ GPU détecté : {gpu_name} ({gpu_mem:.1f} GB)')\n",
|
||
"else:\n",
|
||
" print('⚠️ Pas de GPU — entraînement plus lent (CPU). Active le GPU dans Runtime → Modifier le type de Runtime.')\n",
|
||
"\n",
|
||
"# Variables d'env pour AHAD QUANT\n",
|
||
"os.environ.update({\n",
|
||
" 'DATA_SOURCE' : 'yfinance', # source données (gratuit)\n",
|
||
" 'EXCHANGE' : 'paper', # pas de broker réel\n",
|
||
" 'PAPER_MODE' : 'true',\n",
|
||
" 'PAPER_INITIAL_BALANCE' : '10000',\n",
|
||
" # Mémoire : RAM SYSTÈME Colab (~12 Gi), indépendante de la VRAM du GPU\n",
|
||
" 'MAX_SEQ_PER_PAIR' : '4000',\n",
|
||
" # Batch size TFT/TGRU — GPU peut prendre 512, CPU limité\n",
|
||
" 'AHAD_QUANT_BATCH_SIZE' : '512' if GPU_OK else '128',\n",
|
||
" 'AHAD_QUANT_MAX_EPOCHS' : '20' if GPU_OK else '10',\n",
|
||
"})\n",
|
||
"\n",
|
||
"print('\\n📋 Config active :')\n",
|
||
"for k in ['DATA_SOURCE','PAPER_MODE','MAX_SEQ_PER_PAIR','AHAD_QUANT_BATCH_SIZE','AHAD_QUANT_MAX_EPOCHS']:\n",
|
||
" print(f' {k} = {os.environ[k]}')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "52yD66NnCtLE",
|
||
"outputId": "b0b873ed-ca6b-4642-b2e1-2566a2ec43ac"
|
||
},
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n",
|
||
"✅ Drive monté — sauvegarde dans : /content/drive/MyDrive/AhadQuant_Models\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# ── 1.4 (Optionnel) Montage Google Drive pour sauvegarde auto ────────────────\n",
|
||
"USE_DRIVE = True # ← Mettre False si tu ne veux pas utiliser Drive\n",
|
||
"\n",
|
||
"DRIVE_SAVE_DIR = None\n",
|
||
"if USE_DRIVE:\n",
|
||
" from google.colab import drive\n",
|
||
" drive.mount('/content/drive')\n",
|
||
" DRIVE_SAVE_DIR = '/content/drive/MyDrive/AhadQuant_Models'\n",
|
||
" os.makedirs(DRIVE_SAVE_DIR, exist_ok=True)\n",
|
||
" print(f'✅ Drive monté — sauvegarde dans : {DRIVE_SAVE_DIR}')\n",
|
||
"else:\n",
|
||
" print('ℹ️ Drive désactivé — les modèles seront téléchargés en fin de notebook')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "KnX1i32NCtLE"
|
||
},
|
||
"source": [
|
||
"## 📊 ÉTAPE 2 — Téléchargement des données Forex"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 18,
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "Gvan90BJCtLE",
|
||
"outputId": "745ad356-e0cb-43c6-fa02-dc8c14abc111"
|
||
},
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"📡 Téléchargement des données historiques Forex (Yahoo Finance)...\n",
|
||
" 25 paires × 730 jours × 1h ≈ 17 000 bougies/paire\n",
|
||
"\n",
|
||
"\n",
|
||
"✅ Données téléchargées : 25 fichiers dans data/\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# ── 2.1 Download 730 jours de données 1h pour les 25 paires ─────────────────\n",
|
||
"import subprocess, sys\n",
|
||
"\n",
|
||
"print('📡 Téléchargement des données historiques Forex (Yahoo Finance)...')\n",
|
||
"print(' 25 paires × 730 jours × 1h ≈ 17 000 bougies/paire\\n')\n",
|
||
"\n",
|
||
"result = subprocess.run(\n",
|
||
" [sys.executable, 'download_data.py'],\n",
|
||
" capture_output=False,\n",
|
||
" text=True\n",
|
||
")\n",
|
||
"\n",
|
||
"if result.returncode == 0:\n",
|
||
" data_files = [f for f in os.listdir('data') if f.endswith('.json')] if os.path.exists('data') else []\n",
|
||
" print(f'\\n✅ Données téléchargées : {len(data_files)} fichiers dans data/')\n",
|
||
"else:\n",
|
||
" print(f'\\n❌ Erreur download (code {result.returncode})')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "fs-zpnwoCtLF"
|
||
},
|
||
"source": [
|
||
"## 🤖 ÉTAPE 3 — Entraînement ML (Ensemble + Deep Learning)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 19,
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "otoBfmYuCtLF",
|
||
"outputId": "a0cfa69b-d997-4f1b-df85-50f107e94998"
|
||
},
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"🧠 Lancement entraînement ML...\n",
|
||
" LightGBM + XGBoost + RF + TFT + TransformerGRU + Meta-learner\n",
|
||
"\n",
|
||
"[DL] PyTorch 2.11.0+cu128 detected — TFT + TGRU enabled\n",
|
||
"=================================================================\n",
|
||
" AHAD QUANT — Ensemble Training Pipeline (V5)\n",
|
||
" LightGBM + XGBoost + RF + TFT + TransformerGRU + Meta-learner\n",
|
||
"=================================================================\n",
|
||
"\n",
|
||
"[0/7] Données disponibles : 25 fichiers dans data/\n",
|
||
"\n",
|
||
"[1/7] Loading data and building tabular features...\n",
|
||
" ✅ EURUSD — 17,222 samples (719 jours)\n",
|
||
" ✅ GBPUSD — 17,223 samples (719 jours)\n",
|
||
" ✅ USDJPY — 17,109 samples (714 jours)\n",
|
||
" ✅ USDCHF — 17,163 samples (716 jours)\n",
|
||
" ✅ AUDUSD — 17,321 samples (723 jours)\n",
|
||
" ✅ NZDUSD — 17,314 samples (723 jours)\n",
|
||
" ✅ USDCAD — 17,326 samples (723 jours)\n",
|
||
" ✅ EURGBP — 17,248 samples (720 jours)\n",
|
||
" ✅ EURJPY — 17,234 samples (719 jours)\n",
|
||
" ✅ EURCAD — 17,249 samples (720 jours)\n",
|
||
" ✅ EURCHF — 17,233 samples (719 jours)\n",
|
||
" ✅ EURAUD — 17,247 samples (720 jours)\n",
|
||
" ✅ EURNZD — 17,240 samples (720 jours)\n",
|
||
" ✅ GBPJPY — 17,233 samples (719 jours)\n",
|
||
" ✅ GBPCAD — 17,313 samples (723 jours)\n",
|
||
" ✅ GBPCHF — 17,230 samples (719 jours)\n",
|
||
" ✅ GBPAUD — 17,309 samples (723 jours)\n",
|
||
" ✅ AUDCAD — 17,318 samples (723 jours)\n",
|
||
" ✅ AUDNZD — 17,315 samples (723 jours)\n",
|
||
" ✅ AUDJPY — 17,234 samples (719 jours)\n",
|
||
" ✅ AUDCHF — 17,283 samples (722 jours)\n",
|
||
" ✅ CADJPY — 17,237 samples (720 jours)\n",
|
||
" ✅ CHFJPY — 17,221 samples (719 jours)\n",
|
||
" ✅ NZDJPY — 17,230 samples (719 jours)\n",
|
||
" ✅ NZDCAD — 17,309 samples (723 jours)\n",
|
||
"\n",
|
||
" Total: 431,361 samples | 62 features | 50.40% long labels\n",
|
||
"\n",
|
||
"[1b/7] Building sequence dataset (SEQ_LEN=168)...\n",
|
||
" → sampled 4,000 / 17,051 sequences (RAM cap)\n",
|
||
" [EURUSD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,052 sequences (RAM cap)\n",
|
||
" [GBPUSD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 16,938 sequences (RAM cap)\n",
|
||
" [USDJPY ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 16,992 sequences (RAM cap)\n",
|
||
" [USDCHF ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,150 sequences (RAM cap)\n",
|
||
" [AUDUSD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,143 sequences (RAM cap)\n",
|
||
" [NZDUSD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,155 sequences (RAM cap)\n",
|
||
" [USDCAD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,077 sequences (RAM cap)\n",
|
||
" [EURGBP ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,063 sequences (RAM cap)\n",
|
||
" [EURJPY ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,078 sequences (RAM cap)\n",
|
||
" [EURCAD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,062 sequences (RAM cap)\n",
|
||
" [EURCHF ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,076 sequences (RAM cap)\n",
|
||
" [EURAUD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,069 sequences (RAM cap)\n",
|
||
" [EURNZD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,062 sequences (RAM cap)\n",
|
||
" [GBPJPY ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,142 sequences (RAM cap)\n",
|
||
" [GBPCAD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,059 sequences (RAM cap)\n",
|
||
" [GBPCHF ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,138 sequences (RAM cap)\n",
|
||
" [GBPAUD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,147 sequences (RAM cap)\n",
|
||
" [AUDCAD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,144 sequences (RAM cap)\n",
|
||
" [AUDNZD ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,063 sequences (RAM cap)\n",
|
||
" [AUDJPY ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,112 sequences (RAM cap)\n",
|
||
" [AUDCHF ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,066 sequences (RAM cap)\n",
|
||
" [CADJPY ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,050 sequences (RAM cap)\n",
|
||
" [CHFJPY ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,059 sequences (RAM cap)\n",
|
||
" [NZDJPY ] 4,000 sequences\n",
|
||
" → sampled 4,000 / 17,138 sequences (RAM cap)\n",
|
||
" [NZDCAD ] 4,000 sequences\n",
|
||
"\n",
|
||
" Total : 100,000 sequences | shape X_seq=(100000, 168, 62) | 50.00% long labels\n",
|
||
" Sequence samples : 100,000 | shape: (100000, 168, 62)\n",
|
||
" Seq splits: train=70,000 | val=15,000 | test=15,000\n",
|
||
"\n",
|
||
"[2/7] Walk-forward cross-validation (tabular)...\n",
|
||
"\n",
|
||
" Walk-forward CV (4 windows):\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[500]\tvalid_0's binary_logloss: 0.537108\n",
|
||
" Window 1: train=215,680 test=53,920 acc=0.7102\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[500]\tvalid_0's binary_logloss: 0.536396\n",
|
||
" Window 2: train=269,600 test=53,920 acc=0.7090\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[500]\tvalid_0's binary_logloss: 0.529891\n",
|
||
" Window 3: train=323,520 test=53,920 acc=0.7125\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[500]\tvalid_0's binary_logloss: 0.530677\n",
|
||
" Window 4: train=377,440 test=53,920 acc=0.7138\n",
|
||
" CV accuracy: 0.7114 ± 0.0019\n",
|
||
"\n",
|
||
" Tabular splits: train=301,952 | val=64,704 | test=64,705\n",
|
||
"\n",
|
||
"[3/7] Hyperparameter search (30 Optuna trials)...\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[306]\tvalid_0's binary_logloss: 0.531696\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[256]\tvalid_0's binary_logloss: 0.536046\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.531071\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[176]\tvalid_0's binary_logloss: 0.534405\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[320]\tvalid_0's binary_logloss: 0.532699\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.529014\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.533946\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[997]\tvalid_0's binary_logloss: 0.531701\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[540]\tvalid_0's binary_logloss: 0.535239\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[519]\tvalid_0's binary_logloss: 0.530513\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[996]\tvalid_0's binary_logloss: 0.530435\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[725]\tvalid_0's binary_logloss: 0.529296\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[998]\tvalid_0's binary_logloss: 0.528426\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[987]\tvalid_0's binary_logloss: 0.529308\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[998]\tvalid_0's binary_logloss: 0.528884\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.529866\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[815]\tvalid_0's binary_logloss: 0.528147\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[469]\tvalid_0's binary_logloss: 0.530574\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.532568\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[897]\tvalid_0's binary_logloss: 0.528453\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.528905\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[695]\tvalid_0's binary_logloss: 0.529934\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[833]\tvalid_0's binary_logloss: 0.529049\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.52851\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[761]\tvalid_0's binary_logloss: 0.531103\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.529707\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.531528\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.529486\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Did not meet early stopping. Best iteration is:\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.532756\n",
|
||
"Training until validation scores don't improve for 30 rounds\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[310]\tvalid_0's binary_logloss: 0.531737\n",
|
||
" Best LightGBM accuracy (Optuna): 0.7147\n",
|
||
"\n",
|
||
"[4/7] Training tabular base models (LightGBM + XGBoost + RF)...\n",
|
||
" Training LightGBM...\n",
|
||
"Training until validation scores don't improve for 50 rounds\n",
|
||
"[200]\tvalid_0's binary_logloss: 0.537594\n",
|
||
"[400]\tvalid_0's binary_logloss: 0.532418\n",
|
||
"[600]\tvalid_0's binary_logloss: 0.530308\n",
|
||
"[800]\tvalid_0's binary_logloss: 0.529119\n",
|
||
"[1000]\tvalid_0's binary_logloss: 0.528434\n",
|
||
"Early stopping, best iteration is:\n",
|
||
"[1018]\tvalid_0's binary_logloss: 0.528387\n",
|
||
" Training XGBoost...\n",
|
||
" Training RandomForest...\n",
|
||
"\n",
|
||
"[4b/7] Training Deep Learning models (TFT + TransformerGRU)...\n",
|
||
" Training Temporal Fusion Transformer...\n",
|
||
" [TFT] device=cuda | train=70,000 val=15,000 samples\n",
|
||
" [TFT] parameters: 669,305\n",
|
||
"/content/ahad_quant/ahad_quant_v32_fixed/tft_model.py:280: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.\n",
|
||
" scaler_amp = torch.cuda.amp.GradScaler(enabled=torch.cuda.is_available())\n",
|
||
"/content/ahad_quant/ahad_quant_v32_fixed/tft_model.py:297: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.\n",
|
||
" with torch.cuda.amp.autocast(enabled=torch.cuda.is_available()):\n",
|
||
" [TFT] epoch 1/30 train=0.6950 val=0.6942 val_acc=0.5015 (82s)\n",
|
||
" [TFT] epoch 2/30 train=0.6936 val=0.6953 val_acc=0.5015 (163s)\n",
|
||
" [TFT] epoch 3/30 train=0.6935 val=0.6931 val_acc=0.5053 (243s)\n",
|
||
" [TFT] epoch 4/30 train=0.6934 val=0.6935 val_acc=0.5007 (324s)\n",
|
||
" [TFT] epoch 5/30 train=0.6931 val=0.6934 val_acc=0.5003 (405s)\n",
|
||
" [TFT] epoch 6/30 train=0.6934 val=0.6940 val_acc=0.5033 (486s)\n",
|
||
" [TFT] epoch 7/30 train=0.6932 val=0.6936 val_acc=0.5002 (566s)\n",
|
||
" [TFT] epoch 8/30 train=0.6929 val=0.6938 val_acc=0.5059 (647s)\n",
|
||
" [TFT] epoch 9/30 train=0.6926 val=0.6932 val_acc=0.5073 (728s)\n",
|
||
" [TFT] early stopping at epoch 9\n",
|
||
" [TFT] training complete — best val_loss=0.6931\n",
|
||
"\n",
|
||
" Training TransformerGRU...\n",
|
||
" [TGRU] device=cuda | train=70,000 val=15,000 samples\n",
|
||
" [TGRU] parameters: 588,033\n",
|
||
"/content/ahad_quant/ahad_quant_v32_fixed/transformer_gru_model.py:275: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.\n",
|
||
" scaler_amp = torch.cuda.amp.GradScaler(enabled=torch.cuda.is_available())\n",
|
||
"/content/ahad_quant/ahad_quant_v32_fixed/transformer_gru_model.py:292: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.\n",
|
||
" with torch.cuda.amp.autocast(enabled=torch.cuda.is_available()):\n",
|
||
" [TGRU] epoch 1/30 train=0.6419 val=0.5863 val_acc=0.6751 (32s)\n",
|
||
" [TGRU] epoch 2/30 train=0.5910 val=0.5618 val_acc=0.6897 (64s)\n",
|
||
" [TGRU] epoch 3/30 train=0.5722 val=0.5608 val_acc=0.6885 (96s)\n",
|
||
" [TGRU] epoch 4/30 train=0.5620 val=0.5450 val_acc=0.7002 (128s)\n",
|
||
" [TGRU] epoch 5/30 train=0.5493 val=0.5325 val_acc=0.7067 (160s)\n",
|
||
" [TGRU] epoch 6/30 train=0.5413 val=0.5278 val_acc=0.7097 (192s)\n",
|
||
" [TGRU] epoch 7/30 train=0.5334 val=0.5214 val_acc=0.7157 (224s)\n",
|
||
" [TGRU] epoch 8/30 train=0.5262 val=0.5349 val_acc=0.7007 (255s)\n",
|
||
" [TGRU] epoch 9/30 train=0.5193 val=0.5043 val_acc=0.7247 (287s)\n",
|
||
" [TGRU] epoch 10/30 train=0.5138 val=0.4947 val_acc=0.7262 (319s)\n",
|
||
" [TGRU] epoch 11/30 train=0.5041 val=0.4924 val_acc=0.7249 (351s)\n",
|
||
" [TGRU] epoch 12/30 train=0.4999 val=0.4847 val_acc=0.7306 (383s)\n",
|
||
" [TGRU] epoch 13/30 train=0.4940 val=0.4821 val_acc=0.7277 (415s)\n",
|
||
" [TGRU] epoch 14/30 train=0.4866 val=0.4807 val_acc=0.7303 (447s)\n",
|
||
" [TGRU] epoch 15/30 train=0.4838 val=0.4750 val_acc=0.7301 (479s)\n",
|
||
" [TGRU] epoch 16/30 train=0.4791 val=0.4762 val_acc=0.7289 (511s)\n",
|
||
" [TGRU] epoch 17/30 train=0.4743 val=0.4713 val_acc=0.7335 (543s)\n",
|
||
" [TGRU] epoch 18/30 train=0.4716 val=0.4662 val_acc=0.7378 (575s)\n",
|
||
" [TGRU] epoch 19/30 train=0.4677 val=0.4707 val_acc=0.7357 (606s)\n",
|
||
" [TGRU] epoch 20/30 train=0.4647 val=0.4677 val_acc=0.7347 (639s)\n",
|
||
" [TGRU] epoch 21/30 train=0.4622 val=0.4695 val_acc=0.7347 (670s)\n",
|
||
" [TGRU] epoch 22/30 train=0.4592 val=0.4632 val_acc=0.7383 (702s)\n",
|
||
" [TGRU] epoch 23/30 train=0.4566 val=0.4646 val_acc=0.7373 (734s)\n",
|
||
" [TGRU] epoch 24/30 train=0.4544 val=0.4650 val_acc=0.7395 (766s)\n",
|
||
" [TGRU] epoch 25/30 train=0.4532 val=0.4667 val_acc=0.7361 (798s)\n",
|
||
" [TGRU] epoch 26/30 train=0.4521 val=0.4632 val_acc=0.7394 (830s)\n",
|
||
" [TGRU] epoch 27/30 train=0.4504 val=0.4662 val_acc=0.7381 (862s)\n",
|
||
" [TGRU] epoch 28/30 train=0.4497 val=0.4646 val_acc=0.7389 (894s)\n",
|
||
" [TGRU] epoch 29/30 train=0.4478 val=0.4644 val_acc=0.7387 (926s)\n",
|
||
" [TGRU] epoch 30/30 train=0.4488 val=0.4646 val_acc=0.7397 (958s)\n",
|
||
" [TGRU] training complete — best val_loss=0.4632\n",
|
||
"\n",
|
||
"❌ Échec entraînement ML (code -9) — 88.7 min\n",
|
||
"\n",
|
||
"--- Dernières lignes du log ---\n",
|
||
" [TFT] training complete — best val_loss=0.6931\n",
|
||
"\n",
|
||
" Training TransformerGRU...\n",
|
||
" [TGRU] device=cuda | train=70,000 val=15,000 samples\n",
|
||
" [TGRU] parameters: 588,033\n",
|
||
"/content/ahad_quant/ahad_quant_v32_fixed/transformer_gru_model.py:275: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.\n",
|
||
" scaler_amp = torch.cuda.amp.GradScaler(enabled=torch.cuda.is_available())\n",
|
||
"/content/ahad_quant/ahad_quant_v32_fixed/transformer_gru_model.py:292: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.\n",
|
||
" with torch.cuda.amp.autocast(enabled=torch.cuda.is_available()):\n",
|
||
" [TGRU] epoch 1/30 train=0.6419 val=0.5863 val_acc=0.6751 (32s)\n",
|
||
" [TGRU] epoch 2/30 train=0.5910 val=0.5618 val_acc=0.6897 (64s)\n",
|
||
" [TGRU] epoch 3/30 train=0.5722 val=0.5608 val_acc=0.6885 (96s)\n",
|
||
" [TGRU] epoch 4/30 train=0.5620 val=0.5450 val_acc=0.7002 (128s)\n",
|
||
" [TGRU] epoch 5/30 train=0.5493 val=0.5325 val_acc=0.7067 (160s)\n",
|
||
" [TGRU] epoch 6/30 train=0.5413 val=0.5278 val_acc=0.7097 (192s)\n",
|
||
" [TGRU] epoch 7/30 train=0.5334 val=0.5214 val_acc=0.7157 (224s)\n",
|
||
" [TGRU] epoch 8/30 train=0.5262 val=0.5349 val_acc=0.7007 (255s)\n",
|
||
" [TGRU] epoch 9/30 train=0.5193 val=0.5043 val_acc=0.7247 (287s)\n",
|
||
" [TGRU] epoch 10/30 train=0.5138 val=0.4947 val_acc=0.7262 (319s)\n",
|
||
" [TGRU] epoch 11/30 train=0.5041 val=0.4924 val_acc=0.7249 (351s)\n",
|
||
" [TGRU] epoch 12/30 train=0.4999 val=0.4847 val_acc=0.7306 (383s)\n",
|
||
" [TGRU] epoch 13/30 train=0.4940 val=0.4821 val_acc=0.7277 (415s)\n",
|
||
" [TGRU] epoch 14/30 train=0.4866 val=0.4807 val_acc=0.7303 (447s)\n",
|
||
" [TGRU] epoch 15/30 train=0.4838 val=0.4750 val_acc=0.7301 (479s)\n",
|
||
" [TGRU] epoch 16/30 train=0.4791 val=0.4762 val_acc=0.7289 (511s)\n",
|
||
" [TGRU] epoch 17/30 train=0.4743 val=0.4713 val_acc=0.7335 (543s)\n",
|
||
" [TGRU] epoch 18/30 train=0.4716 val=0.4662 val_acc=0.7378 (575s)\n",
|
||
" [TGRU] epoch 19/30 train=0.4677 val=0.4707 val_acc=0.7357 (606s)\n",
|
||
" [TGRU] epoch 20/30 train=0.4647 val=0.4677 val_acc=0.7347 (639s)\n",
|
||
" [TGRU] epoch 21/30 train=0.4622 val=0.4695 val_acc=0.7347 (670s)\n",
|
||
" [TGRU] epoch 22/30 train=0.4592 val=0.4632 val_acc=0.7383 (702s)\n",
|
||
" [TGRU] epoch 23/30 train=0.4566 val=0.4646 val_acc=0.7373 (734s)\n",
|
||
" [TGRU] epoch 24/30 train=0.4544 val=0.4650 val_acc=0.7395 (766s)\n",
|
||
" [TGRU] epoch 25/30 train=0.4532 val=0.4667 val_acc=0.7361 (798s)\n",
|
||
" [TGRU] epoch 26/30 train=0.4521 val=0.4632 val_acc=0.7394 (830s)\n",
|
||
" [TGRU] epoch 27/30 train=0.4504 val=0.4662 val_acc=0.7381 (862s)\n",
|
||
" [TGRU] epoch 28/30 train=0.4497 val=0.4646 val_acc=0.7389 (894s)\n",
|
||
" [TGRU] epoch 29/30 train=0.4478 val=0.4644 val_acc=0.7387 (926s)\n",
|
||
" [TGRU] epoch 30/30 train=0.4488 val=0.4646 val_acc=0.7397 (958s)\n",
|
||
" [TGRU] training complete — best val_loss=0.4632\n",
|
||
"\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# ── 3.1 Entraînement du pipeline ML complet (version diagnostic) ────────────\n",
|
||
"import subprocess, sys, time, os\n",
|
||
"\n",
|
||
"print('🧠 Lancement entraînement ML...')\n",
|
||
"print(' LightGBM + XGBoost + RF + TFT + TransformerGRU + Meta-learner\\n')\n",
|
||
"\n",
|
||
"t0 = time.time()\n",
|
||
"with open('train_log.txt', 'w') as logfile:\n",
|
||
" proc = subprocess.Popen(\n",
|
||
" [sys.executable, '-u', 'train.py'],\n",
|
||
" stdout=subprocess.PIPE,\n",
|
||
" stderr=subprocess.STDOUT,\n",
|
||
" text=True,\n",
|
||
" bufsize=1,\n",
|
||
" )\n",
|
||
" for line in proc.stdout:\n",
|
||
" print(line, end='')\n",
|
||
" logfile.write(line)\n",
|
||
" logfile.flush()\n",
|
||
" proc.wait()\n",
|
||
"\n",
|
||
"elapsed = (time.time() - t0) / 60\n",
|
||
"if proc.returncode == 0:\n",
|
||
" print(f'\\n✅ Entraînement ML terminé en {elapsed:.1f} min')\n",
|
||
"else:\n",
|
||
" print(f'\\n❌ Échec entraînement ML (code {proc.returncode}) — {elapsed:.1f} min')\n",
|
||
" print('\\n--- Dernières lignes du log ---')\n",
|
||
" with open('train_log.txt') as f:\n",
|
||
" lines = f.readlines()\n",
|
||
" print(''.join(lines[-40:]))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"import os, shutil\n",
|
||
"\n",
|
||
"if os.path.exists('model_ensemble.pkl'):\n",
|
||
" print(f\"model_ensemble.pkl : {os.path.getsize('model_ensemble.pkl')/1024/1024:.1f} MB\")\n",
|
||
" if DRIVE_SAVE_DIR:\n",
|
||
" shutil.copy('model_ensemble.pkl', f'{DRIVE_SAVE_DIR}/model_ensemble.pkl')\n",
|
||
" print(f\"💾 Sauvegardé sur Drive : {DRIVE_SAVE_DIR}/model_ensemble.pkl\")\n",
|
||
"else:\n",
|
||
" print(\"⚠️ model_ensemble.pkl non trouvé\")"
|
||
],
|
||
"metadata": {
|
||
"id": "5dy7w3W8IvvS"
|
||
},
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# Nouvelle section"
|
||
],
|
||
"metadata": {
|
||
"id": "_tNgu2SucvZ-"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "SByjDXk4CtLF"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ── 3.2 Résultats ML (backtest rapide) ───────────────────────────────────────\n",
|
||
"import json\n",
|
||
"\n",
|
||
"# Lire les résultats de l'entraînement\n",
|
||
"results_files = ['last_retrain.json', 'backtest_results.json']\n",
|
||
"for rf in results_files:\n",
|
||
" if os.path.exists(rf):\n",
|
||
" with open(rf) as f:\n",
|
||
" data = json.load(f)\n",
|
||
" print(f'\\n📊 {rf}:')\n",
|
||
" for k, v in list(data.items())[:15]:\n",
|
||
" print(f' {k}: {v}')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "pAM1LE6hCtLG"
|
||
},
|
||
"source": [
|
||
"## 🎮 ÉTAPE 4 — Entraînement RL (PPO Agent)\n",
|
||
"\n",
|
||
"> 💡 **En cas de déconnexion Colab** : remonte Drive, retourne dans le dossier projet, puis relance **la cellule 4.2** avec `--resume` — l'entraînement reprend depuis le dernier checkpoint automatiquement."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "s3hvvDF8CtLG"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ── 4.1 Vérification pré-RL ──────────────────────────────────────────────────\n",
|
||
"import os, json\n",
|
||
"\n",
|
||
"print('🔍 Vérification des prérequis RL...')\n",
|
||
"checks = {\n",
|
||
" 'model_ensemble.pkl' : 'Modèle ML',\n",
|
||
" 'data/' : 'Données Forex',\n",
|
||
"}\n",
|
||
"ok = True\n",
|
||
"for path, label in checks.items():\n",
|
||
" exists = os.path.exists(path)\n",
|
||
" print(f' {\"✅\" if exists else \"❌\"} {label} ({path})')\n",
|
||
" if not exists:\n",
|
||
" ok = False\n",
|
||
"\n",
|
||
"if ok:\n",
|
||
" print('\\n✅ Tous les prérequis sont présents — tu peux lancer l\\'entraînement RL')\n",
|
||
"else:\n",
|
||
" print('\\n❌ Lance d\\'abord les étapes 2 et 3 !')\n",
|
||
"\n",
|
||
"# Config RL\n",
|
||
"RL_STEPS = 1_000_000 # ← Modifier ici si besoin (500k = rapide, 2M = meilleure qualité)\n",
|
||
"print(f'\\n⚙️ Steps RL configurés : {RL_STEPS:,}')\n",
|
||
"print(f' Durée estimée : ~{RL_STEPS/60000:.0f} min (GPU T4)')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "huE9Oa6BCtLG"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ── 4.2 Entraînement RL PPO ──────────────────────────────────────────────────\n",
|
||
"# ⚠️ En cas de reprise après déconnexion : changer --steps en 0 et garder --resume\n",
|
||
"import subprocess, sys, time, os, shutil\n",
|
||
"\n",
|
||
"RESUME = os.path.exists('rl_progress.json') and os.path.getsize('rl_progress.json') > 5\n",
|
||
"\n",
|
||
"if RESUME:\n",
|
||
" with open('rl_progress.json') as f:\n",
|
||
" prog = json.load(f)\n",
|
||
" done = prog.get('steps_done', 0)\n",
|
||
" total = prog.get('total_steps', RL_STEPS)\n",
|
||
" print(f'♻️ Reprise détectée : {done:,} / {total:,} steps déjà effectués')\n",
|
||
" resume_flag = ['--resume']\n",
|
||
"else:\n",
|
||
" print('🆕 Nouvel entraînement RL')\n",
|
||
" resume_flag = []\n",
|
||
"\n",
|
||
"print(f'\\n🎮 Lancement PPO ({RL_STEPS:,} steps)...')\n",
|
||
"print(' Checkpoints sauvegardés toutes les 50k steps dans rl_checkpoints/')\n",
|
||
"print(' Durée estimée : 45-90 min (GPU T4)\\n')\n",
|
||
"\n",
|
||
"t0 = time.time()\n",
|
||
"result = subprocess.run(\n",
|
||
" [sys.executable, 'rl_train.py', '--steps', str(RL_STEPS)] + resume_flag,\n",
|
||
" capture_output=False,\n",
|
||
" text=True\n",
|
||
")\n",
|
||
"elapsed = (time.time() - t0) / 60\n",
|
||
"\n",
|
||
"if result.returncode == 0:\n",
|
||
" print(f'\\n✅ Entraînement RL terminé en {elapsed:.1f} min')\n",
|
||
" for f in ['rl_agent.zip', 'rl_scaler.pkl']:\n",
|
||
" if os.path.exists(f):\n",
|
||
" print(f' {f} : {os.path.getsize(f)/1024:.0f} KB')\n",
|
||
"\n",
|
||
" # Sauvegarde Drive\n",
|
||
" if DRIVE_SAVE_DIR:\n",
|
||
" for f in ['rl_agent.zip', 'rl_scaler.pkl']:\n",
|
||
" if os.path.exists(f):\n",
|
||
" shutil.copy(f, f'{DRIVE_SAVE_DIR}/{f}')\n",
|
||
" # Sauvegarder aussi les checkpoints\n",
|
||
" if os.path.exists('rl_checkpoints'):\n",
|
||
" shutil.copytree('rl_checkpoints', f'{DRIVE_SAVE_DIR}/rl_checkpoints', dirs_exist_ok=True)\n",
|
||
" print(f' 💾 Sauvegardé sur Drive : {DRIVE_SAVE_DIR}/')\n",
|
||
"else:\n",
|
||
" print(f'\\n❌ Échec RL (code {result.returncode}) — {elapsed:.1f} min')\n",
|
||
" print(' ➡️ Relance cette cellule avec --resume pour reprendre')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "HDnmu9rRCtLG"
|
||
},
|
||
"source": [
|
||
"## 📦 ÉTAPE 5 — Export modèle unifié & Téléchargement"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "yfCaEDZ9CtLH"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ── 5.1 Export ahad_quant_unified.zip ─────────────────────────────────────────\n",
|
||
"import subprocess, sys, os, shutil\n",
|
||
"\n",
|
||
"print('📦 Export du modèle unifié...')\n",
|
||
"result = subprocess.run(\n",
|
||
" [sys.executable, 'export_unified.py'],\n",
|
||
" capture_output=False,\n",
|
||
" text=True\n",
|
||
")\n",
|
||
"\n",
|
||
"if result.returncode == 0 and os.path.exists('ahad_quant_unified.zip'):\n",
|
||
" size_mb = os.path.getsize('ahad_quant_unified.zip') / 1024 / 1024\n",
|
||
" print(f'✅ ahad_quant_unified.zip créé ({size_mb:.1f} MB)')\n",
|
||
"\n",
|
||
" if DRIVE_SAVE_DIR:\n",
|
||
" shutil.copy('ahad_quant_unified.zip', f'{DRIVE_SAVE_DIR}/ahad_quant_unified.zip')\n",
|
||
" print(f'💾 Sauvegardé sur Drive : {DRIVE_SAVE_DIR}/ahad_quant_unified.zip')\n",
|
||
"else:\n",
|
||
" print('❌ Export échoué')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "nDiLmnW4CtLH"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ── 5.2 Créer un bundle complet avec tous les modèles ────────────────────────\n",
|
||
"import zipfile, os, datetime, shutil\n",
|
||
"\n",
|
||
"timestamp = datetime.datetime.now().strftime('%Y%m%d_%H%M')\n",
|
||
"bundle_name = f'ahad_quant_trained_{timestamp}.zip'\n",
|
||
"\n",
|
||
"FILES_TO_BUNDLE = [\n",
|
||
" 'ahad_quant_unified.zip', # modèle unifié complet\n",
|
||
" 'model_ensemble.pkl', # modèle ML ensemble\n",
|
||
" 'rl_agent.zip', # agent RL PPO\n",
|
||
" 'rl_scaler.pkl', # scaler RL\n",
|
||
" 'backtest_results.json', # résultats backtest\n",
|
||
" 'last_retrain.json', # méta-infos entraînement\n",
|
||
"]\n",
|
||
"\n",
|
||
"print(f'📦 Création du bundle {bundle_name}...')\n",
|
||
"with zipfile.ZipFile(bundle_name, 'w', zipfile.ZIP_DEFLATED) as zf:\n",
|
||
" for f in FILES_TO_BUNDLE:\n",
|
||
" if os.path.exists(f):\n",
|
||
" zf.write(f)\n",
|
||
" print(f' + {f} ({os.path.getsize(f)/1024:.0f} KB)')\n",
|
||
" else:\n",
|
||
" print(f' ⚠️ {f} — non trouvé, ignoré')\n",
|
||
"\n",
|
||
" # Ajouter les checkpoints RL\n",
|
||
" if os.path.exists('rl_checkpoints'):\n",
|
||
" for root, dirs, files in os.walk('rl_checkpoints'):\n",
|
||
" for file in files:\n",
|
||
" fp = os.path.join(root, file)\n",
|
||
" zf.write(fp)\n",
|
||
" print(f' + rl_checkpoints/ ({len(os.listdir(\"rl_checkpoints\"))} fichiers)')\n",
|
||
"\n",
|
||
"size_mb = os.path.getsize(bundle_name) / 1024 / 1024\n",
|
||
"print(f'\\n✅ Bundle créé : {bundle_name} ({size_mb:.1f} MB)')\n",
|
||
"\n",
|
||
"if DRIVE_SAVE_DIR:\n",
|
||
" shutil.copy(bundle_name, f'{DRIVE_SAVE_DIR}/{bundle_name}')\n",
|
||
" print(f'💾 Sauvegardé sur Drive : {DRIVE_SAVE_DIR}/{bundle_name}')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "OIgijId7CtLH"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ── 5.3 Téléchargement direct depuis Colab ───────────────────────────────────\n",
|
||
"from google.colab import files\n",
|
||
"import os\n",
|
||
"\n",
|
||
"# Choisir ce qu'on veut télécharger\n",
|
||
"TO_DOWNLOAD = [\n",
|
||
" bundle_name, # bundle complet (recommandé)\n",
|
||
" # 'ahad_quant_unified.zip', # ou juste le modèle unifié\n",
|
||
"]\n",
|
||
"\n",
|
||
"for f in TO_DOWNLOAD:\n",
|
||
" if os.path.exists(f):\n",
|
||
" size_mb = os.path.getsize(f) / 1024 / 1024\n",
|
||
" print(f'⬇️ Téléchargement de {f} ({size_mb:.1f} MB)...')\n",
|
||
" files.download(f)\n",
|
||
" else:\n",
|
||
" print(f'⚠️ {f} non trouvé')\n",
|
||
"\n",
|
||
"print('\\n✅ Terminé ! Place les fichiers .pkl et .zip dans ton dossier ahad_quant_v32_fixed/ sur ta machine.')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "aVRssBSDCtLH"
|
||
},
|
||
"source": [
|
||
"## 🔁 BONUS — Relancer uniquement le RL après déconnexion\n",
|
||
"\n",
|
||
"Si Colab s'est déconnecté **pendant le RL** (étape 4), exécute ces cellules dans l'ordre :"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "RsKDPLODCtLI"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ── REPRISE RAPIDE (après déconnexion pendant le RL) ─────────────────────────\n",
|
||
"# 1. Monte Drive\n",
|
||
"from google.colab import drive\n",
|
||
"drive.mount('/content/drive')\n",
|
||
"\n",
|
||
"# 2. Réinstalle les packages\n",
|
||
"!pip install -q lightgbm xgboost scikit-learn optuna yfinance stable-baselines3[extra] gymnasium shimmy python-dotenv torch --index-url https://download.pytorch.org/whl/cu118\n",
|
||
"\n",
|
||
"# 3. Remonte le projet depuis Drive\n",
|
||
"import os, shutil, json\n",
|
||
"DRIVE_SAVE_DIR = '/content/drive/MyDrive/AhadQuant_Models'\n",
|
||
"PROJECT_DIR = '/content/ahad_quant_resume'\n",
|
||
"os.makedirs(PROJECT_DIR, exist_ok=True)\n",
|
||
"\n",
|
||
"# Copie le ZIP du projet depuis Drive (si tu l'avais mis là)\n",
|
||
"# OU remonte le ZIP original depuis upload\n",
|
||
"# from google.colab import files\n",
|
||
"# uploaded = files.upload() # ← décommente si besoin\n",
|
||
"\n",
|
||
"# 4. Copie les modèles existants depuis Drive\n",
|
||
"for f in ['model_ensemble.pkl', 'rl_agent.zip', 'rl_scaler.pkl', 'rl_progress.json']:\n",
|
||
" src = f'{DRIVE_SAVE_DIR}/{f}'\n",
|
||
" if os.path.exists(src):\n",
|
||
" shutil.copy(src, f'{PROJECT_DIR}/{f}')\n",
|
||
" print(f'✅ Restauré : {f}')\n",
|
||
"\n",
|
||
"if os.path.exists(f'{DRIVE_SAVE_DIR}/rl_checkpoints'):\n",
|
||
" shutil.copytree(f'{DRIVE_SAVE_DIR}/rl_checkpoints', f'{PROJECT_DIR}/rl_checkpoints', dirs_exist_ok=True)\n",
|
||
" print('✅ Restauré : rl_checkpoints/')\n",
|
||
"\n",
|
||
"# 5. Affiche la progression\n",
|
||
"prog_file = f'{PROJECT_DIR}/rl_progress.json'\n",
|
||
"if os.path.exists(prog_file):\n",
|
||
" with open(prog_file) as f:\n",
|
||
" prog = json.load(f)\n",
|
||
" steps_done = prog.get('steps_done', 0)\n",
|
||
" total = prog.get('total_steps', 1_000_000)\n",
|
||
" pct = steps_done / total * 100\n",
|
||
" print(f'\\n📊 Progression RL : {steps_done:,} / {total:,} steps ({pct:.1f}%)')\n",
|
||
" print('➡️ Maintenant exécute la cellule 4.2 — la reprise sera détectée automatiquement')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "VcruETjFCtLI"
|
||
},
|
||
"source": [
|
||
"## 📋 RÉCAPITULATIF — Fichiers à récupérer\n",
|
||
"\n",
|
||
"Après l'entraînement, copie ces fichiers dans ton dossier `ahad_quant_v32_fixed/` sur ta machine Windows :\n",
|
||
"\n",
|
||
"| Fichier | Description | Obligatoire |\n",
|
||
"|---------|-------------|-------------|\n",
|
||
"| `ahad_quant_unified.zip` | Modèle complet (ML + RL) | ✅ |\n",
|
||
"| `model_ensemble.pkl` | Ensemble ML seul | ✅ |\n",
|
||
"| `rl_agent.zip` | Agent PPO seul | ✅ |\n",
|
||
"| `rl_scaler.pkl` | Scaler features RL | ✅ |\n",
|
||
"| `rl_checkpoints/` | Checkpoints intermédiaires | Optionnel |\n",
|
||
"\n",
|
||
"Ensuite, depuis le Web UI : **lance le bot** — il chargera `ahad_quant_unified.zip` automatiquement."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"!free -h"
|
||
],
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "HWq8mt_cc0pa",
|
||
"outputId": "e35c3e7a-7b80-4abf-84b0-7e5ce2d226c6"
|
||
},
|
||
"execution_count": 8,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
" total used free shared buff/cache available\n",
|
||
"Mem: 12Gi 1.1Gi 10Gi 2.0Mi 785Mi 11Gi\n",
|
||
"Swap: 0B 0B 0B\n"
|
||
]
|
||
}
|
||
]
|
||
}
|
||
]
|
||
} |