chore: fix PR template (#487)

* fix PR template

* subfolders of the log displayed on the web page

* fix code error

* add health check

* reformat with isort

* reformat with black

* update README

* fix bug && experience confusing

* reformat with black & update health check code

* reformat with isort

* reformat with black

* update README.md

* Removed duplicates in documentation and health_check && Upgraded code for filtering folders

* update docs

* reformat with black

.github/PULL_REQUEST_TEMPLATE.md

@@ -20,14 +20,12 @@
 
 ## How Has This Been Tested?
 <!---  Put an `x` in all the boxes that apply: --->
-- [ ] Pass the test by running: `pytest qlib/tests/test_all_pipeline.py` under upper directory of `qlib`.
 - [ ] If you are adding a new feature, test on your own test scripts.
 
 <!--- **ATTENTION**: If you are adding a new feature, please make sure your codes are **correctly tested**. If our test scripts do not cover your cases, please provide your own test scripts under the `tests` folder and test them. More information about test scripts can be found [here](https://docs.python.org/3/library/unittest.html#basic-example), or you could refer to those we provide under the `tests` folder. -->
 
 ## Screenshots of Test Results (if appropriate):
-1. Pipeline test:
-2. Your own tests:
+1. Your own tests:
 
 ## Types of changes
 <!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->

README.md

@@ -83,15 +83,44 @@ Users must ensure Docker is installed before attempting most scenarios. Please r
   pip install rdagent
   ```
 
+### 💊 Health check
+- rdagent provides a health check that currently checks two things.
+  - whether the docker installation was successful.
+  - whether the default port used by the [rdagent ui](https://github.com/microsoft/RD-Agent?tab=readme-ov-file#%EF%B8%8F-monitor-the-application-results) is occupied.
+  ```sh
+  rdagent health_check
+  ```
+
+
 ### ⚙️ Configuration
-- You have to config your GPT model in the `.env`
+- The demos requires following ability:
+  - ChatCompletion
+  - json_mode
+  - embedding query
+
+- For example: If you are using the `OpenAI API`, you have to configure your GPT model in the `.env` file like this.
   ```bash
   cat << EOF  > .env
-  OPENAI_API_KEY=<your_api_key>
+  OPENAI_API_KEY=<replace_with_your_openai_api_key>
   # EMBEDDING_MODEL=text-embedding-3-small
   CHAT_MODEL=gpt-4-turbo
   EOF
   ```
+- However, not every API services support these features by devault. For example: `AZURE OpenAI`, you have to configure your GPT model in the `.env` file like this.
+  ```bash
+  cat << EOF  > .env
+  USE_AZURE=True
+  EMBEDDING_OPENAI_API_KEY=<replace_with_your_azure_openai_api_key>
+  EMBEDDING_AZURE_API_BASE=<replace_with_your_azure_endpoint>
+  EMBEDDING_AZURE_API_VERSION=<replace_with_the_version_of_your_azure_openai_api>
+  EMBEDDING_MODEL=text-embedding-3-small
+  CHAT_OPENAI_API_KEY=<replace_with_your_azure_openai_api_key>
+  CHAT_AZURE_API_BASE=<replace_with_your_azure_endpoint>
+  CHAT_AZURE_API_VERSION=<replace_with_the_version_of_your_azure_openai_api>
+  CHAT_MODEL=<replace_it_with_the_name_of_your_azure_chat_model>
+  EOF
+  ```
+- For more configuration information, please refer to the [documentation](https://rdagent.readthedocs.io/en/latest/installation_and_configuration.html).
 
 ### 🚀 Run the Application
 
@@ -170,9 +199,18 @@ The **[🖥️ Live Demo](https://rdagent.azurewebsites.net/)** is implemented b
   > - The **Competition List Available** can be found [here](https://rdagent.readthedocs.io/en/latest/scens/kaggle_agent.html#competition-list-available). <br />
 
 ### 🖥️ Monitor the Application Results
-- You can serve our demo app to monitor the RD loop by running the following command:
+- You can run the following command for our demo program to see the run logs.
+
   ```sh
-  rdagent ui --port 80 --log_dir <your log folder like "log/">
+  rdagent ui --port 19899 --log_dir <your log folder like "log/">
+  ```
+
+  **Note:** Although port 19899 is not commonly used, but before you run this demo, you need to check if port 19899 is occupied. If it is, please change it to another port that is not occupied.
+
+  You can check if a port is occupied by running the following command.
+
+  ```sh
+  rdagent health_check
   ```
 
 # 🏭 Scenarios
@@ -208,11 +246,7 @@ The supported scenarios are listed below:
 
 Different scenarios vary in entrance and configuration. Please check the detailed setup tutorial in the scenarios documents.
 
-Here is a gallery of [successful explorations](https://github.com/SunsetWolf/rdagent_resource/releases/download/demo_traces/demo_traces.zip) (5 traces showed in **[🖥️ Live Demo](https://rdagent.azurewebsites.net/)**). You can download and view the execution trace using the command below:
-
-```bash
-rdagent ui --port 80 --log_dir ./demo_traces
-```
+Here is a gallery of [successful explorations](https://github.com/SunsetWolf/rdagent_resource/releases/download/demo_traces/demo_traces.zip) (5 traces showed in **[🖥️ Live Demo](https://rdagent.azurewebsites.net/)**). You can download and view the execution trace using [this command](https://github.com/microsoft/RD-Agent?tab=readme-ov-file#%EF%B8%8F-monitor-the-application-results) from the documentation.
 
 Please refer to **[📖readthedocs_scen](https://rdagent.readthedocs.io/en/latest/scens/catalog.html)** for more details of the scenarios.
 

docs/installation_and_configuration.rst

@@ -38,17 +38,18 @@ Azure OpenAI
 The following environment variables are standard configuration options for the user using the OpenAI API.
 
    .. code-block:: Properties
-      
+
       USE_AZURE=True
 
-      OPENAI_API_KEY=<replace_with_your_openai_api_key>
-      
+      EMBEDDING_OPENAI_API_KEY=<replace_with_your_azure_openai_api_key>
+      EMBEDDING_AZURE_API_BASE=  # The endpoint for the Azure OpenAI API.
+      EMBEDDING_AZURE_API_VERSION=  # The version of the Azure OpenAI API.
       EMBEDDING_MODEL=text-embedding-3-small
-      EMBEDDING_AZURE_API_BASE= # The base URL for the Azure OpenAI API.
-      EMBEDDING_AZURE_API_VERSION = # The version of the Azure OpenAI API.
 
-      CHAT_MODEL=gpt-4-turbo
-      CHAT_AZURE_API_VERSION = # The version of the Azure OpenAI API.
+      CHAT_OPENAI_API_KEY=<replace_with_your_azure_openai_api_key>
+      CHAT_AZURE_API_BASE=  # The endpoint for the Azure OpenAI API.
+      CHAT_AZURE_API_VERSION=  # The version of the Azure OpenAI API.
+      CHAT_MODEL=  # The model name of the Azure OpenAI API.
 
 Use Azure Token Provider
 ------------------------

rdagent/app/cli.py

@@ -25,10 +25,11 @@ from rdagent.app.kaggle.loop import main as kaggle_main
 from rdagent.app.qlib_rd_loop.factor import main as fin_factor
 from rdagent.app.qlib_rd_loop.factor_from_report import main as fin_factor_report
 from rdagent.app.qlib_rd_loop.model import main as fin_model
+from rdagent.app.utils.health_check import health_check
 from rdagent.app.utils.info import collect_info
 
 
-def ui(port=80, log_dir="", debug=False):
+def ui(port=19899, log_dir="", debug=False):
     """
     start web app to show the log traces.
     """
@@ -52,6 +53,7 @@ def app():
             "med_model": med_model,
             "general_model": general_model,
             "ui": ui,
+            "health_check": health_check,
             "collect_info": collect_info,
             "kaggle": kaggle_main,
         }

rdagent/app/general_model/general_model.py

@@ -42,7 +42,6 @@ def extract_models_and_implement(report_file_path: str) -> None:
     with logger.tag("d"):
         exp = QlibModelCoSTEER(scenario).develop(exp)
         logger.log_object(exp, tag="developed_experiment")
-    return exp
 
 
 if __name__ == "__main__":

rdagent/app/utils/health_check.py

@@ -0,0 +1,49 @@
+import socket
+
+import docker
+
+from rdagent.log import rdagent_logger as logger
+
+
+def check_docker() -> None:
+    try:
+        client = docker.from_env()
+        client.images.pull("hello-world")
+        container = client.containers.run("hello-world", detach=True)
+        logs = container.logs().decode("utf-8")
+        print(logs)
+        container.remove()
+        logger.info(f"The docker status is normal")
+    except docker.errors.DockerException as e:
+        logger.error(f"An error occurred: {e}")
+        logger.warning(
+            f"Docker status is exception, please check the docker configuration or reinstall it. Refs: https://docs.docker.com/engine/install/ubuntu/."
+        )
+
+
+def is_port_in_use(port):
+    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+        return s.connect_ex(("127.0.0.1", port)) == 0
+
+
+def check_and_list_free_ports(start_port=19899, max_ports=10) -> None:
+    is_occupied = is_port_in_use(port=start_port)
+    if is_occupied:
+        free_ports = []
+        for port in range(start_port, start_port + max_ports):
+            if not is_port_in_use(port):
+                free_ports.append(port)
+        logger.warning(
+            f"Port 19899 is occupied, please replace it with an available port when running the `rdagent ui` command. Available ports: {free_ports}"
+        )
+    else:
+        logger.info(f"Port 19899 is not occupied, you can run the `rdagent ui` command")
+
+
+def health_check():
+    """
+    Check that docker is installed correctly,
+    and that the ports used in the sample README are not occupied.
+    """
+    check_docker()
+    check_and_list_free_ports()

rdagent/log/ui/app.py

@@ -61,9 +61,25 @@ QLIB_SELECTED_METRICS = [
 
 SIMILAR_SCENARIOS = (QlibModelScenario, DMModelScenario, QlibFactorScenario, QlibFactorFromReportScenario, KGScenario)
 
+
+def filter_log_folders(main_log_path):
+    """
+    The webpage only displays valid folders.
+    If the __session__ folder exists in a subfolder of the log folder, it is considered a valid folder,
+    otherwise it is considered an invalid folder.
+    """
+    folders = [
+        folder.relative_to(main_log_path)
+        for folder in main_log_path.iterdir()
+        if folder.is_dir() and folder.joinpath("__session__").exists() and folder.joinpath("__session__").is_dir()
+    ]
+    folders = sorted(folders, key=lambda x: x.name)
+    return folders
+
+
 if "log_path" not in state:
     if main_log_path:
-        state.log_path = next(main_log_path.iterdir()).relative_to(main_log_path)
+        state.log_path = filter_log_folders(main_log_path)[0]
     else:
         state.log_path = None
         st.toast(":red[**Please Set Log Path!**]", icon="⚠️")
@@ -505,7 +521,7 @@ def research_window():
             # pdf image
             if pim := state.msgs[round]["r.extract_factors_and_implement.load_pdf_screenshot"]:
                 for i in range(min(2, len(pim))):
-                    st.image(pim[i].content, use_column_width=True)
+                    st.image(pim[i].content, use_container_width=True)
 
             # Hypothesis
             if hg := state.msgs[round]["r.hypothesis generation"]:
@@ -526,7 +542,7 @@ def research_window():
             with c1:
                 if pim := state.msgs[round]["r.pdf_image"]:
                     for i in range(len(pim)):
-                        st.image(pim[i].content, use_column_width=True)
+                        st.image(pim[i].content, use_container_width=True)
 
             # loaded model exp
             with c2:
@@ -679,8 +695,7 @@ with st.sidebar:
             if manually:
                 st.text_input("log path", key="log_path", on_change=refresh, label_visibility="collapsed")
             else:
-                folders = [folder.relative_to(main_log_path) for folder in main_log_path.iterdir() if folder.is_dir()]
-                folders = sorted(folders, key=lambda x: x.name)
+                folders = filter_log_folders(main_log_path)
                 st.selectbox(f"**Select from `{main_log_path}`**", folders, key="log_path", on_change=refresh)
         else:
             st.text_input(":blue[**log path**]", key="log_path", on_change=refresh)
@@ -763,7 +778,7 @@ with st.container():
     image_c, scen_c = st.columns([3, 3], vertical_alignment="center")
     with image_c:
         img_path = rfiles("rdagent.log.ui").joinpath("flow.png")
-        st.image(str(img_path), use_column_width=True)
+        st.image(str(img_path), use_container_width=True)
     with scen_c:
         st.header("Scenario Description📖", divider="violet", anchor="_scenario")
         if state.scenario is not None: