NexQuant/rdagent/log/server/README.md

API

A. Controls

1. /upload [POST]

Request

• "scenario": one of six values
  1. "Finance Data Building"
  2. "Finance Data Building (Reports)"
  3. "Finance Model Implementation"
  4. "General Model Implementation"
  5. "Medical Model Implementation"
  6. "Data Science"
• "files": 2 scenarios need this
  1. in "Finance Data Building (Reports)" Scenario, one or more pdf files.
  2. in "General Model Implementation" Scenario, one pdf file or one pdf link like https://arxiv.org/pdf/2210.09789
• "competition": Data Science Scenario need this, one of 75 competitions.
• "loops": Number of loops after which RD-Agent will automatically stop (optional; if not set, it will not stop automatically and must be stopped manually).
• "all_duration": Total duration (in hours) for which the RD-Agent should run before stopping automatically. If not set, the agent will continue running until stopped manually or by the "loops" parameter.

Response

• "id": a unique identifier string, such as /home/rdagent_log/data_science/competition_A/trace_1 or /home/rdagent_log/finance/trace_1, used to mark the series of logs generated by this RD-Agent run.

2. /control [POST]

Request

• "id": identifier
• "action": one of three values
  1. "pause"
  2. "resume"
  3. "stop"

Response

• "status": "success" / "error: ..."

3. /trace [POST]

Returns the sequence of Messages generated for the current id on the backend that have not yet been returned to the frontend.

Request

• "id": identifier
• "all": True / False. True means all Messages not yet provided to the frontend will be returned; False returns a random 1 to 10 Messages. In most cases, this should be True.
• "reset": True / False. Reset means the pointer for "not yet returned to the frontend" will be set back to the first Message generated for this id, i.e., return from the beginning. In most cases, this should be False.

Response

• a list of Messages

B. Messages

Research

Only 2 Message in one loop

1. hypothesis

{
    "tag": "research.hypothesis",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "content": {
        "hypothesis": "...",
        "reason": "...",
        "component": "...", // only exists in Data Science Scenario
        "concise_reason": "...",
        "concise_justification": "...",
        "concise_observation": "...",
        "concise_knowledge": "...",
    }
}

2. tasks

{
    "tag": "research.tasks",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "content": [    // list of tasks
        {
            "name": "...",
            "description": "...",
            "model_type": "...", // only exists in "Finance Model Implementation", "General Model Implementation", "Medical Model Implementation", or some tasks of "Data Science"
            "architecture": "...", // same as above
            "hyperparameters": "...", // same as above
        },
        {

        }
        //... same as above
    ]
}

evolving

• 1 to 10 pairs of Messages (codes & feedbacks), each identified by an "evo_id" indicating the evolving round.
• In the Data Science scenario, each evolving round contains only one task, but the "codes" for that task may include multiple code files.
• In other scenarios, each evolving round may contain multiple tasks, but each task's "codes" will include only one code file.

1. codes

{
    "tag": "evolving.codes",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "evo_id": "0",
    "content": [ // list of task_name & codes
        {
            "target_task_name": "task_1",
            "codes": { // one or more codes
                "a.py": "...<python codes>",
                "b.py": "...<python codes>",
                //...
            }
        },
        {
            "target_task_name": "task_2",
            "codes": {
                "a.py": "...<python codes>",
                //...
            }
        }
        //... same as above
    ]
}

{
    "tag": "evolving.codes",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "evo_id": "1",
    "content": [
        //... same as above
    ]
}

2. feedbacks

{
    "tag": "evolving.feedbacks",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "evo_id": "0",
    "content": [ // list of feedbacks
        {
            "final_decision": "True", // True or False
            "execution": "...",
            "code": "...",
            "return_checking": "..."
        },
        //... same as above
    ]
}

{
    "tag": "evolving.codes",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "evo_id": "1",
    "content": [
        //... same as above
    ]
}

feedback

Each tag below appears only once per loop.

1. config (only exists in "Finance Data Building"/"Finance Data Building (Reports)"/"Finance Model Implementation")

{
    "tag": "feedback.config",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "content": {
        "config": "a markdown string",
    }
}

2. return_chart (only exists in "Finance Data Building"/"Finance Data Building (Reports)"/"Finance Model Implementation")

{
    "tag": "feedback.return_chart",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "content": {
        "chart_html": "chart html codes string",
    }
}

3. metric

{
    "tag": "feedback.metric",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "content": {
        "result": "{ \"<metric_name>\": <value>, ... }" // A JSON string containing metric names and their corresponding values.
    }
}

4. hypothesis_feedback

{
    "tag": "feedback.hypothesis_feedback",
    "timestamp": "<isoformat>",
    "loop_id": "1",
    "content": {
        "decision": "True",
        "reason": "...",
        "exception": "...",
        "observations": "...", // may not exists
        "hypothesis_evaluation": "...", // may not exists
        "new_hypothesis": "...", // may not exists
    }
}