diff --git a/README.md b/README.md index d8e26a41..64022a3c 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ https://github.com/user-attachments/assets/3eccbecb-34a4-4c81-bce4-d3f8862f7305 # ๐Ÿ“ฐ News | ๐Ÿ—ž๏ธ News | ๐Ÿ“ Description | | -- | ------ | +| Support LiteLLM Backend | We now fully support **[LiteLLM](https://github.com/BerriAI/litellm)** as a backend for integration with multiple LLM providers. | | More General Data Science Agent | ๐Ÿš€Coming soon! | | Kaggle Scenario release | We release **[Kaggle Agent](https://rdagent.readthedocs.io/en/latest/scens/kaggle_agent.html)**, try the new features! | | Official WeChat group release | We created a WeChat group, welcome to join! (๐Ÿ—ช[QR Code](docs/WeChat_QR_code.jpg)) | @@ -126,6 +127,18 @@ Users must ensure Docker is installed before attempting most scenarios. Please r CHAT_MODEL= EOF ``` + +- We now support LiteLLM as a backend for integration with multiple LLM providers. If you use LiteLLM Backend to use models, you can configure as follows: + ```bash + cat << EOF > .env + BACKEND=rdagent.oai.backend.LiteLLMAPIBackend + # It can be modified to any model supported by LiteLLM. + CHAT_MODEL=gpt-4o + EMBEDDING_MODEL=text-embedding-3-small + # The backend api_key fully follow the convention of litellm. + OPENAI_API_KEY= + ``` + - For more configuration information, please refer to the [documentation](https://rdagent.readthedocs.io/en/latest/installation_and_configuration.html). ### ๐Ÿš€ Run the Application diff --git a/docs/installation_and_configuration.rst b/docs/installation_and_configuration.rst index 4363486c..a97f50b6 100644 --- a/docs/installation_and_configuration.rst +++ b/docs/installation_and_configuration.rst @@ -12,8 +12,49 @@ Installation **Install Docker**: RDAgent is designed for research and development, acting like a human researcher and developer. It can write and run code in various environments, primarily using Docker for code execution. This keeps the remaining dependencies simple. Users must ensure Docker is installed before attempting most scenarios. Please refer to the `official ๐ŸณDocker page `_ for installation instructions. -Configuration -============= +LiteLLM Backend Configuration +============================= + +Please create a `.env` file in the root directory of the project and add environment variables. + +Here is a sample configuration for using OpenAI's gpt-4o via LiteLLM. + + .. code-block:: Properties + + BACKEND=rdagent.oai.backend.LiteLLMAPIBackend + # It can be modified to any model supported by LiteLLM. + CHAT_MODEL=gpt-4o + EMBEDDING_MODEL=text-embedding-3-small + # The backend api_key fully follows the convention of litellm. + OPENAI_API_KEY= + +Necessary parameters include: + +- `BACKEND`: The backend to use. The default is `rdagent.oai.backend.DeprecBackend`. To use the LiteLLM backend, set it to `rdagent.oai.backend.LiteLLMAPIBackend`. + +- `CHAT_MODEL`: The model name of the chat model. + +- `EMBEDDING_MODEL`: The model name of the embedding model. + +The `CHAT_MODEL` and `EMBEDDING_MODEL` parameters will be passed into LiteLLM's completion function. + +Therefore, when utilizing models provided by different providers, first review the interface configuration of LiteLLM. The model names must match those allowed by LiteLLM. + +Additionally, you need to set up the the additional parameters for the respective model provider, and the parameter names must align with those required by LiteLLM. + +For example, if you are using a DeepSeek model, you need to set as follows: + + .. code-block:: Properties + + # For some models LiteLLM requires a prefix to the model name. + CHAT_MODEL=deepseek/deepseek-chat + DEEPSEEK_API_KEY= + +For more details on LiteLLM requirements, refer to the `official LiteLLM documentation `_. + + +Configuration(deprecated) +========================= To run the application, please create a `.env` file in the root directory of the project and add environment variables according to your requirements.