gavindiaz/NexQuant
1
0
Fork 0
mirror of https://github.com/NicolasBohn/NexQuant.git synced 2026-08-07 20:17:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: update readme and installation guide for litellm backend addition (#703)

Browse Source 
* docs: Update README and installation guide for LiteLLM backend addition

* docs: update LiteLLM Backend configuration guide

* remove and add some llm settings docs

* CI Issue

* CI Issue

* update docs

---------

Co-authored-by: Young <afe.young@gmail.com>
This commit is contained in:
Haoran Pan
2025-03-21 14:55:30 +08:00
committed by GitHub
parent 837300c4d2
commit 021e96b12b
2 changed files with 56 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/installation_and_configuration.rst
+43 -2
View File
@@ -12,8 +12,49 @@ Installation
**Install Docker**: RDAgent is designed for research and development, acting like a human researcher and developer. It can write and run code in various environments, primarily using Docker for code execution. This keeps the remaining dependencies simple. Users must ensure Docker is installed before attempting most scenarios. Please refer to the `official 🐳Docker page <https://docs.docker.com/engine/install/>`_ for installation instructions.
Configuration
=============
LiteLLM Backend Configuration
=============================
Please create a `.env` file in the root directory of the project and add environment variables.
Here is a sample configuration for using OpenAI's gpt-4o via LiteLLM.
.. code-block:: Properties
BACKEND=rdagent.oai.backend.LiteLLMAPIBackend
# It can be modified to any model supported by LiteLLM.
CHAT_MODEL=gpt-4o
EMBEDDING_MODEL=text-embedding-3-small
# The backend api_key fully follows the convention of litellm.
OPENAI_API_KEY=<replace_with_your_openai_api_key>
Necessary parameters include:
- `BACKEND`: The backend to use. The default is `rdagent.oai.backend.DeprecBackend`. To use the LiteLLM backend, set it to `rdagent.oai.backend.LiteLLMAPIBackend`.
- `CHAT_MODEL`: The model name of the chat model.
- `EMBEDDING_MODEL`: The model name of the embedding model.
The `CHAT_MODEL` and `EMBEDDING_MODEL` parameters will be passed into LiteLLM's completion function.
Therefore, when utilizing models provided by different providers, first review the interface configuration of LiteLLM. The model names must match those allowed by LiteLLM.
Additionally, you need to set up the the additional parameters for the respective model provider, and the parameter names must align with those required by LiteLLM.
For example, if you are using a DeepSeek model, you need to set as follows:
.. code-block:: Properties
# For some models LiteLLM requires a prefix to the model name.
CHAT_MODEL=deepseek/deepseek-chat
DEEPSEEK_API_KEY=<replace_with_your_deepseek_api_key>
For more details on LiteLLM requirements, refer to the `official LiteLLM documentation <https://docs.litellm.ai/docs>`_.
Configuration(deprecated)
=========================
To run the application, please create a `.env` file in the root directory of the project and add environment variables according to your requirements.