gavindiaz/NexQuant
1
0
Fork 0
mirror of https://github.com/NicolasBohn/NexQuant.git synced 2026-07-27 23:47:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: update explanation for separate config use in litellm (#958)

Browse Source 
* docs: update explanation for separate config use in litellm

* docs: update default backend to `rdagent.oai.backend.LiteLLMAPIBackend`

* docs: update .rst format

* Update installation_and_configuration.rst
This commit is contained in:
amstrongzyf
2025-06-13 18:32:39 +08:00
committed by GitHub
parent 3bb4276595
commit 9165d64464
5 changed files with 106 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+25 -3
View File
@@ -153,16 +153,38 @@ Ensure the current user can run Docker commands **without using sudo**. You can
You can set your Chat Model and Embedding Model in the following ways:
- **Using LiteLLM (Recommended)**: We now support LiteLLM as a backend for integration with multiple LLM providers. You can configure as follows:
- **Using LiteLLM (Default)**: We now support LiteLLM as a backend for integration with multiple LLM providers. You can configure in two ways:
**Option 1: Unified API base for both models**
```bash
cat << EOF > .env
BACKEND=rdagent.oai.backend.LiteLLMAPIBackend
# Set to any model supported by LiteLLM.
CHAT_MODEL=gpt-4o
EMBEDDING_MODEL=text-embedding-3-small
# Then configure the environment variables required by your chosen model in the convention of LiteLLM here.
# Configure unified API base
OPENAI_API_BASE=<your_unified_api_base>
OPENAI_API_KEY=<replace_with_your_openai_api_key>
```
**Option 2: Separate API bases for Chat and Embedding models**
```bash
cat << EOF > .env
# Set to any model supported by LiteLLM.
# Configure separate API bases for chat and embedding
# CHAT MODEL:
CHAT_MODEL=gpt-4o
OPENAI_API_BASE=<your_chat_api_base>
OPENAI_API_KEY=<replace_with_your_openai_api_key>
# EMBEDDING MODEL:
# TAKE siliconflow as an example, you can use other providers.
# Note: embedding requires litellm_proxy prefix
EMBEDDING_MODEL=litellm_proxy/BAAI/bge-large-en-v1.5
LITELLM_PROXY_API_KEY=<replace_with_your_siliconflow_api_key>
LITELLM_PROXY_API_BASE=https://api.siliconflow.cn/v1
```
Notice: If you are using reasoning models that include thought processes in their responses (such as \<think> tags), you need to set the following environment variable:
```bash
REASONING_THINK_RM=True