mirror of
https://github.com/NicolasBohn/NexQuant.git
synced 2026-08-07 12:07:43 +00:00
fix: support experimental support for Deepseek models and update docs about configuration (#1024)
* fix qlib running bug for deepseek and add configuration docs about deepseek
This commit is contained in:
@@ -16,6 +16,9 @@ Ensure the current user can run Docker commands **without using sudo**. You can
|
||||
LiteLLM Backend Configuration (Default)
|
||||
=======================================
|
||||
|
||||
.. note::
|
||||
🔥 **Attention**: We now provide experimental support for **DeepSeek** models! You can use DeepSeek's official API for cost-effective and high-performance inference. See the configuration example below for DeepSeek setup.
|
||||
|
||||
Option 1: Unified API base for both models
|
||||
------------------------------------------
|
||||
|
||||
@@ -48,6 +51,23 @@ Option 2: Separate API bases for Chat and Embedding models
|
||||
LITELLM_PROXY_API_KEY=<replace_with_your_siliconflow_api_key>
|
||||
LITELLM_PROXY_API_BASE=https://api.siliconflow.cn/v1
|
||||
|
||||
Configuration Example: DeepSeek Setup
|
||||
-------------------------------------
|
||||
|
||||
Many users encounter configuration errors when setting up DeepSeek. Here's a complete working example:
|
||||
|
||||
.. code-block:: Properties
|
||||
|
||||
# CHAT MODEL: Using DeepSeek Official API
|
||||
CHAT_MODEL=deepseek/deepseek-chat
|
||||
DEEPSEEK_API_KEY=<replace_with_your_deepseek_api_key>
|
||||
|
||||
# EMBEDDING MODEL: Using SiliconFlow for embedding since DeepSeek has no embedding model.
|
||||
# Note: embedding requires litellm_proxy prefix
|
||||
EMBEDDING_MODEL=litellm_proxy/BAAI/bge-m3
|
||||
LITELLM_PROXY_API_KEY=<replace_with_your_siliconflow_api_key>
|
||||
LITELLM_PROXY_API_BASE=https://api.siliconflow.cn/v1
|
||||
|
||||
Necessary parameters include:
|
||||
|
||||
- `CHAT_MODEL`: The model name of the chat model.
|
||||
|
||||
Reference in New Issue
Block a user