TPTBusiness faa891eb41 feat: Intelligent embedding chunking instead of truncation ...

Change: Instead of truncating texts, now using intelligent chunking:

1. Content ≤ 20,000 characters: Single embedding (complete)
2. Content > 20,000 characters: Split into 20k chunks
   - Each chunk gets its own embedding
   - All embeddings are averaged
   - No information loss!

Benefits:
- No more text truncation
- Full information preserved
- Stays under 8192 token limit (nomic-embed-text)
- Average embedding represents entire text

Affected files:
- rdagent/components/knowledge_management/vector_base.py

2026-04-02 19:22:50 +02:00

..

graph.py

fix(graph): using assignment expression to avoid repeated function call (#1174)

2025-08-13 17:16:58 +08:00

vector_base.py

feat: Intelligent embedding chunking instead of truncation

2026-04-02 19:22:50 +02:00