feat: add type checker to api backend & align litellm and old backend (#647)

* move cache auto continue and retry to all api backend

* add type checker to json mode output

* fix CI

* feat: Add json_mode handling and streaming support in chat completion function

* lint

* fix a bug when returning a dict which value could contain int or bool

* remove litellm

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>

rdagent/utils/__init__.py

@@ -120,7 +120,10 @@ def filter_progress_bar(stdout: str) -> str:
 
         response = json.loads(
             APIBackend().build_messages_and_create_chat_completion(
-                user_prompt=user_prompt, system_prompt=system_prompt, json_mode=True
+                user_prompt=user_prompt,
+                system_prompt=system_prompt,
+                json_mode=True,
+                json_target_type=dict,
             )
         )
         needs_sub = response.get("needs_sub", True)