Multimodal QA with Eval and Image Gen

Created by team Agent Artificial on December 27, 2023

LiteLLM manages a hot-swap between, by default, Gemini and GPT-4-V (but any of the 100+ litellm models could be chosen) for comparison of behaviors. Litellm enables any open source model to call functions through it's openai proxy server. The included function is used to generate images via's realtime lcm endpoint in response to user requests, resulting in a rapid iteration cycle between user and vision model -- generate an image, show it to vision model with requested changes, model re-generates on a realtime endpoint. Trulens evals allow for metrics comparisons between various models on vision tasks in this iteration cycle.

Category tags: