Would you consider a DI approach for your OpenAI adapters?

[Adrian-Samuel]

First of all, great work! Love what you're doing here with the library; I'm excited to see this grow and develop.

From the examples provided, I noticed OpenAI's API within your codebase, calls your own instance of openai under the hood which you've subclassing and defined interceptors etc.

Although it works well, there's a strong preference for users to manage their own dependencies which avoids vendor/package lock-in for numerous reasons including security, maintainability etc.

Would you explore enabling this by implementing some kind of dependency injection pattern to your cache adapter layer?

Keen to hear your thoughts!

[xiaofan-luan]

First of all, great work! Love what you're doing here with the library; I'm excited to see this grow and develop.

From the examples provided, I noticed OpenAI's API within your codebase, calls your own instance of openai under the hood which you've subclassing and defined interceptors etc.

Although it works well, there's a strong preference for users to manage their own dependencies which avoids vendor/package lock-in for numerous reasons including security, maintainability etc.

Would you explore enabling this by implementing some kind of dependency injection pattern to your cache adapter layer?

Keen to hear your thoughts!

Hi Adrian! glad to know you like GptCache.

With langchain adapter you should be able to use any LLMs as the background, and we are working on supporting LLMs like LLaMa and Dolly by hosting it on local machine. Is that what you are looking for?

[Adrian-Samuel]

Hey @xiaofan-luan! I believe Langchain makes the same design decisions.

Essentially my preference would be something like this:

import openai
from gptcache import cache
from gptcache.adapter import OpenAI
 
openai.api_key = "secret"

# some kind of adapter intialiser, similar to how Langchain does it
llm = OpenAI(openai)
# a way to initialise your cache with the specific llm adapter chosen
instance = cache.init(llm) 

response = instance.ChatCompletion.create(
      model='gpt-3.5-turbo',
      messages=[
        {
            'role': 'user',
            'content': question
        }
      ],
    )
...

[SimFG]

So sorry for the late reply. I guess you want to specify a different cache for each llm model.

from gptcache import Cache
from gptcache.adapter import openai
from gptcache.manager import manager_factory

llm_str = "openai"
data_manager = manager_factory("map", data_dir=llm_str)

openai_cache = Cache()
openai_cache.init(data_manager=data_manager)
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": question}],
    cache_obj=openai_cache,
)

Surely, your suggestion also provides us with another way of thinking, thank you very much, we will also try to adjust our adaptation method.