Integration with Amazon Bedrock and Nova models

[ebaizel]

I am considering creating a PR that would enable Eliza agents to work with Amazon Bedrock which would enable even more LLMs and an additional provider. Before doing that, I want to confirm that there is interest in this and it would be included (assuming it's properly designed and implemented).

[LanreDaDev]

Great idea, I have more Claude tokens through bedrock than I do on anthropic's platform itself. I was thinking of building it out myself locally but this would be a great addition

[rijul-newalkar]

would be interested!

[ebaizel]

Hey @lalalune thoughts on this? Would you consider merging this PR if I created it?

[jmikedupont2]

OK i will start working on this unless there is code published? can you please share?

[jmikedupont2]

https://www.npmjs.com/package/@ai-sdk/amazon-bedrock it appears to be as simple to integrate this

[jmikedupont2]

meta-introspector#5 here is a starting point, more work needed, untested

[ebaizel]

PR has been merged, adding Bedrock as a provider and making available the Amazon Nova family of LLMs #2769

[ebaizel]

@LanreDaDev and @rijul-newalkar, let me know if you try it out and how it works for you

[jmikedupont2]

cool

…

On Mon, Jan 27, 2025, 08:29 emile baizel ***@***.***> wrote: PR has been merged, adding Bedrock as a provider and making available the Amazon Nova family of LLMs #2769 <#2769> — Reply to this email directly, view it on GitHub <#1975 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5KQ2NMP3JY43OTG7L4IXT2MYYEVAVCNFSM6AAAAABUYIA5P2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCOJWHE3TINY> . You are receiving this because you commented.Message ID: ***@***.***>

[retdude]

It has not been working for me... Is it possible I am missing something? All the changes are there how did you get it working?

[jmikedupont2]

I have not been testing it but this PR is not the right one I think someone fixed it in a different PR

…

On Tue, Feb 4, 2025, 20:22 Jarret F Beauregard ***@***.***> wrote: It has not been working for me... Is it possible I am missing something? All the changes are there how did you get it working? — Reply to this email directly, view it on GitHub <#1975 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5KQ2LDJAGOHSRA2ZDXQUT2OFRWLAVCNFSM6AAAAABUYIA5P2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMBWGE2DOOI> . You are receiving this because you commented.Message ID: ***@***.***>

[ebaizel]

this is the pr #2769

[retdude]

I was looking at this PR #2769 on the official release... Have you been able to test it? When I run my agent with all this code the ModelProvider refuses to appear:

When I set the modelProvider to BEDROCK, it all does not appear:

 ["ℹ Setting model provider..."] 

 ℹ INFORMATIONS
   Model Provider Selection: 
   {} 

 ℹ INFORMATIONS
   Selected model provider: 
    

 ℹ INFORMATIONS
   Selected image model provider: 
    

 ⛔ ERRORS
   Invalid model provider: 
    

 ⛔ ERRORS
   Available providers: 
   ["openai","eternalai","anthropic","grok","groq","llama_cloud","together","llama_local","google","claude_vertex","redpill","openrouter","ollama","heurist","galadriel","falai","gaianet","ali_bailian","volengine","nanogpt","hyperbolic","venice","akash_chat_api"] 

[retdude]

I have set the model provider in the same way for working ones, and it runs just fine. Here it is with OPENAI:

 ["ℹ Setting model provider..."] 

 ℹ INFORMATIONS
   Model Provider Selection: 
   {"characterModelProvider":"openai","optsModelProvider":"openai","finalSelection":"openai"} 

 ℹ INFORMATIONS
   Selected model provider: 
   openai 

[jmikedupont2]

Okay when I find some time I will test it okay myself I haven't actually tested this new patch and I should

…

On Tue, Feb 4, 2025, 21:01 emile baizel ***@***.***> wrote: this is the pr #2769 <#2769> — Reply to this email directly, view it on GitHub <#1975 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5KQ2OEYST65UQ5BKU4KOL2OFWH5AVCNFSM6AAAAABUYIA5P2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMBWGE3TEMQ> . You are receiving this because you commented.Message ID: ***@***.***>

[retdude]

Thanks!
Let me know how it goes!

[ebaizel]

@retdude have you tried with setting it to bedrock (lowercase)?

[retdude]

Yes. I write it as modelProviderName.BEDROCK, which uses that enum equaling the string “bedrock”. I have tried just the raw string “bedrock” as well.

[ebaizel]

@retdude I think I found the issue, which is that I somehow didn't check in the Bedrock code within generate.ts.

Can you try adding the code below to ~line 1000 of packages/core/src/generation.ts? I tested it successfully by creating a new character.json and setting "modelProvider": "bedrock"

            case ModelProviderName.BEDROCK: {
                elizaLogger.debug("Initializing Bedrock model.");

                const { text: bedrockResponse } = await aiGenerateText({
                    model: bedrock(model),
                    prompt: context,
                    system:
                        runtime.character.system ??
                        settings.SYSTEM_PROMPT ??
                        undefined,
                    tools: tools,
                    onStepFinish: onStepFinish,
                    maxSteps: maxSteps,
                    temperature: temperature,
                    maxTokens: max_response_length,
                    frequencyPenalty: frequency_penalty,
                    presencePenalty: presence_penalty,
                    experimental_telemetry: experimental_telemetry,
                });

                response = bedrockResponse;
                elizaLogger.debug("Received response from Bedrock model.");
                break;
            }

[ebaizel]

PR for bug fix #3329