OpenAI extension does not accept any setting for over 200 tokens

[SatoshiReport]

@matatonic

OpenAI extension does not accept any setting for over 200 tokens. I had to change the default in script.py to get this to work. max_new_tokens and new_tokens both did not work.

Thanks for all your great work, making this.

[matatonic]

Can you update your install and try this again? I think this issue is fixed in the current release.

[matatonic]

Can you give me an example of what's not working? Ie. can you share the code? I'm testing now with TheBloke/Wizard-Vicuna-13B-Uncensored-GPTQ .

[matatonic]

I was just testing via the streaming chat api and I had no problems getting the context over 2000 tokens.
llm-text-generation-webui-1 | RuntimeError: start (0) + length (2049) exceeds dimension size (2048).
llm-text-generation-webui-1 | Output generated in 4.11 seconds (69.13 tokens/s, 284 tokens, context 1765, seed 1737398157)

[SatoshiReport]

Apologies, I edited my comment a few minutes after posting but it was too late.

[SatoshiReport]

Going back to the original issue, I downloaded the latest code, and it still stops after 200 tokens. Here is how I set my parameter list in R.

max_tokens <- 2048
model_tokens <- round(max_tokens*0.9 - tik(content), 0) 
parameter_list <- list(messages = messages, stream = stream, max_new_tokens = model_tokens)

What should I be doing to get over 200 tokens? Thanks

[SatoshiReport]

got it! Used max_tokens. Thanks for all your help