Use CAI's mobile version endpoints #104

realcoloride · 2023-10-07T08:56:28Z

Hello,

As you may have noticed, characterAI released their mobile app and I would like to know if it would be interesting to perhaps try to see if using their endpoints or if they have useragents or endpoints that could be used for easier automation.

Parking-Master · 2023-10-07T12:54:55Z

I never thought of that, but that actually sounds like a good idea

Parking-Master · 2023-10-07T13:01:50Z

How would you find those endpoints though?

realcoloride · 2023-10-07T19:48:13Z

How would you find those endpoints though?

Use an android emulator and a reverse proxy like Fiddler or WireShark

Parking-Master · 2023-10-07T20:52:25Z

Yeah that's what I was thinking. An emulator like https://appetize.io and inspect HTTP requests

Parking-Master · 2023-10-07T20:52:51Z

For free users a session doesn't last long so you'd need to install the app quick and sign in

Parking-Master · 2023-10-07T21:00:06Z

Nevermind, you can't install apps on there for whatever reason

catvinyl · 2023-10-11T07:06:59Z

Character AI without Puppeteer
https://gist.github.com/catvinyl/f7fa537954265f0ac3b940a114745b04
You need to install ws.
And you need to put your data to test:

How to get WebSocket token and chat id?

Create chat with character.
Open DevTools.
Reload webpage.
Write something to Character AI.
Find in Network tab URL /ws/ or wss://neo.character.ai/ws/
Click on that URL and open tab Response/Messages.
Copy chat_id
Copy robot's author_id (optional).
Open tab Cookies/Payload.
Copy Token from HTTP_AUTHORIZATION

Parking-Master · 2023-10-11T12:40:13Z

@catvinyl you're joking, right???

How the heck did you actually make an API without cloudflare catching you?

realcoloride · 2023-10-11T15:51:06Z

@catvinyl you're joking, right???

How the heck did you actually make an API without cloudflare catching you?

its only a matter of time till the staff catches up..

Parking-Master · 2023-10-11T15:57:29Z

@realcoloride lol, true... I've got to test out this script

realcoloride · 2023-10-11T15:57:50Z

Alright, keep me updated!

Parking-Master · 2023-10-11T16:17:46Z

Ok. It seems to work but I'm getting an error because I didn't provide the right chat_id. I inspected the /ws/ request in the Network tab of Firefox, but it's empty. There's no response data from the websocket in the console. Is there any other way to get the chat id?

catvinyl · 2023-10-11T18:45:44Z

@Parking-Master you need send message to AI, then check out for GET request with status 101. You can also choose filter for WS requests. Check that you're using https://beta.character.ai/chat2?char=... but not https://beta.character.ai/chat?char=...

catvinyl · 2023-10-11T19:12:44Z

My implementation is simple, but it can be broken with async requests, so you need to use await, and it skips to the fully generated answer. You can get a streamed answer (but it needs to be implemented).
It is also possible to create rooms with WS.

{"command":"create_chat","request_id":"a","payload":{"chat":{"chat_id":"b","creator_id":"c","visibility":"VISIBILITY_PRIVATE","character_id":"d","type":"TYPE_ONE_ON_ONE"},"with_greeting":true}

{"command":"abort_generation","request_id":"x","origin_id":"y"}

{"command":"generate_turn_candidate","request_id":"a","payload":{"character_id":"b","turn_key":{"turn_id":"c","chat_id":"d"}},"origin_id":"e"}

It is possible that the WS API has more commands.

https://beta.character.ai/chat2?char=...
This uses WS and doesn't have image uploading or generation. So you need to use this to get IDs, tokens, etc.
Rating is done using POST.

https://beta.character.ai/chat?char=...
POST request.

Parking-Master · 2023-10-11T19:48:05Z

@Parking-Master you need send message to AI, then check out for GET request with status 101. You can also choose filter for WS requests. Check that you're using https://beta.character.ai/chat2?char=... but not https://beta.character.ai/chat?char=...

Oh, I got it. That's why. I was only using the first chat

Parking-Master · 2023-10-11T19:53:42Z

Yep, I'm still getting an empty response

catvinyl · 2023-10-12T13:06:53Z

@Parking-Master I upgraded the script. Now it can create chat. Try only putting in a token. You can also try "debug = true" to see what's wrong.

Parking-Master · 2023-10-12T13:08:05Z

Nice! I'll try it when I get a chance. I was able to get the token pretty easily

Parking-Master · 2023-10-12T13:58:32Z

@catvinyl it works awesome, and responds really fast! One thing you should add though is like what we currently have, createOrContinueChat, because I'd like to continue the current chat. Other than that, it works great

catvinyl · 2023-10-12T14:20:26Z

@Parking-Master you can continue chat. You need to replace

const chat = await characterAI.createChat(character_id);
const chat_id = getChatId(chat);

with

const chat_id = 'xxxxyyyy-bbbb-cccc-dddd-eeeeffff1111';

Parking-Master · 2023-10-12T14:29:23Z

Yep, but the problem is, I don't know the chat id lol. Besides, even if I did, I want to continue the "original" chat (like "/chat?char=...") and I don't think that one has an id. I've tried inspecting multiple socket requests on the page on Chrome and Firefox and they both say it's empty. I don't know what else I can do.

Parking-Master · 2023-10-12T14:37:18Z

I even asked the AI himself and he gave me a valid-looking chat_id but it's still invalid

catvinyl · 2023-10-12T14:54:56Z

I don't think AI have information about chat_id. When you run script it says

Ready!
user_id (some numbers)
chat_id 9e5fe1a7-7878-42a5-a43e-e1421cefe027
...

Then you need put that to chat_id.
const chat_id = '9e5fe1a7-7878-42a5-a43e-e1421cefe027';

"?char=YntB_ZeqRq2l_aVf2gWDCZl4oBttQzDvhj9cXafWcF8" This is not chat_id.
That is character_id.

Parking-Master · 2023-10-13T02:57:21Z

@catvinyl ok, so I did in fact find the chat ID on the page. It worked, but the stream stops every time the AI stutters, so I found a way to return the response soon as the AI is done talking. It works great, less resources, no puppeteer, and lightning fast! Thank you so much for making this small script. I've adapted it to my own needs.

Mr-TIME-c · 2023-10-17T13:22:11Z

@catvinyl ok, so I did in fact find the chat ID on the page. It worked, but the stream stops every time the AI stutters, so I found a way to return the response soon as the AI is done talking. It works great, less resources, no puppeteer, and lightning fast! Thank you so much for making this small script. I've adapted it to my own needs.

do you mind uploading the package, because rn with puppeteer it takes too long to get a response and mostly failed response

Parking-Master · 2023-10-17T15:41:00Z

@Mr-TIME-c I don't mind uploading a package for this, but I don't want to take credit for realcoloride's work. I will upload a package soon but I will also try it help with this package to make it without puppeteer and make it faster.

realcoloride · 2023-10-19T20:22:29Z

Hello again, this looks good and could help a lot in the process.
I will try this when I'll be home.

However, I would like to probably make a way to use the old puppeteer method and this new one. But judging by how it looks like it works, this would require much more effort to convert and find all new endpoints

realcoloride · 2023-10-19T21:24:07Z

Could you just list them all from the current code base? (see chat and message and client)
I also need to be aware of how different API responses and so on

Parking-Master · 2023-10-19T23:33:38Z

I've published my own package!

@parking-master/node_characterai

https://github.com/Parking-Master/characterai

realcoloride · 2023-10-19T23:41:39Z

Worth the look but looks very messy in my opinion.
I'll stick this as a reminder but I would still like to know if it is possible to scrap the mobile version of the endpoints.

Parking-Master · 2023-10-20T00:30:20Z

@realcoloride okay, just a recommendation haha :)

I hope there's an easy way to get the mobile version endpoints though... have you experimented yet?

realcoloride · 2023-10-20T00:33:18Z

@realcoloride okay, just a recommendation haha :)

I hope there's an easy way to get the mobile version endpoints though... have you experimented yet?

I wish I could talk in-depth about my searches (mainly reverse engineering) of the mobile app but I fear of if it complies legally or with their ToS.
So for now I'll just do my searches, and even if i found some interesting things, I am not sure if I can legally show it here

Parking-Master · 2023-10-20T00:35:39Z

Yep, I always do a deep search in the company's ToS before making unofficial APIs like this, because I don't want to get in trouble. I'll do some research too.

Jeffgrey91 · 2024-01-08T18:31:33Z

If you got the mobile version endpoints make sure to get the new group chat endpoints as well as it uses a different and improved chat model that's not available through the browser.

realcoloride · 2024-07-23T03:07:23Z

The mobile endpoints use new the endpoints. See #180

realcoloride closed this as completed Jul 23, 2024

realcoloride mentioned this issue Dec 19, 2024

Stuck in docs #190

Closed

Use CAI's mobile version endpoints #104

Use CAI's mobile version endpoints #104

Comments

realcoloride commented Oct 7, 2023

Parking-Master commented Oct 7, 2023

Parking-Master commented Oct 7, 2023

realcoloride commented Oct 7, 2023

Parking-Master commented Oct 7, 2023

Parking-Master commented Oct 7, 2023

Parking-Master commented Oct 7, 2023

catvinyl commented Oct 11, 2023

Parking-Master commented Oct 11, 2023

realcoloride commented Oct 11, 2023

Parking-Master commented Oct 11, 2023

realcoloride commented Oct 11, 2023

Parking-Master commented Oct 11, 2023 • edited Loading

catvinyl commented Oct 11, 2023 • edited Loading

catvinyl commented Oct 11, 2023

Parking-Master commented Oct 11, 2023

Parking-Master commented Oct 11, 2023

catvinyl commented Oct 12, 2023

Parking-Master commented Oct 12, 2023

Parking-Master commented Oct 12, 2023

catvinyl commented Oct 12, 2023

Parking-Master commented Oct 12, 2023

Parking-Master commented Oct 12, 2023

catvinyl commented Oct 12, 2023 • edited Loading

Parking-Master commented Oct 13, 2023

Mr-TIME-c commented Oct 17, 2023

Parking-Master commented Oct 17, 2023

realcoloride commented Oct 19, 2023

realcoloride commented Oct 19, 2023

Parking-Master commented Oct 19, 2023

realcoloride commented Oct 19, 2023

Parking-Master commented Oct 20, 2023

realcoloride commented Oct 20, 2023

Parking-Master commented Oct 20, 2023

Jeffgrey91 commented Jan 8, 2024

realcoloride commented Jul 23, 2024

Parking-Master commented Oct 11, 2023 •

edited

Loading

catvinyl commented Oct 11, 2023 •

edited

Loading

catvinyl commented Oct 12, 2023 •

edited

Loading