Improve WebSockets support

[brlin-tw]

Currently, TetherFi doesn't support WebSockets which is used in several web applications(e.g. Discord and IRCcloud) that implement this relatively modern technology. As more and more applications adopt this feature(and drop support to legacy counterparts like long polling) we need to investigate whether we can support such usecase.

Reverse proxy implementations like NGINX seems to have already implemented such support, maybe it is also applicable to forward proxies as well?

Implement SOCKS proxy(#101) may make this feature request obsolete, though...

[pyamsoft]

So there's two pieces here, the WS and wss protocol. For WS, we would have to implement the HTTP 1.1 upgrade flow, which I believe is straightforward but not the easiest.

For wss, I think all we are able to do is respond to the initial CONNECT tunnel, past that we are just a delegate for proxies network data. In theory, wss protocol should already work, as evidenced by web chat usually working when accessed through a mobile browser.

For fun, here are some notes about apps which I wrote up while using the proxy on a plane.

Line yes

Discord yes

Spotify yes but slow

Messenger
send on proxy no
Receive yes

Instagram
Receive notification yes
But on proxy send receive no
Pictures seem to load

Threads no

Youtube app no
Youtube website ads load lol
Website works but quality is shit


Apple mail no (IMAP)
 
Facebook hates me and I hate facebook

[pyamsoft]

Testing using the node.js powered wscat and https://websocketman.com/, it looks like the existing proxy server code does support the WS and WSS protocol

Using wscat --proxy "http://192.168.49.1:8228" -c "wss://echo.websocket.in" I am able to connect to the websocket normally. I do not believe there is any further work to do to support Websockets.

Sadly, since third-party apps on the device are allowed by the OS to "go around" the proxy, things like WSS requests from a client library would not be intercepted by the hotspot proxy in all cases, which leads to the inconsistent behavior in chat apps like Discord. Not sure what else I can do here unfortunately.

[brlin-tw]

@pyamsoft

it looks like the existing proxy server code does support the WS and WSS protocol

Unfortunately I unable to reproduce the same behavior while using the Discord web app:

I wonder whether it is rather caused by #298, would you mind check it out?

[brlin-tw]

Another case using the IRCCloud web app:

[brlin-tw]

debug log: tetherfi-debug-0202405241408.log.gz

[pyamsoft]

I am so far unable to reproduce either of these issues using the production or latest Dev version.

That being said, I think this behavior is a memory leak on the server side of TetherFi.

If you turn off the hotspot, kill the app, and then try again, does it work? Seems to only leak after a "long time"

[pyamsoft]

Confirmed a memory leak from never closing sockets does at least effect this issue.

The next version will have a tweak to close inactive socket connections after 1 minute (if a network connection is started, but no data is transferred for a full minute, it will close the connection). This should help us clean up and reduce memory usage, which should avoid the IOException that causes these rejected connections.

[pyamsoft]

Version 43 was just approved on the store and includes a new option:

While this is OFF by default, I recommend you turn on the Enable Idle Timeout tweak, which will clean up memory after a connection has not had any activity for a whole minute.

I will continue to investigate memory usage, but as far the original issue about websockets in third-party apps there is very little we can do. Mobile websites will usually work, but the actual apps themselves do NOT use the system proxy on Android (only the default browser app does). Sadly, this may be all that we can do for now, until we find a more encompassing solution. Sorry :(

Thank you for your contribution to the project!

[brlin-tw]

Thanks for your work! I no longer have problems using the web versions of the WebSockets applications at the moment so it's all fine.