Remote closing connection leads to unhandled exceptions in _sync

[g4borg]

I just wanted to point out, that without an error callback, the current implementation of having listen_forever in a thread created by the library can lead to unhandled exceptions that cannot be intercepted.

Following traceback sometimes occurs during running my currently developed bridge software, which atm. lets matrix clients run with start_listener_thread() - it happens very occasionally. I only get a real palpable error later, when i try to send with one of those clients - so until then it does not get noticed. i can also provide more information, but i think this is a more general issue.
I also should mention, my client was connected via https externally

I am of course rewriting it to handle threading on my own anyway so it is not an issue for me i hope in the future - however documentation should probably point out problems like this to the user, so he can prepare or set a general exception handler, and maybe if you keep a "simple we do the threading" solution, an exception callback for the thread would also be a solution.

from pip:
used package: matrix-client==0.3.2

the error logged:
matrix_client.client [ERROR]: Exception thrown during sync

the traceback:

Traceback (most recent call last):
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.5/http/client.py", line 1198, in getresponse
    response.begin()
  File "/usr/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.5/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/myproject/directory/lib/python3.5/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/myproject/directory/lib/python3.5/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.5/http/client.py", line 1198, in getresponse
    response.begin()
  File "/usr/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.5/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/myproject/directory/lib/python3.5/site-packages/matrix_client/api.py", line 670, in _send
    verify=self.validate_cert
  File "/myproject/directory/lib/python3.5/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/myproject/directory/lib/python3.5/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/myproject/directory/lib/python3.5/site-packages/requests/adapters.py", line 495, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/myproject/directory/lib/python3.5/site-packages/matrix_client/client.py", line 473, in listen_forever
    self._sync(timeout_ms)
  File "/myproject/directory/lib/python3.5/site-packages/matrix_client/client.py", line 555, in _sync
    response = self.api.sync(self.sync_token, timeout_ms, filter=self.sync_filter)
  File "/myproject/directory/lib/python3.5/site-packages/matrix_client/api.py", line 105, in sync
    api_path=MATRIX_V2_API_PATH)
  File "/myproject/directory/lib/python3.5/site-packages/matrix_client/api.py", line 673, in _send
    raise MatrixHttpLibError(e, method, endpoint)
matrix_client.errors.MatrixHttpLibError: Something went wrong in GET requesting https://myproject.host/_matrix/client/r0/sync: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

[Artiom-M]

I am using the same client SDK : matrix-client (0.3.2)
And getting the same type of error in case of some network issue:

Exception thrown during sync
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 384, in _make_request
six.raise_from(e, None)
File "", line 2, in raise_from
File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.4/http/client.py", line 1227, in getresponse
response.begin()
File "/usr/lib64/python3.4/http/client.py", line 386, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.4/http/client.py", line 356, in _read_status
raise BadStatusLine(line)
http.client.BadStatusLine: ''

	During handling of the above exception, another exception occurred:
	
	Traceback (most recent call last):
	File "/usr/lib/python3.4/site-packages/requests/adapters.py", line 445, in send
    	timeout=timeout

[Artiom-M]

I am using on_exception handler in order to just reconnect in case of any exception happens, but this one seems to not be captured at all:

    def on_exception(self,e):
    self.client.logout()
    self.log.error(e)
    time.sleep(random.random() * 100)
    self.connect()

[makefu]

i am struggeling with the same exception, any way to fix it?

[Artiom-M]

I don't think so. This SDK is kind of abandoned. I have switched to matrix-nio and it works much better.

[makefu]

@Artiom-M thanks for the heads-up, i am currently using https://gitlab.com/gibberfish/python-matrixbot and i was not planning to write my own bot as of right now :)
i will try to patch the matrix-python-sdk sources to meet my use case and hopefully work around the issue.
cheers

[ahvahsky2008]

I catch same issue(( How solve it? Who know?