Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"CodeChecker store" on a large quantity of files failed with an unhelpful error message #1120

Open
sylvestre opened this issue Nov 14, 2017 · 14 comments
Open

"CodeChecker store" on a large quantity of files failed with an unhelpful error message #1120

sylvestre opened this issue Nov 14, 2017 · 14 comments
Labels
bug 🐛 server 🖥️

Comments

@sylvestre
Copy link
Contributor

My store directory is about 2.1G.
CodeChecker store fails on this directory with Request failed to localhost:8001
Which is not super useful to debug or understand anything!
@dkrupp
Copy link
Member

dkrupp commented Nov 14, 2017

could you please run the store with --verbose debug and attach the logs?

@sylvestre
Copy link
Contributor Author

here it is:

[11:12:57] {PLIST_PARSER} [13723] <140623444686592> - plist_parser.py:78 parse_plist() - Parsing plist: /var/lib/jenkins/codechecker-results-full/uspoof_build.cpp_13601f5cd35d4f5636dcb7619ae9370a.plist
[11:12:57] {STORE} [13723] <140623444686592> - store.py:230 assemble_zip() - Copying file '/var/lib/jenkins/codechecker-results-full/uspoof_build.cpp_13601f5cd35d4f5636dcb7619ae9370a.plist' to ZIP assembly dir...
[11:12:57] {THRIFT HELPER} [13723] <140623444686592> - thrift_helper.py:76 wrapper() - Thrift invalid data error.
[11:12:57] {THRIFT HELPER} [13723] <140623444686592> - thrift_helper.py:83 wrapper() - getMissingContentHashes
[11:12:57] {THRIFT HELPER} [13723] <140623444686592> - thrift_helper.py:84 wrapper() - (['299d281b473d4f2598a2625676d39ddb2b8a3d285dab70bcf50bc4d06acc9492', [...] 'fae3838a554958a557a69213812d93eb25ae9cc5b2c3f8e1938c56\
70f1b167b9'],)
[11:12:57] {THRIFT HELPER} [13723] <140623444686592> - thrift_helper.py:85 wrapper() - {}
[11:12:57] {THRIFT HELPER} [13723] <140623444686592> - thrift_helper.py:86 wrapper() - Unexpected character: <
Request failed to localhost:8001

@whisperity
Copy link
Contributor

The clientside log won't help in this case. 2.1GB is too large and it looks like Thrift decapitates the connection.

The unexpected < char is the first character of the HTML output (<html><head>...) the server gives. Which is of course not a json, so the clientside Thrift handler says it's an invalid response.

@whisperity
Copy link
Contributor

Not sure if this is a bug.

@gyorb @bruntib Was there a decision made on how big the transported ZIP can be and/or how long the storage could last at max?

@Xazax-hun
Copy link
Contributor

I wonder how many results are in that zip? Maybe it would make sense to turn some checks off?

@Xazax-hun
Copy link
Contributor

@whisperity even if the limit on the size is intentional, the unhelpful error message is a bug in my opinion.

@whisperity
Copy link
Contributor

I agree, but I'm not sure if Thrift is able to gracefully handle receiving a HTTP error code when it expected a valid HTTP transport with a JSON in it.

@sylvestre Can you try this again with a --verbose debug server and add its log too for us?

@sylvestre
Copy link
Contributor Author

Looks like it is a different issue. The login timeouts:

[15:08:20] {SERVER} [24142] <140230194267904> - server.py:124 __check_auth_in_request() - 127.0.0.1:39224 Invalid access, credentials not found - session refused.
[15:08:20] {SERVER} - server.py:244 do_POST() - 127.0.0.1:39224 -- [Anonymous] POST /firefox/v6.2/CodeCheckerService
[15:08:20] {SERVER} [24142] <140230194267904> - server.py:274 do_POST() - 127.0.0.1:39224 Invalid access, credentials not found - session refused.

@whisperity
Copy link
Contributor

whisperity commented Nov 15, 2017 •
edited
Loading

It's strange, I recall specifically making the connection to the server initialise only after the ZIP is created in an earlier patch.

You can alter the session timeouts serverside in the servers configuration. It's also good if you create a password file (see the authentication docs for a how-to) that stores your credentials so the client can automatically authenticate you when it is needed.

@gyorb
Copy link
Contributor

gyorb commented Nov 16, 2017

We should definitely handle these types of errors better. Currently we only give a warning if the zip is more than 1GB but we do not stop the client.

@sylvestre did the session timeout increase in the server config help?

@whisperity maybe we should increase the default session timeout.

@sylvestre
Copy link
Contributor Author

@gyorb Nope, a 10 times increase didn't change anything :/

@sylvestre
Copy link
Contributor Author

I also tried to remove the need of authentication for localhost operation but I could not figure out how to do that reading the documentation.

@whisperity
Copy link
Contributor

@sylvestre Start the server without specifying --force-authentication and edit the ~/.codechecker/session_config.json file. In this file, there is an enabled key in the authentication dict, specify it to false and the server will run in non-authenticative mode.

@d-led
Copy link

d-led commented Apr 20, 2018

getting this error too:

----=================----
Total number of reports: 577
----=================----
[INFO 2018-04-20 10:45] - Storing analysis results for run 'appsmake-1524212626'
[INFO 2018-04-20 10:45] - Authenticated using pre-configured credentials.
[ERROR 2018-04-20 10:45] - Thrift invalid data error.
[ERROR 2018-04-20 10:45] - getPackageVersion
[ERROR 2018-04-20 10:45] - ()
[ERROR 2018-04-20 10:45] - {}
[ERROR 2018-04-20 10:45] - Unexpected character: <
[ERROR 2018-04-20 10:45] - Request failed.

since CodeChecker 6, the browser session expires really quickly. Set the session expiration in the config to a larger value now. Will see if it appears again

@scphantm scphantm mentioned this issue Oct 25, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug 🐛 server 🖥️
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@sylvestre @Xazax-hun @d-led @whisperity @dkrupp @gyorb