Problem with uploading a lot of files.

[dddmark]

Hello! Pls help! :)
When i try to Uploading of lot files (1500 (*.jpg) files and 71 folder) it uploads a small amount and then produces errors.

But there is no problem if I upload 4.5 Gb file.
Ver. Nextcloud: 16.0.4

Video 1:
https://www.youtube.com/watch?v=oUN2s_zbPTI
Video 2:
https://www.youtube.com/watch?v=a3O8xCWodwI
I hope for your help! Thanks!

[kesselb]

Note: This is the issue tracker of Nextcloud, please do NOT use this to get answers to your questions or get help for fixing your installation. This is a place to report bugs to developers, after your server has been debugged. You can find help debugging your system on our home user forums: https://help.nextcloud.com or, if you use Nextcloud in a large organization, ask our engineers on https://portal.nextcloud.com. See also https://nextcloud.com/support for support options.

Thanks ;) I guess there are some issues with uploading many files via web but without the issue template and some logs it's quite hard to triage your problem.

[dddmark]

As I saw, there is nothing useful in the logs. Maybe I'm wrong! :)

[dddmark]

There is a suggestion that CSRF token is expired while uploading a lot of files (as errors on screen says)

[zorn-v]

I can confirm

After pressing "Leave page" there is logout happened

Debug logs

Nextcloud 16.0.5

[JonathanTreffler]

Just use Flowupload
It works really well for thousands of files.

Usage of Flowupload:

(i know thats not a fix , but i dont think this a bug in nextcloud.)

[zorn-v]

Thanks for proposal, I will try

but i dont think this a bug in nextcloud

If something works not as expected - it is a bug 😉

[JonathanTreffler]

If something works not as expected - it is a bug

Ok. Youre right.
I am not sure how nextclouds implements uploading (directly (web)dav or something different ?) but i had some bad experiences too. Especially with big or many files. My server is pretty slow (4 cores for 15 Dockers) and before i used flowupload wich uses many independent chunks (each 1.04 MB) i got many timeout errors (i am not saying its nextclouds fault- but being able to retry failed uploads seems like a basic and and needed feature to me)

I think a lot more changes could be made to uploading. Especially a List of currently uploading files would be great (I made a app to implement that , but its pretty ugly/still a lot of work to do)

[dgtlmoon]

You may run into issues where the server runs out of 'power' whilst trying to build the preview thumbnails for all of the existing images... #13709 (comment)

[szaimen]

As I cannot reproduce the original issue anymore, I will close this ticket. If this is still happening please make sure to upgrade to the latest version. After that, feel free to reopen. The latest version can be seen by checking https://nextcloud.com/install/#install-clients and https://nextcloud.com/install/#instructions-server respectively.