Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dump command problems #5483

Open
timfeirg opened this issue Dec 30, 2024 · 3 comments · Fixed by #5493
Open

dump command problems #5483

timfeirg opened this issue Dec 30, 2024 · 3 comments · Fixed by #5493
Assignees

Comments

@timfeirg
Copy link
Contributor

timfeirg commented Dec 30, 2024

juicefs version 1.2.2+2024-12-02.92e1db2

  • dump command did not finish, even after dumped inodes exceed file system usage
  • --fast option breaks database connection (without --fast, dump works normally)
2024/12/30 05:24:45.086469 juicefs[31691] <INFO>: Meta address: mysql://xxx:***@xxx:3306)/juicefs
Snapshot keys: 0/9920492 [-----------] 0.0/s ETA: 0s
2024/12/30 05:24:50.492201 juicefs[31691] <FATAL>: Fetoch all metadata from DB: invalid connection [main.go:31]

this error is accompanied by MySQL errors as well:

2024-12-30T05:06:02.293563-08:00 760724 [Note] [MY-010914] [Server] Aborted connection 760724 to db: 'xxx' user: 'xxx' host: '[xxx](http://xxx/)' (Got an error reading communication packets).
  • ETA is mostly meaningless information as it's always "0s", in reality it's going to run for hours, consider using file system used inodes as denominator to get a more useful estimation
@anysql
Copy link
Contributor

anysql commented Dec 30, 2024

This is due to incorrect user of database connection, a coding bug I think.

If a connection is reading the result from mysql server, this connection cannot be used to run another query, else the packet sequence is broken and the connection is broken.

@zhijian-pro
Copy link
Contributor

  1. invalid connection is because the us3 database restarted due to slow queries, causing all sessions to be closed.
  2. We will use total inodes to set the total number of the progress bar

@timfeirg
Copy link
Contributor Author

timfeirg commented Jan 2, 2025

dump command did not finish, even after dumped inodes exceed file system usage

don't forget to investigate this as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants