Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebalance Disks Broker by Broker #2237

Open
imans777 opened this issue Jan 4, 2025 · 0 comments
Open

Rebalance Disks Broker by Broker #2237

imans777 opened this issue Jan 4, 2025 · 0 comments

Comments

@imans777
Copy link

imans777 commented Jan 4, 2025

Hi,
Currently when calling rebalance_disk=true, it rebalances all disks of all brokers. When #2145 merges, it resolves part of the problem that we can throttle kafka I/O to avoid cluttering the cluster.
But our kafka cluster is not always updated. This feature is added in kafka 3.9.X, and I believe there are tons of clusters out there which are not updated, and possibly not even close to updating.
So for these clusters, even after #2145 merges, it doesn't help. But I propose that if we can do rebalance_disk broker by broker, it would be much better for the cluster as it lags only one broker at a time, not the whole cluster. This can be done with a flag to enable/disable this method, or with specifying brokerid and run rebalance multiple times for each broker.
This way, we can use rebalance disk feature with lowest impact. Without this, this feature is completely useless and in JBOD deployments where disks matters, CC loses lots of its values.
Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant