You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
Currently when calling rebalance_disk=true, it rebalances all disks of all brokers. When #2145 merges, it resolves part of the problem that we can throttle kafka I/O to avoid cluttering the cluster.
But our kafka cluster is not always updated. This feature is added in kafka 3.9.X, and I believe there are tons of clusters out there which are not updated, and possibly not even close to updating.
So for these clusters, even after #2145 merges, it doesn't help. But I propose that if we can do rebalance_disk broker by broker, it would be much better for the cluster as it lags only one broker at a time, not the whole cluster. This can be done with a flag to enable/disable this method, or with specifying brokerid and run rebalance multiple times for each broker.
This way, we can use rebalance disk feature with lowest impact. Without this, this feature is completely useless and in JBOD deployments where disks matters, CC loses lots of its values.
Thanks.
The text was updated successfully, but these errors were encountered:
Hi,
Currently when calling rebalance_disk=true, it rebalances all disks of all brokers. When #2145 merges, it resolves part of the problem that we can throttle kafka I/O to avoid cluttering the cluster.
But our kafka cluster is not always updated. This feature is added in kafka 3.9.X, and I believe there are tons of clusters out there which are not updated, and possibly not even close to updating.
So for these clusters, even after #2145 merges, it doesn't help. But I propose that if we can do
rebalance_disk
broker by broker, it would be much better for the cluster as it lags only one broker at a time, not the whole cluster. This can be done with a flag to enable/disable this method, or with specifying brokerid and run rebalance multiple times for each broker.This way, we can use rebalance disk feature with lowest impact. Without this, this feature is completely useless and in JBOD deployments where disks matters, CC loses lots of its values.
Thanks.
The text was updated successfully, but these errors were encountered: