HDFS-14973

The mechanism built to throttle data balancing in the cluster is broken, thus when many datanodes perform getBlock calls, they flood the namenode.

The intent was initially to be able to throttle the number of RPCs the namenode will receive, but that balancing got broken by other modifications. This ends up flooding the namenode. So this problem impacts the amount of load the namenode will receive.

Fix: Throttle load balancer so that the number of RPCs to the namenode does not exceed a limit at certain point in time.

2 1 - 1 Mar. 18, 2024, 10:09 AM

Authors

Launch on Chameleon

Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.

Download Archive

Download an archive containing the files of this artifact.

Version Stats

2 1 -