YARN-9565 RMAppImpl#ranNodes not cleared on FinalTransition
This is a reproduction and graph generation for YARN-9565 bug, a minor memory leak.
Code modification such that it is easily reproducible
Note line 1503 in RMAppImpl.java, there is a System.getenv("APP_RANNODES_CLEAR") if statement. Therefore, setting this environmental variable to "true" would be applying the patch. Setting it to any other thing would mean not applying the patch.
How to measure the effect of patch (follow the markdown sections in the notebook)
Repetitively run wordcount mapreduce task, and take java heap dump afterward. Try to find the memory usage of CustomConcurrentSkipListSet class which is an inherited class of the original ConcurrentSkipListSet. This allows easier searching in the java heap dump.
Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.
Download ArchiveDownload an archive containing the files of this artifact.
Download with git
Clone the git repository for this artifact, and checkout the version's commit
git clone https://github.com/jiajunmao/hadoop-repro
# cd into the created directory
git checkout 577017b416aa8757ecdd68aba3a85a6e50745762Submit feedback through GitHub issues