YARN-9565 RMAppImpl#ranNodes not cleared on FinalTransition
This is a reproduction and graph generation for YARN-9565 bug, a minor memory leak.
Code modification such that it is easily reproducible
Note line 1503 in RMAppImpl.java, there is a System.getenv("APP_RANNODES_CLEAR") if statement. Therefore, setting this environmental variable to "true" would be applying the patch. Setting it to any other thing would mean not applying the patch.
How to measure the effect of patch (follow the markdown sections in the notebook)
Repetitively run wordcount mapreduce task, and take java heap dump afterward. Try to find the memory usage of CustomConcurrentSkipListSet class which is an inherited class of the original ConcurrentSkipListSet. This allows easier searching in the java heap dump.
Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.
Request daypassIf you do not have an active Chameleon allocation, or would prefer to not use your allocation, you can request a temporary one from the PI of the project this artifact belongs to.
Download ArchiveDownload an archive containing the files of this artifact.
Download with git
Clone the git repository for this artifact, and checkout the version's commit
git clone https://github.com/jiajunmao/yarn-repro-trovi.git
# cd into the created directory
git checkout 4791512981448b841fdcf316c35cfce8613e0741Submit feedback through GitHub issues