Spark-29351-Bug-Reproduce
The ShuffleStatus serialization is implemented in synchronized block, which scale badly when number of executors increases. The detail description from the reporter is: https://issues.apache.org/jira/browse/SPARK-29351
Launch on Chameleon
Feedback
Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.
Download ArchiveDownload an archive containing the files of this artifact.
Download with git
Clone the git repository for this artifact, and checkout the version's commit
git clone https://github.com/QinPR/spark_bug_reproduce.git
# cd into the created directory
git checkout 58a5c0f98bf7dac9d8f38efbb66815f2cc5eff70Submit feedback through GitHub issues