We are excited to announce our Virtual Reproducibility Hackathon taking place on December 15, 2023, hosted by the Chameleon Team! See the details below for more information and to register.
About the Event
Reproducibility is not just a sound scientific practice – it also has many potential practical applications in teaching and creating new research results. At this event, the attendees will learn about how to package their computer science research experiments in such a way that they are not only reproducible – but also practically reproducible, i.e., capable of being reproduced easily enough to be used as a mainstream method of interactive scientific exploration and exchange.
Sign up to learn about practical reproducibility on Chameleon, receive guidance on packaging your experiments, share insights and feedback on the reproducibility process, and much more!
We hope that participants can use their initial work and what they learn at the Hackathon to make future contributions to reproducibility workshops and conferences such as ACM REP.
The Hackathon will consist of three sessions held throughout the day to accommodate folks outside of the Central time zone:
Session I: Tutorial – Packaging Experiments for Practical Reproducibility - 10 CT AM to 12 CT PM
Session II: Open Work Period – Packaging Your Experiments - 1 CT PM to 4 CT PM
Session III: Open Discussion – Share Your Experience - 4:10 CT PM to 5 CT PM
Participants are encouraged, but not required, to attend all three sessions.
Tutorial: Packaging Experiments for Practical Reproducibility
Session I will include an overview of Chameleon services, how to run experiments on Chameleon, how to package experiments for reproducibility, how to make such artifacts searchable and shareable, and other best practices for practical reproducibility. If you are interested in learning more about Chameleon Cloud and our digital artifact hub (Trovi), this session is for you!
Concepts may include how to use the NSF-funded Chameleon platform, one of the largest academic clouds, and how to package experiments for reproducibility using this platform; tools and services Chameleon provides to share experiments, including using Jupyter to manage the full experimental workflow including creating the environment, implementing the experiment body, and data analytics; using Chameleon daypass to give access to the testbed for reproducibility; as well as Trovi, an experiment sharing portal integrated with the testbed. The tutorial will also cover the existing experiment patterns available via Trovi representing common elements of experimental configurations such as e.g., configuring storage with RAID, NFS, or RDMA.
Open Work Period: Packaging Actual Experiments
During Session II, participants will work on packaging their experiments to share on the Chameleon digital artifact hub, Trovi, following the guidelines covered in Session I. Technical support and “office hours” will be provided for individuals and teams during this session. If you have an experiment that you have been wanting to share with others, this is a great opportunity to finally get that artifact online with some extra help and support from the Chameleon team!
No experiment to share at the moment? No problem! Participants are also encouraged to use this session to reproduce existing artifacts from a curated list of recommended experiments that we will post two weeks before the event.
Open Discussion: Share Your Experience
Participants will share their experiences during the hackathon and have time to ask more questions about the platform, give feedback on the Chameleon and Trovi sites, and brainstorm ideas to facilitate experiment reproducibility on public testbeds.
We hope to see you all on December 15!
Chameleon’s JupyterHub is a great way to organize your experiments for practical reproducibility. To overcome its resource limitations, we describe how to extend the Jupyter Server Trovi artifact so that you can run your full experiment inside a Jupyter notebook.
We are delighted to offer a limited number of registration reimbursement grants for the upcoming Practical Reproducibility Tutorial/Hackathon at the IC2E Conference, which takes place in Boston, MA on September 25, 2023. Read on for details of this offer and how to apply.
Calling all Chameleon users interested in reproducibiliity! As you have seen in our last month's changelog we are trying to make reproducible experiments on Chameleon more visible -- and now we are also organizing a hackathon that will help you package your results to be easy to reproduce on Chameleon via Trovi. Please, take a look and see if you can join us in Boston for a day of good ideas, good discussion, and good fun making experiments (including your experiments) more accessible!