As another semester begins, we’ve rounded up a series of fully packaged experiments on Chameleon. These experiments are all publicly available on Trovi, Chameleon’s sharing platform. The experiments can be used for classes to introduce different topics, tools, or datasets, or serve as an introduction to provisioning resources on Chameleon with Jupyter Notebook. Once you launch an experiment, you can edit the notebook, allowing individual experimentation, and letting you introduce variation, such as trying different resources or datasets. 

The structure used in the example notebooks to provision and launch resources can also be adapted to your needs, saving time to help you experiment! Simply copy the relevant commands to a notebook of your choosing, and begin your own experiments. 

Read on to learn more about the experiments currently available, how to access them, and what you need to do to begin experimenting! At the end of this blog, you’ll find steps, complete with pictures, to guide you through launching an experiment on Trovi with Jupyter Notebook.

 

Chameleon Quick Start:

If you’re unfamiliar with using Jupyter Notebook to provision resources with Jupyter Notebook, the Chameleon team has created a few notebooks designed to introduce this and the various capabilities available. You can also read the documentation on using Chameleon with Jupyter Notebook.

Openflow Quick Start Example

An artifact designed to help you get started using OpenFlow on Chameleon and can be used as a base for OpenFlow experiments or advanced network appliances. 

Jupyter Usage Metric Exploration

This notebook is an example data analysis notebook looking at usage patterns on Chameleon. Feel free to use as a model for your own data analysis needs.

Power Management Experiment Example

This example illustrates how to create a reproducible experiment in power management and describes tools available within the Chameleon base images, as well as the orchestration and snapshot capabilities.

 

Example Experiments on Chameleon:

1. Tiny-Tail Flash: Near Perfect Elimination of Garbage Collection Tail Latencies in NAND SSDs Reproduction

For this experiment, a Jupyter notebook is packaged and available from Zenodo, reproducing the Dev Tools Release experiment from this paper.

Experiment Estimated Time: 30 minutes

 

2. Training Convolutional Neural Networks with Pytorch on MNIST Dataset

This packaged notebook reproduces a simple benchmark experiment that trains a convolutional neural network with the MNIST dataset using Pytorch. The MNIST dataset contains 60,000 training images and 10,000 testing images of hand-written digits. 

Experiment Estimated Time: 45 minutes

 

3. Training Convolutional Neural Networks with TensorFlow on MNIST Dataset

This packaged notebook reproduces a simple benchmark experiment that trains a convolutional neural network with the MNIST dataset using TensorFlow. This notebook serves as a way to get hands-on experience with Chameleon and the basics of Machine Learning. The MNIST dataset contains 60,000 training images and 10,000 testing images of hand-written digits.

Experiment Estimated Time: 45 minutes

 

4. Using FlyMC to Explore Data Center and Cloud System Bugs

FlyMC is a fast and scalable testing approach for data center and cloud systems like Cassandra, Hadoop, Spark, and ZooKeeper, developed by the University of Chicago’s systems research group, UCARE. In this packaged notebook, you’ll use FlyMC to capture the Cassandra bug-5925, step by step. There’s also options to try other Cassandra, ZooKeeper, Spark and MapReduce bugs. 

Experiment Estimated Time: 1 hour

 

5. Image Classification with Network-in-Network model on MNIST Dataset

This packaged notebook illustrates how to create a reproducible experiment using machine learning libraries, models, and datasets and describes tools available within Chameleon and OpenStack. It replicates a Network in Network model implementation found on Kaggle and reproduces the original Network in Network model.

Experiment Estimated Time: 1 hour

 

6. Image Classification with AlexNet on the Stanford Dogs Dataset

A machine learning experiment packaged in Jupyter Notebook, designed to be run with tools available within Chameleon and OpenStack. The packaged notebook is available on Zenodo, making it easy to reproduce and perfect to use to teach machine learning or how to use the Chameleon testbed.

Experiment Estimated Time: 1 hour

 

7. LinnOS: Predictability on Unpredictable Flash Storage with a Light Neural Network

This experiment, featured as the October user experiments blog with its respective paper published at OSDI ‘20, is available on Trovi as a fully packaged notebook. Experiment with the notebook to create end-to-end baseline and LinnOS workflows. 

Experiment Estimated Time: >1 hour

 

8. Accuracy Levels with DAWNBench and TensorFlow End-to-End Training

This packaged notebook reproduces Figure 1 of the DAWNBench experiment using TensorFlow. Created at Stanford University, DAWNBench introduces a benchmark and competition that focuses on end-to-end training time for a model to reach a fixed accuracy level. The graph reproduced in this experiment illustrated the relationship between the accuracy level and the end-to-end training time for three different batch sizes. 

Experiment Estimated Time: 24 hours

 

Launching an Experiment from Trovi: All You Need to Know

One of the experiments has piqued your interest, now what? All it takes is 3 simple steps to get started experimenting!

1. After you click on any of the packaged notebook links mentioned in this blog, you’ll be taken to its shared Trovi page. From here, click ‘Launch on Chameleon’. If you aren’t logged into Chameleon, you’ll have to log in, but then the Jupyter Notebook will begin to load.

2. Once the Jupyter Notebook loads, you’ll see all the files associated with that experiment. On some experiments, you might have to click through folders first to navigate to the folder with the experimental files Most of the experiments contain a reservation script, GPU or CPU setup scripts, a yaml setup file, the experiment in a python file, and run_experiment and analysis python notebooks. 

3. To run the experiment, click on the Python notebook labeled as such (a variation of run_experiment), and begin running cells. 

Other Notes:

• Replace the project name with the project name you’re associated with that has an active allocation.

• You can adjust the Chameleon site where you use resources from by changing OS_REGION_NAME to CHI@TACC, CHI@UC, or CHI@NU.