Frequently Asked Questions

 

General

What is Chameleon?

Chameleon is an experimental testbed for Computer Science funded by the NSF FutureCloud program. Chameleon is built over two sites, the University of Chicago and TACC, offering a total of over 550 nodes and 5 PB of space in twelve Standard Cloud Unit (SCU) racks. To effectively support Computer Science experiments, Chameleon offers bare metal reconfigurability on most of the hardware. To provide easy access to educational users, two SCUs at TACC (one sixth of the testbed) are configured with OpenStack KVM. You can read more about Chameleon here.

 

How should I acknowledge Chameleon?

An acknowledgement of support from the Chameleon project and the National Science Foundation should appear in any publication of material, whether copyrighted or not, that describes work which benefited from access to Chameleon cyberinfrastructure resources. The suggested acknowledgement is as follows: “Results presented in this paper were obtained using the Chameleon testbed supported by the National Science Foundation”.

In addition, it helps us a lot when you cite the Chameleon paper (see below). <meta charset="UTF-8">This makes it much easier for us to find the specific instances of research produced using Chameleon, and understand better how the testbed is used to support specific experiments. 

 

How should I cite Chameleon? 

If you use Chameleon in a publication, we would appreciate citations. The best reference for Chameleon is:

Kate Keahey, Jason Anderson, Zhuo Zhen, Pierre Riteau, Paul Ruth, Dan Stanzione, Mert Cevik, Jacob Colleran, Haryadi S. Gunawi, Cody Hammock, Joe Mambretti, Alexander Barnes, François Halbach, Alex Rocha and Joe Stubbs. "Lessons Learned from the Chameleon Testbed". In Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC '20). USENIX Association. July 2020.  Full paper

BibTex entry:

@incollection{keahey2020lessons,
  title={Lessons Learned from the Chameleon Testbed},
  author={Kate Keahey and Jason Anderson and Zhuo Zhen and Pierre Riteau and Paul Ruth and Dan Stanzione and Mert Cevik and Jacob Colleran and Haryadi S. Gunawi and Cody Hammock and Joe Mambretti and Alexander Barnes and Fran\c{c}ois Halbach and Alex Rocha and Joe Stubbs},
  booktitle={Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC '20)},
  publisher={USENIX Association},
  month={July},
  year={2020}
}

Visit the papers page for more Chameleon papers.

 

Where do I find help?

If you need any help with specific questions related to Chameleon, the Chameleon help desk staff is here to support you. You can easily reach the help desk by submitting tickets through the help desk portal, visible next to your username on the Chameleon home page.

 

Project and Allocation Management

How do I apply for a Chameleon project?

Project applications may be filled out here. If you want to apply for a project you have to be PI eligible; if you fulfill the PI eligibility criteria but did not request PI eligibility when you applied for a Chameleon account you can request it by modifying options in your profile. An application for a project has to include a description of the research or education project to be performed using the testbed and the type of resources needed (see below). Each Chameleon project is awarded an allocation of service units for a specific amount of time. Users can expect a project decision within one business day.

 

My PI/Professor/Colleague already has a Chameleon Project. How do I get added as a user on the project?

You will need to contact the project PI and request that they add you as a user. Provide the PI with your Chameleon username. The project PI should visit the Chameleon Project Management page. From there, the PI may follow the instructions on how to manage users.

 

What are the units of an allocation, and how am I charged?

Chameleon allocations can consist of several components of the system. Users can request allocation of individual compute nodes, storage servers, or complete Standard Cloud Unit (SCU) racks which contain compute servers, storage nodes, and an open flow switch.

Compute servers are allocated in Service Units (SUs), which equates to one hour of wall clock time on a single bare metal server. Note this unit differs from traditional HPC or cloud service units that are charged in core-hours; a Chameleon SU is a full server, as the type of experiments and performance measurements users may wish to do may be contaminated by sharing nodes.

Storage, GPU and FPGA nodes are also charged in SUs, at 2x the rate of compute servers (i.e., 1 hour allocation of 1 storage server == 2 SUs). SCU racks are charged at the rate of 50 SUs per wall clock hour (42 compute servers, 4 storage nodes, plus one OpenFlow switch).

An allocation may make use of multiple SCUs, up to the size of the full testbed.

For example, a user wishing to provision a 10 node cluster +1 storage server for a 1 week experiment should budget [(10 + 2) SUs per hour] * [7 days * 24 hours/day] = 2,016 SUs for that experiment.

Network resources are charged in SUs as well, at 1x the rate of compute servers, except for ExoGENI VLANs. For example, 1 hour allocation of 1 floating IP costs 1 SU. ExoGeni VLANs are charged at 4x the rate of compute servers.

SUs are charged the same regardless of use case — you are charged for the fraction of the resource your experiment occupies, regardless of the type of the experiment.

The basic principle for charging service units for Chameleon resources is to evaluate the amount of time a fraction of the resource is unavailable to other users. If a reservation is made through the portal for a particular date/time in the future, the user will be charged for this time regardless of whether the reservation is actually used, as the Chameleon scheduling system will have to drain the appropriate part of the system to satisfy the reservation, even if the nodes requested are not actually used. A reservation request may be cancelled in which case no charges will apply.

 

What are the project allocation sizes and limits?

In the initial phase Chameleon is operating on a “soft allocation model” where each project, if approved, will receive a startup allocation of 20,000 SUs for six months that can be both recharged (i.e., more SUs can be added) and renewed (i.e., the duration can be extended) via submitting a renew/recharge request. This startup allocation value has been designed to respond to both PI needs (i.e., cover an amount of experimentation needed to obtain a significant result) and balance fairness to other users (it represents roughly 1% of testbed six months’ capacity). Requests for these startup projects will receive a fast track internal review (i.e., users can expect them to be approved within a few days).

A PI can apply for multiple projects/allocations; however, the number of held allocations will be taken into account during review.

As our understanding of user need grows we expect the Chameleon allocation model to evolve towards closer reflection of those needs in the form of more differentiated allocations that will allow us to give larger allocations to users for longer time.

 

What is the allocation model for KVM virtual machines?

KVM resources do not use up your SU allocation, but projects are bound to the following resource limits on KVM:

  • 10 instances
  • 20 VCPUs
  • 50 GB RAM
  • 10 volumes
  • 50 floating IP addresses

If your project requires more resources than this, your quota can be increased upon request via the Help Desk.

 

What is the format of an allocation proposal?

A Chameleon Allocation request consists of the following components:

  • Project Title
  • Project abstract describing the proposed experiments including the type of resources needed; this part is required and may be published on Chameleon website (~200 words)
  • Supplemental details; this is an optional extension of the project abstract, potentially including details that the PI does not wish to publish such as e.g., sources of funding that support the proposed research (500 words maximum)

 

What criteria is used to review project proposals?

Requests for projects and allocations are currently reviewed for merit by project operators with a future move towards review by independent review board composed of Chameleon Science Advisory Board members. The following criteria are used:

  • PI eligibility
  • Relevance of the proposed experiment to cloud computing research; scientific merit and significance of the proposed experiments
  • Demonstrated need for Chameleon resources, methodology appropriate to the use of the Chameleon resource, justification of the requested allocation
  • Success of prior or other existing allocations (for renewals) in terms of published research results and new funding
  • Technical feasibility (i.e, can the project succeed in the Chameleon environment?)
  • Any funded support for the project (optional, but we want to make certain that we give allocations to NSF CISE-supported cloud computing research!)

 

Policies

What are the security best practices on Chameleon?

The security best practices are described in this article. 

 

What are the consequences of not following the security best practices and causing a security incident?

The Chameleon team will terminate all instances involved in a security incident and place an administrative lock on all interactions from the project under which the incident was caused that will last until the incident is resolved. This means that your negligence will affect not only you but also your collaborators on the project for the duration of the incident. Depending on the severity of incident Chameleon personnel may further suspend Chameleon access for anybody on the affected project for a week. In the case of repeat offenders Chameleon access will be withdrawn until further notice. 

 

What are the policies on Chameleon resource usage?

Allocation:

Chameleon projects are granted allocations of "service units", currently set to 20,000 service units for 6 months. Leases are charged against the allocation's balance of service units. Allocations can be renewed or extended. See the project management documentation for more details on Chameleon allocations

Leases:
Specialized hardware on Chameleon can sometimes be in high demand and low supply. Like fishing boats sharing a pond, we must beware to avoid a tragedy of the commons and adopt community practices that encourage efficient and fair resource use.

To ensure fairness to all users, resource reservations (leases) are subject to the following policies:

  • A new lease cannot conflict with a pre-existing one, but can be scheduled in the future if needed.
  • You can request a lease duration of up to 7 days. 
  • Within 48 hours of lease expiry, you can request an extension, for up to an additional 7 days. 
  • KVM resources do not require a lease, and will remain active as long as your allocation is active.

In addition, we actively discourage “lease stacking”—the practice of obtaining multiple overlapping reservations for resources (e.g., compute instances, storage, or networking components) to extend project usage beyond typical limits or to ensure continuous access. Please do not try to circumvent the one week lease restriction by stacking leases. Engaging in such stacking could lead to resource hoarding, impacting the availability for other users. Our experience has shown that such conduct results in idle hardware capacity and creates negative incentives for all users to “overfish” the pond out of fear that others will do so first.

We monitor the number and duration of leases across all projects for a given resource type. Projects with excessive reservations will be flagged for review. The project’s Principal Investigator (PI) will be contacted to provide justification or remove the violating leases within 3 business days. Failure to respond in a timely manner will result in the violating leases being terminated by Chameleon staff.

Exceptions for extended access or intensive resource use may be granted for projects with significant computational needs, such as large-scale experiments and educational use cases. Users can submit exception requests through the Help Desk.

Requests for exceptions should be made by the Project's PI and must detail the project’s goals, the need for additional resources, and how impact on the broader user community will be minimized. If you require a longer lease than 7 days or if you would like to extend your current lease, see the relevant FAQ items below.

Exceptions will be made sparingly. Chameleon offers other tools and methods for saving your work at the end of a lease and automating relaunching on a new lease. See documentation on cc-snapshot (which simplifies the process of saving the state of a bare-metal instance) and our blog post on making instance snapshots on Chameleon.

 

What are the best practices for using Chameleon bare metal partitions?

  • While it is possible to repeatedly request a lease for the same resource, we Strongly discourage this. Instead, save your work between leases using the CC-Snapshot instructions below.
  • Do not reserve more resources that you need at any given time. For example, if you need relatively few resources to develop something that you will then test at large scale, start with a small reservation for the development phase of your work and enlarge it later to test at scale – as opposed to making a large reservation up front.
  • Always release the reservation if you will not use the testbed for an extended period of time. For example, when you leave for the weekend, holidays, or simply need to take a break from experimentation and analyze your experiment, the resources could be used by others.
  • Automate creating your experimental environment. You can use scripting or some of the tools we provide that let you save your appliances/images between sessions (cc-snapshot ) or to orchestrate the deployment of complex environments (Complex Appliances) – if you need more ideas read our article on How to Make the Most of your Seven Day Lease. This will make reestablishing your experiment in a new lease easier and also makes it easier for you to reproduce your work and eventually share it with colleagues.

 

Who can use Chameleon?

Chameleon is broadly available to members of the US Computer Science research community and its international collaborators working in the open community on cloud research. By emphasizing “open” we mean that the expectation is that any research performed on Chameleon will result in publication in a broadly available journal or conference.

 

Who is eligible to be Chameleon PI?

Chameleon PIs carry significant responsibility for the users on their projects; we therefore limit PI eligibility to individual from the following groups:

  • Academic institutions: This eligibility criterion covers research scientists, research staff, and faculty members in supervisory positions at academic institutions. Graduate and PhD student researchers (including those serving as paid research assistants) are not typically considered eligible for PI status on Chameleon. Students should instead ask their faculty advisor to request PI status and give them access to a project.
  • Federal agencies such as national labs, R&D centers, and institutes: Research staff employed by federal agencies or non-NSF Federally Funded R&D Centers (FFRDCs) are eligible to apply for an allocation.
  • Independent museums, observatories, libraries, research laboratories, professional societies and similar organizations in the United States that are directly associated with educational or research activities are eligible.
  • International research institutions: to promote intellectual exchange and federation with institutions abroad we support a limited number of international PIs with ongoing, active collaborations with scientists in the US.
  • NSF Graduate Student Fellows: While in most cases, a graduate student is ineligible to be PI of an allocation request, an exception is made for NSF Graduate Student Fellows. Recipients of these NSF awards can submit requests for Startup allocations as long as they include supporting documentation (grant number or an award letter) as part of the request submission.
  • State educational offices or organizations and local school districts may submit allocation requests intended to broaden the impact, accelerate the pace, and increase the effectiveness of improvements in science, mathematics, and engineering education in both K-12 and post-secondary levels. A teacher or educator at an accredited public or private K-12 school is eligible to apply for an allocation as PI.

We do occasionally provide case-by-case exceptions to this guideline in well-justified cases.

 

Tips and Tricks

How do I make sure that my PI status is reflected in my profile?

If you are eligible to be PI (you can verify in this section), in order to apply for a project you need to make sure that your Chameleon profile reflects your status. You can do so on the Edit Account Profile page. Simply check the "Request PI Eligibility" checkbox and save you Account Profile. 

 

How can I extend a Chameleon lease?

An active Chameleon lease within 48 hours of its end time can be prolonged by up to 7 days from the moment of request if resources are available. To prolong a lease, click on the “Update Lease” button in the Reservations panel of the CHI OpenStack dashboard, and enter the additional duration requested in the “Prolong for” boxes. If there is an advance reservation blocking your lease prolongation that could potentially be moved, you can interact through the users mailing list to coordinate with others users.

 

What if I need a lease that is longer than the limitation (i.e., 7 days)?

If you know from the start that your lease has will require more than a week and cannot be broken into two or more 7 day leases because of the nature of the experiment, you can contact Chameleon staff via the ticketing system to request a one-time exception to create a longer lease. The request has to be submitted by the project PI and should contain a detailed justification for why a contiguous lease is needed. Please note, that these requests may take a longer time to consider as needed to understand all the details.

 

I am new to SSH, how do I create my own SSH key pairs on Linux/macOS?

Whenever you are creating an instance in Chameleon, you will have an option to select an Public SSH Key imported from your desktop. Once selected, this public key will be inserted into the instance's ~/.ssh/known_hosts file. When a user attempts to connect to the instance, the private key provided by the user will be validated against this public key in the known_hosts file. These instructions will help you create an SSH key pair and log in to your instance on Chameleon.

For Linux/ Mac OS X

Open a terminal window:

  • In a Mac OS X system, click on your launchpad and search for terminal
  • In an Ubuntu system you can use the keys Ctrl+Alt+T (for desktop version)

Access the SSH key pairs directory; in your terminal type the command:

cd ~/.ssh

Create your ssh key pair (public and private keys); in the .ssh directory, type the command:

ssh-keygen

Press the enter key, then enter a name for your key.

After completing the previous step, a message stating “Enter file in which to save the Key” will be displayed. Enter the name of your preference. I will use in this example the name “sample-key”. Then press the enter key.

Then, you will be requested to enter a passphrase for your key. Entering a passphrase is not necessary, so you can proceed to leave it blank and press enter. You will receive a message “Enter same passphrase again:” so just leave it blank and press enter.

Since we are still in the .ssh directory, now you can see your newly created key by typing ls.

You will see two files:

  • sample-key (containing the private key)
  • sample-key.pub (containing the public key)

You may view your sample-key.pub contents by typing:

cat sample-key.pub

Select and copy the contents displayed starting ssh-rsa all the way to the end. To add a key pair in Chameleon, follow the instructions for importing key pair and paste the contents of the key in the Public Key text entry.

After you have created a key pair and imported it in Chameleon, you can connect to any instance configured with this key pair. To do so you can use the command:

ssh -i ~/.ssh/sample-key cc@<instance ip address>

For Windows

First, download and install PuTTY and PuTTYgen from here. Once downloaded, opening PuTTYgen will open a key generator window, seen below.

I can't ping or SSH to my instance, what are some good things to try?

While the possibility that the system is being taking over by nanites should not be discounted too easily, it is always prudent to first check for the following issues:

  • Do you have a floating IP associated with your instance? By default, instances do not have publicly-accessible IP addresses assigned. See our documentation on associating a floating ip.
  • KVM only: Does your security group allow incoming ICMP (e.g. ping) traffic? By default, firewall rules do not allow ping to your instances. If you wish to enable it, see our documentation on security group.
  • KVM only: Does your security group allow incoming SSH (TCP port 22) traffic? By default, firewall rules do not allow SSH to your instances. If you wish to enable it, see our documentation on security group.

If none of these solve your problem, please open a ticket with our help desk, and send us the results of the above (and any evidence of nanites you find as well).

 

Why are my instances failing to launch?

The Chameleon Bare Metal clouds require users to reserve resources before allowing them to launch instances. Please follow the documentation on making reservations and make sure that:

  • You have created a lease with physical nodes and it has started (the associated reservation is shown as 'Active')
  • You have selected your reservation in the Launch Instance panel
  • You don't over-use your reservation

If you get an error stating that 'No valid host was found', it might be caused by a lack of resources in the cloud. The Chameleon staff continuously monitors the utilization of the testbed, but there might be times when no more resources are available. If the error persists, please open a ticket with our help desk.

 

What does CHI mean?

CHI stands for Chameleon Infrastructure, and refers to the technology powering our bare-metal clouds: a combination of software components from OpenStack, Grid'5000, and our own developments.

 

What is CHI-in-a-box?

CHI-in-a-box is a packaging of the implementation of the core services that together constitute the Chameleon testbed for experimental Computer Science research. These services allow Chameleon users to discover information about Chameleon resources, allocate those resources for present and future use, configure them in various ways, and monitor various types of metrics.

While a large part of CHI (CHameleon Infrastructure) is based on an open source project (OpenStack), and all the extensions we made are likewise open source, without proper packaging there was no clear recipe on how to combine them and configure a testbed of this type. CHI-in-a-box is composed of the following three components: (a) open source dependencies supported by external projects (e.g., OpenStack and Grid’5000), (b) open source extensions made by the Chameleon team, both ones that are scheduled to be integrated into the original project (but have not been yet) and ones that are specific to the testbed, and (c) new code written by the team released under the Apache License 2.0.

We have identified demand for three types of scenarios in which users would like to use a packaging of Chameleon infrastructure:

Chameleon Associate: In this scenario a provider wants to add resources to the Chameleon testbed such that they are discoverable and available to all Chameleon users while retaining their own project identity (via branding, usage reports, some of the policies, etc.). This type of provider will provide system administration of their resources (hardware configuration and operation as well as CHI administration with the support of the Chameleon team) and use the Chameleon user services (user/project management, etc.), user portal, resource discovery, and appliance catalog. All user support will be provided by the Chameleon team.

Chameleon Part-time Associate: This scenario is similar to the Chameleon Associate but while the resources are available to the testbed users most of the time, the provider anticipates that they may want to take them offline for extended periods of time for other uses. In this scenario Chameleon support extends only to the time resources are available to the testbed.

Independent Testbed: In this scenario a provider wants to create a testbed that is in every way separate from Chameleon. This type of provider will use CHI for the core testbed services only and operate their user services (i.e., manage their own user accounts and/or projects, help desk, mailing lists and other communication channels, etc.), user portal, resource discovery, and appliance catalog (some of those services can in principle be left out at the cost of providing a less friendly interface to users). This scenario will be supported on a best effort basis only.

 

What is a Jupyter Notebook?

Developed by Project Jupyter, the Jupyter Notebook is an open-source web application where you can create rich documents that marry code, data, documentation, and visualization. Jupyter Notebooks are used in many fields for the collection and analysis of data, and we are now seeing explorations of their use in the research sphere. All Chameleon users can get their own Jupyter Notebook server provisioned automatically by going to the Chameleon JupyterHub server and logging in with their Chameleon credentials. Chameleon Notebook servers come pre-installed with some convenience libraries to make it easier to interact with the Chameleon testbed. See the Jupyter Notebook documentation for more details.

 

What is Trovi?

Chameleon Trovi is a sharing portal that allows you to share digital research and education artifacts, such as packaged experiments, workshop tutorials, or class materials. Each research artifact is represented as a deposition (a remotely accessible folder) where a user can put Jupyter notebooks, links to images, orchestration templates, data, software, and other digital representations that together represent a focused contribution that can be run on Chameleon. Users can use these artifacts to recreate and rerun experiments or class exercises on a Jupyter Notebook within Chameleon. They can also create their own artifacts and publish them directly to Trovi from within Chameleon’s Jupyter server. You can learn more in the Trovi documentation.

 

How do I reserve GigaIO Composable Hardware?

To reserve our new composable hardware, please follow these steps:

  1. Review the background information

    Read our blog post on GigaIO Hardware for an overview of the technology and its capabilities.

  2. Understand the reservation process

    • You need to reserve multiple nodes to access multiple GPUs, even if you plan to compose a system with multiple GPUs on one node.
    • Each node's "default" configuration is available in the reference-api and for reservation.
    • Example: To reserve 4 GPUs and 1 node, you'll need to reserve all 4 nodes, as each node "comes with" one GPU.

  3. Make your reservation

    Use the standard reservation process to book the required number of nodes.

  4. Request reconfiguration

    After making your reservation, submit a helpdesk ticket with the title "Composable Hardware Configuration Request." We will trigger the re-composition of the resources when your reservation starts.

Note: Due to the complexity of reconfiguring the switch fabric, we do not currently have an API for direct reconfiguration. Our team will handle the reconfiguration based on your helpdesk ticket.

For any questions or assistance, please don't hesitate to contact our support team.

 

Federated Login

What is federated login?

Federated login enables users to use a single set of credentials to log into many different services. For example, federated login allows you to use your university or other institutional credentials to log into Chameleon -- there is no need to create a new account. In addition, since federated login is supported by many testbeds and services across scientific infrastructures you will be able to sign in once and use multiple services. For example, users who have allocation on both the CloudBank and Chameleon testbeds will be able to sign into one of them and use both. Users who have allocation on both GENI and Chameleon testbeds will likewise be able to access both by signing in only once. And users who use Globus services and Chameleon at the same time will be able to use both as well. 

 

Why did Chameleon move to federated login?

Federated login has many advantages for our users: it makes it easier to log into Chameleon, reduces the number of accounts a user has to create to use scientific infrastructure, and makes using testbeds and tools in conjunction much easier by supporting single sign-on across multiple services. The latter also makes it easier for Chameleon to add new integrated services -- for example, this move to federated login will enable support for single sign-on for Jupyter on Chameleon. Finally, long-term it will allow us to scale the testbed to many sites more efficiently and securely. 

 

How is federated login implemented on Chameleon?

Chameleon uses Globus Auth, a popular authentication service, to implement federated login. New users can sign up via their existing host institution account, use their Google account, or create a Globus ID tied to an email and password that they provide.

 

It looks like I have multiple set of credentials that I could log in with: which one should I choose?

In many cases, our uses will be able to use multiple identities to gain access to the testbed: this happens for example if your institution is part of InCommon and you also have a Google account. Is one set of credentials better to use than another? This largely depends on your preferences and the nature of your work. Many users like to use their Google account in a “lingua franca”, access everything capacity: it has the advantage of simplicity. On the other hand, if you use infrastructure based on InCommon login (such as GENI or CloudBank) in conjunction with Chameleon, logging in via credentials linked to one of the InCommon institutions will have the advantage that you need to authenticate only once in order to access both infrastructures. Note that no specific set of credentials is permanently linked to your CHameleon access: so if you you choose to log in with Google account one time, you may choose your institutional credentials another time without changes to your authentication workflow.


What entities is Chameleon federated with?

Since Chameleon uses Globus Auth, it is federated with identities supported via InCommon, Google, and Globus ID. Additionally, we also support the TAS entity. For most of you, the good news is probably that you can log in with your google account! 

 

What will moving to federated identity mean for me in the short-term?

The move to federated identity effectively creates a new account for each user. This means that data associated with your current Chameleon account, such as keypairs, any disk image snapshots published to your project(s) via cc-snapshot, experimental metrics and Experiment Precis data for old experiments and leases, RC files for CLI usage, as well as active instances and leases will NOT be available from your new account. 

We implemented a migration process that will automatically copy your keypairs and share your images with the new account. Please, bear in mind that the migration process works only one way -- from your old to your new account -- in other words, new images and keypairs created under your new account will not be available under your old account. You will however be able to apply the migration process multiple times throughout the migration period to apply updates made under your old account.  Migrating any other data will be your responsibility -- but as always, please contact us via the help desk (the sooner the better) and we will do our best to help!

Due to the fact that data will not be shared automatically between accounts, users working on collaborative projects may want to coordinate migration activities to ensure that everybody uses either old or federated accounts. 

Please, note that during the migration period (October and November of 2020) you will be able to log in to both the old and the new account so that you can finish your work and migrate data as needed. However, after the migration period the data associated with your old account may be lost. 

 

What is the schedule for migration to federated identity?

October 12th:
New federated login with migration support becomes available; users can use it on an opt-in basis since old login remains supported till the end of November

October 19th: 
The existing Chameleon account creation feature becomes disabled; all new Chameleon users will be logging in with federated login

November 1st:
The “Log in with GENI” feature goes away (users can use federated login to log into Chameleon with their GENI credentials) and federated login becomes the default login option (though you will also be able to log in “the old way” by following a link)

December 1st:
Federated login becomes the only login option; please make sure to migrate to the new account by then! 

 

How do I migrate to federated identity on Chameleon?

To migrate, please follow our migration instructions. 

 

What changes should I expect when working with a federated identity management?

A federated identity creates a login context over potentially multiple infrastructures and tools; these tools refresh their login information based on this context. This provides a single sign-on over multiple tools (it may take a few clicks in some cases though you will likely not have to enter a password). It may also mean that logging out of one of those tools will not affect your login status with others, and when you try to log in again your context will be refreshed based on the federated login context (i.e., you won’t be prompted for a password). In order to ensure that you have logged out completely, after you log out of Chameleon, you should also terminate your session with your federated identity provider--this could mean logging out of your Google account, or perhaps your InCommon institution.

 

Appliances

What is an appliance?

An appliance is an application packaged together with the environment that this application requires. For example, an appliance can consists of the operating system, libraries and tools used by the application, configuration features such as environment variable settings, and the installation of the application itself. Examples of appliances might include a KVM virtual machine image, a Docker image, or a bare metal image. Chameleon appliance refers to bare metal images that can be deployed on the Chameleon testbed. Since an appliance captures the experimental environment exactly, it is a key element of reproducibility; publishing an appliance used to obtain experimental results will go a long way to allowing others to reproduce and build on your research easily.

To deploy distributed applications on several Chameleon instances, complex appliances combine an image and a template describing how the cluster should be configured and contextualized. You can read more about them in the complex appliance documentation.

 

What is the Chameleon Appliance Catalog?

The Chameleon Appliance Catalog is a repository that allows users to discover, publish, and share appliances. The appliance catalog contains useful images of both bare metal and virtual machine appliances supported by the Chameleon team as well appliances contributed by users.

 

How to build or customize a Chameleon appliance?

There are two options to build or customize a Chameleon appliance -- the cc-snapshot utility and the OpenStack diskimage-builder.

The cc-snapshot-utility
The cc-snapshot tool is pre-installed in all Chameleon supported appliances and it provides a quick and easy way to customize a Chameleon appliance. To start, spin up an instance with the Chameleon appliance you would like to customize. Then install the libraries and tools you would like to add into your new appliance, or uninstall things you want to exclude from your new appliance. Finally, take a snapshot by running the cc-snapshot command.

The OpenStack diskimage-builder
You can use diskimage-builder to build your appliance from scratch or customize the Chameleon appliances by using the code on Github as templates (CC-CentOS7, CC-Ubuntu14.04, CC-Ubuntu16.04). The OpenStack diskimage-builder provides a more manageable way of building appliances. For more information about OpenStack diskimage-builder, please see the OpenStack documentation.

 

How do I publish an appliance in the Chameleon Appliance Catalog?

The new Appliance Catalog allows you to easily publish and share your own appliances so that others can discover them and use them either to reproduce the research of others or as a basis for their own research. Before creating your own appliance it is advisable to review other appliances on the Chameleon Appliance Catalog in order to get an idea of the categories you will want to contribute and what others have done.

Two methods exist to submit an appliance to the Appliance Catalog. They can be added using the simplified process available through the Images view. They can also be added using the manual process as described below:

  1. Create the appliance itself. You may want to test it as well as give some thought to what support you are willing to provide for the appliance (e.g., if your group developed and supports a software package, the appliance may be just a new way of packaging the software and making it available, in which case your standard support channels may be appropriate for the appliance as well).
  2. Upload the appliance to the Chameleon Image Repository (Glance) and make the image public. In order to enter the appliance into the Catalog you will be asked to provide the Glance ID for the image. These IDs are per-cloud, so that there are three options right now: bare metal/CHI at University of Chicago, bare metal/CHI at TACC, and OpenStack/KVM at TACC. You will need to provide at least one appliance, but may want to provide all three.
  3. Go to the Appliance Catalog Create Appliance web form, fill out, and submit the form. Be prepared to provide the following information: a descriptive name (this sometimes requires some thought!), author and support contact, version, and an informative description. The description is a very important part of the appliance record; others will use it to evaluate if the appliance contains tools they need for their research so it makes sense to prepare it carefully. To make your description effective you may want to think of the following questions: what does the appliance contain? what are the specific packages and their versions? what is it useful for? where can it be deployed and/or what restrictions/limitations does it have? how should users connect to it / what accounts are enabled?

If you are adding a complex appliance, skip the image ID fields and enter your template instead in the dedicated text box.

As always, if you encounter any problems or want to share with us additional improvements we should do to the process, please don’t hesitate to submit a ticket.

 

How can I manage an appliance on Chameleon Appliance Catalog?

If you are the owner of the appliance, you can edit the appliance data, such as the description or the support information. Browse to the appliance that you want to edit and view its Details page. At the top right of the page is an Edit button. You will be presented with the same web form as when creating the appliance, pre-filled with the appliances current information. Make changes as necessary and click Save at the bottom of the page.

And finally, you can delete appliances you had made available. Browse to the appliance that you want to delete and click Edit on the Appliance Details page. At the bottom of the page is a Delete button. You will be asked to confirm once more that you do want to delete this appliance. After confirming, the appliance will be removed and no longer listed on the Appliance Catalog.

 

Why are there different image IDs for KVM@TACC, CHI@TACC, and CHI@UC for the same appliance?

The three clouds forming the Chameleon testbed are fully separated, each having its own Glance image repository. The same appliance image uploaded to the three clouds will produce three different image IDs. In addition, it is sometimes needed to customize an appliance image for each site, resulting in slightly different image files.

 

Can I use Ubuntu, Debian, or another operating system rather than CentOS on bare-metal?

The recommended appliance for Chameleon is CentOS 7 (supported by Chameleon staff), or appliances built on top of it.
These appliances provide Chameleon-specific customizations, such as login using the cc account, the cc-checks utility to verify hardware against our resource registry, gathering of metrics, etc. Since 2016, we also provide and support Ubuntu 14.04 and 16.04 appliances with the same functionality.

 

The cc-snapshot tool doesn't work on previously snapshot images.

The cc-snapshot is occasionally updated to accommodate changes to the infrastructure and distributions. To replace the script in your image, follow our instructions for updating cc-snapshot.

 

How do I move images between sites?

Chameleon bare-metal sites -- CHI@TACC and ``CHI@UC`` -- belong to a single OpenStack deployment as two :ref:`independent <bare-metal-sites-independent>` regions.
You can move images between sites by using the :ref:`command line interface <cli>`. Make sure you have :ref:`installed CLI properly <cli-installing>` and :ref:`configured the environment variables using the rc script <cli-rc-script>`.

  • Download the image from the source site to local

openstack --os-region-name <source_site [CHI@TACC or CHI@UC]> image save <image_name> --file <filename>

  • Upload the image to the target site from local

openstack --os-region-name <target_site [CHI@TACC or CHI@UC]> image create --file <filename> --disk-format <format> <image_name>

You can get disk-format from the output of the following command:

openstack --os-region-name <source_site [CHI@TACC or CHI@UC]> image show <image_name>