So your experiment server has been hacked. Not only is this a bit embarrassing, it can lead to your experiments being terminated by Chameleon support staff (to prevent further infection or malicious traffic), and you can lose valuable time setting things up securely again! Furthermore, if there is a pattern of bad security hygiene, we may need to disable your account to prevent future incidents.

 

It doesn't have to be this way! In fact, there are a few simple steps you can take to ensure that your experiment environment is as secure as possible. Here are a few do's and don'ts to keep in mind.

 

DO change the passwords. If you are basing your experiment on an image or software downloaded from the internet with a well-known default or admin password, change it! Consider that you move into a new apartment, and the key to your front door has already been copied by everybody in the city. Wouldn't you feel more comfortable changing the locks?

 

DO NOT change firewall rules or other security defaults. All Chameleon instances that are built using the Chameleon base images (for, e.g. CentOS or Ubuntu) come pre-configured with SSH settings and a firewall policy that is designed to protect your experimental environment from intrusion. In particular, do not alter SSH to allow password authentication; use strong public/private key pairs instead.

 

DO use a bastion host for sensitive experiments. Sometimes you need to run software you know is insecure, but you don't have a choice, because you need to test a specific version or configuration. In this case, it is useful to reserve an additional node to serve as a login bastion. The bastion is the only host that needs a public IP address; after connecting to the bastion, you simply jump through to other hosts on your private network from there. Pro tip: a bastion is also a good pattern to follow on a multi-node cluster setup! While it's possible to assign public IP addresses to each node in a multi-node cluster, this is wasteful (we only have so many public IPs!) and unnecessarily exposes each node to intrusion. Instead, designate a single node to be your login bastion and always log in to your cluster from that node.

 

DO NOT leave internet-facing services running longer than necessary. Automated scanners (e.g. Shodan) are running constantly to find and exploit new services that appear on the web. For a humorous demonstration of this, refer to this Atlantic article describing how a fake IoT toaster was hacked within an hour of making its internet debut. Combined with the tendency of security researchers and black hat hackers to find application/framework vulnerabilities at an impressive pace, it logically follows that keeping any service on the web for some significant amount of time presents risk. Therefore, consider shutting down any internet-facing applications when you're not using your node. If you want an automated solution, you could leverage the .bash_logout file, which allows configuring commands to run when you log out of the machine.

 

DO NOT bind web services on all interfaces unless absolutely necessary. Most web services can be configured to listen on only a specific IP address, but some default to listening on, e.g., 0.0.0.0, which effectively means "any address". What this means in practice is that a server that you may think is only running locally can actually be reached over the internet! Ensure that your application/service is set to only listen on a local loopback address (127.0.0.1) to avoid this, or at least set it to a private IP address on your network.

 

As Ben Franklin said, an ounce of prevention is worth a pound of cure. So it is with network security: by taking a few steps, thinking about risk, and using tools already at your disposal, you can ensure your experiments are properly secured, now and in the future.