Reported Outages

RESOLVED: The upgrade is complete! All services are operating normally.

On June 21st, Chameleon services at TACC will be unavailable to permit an OpenStack version upgrade. During this time, the CHI@TACC dashboard and API will be unavailable. Network access to running instances will also be interrupted, but instances will continue to run and will become available again after the upgrade. KVM@TACC will not be impacted.

UPDATE: The Network Maintenance scheduled for Sunday, June 26th, 2022 has been suspended. 
 

Network maintenance will be carried out between 10:00 AM and 4:00 PM (CST) on Sunday, June 26th. Access to all TACC systems will be unavailable during this time, including CHI@TACC, KVM@TACC, and the Chameleon Portal. Instances will continue to run, but users will have no access to TACC services and systems until the upgrade is complete.

Please submit any questions you may have via the TACC User Portal.

Update: 4:36 PM. NU has restored network connectivity, and the site is back up.

The network providing access to CHI@NU has gone down, preventing access to the site. The web UI and all instances are inaccessible.

Site staff are investigating, and we'll update here with a timeline for resolution.

Resolved: The system is now operating normally. Thank you for your patience.

CHI@TACC is currently experiencing an outage in provisioning baremetal nodes. This does not affect currently running instances, but prevents the launch of new ones. The team is working to resolve the issue.

Update 5:30 pm: Issues are resolved, all nodes are usable again.

Update 4pm June 7th: Provisioning of baremetal nodes is restored. We're seeing failures to create leases for P2 nodes (types compute_skylake, gpu_rtx_6000), but reservation of P3 nodes is succeeding.

Due to an unknown cause, users are unable to login to Chameleon.

We are experiencing an outage of the Jupyter environment and are working to restore service shortly, stay tuned, and apologies for the lack of notice.

Update: 4PM 05/11/2022: This issue is now resolved, provisioning and connectivity should be restored for all UC nodes.

An issue is affecting the provisioning of new instances on P3 nodes at UC. Existing nodes are unaffected.

Update: 6pm 05/10/22: The outage is now resolved. Both switches are now functional, and P2 nodes from nc01-nc64 are back online. New instances have no issues, existing instances may still have connectivity issues. If you have those issues, please try removing and re-attaching the network port to your instance.

Update 05/03/22: This issue is now resolved. It was caused by a combination of two factors: misconfiguration of the DHCP behavior for out-of-band interfaces, and a failure causing an out of band switch to power off.

All affected nodes should be reservable again. If you have an instance that has become inaccessable, please get in touch with us via the helpdesk.