| Outage start | Saturday, June 07, 2025 6 a.m. |
| Expected end | Monday, June 09, 2025 6 p.m. |
It appears that early morning on Saturday, container launches on CHI@Edge began timing out. We are currently investigating the cause, and will update here.
| Outage start | Sunday, June 01, 2025 9 a.m. |
| Expected end | Tuesday, June 10, 2025 4:28 p.m. |
Update June 10th: The datacenter maintenance has concluded, and CHI@NCAR is back online.
| Outage start | Wednesday, May 28, 2025 6 p.m. |
| Expected end | Wednesday, May 28, 2025 10 p.m. |
Tonight, from 6pm - 10pm US Central time, network maintenance at ANL will cause rolling outages impacting access to CHI@UC, Trovi, and to Authentication services for all Chameleon sites.
We expect the interruptions to be in the range of minutes to <= 1 hour, sometime during this window.
Connections (e.g. over ssh) to instances that are already running at other sites will not be affected, but you may observe interuptions in the ability to launch new instances due to the authentication outage.
| Outage start | Monday, May 12, 2025 4:30 p.m. |
| Expected end | Monday, May 12, 2025 6:56 p.m. |
Resolved: 7:00 pm Monday:
Reservations are workign again, and tests to launch a container and access it via floating IP are succeeding, implying that the network underlay is also healthy once again.
Please let us know if you continue to observe issues.
| Outage start | Wednesday, April 30, 2025 5:18 p.m. |
| Expected end | Wednesday, April 30, 2025 5:30 p.m. |
Update: 5:30pm: services appear to be reachable again, we saw a downtime of ~ 10 minutes. Our aplogies for the interruption.
We're waiting for more information about whether to excpect any more interruptions.
| Outage start | Thursday, April 24, 2025 12 p.m. |
| Expected end | Friday, April 25, 2025 9:27 a.m. |
9:00 AM 04/25/25: A fix has been deployed and we belive the issue to be resolved, but please reach out if you observe further issues.
Currently, authenticating from jupyter to KVM@TACC, or using "CLI" auth with federated ID credentials to KVM@TACC is failing.
| Outage start | Tuesday, April 22, 2025 9 a.m. |
| Expected end | Tuesday, April 29, 2025 5:19 p.m. |
Update: All work associated with this outage is complete.
| Outage start | Tuesday, April 29, 2025 9 a.m. |
| Expected end | Tuesday, April 29, 2025 9 a.m. |
On April 29th, CHI@Edge will be inaccessible as we migrate the control-plane to more robust hosting and upgrade several dependent services. During the maintenance window, the dashboard, running containers, and device enrollement will be unavailable.
| Outage start | Thursday, April 10, 2025 9 a.m. |
| Expected end | Thursday, April 10, 2025 4:23 p.m. |
Resolved
To prepare for upgrades to KVM@TACC, we are scheduling maintenance starting at 9:00 AM CDT on Thursday April 10th.
During this time, Staff will Stop All Running Instances, perform upgrades, then restart those instances.
This work will ensure that VMs are compatible with following upgrades, preventing further downtime.
| Outage start | Monday, March 17, 2025 11:35 a.m. |
| Expected end | Monday, March 17, 2025 6 p.m. |
We're observing issues with leases for some devices in CHI@edge, and have identified some configuration inconistency on the backend that may be contributing. While debugging, there was a brief outage in the reservation api service, which manifested as a failure to load the "leases" dialog.
Thing seem to be back, but we're still investigating.