| Outage start | Wednesday, May 11, 2022 8 a.m. |
| Expected end | Wednesday, May 11, 2022 4:13 p.m. |
Update: 4PM 05/11/2022: This issue is now resolved, provisioning and connectivity should be restored for all UC nodes.
For those who want the details, this issue was triggered by a (suspected) switch operating system bug. The switches had Per-VLAN Spanning-Tree enabled, and we provision isolated networks by configuring a range of vlans on each top of rack switch. However, once a certain threshold # of vlans is configured, the switch ceases to create more spanning-tree daemons. The observed behavior is that newly active ports no longer pass traffic, but already provisioned ports show no issues.
Forcing the switches to use a single spanning-tree daemon (RSTP mode), restored connectivity on affected ports.
An issue is affecting the provisioning of new instances on P3 nodes at UC. Existing nodes are unaffected.