During a major upgrade of our infrastructure, the new servers were not performing like excepted. Immediately our monitoring systems informed us and we took action to switch all traffic back to the old infrastructure. During the migration back to the old servers there was accidentally a firewall rule added onto our load balancers. This caused that only our monitoring systems and company members were able to reach the websites. Customers were blocked out due to this. Our monitoring systems showed us an anomaly in the amount of data packages sent. After 1h 30m we were able to find the underlying reason and resolved it.
We apologize for any inconvenience caused.
Posted Jun 26, 2020 - 02:13 UTC
A fix has been implemented and we are monitoring the results.
Posted Jun 26, 2020 - 01:29 UTC
We are continuing to work on a fix for this issue.
Posted Jun 26, 2020 - 01:10 UTC
During a major upgrade of our infrastructure, the new infrastructure was not performing like excepted. Our monitoring systems informed us immediately and we currently take action to switch all traffic back to the older cluster.
Posted Jun 25, 2020 - 23:32 UTC
The issue has been identified and a fix is currently being implemented.
Posted Jun 25, 2020 - 23:20 UTC
We are currently experiencing a service disruption.
Some users may be experiencing trouble when visiting our pages.
Our infrastructure team is working to identify the root cause and implement a solution.
We will send an additional update as soon as we found the root cause.
Posted Jun 25, 2020 - 23:10 UTC
This incident affected: WFM, Forecast, Me, Learn, Account, Support, Login, Analyze (Dashboards), Integrations (On-Premise Integrations, Cloud Integrations), Plan (Meetings), and Intraday (Adherence).