Plan - Shift Center closes unexpectedly and does not (re)start
Incident Report for injixo

On Wendnesday 03.05.2023 around 15:45 CET we noticed that customers were unable to work with Plan/ShiftCenter.
16:41 we managed to rollback our changes.
Unexpectedly, however, this did not solve the problem and further investigations became necessary.
17:15 we involved our platform engineering team to downgrade the machine to a previous version.
The machine rollback resolved the issue instantly.

Any attempt to open a ShiftCenter leads to the error message about application load failed.

Starting 03.05.2023 15:45 CET until 03.05.2023 18:25 CET all customers using Plan/ShiftCenter were unable to use it.

The incident was detected by our development team first.

We responded to customers on 03.05.2023 15:45 CET about a partial outage of Plan/ShiftCenter at and
raise the outage to major outage at 03.05.2023 16:41 CET.

We rolled back all code changes made by our team first. As this did not resolved the outage we rolled back all changes made on the machine.

Root cause
We accidentally deployed an insufficiant tested version with security updates that blocked starting the application.

Corrective actions
We developed peer review approval process to ensure, that accidentally deployments will not happend again.

We apologise for the service disruption.

Posted May 05, 2023 - 14:06 UTC

This incident has been resolved.
Posted May 03, 2023 - 16:25 UTC
A fix has been implemented and we are monitoring the results.
Posted May 03, 2023 - 15:30 UTC
The issue has been identified and a fix is being implemented.
Posted May 03, 2023 - 15:15 UTC
We are continuing to investigate this issue.
Posted May 03, 2023 - 14:41 UTC
We are currently investigating this issue.
Posted May 03, 2023 - 13:48 UTC
This incident affected: Plan (Shift Center).