Iwfm page not loading correctly
Incident Report for injixo
Postmortem

WFM not accessible through Microsoft Internet Explorer

Status: Resolved

Summary: WFM could not be used by customers with Internet Explorer

Impact: All customers using Internet Explorer were affected

Root Causes: The landing page of WFM was accidentally served with a generic mime type. The mime type defines the type of data sent. Browsers use this information to load the data. Microsoft Internet Explorer does not try to detect the content automatically. This produced a downloadable file for users visiting our website instead of a rendered page. Our monitoring systems did not detect this behavior because all requests seemed to be successful on our side. **

Detection: Customer support ticket was created in our community component

Resolution: We updated the mime type we send during requests and patched all of our systems

Lessons Learned: We need to improve our testing for all common browsers

Action Items: We will improve our testing regarding Internet Explorer. In this case the issue was neither detected through automatic testing nor through manual efforts. We have an automatic test suite that has about 20.000 test cases which runs on every change of the affected software component. With this system we try to guarantee that our software is always delivered with best quality.

We take all outages seriously, but we are particularly concerned with outages which affect all customers at once. This incident report is both longer and more detailed than usual precisely because we consider this event so important. We want you to understand why it happened and what we are doing about it. It is our hope that, by being transparent and providing considerable detail, we demonstrate our ongoing commitment to offering you the best product.

Timeline

  • 05:16 UTC - First support issue was created describing the problem
  • 05:40 UTC - Support employee escalated this to third level support
  • 05:50 UTC - Status page was updated
  • 06:49 UTC - First patch in place which was not able to solve the issue
  • 07:21 UTC - First customer is patched successfully
  • 08:31 UTC - All customers are patched successfully
Posted Jul 11, 2019 - 12:04 UTC

Resolved
This incident has been resolved.

We apologize for any inconvenience caused.
Posted Jul 11, 2019 - 08:36 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jul 11, 2019 - 08:23 UTC
Identified
The issue has been identified and a fix is currently being implemented.
Posted Jul 11, 2019 - 06:52 UTC
Update
We are continuing to investigate the issue.
Posted Jul 11, 2019 - 06:30 UTC
Investigating
We are currently experiencing a service disruption.

Some users may be experiencing trouble when clicking the iwfm link in the navigation.

Our development team is working to identify the root cause and implement a solution.

We will send an additional update as soon as we found the root cause.
Posted Jul 11, 2019 - 05:47 UTC
This incident affected: WFM.