Websocket API Service Impaired
Incident Report for BitMEX
Postmortem

From 06:53 to 07:03 UTC today, September 3rd, 2018, the service of our websocket API was impaired, which impacted the live data updates on the bitmex.com website as well as clients connected directly to our websocket API.

As part of a scheduled software release on the trading engine, our market data distribution component became saturated with a cascade of updates during the restart process which temporarily prevented downstream data distribution on the websocket API.

We apologise for the disruption. The root cause has been identified and mitigated in the production environment. BitMEX engineers have already deployed a fix for the underlying issue to prevent a re-occurrence. We are also improving a number of alerting mechanisms to enable us to recover from potential issues in a more timely manner.

Posted Jan 15, 2020 - 01:24 UTC

Resolved
The issue has been resolved. We apologise for any inconvenience this may be causing. If you have any further questions, please contact us via https://www.bitmex.com/app/support/contact.
Posted Sep 03, 2018 - 06:03 UTC
Investigating
We have identified an interruption of the feeds through the websocket. We apologise for any inconvenience this may be causing. We are looking into the cause of this issue, and will be sharing an update as soon as possible. If you have any further questions, please contact us via https://www.bitmex.com/app/support/contact.
Posted Sep 03, 2018 - 05:53 UTC
This incident affected: BitMEX.com (REST API).