Service Disruption - ID3global / Platform Issue
Incident Report for GBG

When GBG release a significant volume of new Item Checks to the ID3global platform. GBG releases changes in a planned seamless manner - delivering to the Disaster Recovery site first, signing off and updating the Production site after moving live customers to the proven/tested DR site.

Root Cause

The delivery of these new Item Checks require more result codes to be available in requests for getting these results. These additional result codes were loaded into the Database as part of this change.

This meant that the 'size' of the data requested in the UI was significantly increased. Meaning that the volume of the data retrieved exceeded the maximum buffer size configured on the Production platform.

On DR, where this change was delivered first, various configuration settings were changed (as part of this release) to manage and accommodate for this increase size.

However, these changes are local to the platform where the changes were made and did not benefit the Production site.

It was not expected during the presentation of the change that this would impact the live service through our Production site. We expected that this would only impact the speed of transactions.

So, to restore service the Support Teams failed over to DR where the new settings existed. When the Production site is deployed to, the new settings will also be applied there, restoring normal service across all sites.

Posted May 22, 2023 - 12:38 BST

We’re pleased to inform you that the service disruption we reported earlier is now resolved. Thank you for your patience whilst we restored service.

The incident duration was between 10:47-11:07 BST on 22/05/2023
Posted May 22, 2023 - 11:10 BST
We are currently experiencing a service disruption that may be preventing you from using our service. We’re sorry for the impact this may be having on your Business and we’re working hard to get this back up and running.
Posted May 22, 2023 - 11:04 BST
This incident affected: GBG ID3global (ID3global Portal, URU Portal, ID3global Service, URU Service).