From 9:23 AM EST to 10:32 AM EST on January 5th, 2022, storefronts powered by Shogun Frontend were unavailable due to a misconfiguration within Shogun’s content delivery network. Affected consumers would have been presented with an unbranded error message during this timeframe.
Shogun’s Engineering team regularly releases new functionality, as well as bug fixes and security updates. Shogun also utilizes a premier content delivery network to ensure the fastest delivery of content to shoppers.
As part of the software update delivered on January 5th, 2022, Shogun’s Engineering team modified the storage location of credentials and other configuration data as part of routine maintenance to the platform. This change was tested in our pre-production environment with no ill effects prior to this release. Upon release to production it was discovered that the automated steps to remove the configuration data from the existing storage location and the automated steps to add the configuration data to the new location were not guaranteed to run in the correct order. When ran out of order the configuration data had not yet been added to the new storage location, but was removed prematurely from the existing storage location causing the storefront to fail to load.
After troubleshooting the issue and evaluating several options Shogun’s Engineering team began manually correcting the configuration to restore service.
All future configuration storage changes will be completed in two phases. The first phase will put the configuration in the new storage location and update the code to read from both locations preferring the new location. The second phase will remove the configuration from the old storage location.
Shogun’s Engineering team is continually exploring additional ways to make our deployments more reliable.