In an attempt to optimise our network traffic routing, we provisioned some config changes to our load balancer. The change caused our Kubernetes routing to become unhealthy and thus blocking traffic coming into our app.herondata.io domain. This was tested in staging environments and worked correctly, which means production-level traffic made it impossible to apply this change properly. We quickly reverted the config to stabilise the API which is now operating normally.