This was a series of decisions with good intentions that went poorly in the long run.

Our customer wanted us to setup a system so their users could track their products from their site from a variety of carriers; but their backend was very old and difficult to work with, and their network very locked down.

We were struggling to setup a single carrier, so we eventually decided to setup a new server with modern tooling on our own network so we could develop this and other “complicated” features with less pain, and they would only have to make a single exception to their firewall.

Fast forward a year and:

• They didn’t request any more “difficult” features, so the server was serving a single API
• One of our carrier’s API keys had expired and nobody noticed because they weren’t using it, and they didn’t request support for additional carriers either
• Somebody on their security team noticed the strange calls to our servers and demanded we moved the API to their infrastructure anyway