API Metadata Micro-instrumentation

Attach AI-powered API security and insights directly to your microservices

As someone once famously observed, “All problems in computer science can be solved by another level of indirection…except for the problem of too many layers of indirection.” APIs are a prime example, as they are essentially an externally facing abstraction to your applications, and those APIs themselves are often consumed through many layers of indirection. API transactions will navigate from client application through a number of gates such as:
 

• Proxy
• CDN caching
• Cloud-front/Load-balancer
• API Gateway
• Service mesh
   

Eventually, an API request reaches the actual API implementation, which interprets the various inputs and does something. Often this “last” step includes calling other APIs, further contributing to the indirection layers.

To analyze your API traffic and detect anomalies, you need to tap into one or more of those layers.

Micro-tapping

Gateways have long been a common component participating in these API transactions. However, they are not always part of the picture. As Christian Posta wrote earlier this year, API Gateways are going through an identity crisis. The emerging Service Mesh is taking a chunk of responsibility that used to be in the domain of the API Gateway. When used as an entry point to a microservice architecture, the API Gateway might see only a portion of the API traffic: the portion that crosses this layer. But more API traffic is happening behind the scenes, and Zero Trust teaches us to keep an eye on that traffic too.

Purely from a practical perspective, you sometimes don’t need a gateway for the task at hand, but you still want to tap into this API traffic metadata. Choice is good, and when PingIntelligence for APIs released a major version (4.0) this summer, I was eager to start playing with the latest features that came with it. Its API sequence modeling and ability to correlate user identity with API traffic unlocks new insights about how your APIs are consumed. Trying out these capabilities requires a microservice, a synthetic API traffic generator, some attack scripts and a way to feed the API traffic metadata into our AI engine.

Since I’m writing this microservice as part of the project, and the gateway integration is an easy-to-use API known as the sideband API, I can just call this sideband API right from my microservice. This integration pattern gives you the best of both inline and sideband modes; it requires no additional component, yet does not require a metadata collector to be inserted inline of the traffic. For this project, I chose to plug in the sideband API using a spring boot microservice filter. An example of such a microservice filter is available at this public repo.

This simple and lightweight sandbox setup allows me to exercise the sideband API in my own way. It improves my ability to experiment with various API traffic modeling and test the reaction of an AI engine when subjected to different training sets and different simulated attacks on these APIs. 

But does this integration pattern make sense in a “real world” situation? The API Security Enforcer (ASE)—the component in PingIntelligence for APIs responsible for implementing this sideband API—is able to scale on demand. As a result, you could attach as many sideband API clients (as many microservices) as you want. The ability to leverage the sideband API from this layer, as part of a service mesh, for example, is a great asset. And since a single AI engine can receive metadata from multiple ASE deployments, this allows you to get unified insights from multiple microservice deployments, and from multiple API Gateway deployments, all combined into a single data lake. 

Choice Improves API Visibility

Lack of API visibility is a top security concern at many enterprises today. This trend is reflected in the recent Gartner API Security report in which one of the top recommendations is “Discover your APIs before attackers discover them.” APIs are sometimes hidden implementation details of applications and although they may be invisible to your security practices, they are not hidden from attackers who easily reverse-engineer them through your apps. Even when APIs are known, the multiplication of technology stacks means they operate in separate silos, producing inconsistent analytics and decentralized security.

API traffic metadata is an underutilized asset, essential in bridging this visibility gap. This is why PingIntelligence for APIs is focused on providing maximum choice for where to tap this resource from, to optimize API traffic insights, and improve security. More choice means the ability to tap wide—not just one API Gateway technology, but any of the popular ones. More choice also means the ability to tap deep, which is why you can attach PingIntelligence for APIs to a load-balancer, to any API Gateway, inline or all the way at the micro level.

Last week, I discussed the benefits of collecting and analyzing API traffic metadata in a keynote at Nordic API’s 2019 Platform Summit in Stockholm. If you didn’t attend this event and want to deepen your understanding of this topic, please join me at our upcoming webinar The New Gold Rush – Mining Metadata to Secure APIs.