EMT-Mode - API-Gateway-Traffic-Monitor don't show traffic when Pod is recreated

[cwiechmann]

Axway APIM-ELK version
3.1.0

API-Manager Version
2021-7.7-March

Expected behavior
If the API management solution is running in EMT mode, then the traffic monitor must also display traffic for PoDs that no longer exist because they have either been recreated and deactivated.

Actual behavior
Currently, the traffic monitor only shows traffic from currently running pods.
This is apparently due to the fact that the search query contains the ID of the API gateway and this changes with every PoD restart.

Test case sample
The problem is the service-ID part of the search query: processInfo.serviceId":"traffic-7dff459748-kdlxf

{"bool":{"must":[{"exists":{"field":"http"}},{"term":{"processInfo.serviceId":"traffic-7dff459748-kdlxf"}},{"range":{"@timestamp":{"gt":1623858508454}}}]}}

[cwiechmann]

As I thought, how to identify which Pod of a Deployment is the “first”, as there is no such thing like a “first” in a Deployment, I had a better idea to solve this problem:

• Every API-Builder container will talk to the ANM to obtain the current topology and cache it for 5 minutes
  • It is an Array and the order of services are fixed

• For a Search-Request on the Traffic-Monitor for each active Service-ID a request to the API-Builder REST-API is fired
  • The API-Builder will compare the received Service-ID with the cached topology
    • If the received Service-ID is the FIRST entry (and part of the Traffic-Gateways (e.g. traffic-*)), it will take care about the no longer existing services using with a wildcard search, but ignoring still active services (basically an include plus exclude)
  • If the Service-ID is not the first in the topology
    • It takes care about this service-ID only as it works today.

Of course, also with that option, there is a max. 5 minutes time window, where already gone services might not be included in the result-set, as the API-Builder may “think” this Service-ID is still active and handled by another API-Builder. The good point is, there is no such notion anymore like First API-Builder, which is much better.