Use Istio traffic mirroring for quicker debugging

Often when an error occurs, especially in production, one needs to debug the application to create a fix. Unfortunately the input that created the issue is gone. And the test data on file does not trigger the error (otherwise it would have been fixed before delivery).

Likewise if one is creating new code, one often wants to see what values a client can supply (and to be honest I have used more than once WireShark to see what is being sent). Istio’s traffic mirroring feature can help, as it allows an application to receive real traffic, which is processed by the main version. The same request is copied and then sent to the Mirror service.

The difference is though, that the reply from the mirror service is just dropped (by the Envoy proxy in the sidecar) and not returned to the caller. You do not need to modify the mirror-version—Istio does this all for you. This DestinationRule defines two subsets.

Prod is for the production traffic and the other, development for our debugging target. With that we can now use the following VirtualService to define the mirroring:

Source: itnext.io