WebClient Request and Response body logging

Question

I am trying to make a POJO out of request and response data received when making WebClient calls. But I am not getting the request body in string/JSON readable form instead I am getting a BodyInsertor. I am making use of Exchange Filters.

public ExchangeFilterFunction logWebRequest() {
    return (request, next) -> {
      log.info("Entered in logWebRequest for WebClient");
      long startTime = System.currentTimeMillis();
      Mono<ClientResponse> response = next.exchange(request);
      long processingTimeInMs = System.currentTimeMillis() - startTime;

      // request.body() -> Gives Body Insertor

      WebRequestLog webRequestLog = webRequestService.makeWebRequestLog(request, response.block());
      webRequestLog.setProcessingTimeInMs(processingTimeInMs);

      log.info("WebRequest to be produced to kafka topic: " + webRequestLog);
      kafkaService.produceAuditLog(webRequestLog);
      return response;
    };
  }

I followed some articles such as https://andrew-flower.com/blog/webclient-body-logging and https://www.gitmemory.com/issue/spring-projects/spring-framework/24262/570245788 but nothing worked for me.

My end goal is to capture requests and responses with their bodies and produce the data collected for Kafka.

Answer 1

To add to Vicky Ajmera answer best way to get and log a request is with ExchangeFilterFunction.

private ExchangeFilterFunction logRequest() {
        return (clientRequest, next) -> {
            logger.info("Request: {} {} {}", clientRequest.method(), clientRequest.url(), clientRequest.body());
            clientRequest.headers()
                    .forEach((name, values) -> values.forEach(value -> logger.info("{}={}", name, value)));
            return next.exchange(clientRequest);
        }; 
}

But to log a response body you will have to go to the lower level of ClientHttpResponse which then allows you to intercept the body. First extend ClientHttpResponseDecorator like this:

public class LoggingClientHttpResponse extends ClientHttpResponseDecorator {

    private static final Logger logger = LoggerFactory.getLogger(LoggingClientHttpResponse.class);
    private static final DataBufferFactory bufferFactory = new DefaultDataBufferFactory();
    private final DataBuffer buffer = bufferFactory.allocateBuffer();


    public LoggingClientHttpResponse(ClientHttpResponse delegate) {
        super(delegate);
    }

    @Override
    public Flux<DataBuffer> getBody() {
        return super.getBody()
                .doOnNext(this.buffer::write)
                .doOnComplete(() -> logger.info("Response Body: {}", buffer.toString(StandardCharsets.UTF_8)));
    }
}

Then create your implementation of ClientHttpConnector like this:

public class LoggingClientHttpConnector implements ClientHttpConnector {
    private final ClientHttpConnector delegate;

    public LoggingClientHttpConnector(ClientHttpConnector delegate) {
        this.delegate = delegate;
    }

    @Override
    public Mono<ClientHttpResponse> connect(HttpMethod method, URI uri, Function<? super ClientHttpRequest, Mono<Void>> requestCallback) {
        return this.delegate.connect(method, uri, requestCallback).map(LoggingClientHttpResponse::new);
    }
}

And last when building your WebClient add a connector:

HttpClient httpClient = HttpClient.create();
ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);

WebClient.builder()
         .baseUrl("http://localhost:8080")
         .clientConnector(new LoggingClientHttpConnectorDecorator(connector))
         .filter(logRequest())
         .build();

Answer 2

Inside ExchangeFilterFunction, you can access HTTP method, URL, headers, cookies but request or response body can not be accessed directly from this filter.

Refer to the answer here. It provides a way to get access to the request and response body. It also provides a link to This blog post. It explains how to get the body in JSON/String format in Web Client.

Answer 3

You can do tracing of request and response payloads with small manipulations with request and responses:

public class TracingExchangeFilterFunction implements ExchangeFilterFunction {
 
 
        return next.exchange(buildTraceableRequest(request))
                .flatMap(response ->
                        response.body(BodyExtractors.toDataBuffers())
                                .next()
                                .doOnNext(dataBuffer -> traceResponse(response, dataBuffer))
                                .thenReturn(response)) ;
    }

    private ClientRequest buildTraceableRequest( 
            final ClientRequest clientRequest) {
        return ClientRequest.from(clientRequest).body(
                new BodyInserter<>() {
                    @Override
                    public Mono<Void> insert(
                            final ClientHttpRequest outputMessage,
                            final Context context) {
                        return clientRequest.body().insert(
                                new ClientHttpRequestDecorator(outputMessage) {
                                    @Override
                                    public Mono<Void> writeWith(final Publisher<? extends DataBuffer> body) {
                                        return super.writeWith(
                                                from(body).doOnNext(buffer ->
                                                        traceRequest(clientRequest, buffer)));
                                    }
                                }, context);
                    }
                }).build();
    }

    private void traceRequest(ClientRequest clientRequest, DataBuffer buffer) {
        final ByteBuf byteBuf = NettyDataBufferFactory.toByteBuf(buffer);
        final byte[] bytes = ByteBufUtil.getBytes(byteBuf);
        // do some tracing e.g. new String(bytes)
    }


    private void traceResponse(ClientResponse response, DataBuffer dataBuffer) {
        final byte[] bytes = new byte[dataBuffer.readableByteCount()];
        dataBuffer.read(bytes);
        // do some tracing e.g. new String(bytes)
    }
}