How to enable query logging in Spring-data-elasticsearch

Question

I use spring-data-elasticsearch framework to get query result from elasticsearch server, the java code like this:

SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery()).withSearchType(SearchType.COUNT)
.addAggregation(new MinBuilder("min_createDate").field("createDate"))
.build();

List<Entity> list = template.queryForList(searchQuery, Entity.class);

While how can I know the raw http query sent to elasticssearch server? How can I enable the logging, I tried add log4j, but it seems the spring-data-elasticsearch doesn't log the query.

Answer 1

After digging through the spring data code i found this helpful little logger called "tracer" (name not very unique)

By setting the following in application.properties

logging.level.tracer=TRACE

It will print out a full curl statement for the request along with full JSON the response from Elasticsearch.

Answer 2

This one is quite old, but I'd still like to share the solution that worked for me. To log Spring Data Elasticsearch queries executed through the Repository, you need to enable DEBUG logging for the package org.springframework.data.elasticsearch.core.*, e.g. as follows:

logging:
  level:
    org:
      springframework:
        data:
          elasticsearch:
            core: DEBUG

After that, queries will appear in logs:

{
  "from" : 0,
  "size" : 1,
  "query" : {
    "bool" : {
      "should" : [ {
        "query_string" : {
          "query" : "John Doe",
          "fields" : [ "entityName" ],
          "default_operator" : "and"
        }
      }, {
        "query_string" : {
          "query" : "John Doe",
          "fields" : [ "alias" ],
          "default_operator" : "and"
        }
      } ]
    }
  },
  "post_filter" : {
    "bool" : { }
  }
}

One would expect an elegant solution similar to JPA, but it seems that it doesn't simply exist.

Tested with Spring Boot 1.4.0 and Spring Data Elasticsearch 1.7.3.

Answer 3

This works on Spring Boot 2.3.3.RELEASE

logging.level.org.springframework.data.elasticsearch.client.WIRE=trace

Answer 4

If you are using spring boot you can set the following in your application.properties:

logging.level.org.elasticsearch.index.search.slowlog.query=INFO
spring.data.elasticsearch.properties.index.search.slowlog.threshold.query.info=1ms

Answer 5

I don't have an answer for Spring Data Elasticsearch, but in ES itself you can bump up the default settings for slow query logging and see all the queries in the slow log. More details about slow log here.

As to how to change the thresholds, a command like this should be used:

PUT /_settings
{
  "index.search.slowlog.threshold.query.info": "1ms"
}

1ms is kindof the smallest value you can set.

Answer 6

I encountered the same problem, In the ElasticsearchTemplate only a few method have log debug level, E.g:

public <T> Page<T> queryForPage(CriteriaQuery criteriaQuery, Class<T> clazz) {
    QueryBuilder elasticsearchQuery = new CriteriaQueryProcessor().createQueryFromCriteria(criteriaQuery.getCriteria());
    QueryBuilder elasticsearchFilter = new CriteriaFilterProcessor().createFilterFromCriteria(criteriaQuery.getCriteria());
    SearchRequestBuilder searchRequestBuilder = prepareSearch(criteriaQuery, clazz);

    if (elasticsearchQuery != null) {
        searchRequestBuilder.setQuery(elasticsearchQuery);
    } else {
        searchRequestBuilder.setQuery(QueryBuilders.matchAllQuery());
    }

    if (criteriaQuery.getMinScore() > 0) {
        searchRequestBuilder.setMinScore(criteriaQuery.getMinScore());
    }

    if (elasticsearchFilter != null)
        searchRequestBuilder.setPostFilter(elasticsearchFilter);
    if (logger.isDebugEnabled()) {
        logger.debug("doSearch query:\n" + searchRequestBuilder.toString());
    }

    SearchResponse response = getSearchResponse(searchRequestBuilder
            .execute());
    return resultsMapper.mapResults(response, clazz, criteriaQuery.getPageable());
}

Answer 7

@2280258 is correct, and here comes the official doc:

• https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/index.html#elasticsearch.clients.logging

<logger name="org.springframework.data.elasticsearch.client.WIRE" level="trace"/>

Here is the reason: in org.springframework.data.elasticsearch.client.ClientLogger, spring data elasticsearch creates a logger named "org.springframework.data.elasticsearch.client.WIRE"：

    private static final Logger WIRE_LOGGER = LoggerFactory
            .getLogger("org.springframework.data.elasticsearch.client.WIRE");

Answer 8

Just to add my two cents to @AndreiStefan: Now you can set 0ms instead of 1ms. It seems that some very fast queries can be captured using this method.

Simply do:

PUT /_settings
{
  "index.search.slowlog.threshold.query.info": "0ms"
}