Log Elasticsearch query to a logfile for spring data elasticsearch

Question

I am trying to have multiple log files for my spring boot app. I have configured logback.xml file. I can append hibernate spring data query in my log file. But when I tried to append elasticsearch query log on my logfile made by spring-data-elasticsearch repository, I don't get any log in the file. I have added this on my logback.xml file:

 <configuration>

    <property name="LOG_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %logger{40}.%M - %msg%n" />


    <property name="APP_LOG_ROOT" value="/mylog/log"/>

    <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${LOG_PATTERN}</pattern>
        </encoder>
    </appender>

    <appender name="applicationLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${APP_LOG_ROOT}/application.log</file>
        <encoder>
            <pattern>${LOG_PATTERN}</pattern>
        </encoder>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${APP_LOG_ROOT}/application.%d{yyyy-MM-dd}.%i.log</fileNamePattern>

            <timeBasedFileNamingAndTriggeringPolicy

                    class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">

                <maxFileSize>5MB</maxFileSize>

            </timeBasedFileNamingAndTriggeringPolicy>

            <maxHistory>30</maxHistory>
        </rollingPolicy>
    </appender>


    <appender name="hibernateLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${APP_LOG_ROOT}/hibernate.log</file>
        <encoder>
            <pattern>${LOG_PATTERN}</pattern>
        </encoder>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${APP_LOG_ROOT}/hibernate.%d{yyyy-MM-dd}.%i.log</fileNamePattern>

            <timeBasedFileNamingAndTriggeringPolicy

                    class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">

                <maxFileSize>5MB</maxFileSize>

            </timeBasedFileNamingAndTriggeringPolicy>

            <maxHistory>30</maxHistory>
        </rollingPolicy>
    </appender>

    <appender name="databaseLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${APP_LOG_ROOT}/database.log</file>
        <encoder>
            <pattern>${LOG_PATTERN}</pattern>
        </encoder>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${APP_LOG_ROOT}/database.%d{yyyy-MM-dd}.%i.log</fileNamePattern>

            <timeBasedFileNamingAndTriggeringPolicy

                    class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">

                <maxFileSize>5MB</maxFileSize>

            </timeBasedFileNamingAndTriggeringPolicy>

            <maxHistory>30</maxHistory>
        </rollingPolicy>
    </appender>


    <!--<logger name="org.hibernate.type.descriptor.sql" level="trace">-->
    <logger name="org.hibernate.SQL" level="trace" additivity="false">
        <appender-ref ref="hibernateLog" />
    </logger>

    <logger name="org.springframework.data.elasticsearch.client.WIRE" level="trace">
        <appender-ref ref="databaseLog" />
        <appender-ref ref="console" />
    </logger>
    <logger name="com.test" level="info">
        <appender-ref ref="applicationLog" />
        <!--<appender-ref ref="console" />-->
    </logger>

    <root level="info">
        <appender-ref ref="console" />
    </root>


</configuration>

I also tried from here

<logger name="org.springframework.data.elasticsearch.core.*" level="debug" additivity="false">
            <appender-ref ref="databaseLog" />
            <appender-ref ref="console" />
        </logger>

But both didn't work. How to solve this? I don't want to set them from properties file as I don't know how to configure logging for multiple files from .properties file
Another Question: If I refer the same log-appender for multiple package multiple times, will it be any problem?

pcsutar · Answer 1 · 2021-06-02T12:12:03.587

2

You can enable transport layer logging to see what is actually sent to and received from the server.

You can enable it using below logger:

<logger name="org.springframework.data.elasticsearch.client.WIRE" level="trace"/>

OR

You can add following property to your application.properties file:

logging.level.org.springframework.data.elasticsearch.client.WIRE = trace

Please refer spring docs

I have tested this solution on my local machine its printing request and response in logs. I am using RestHighLevelClient and Spring boot version- 2.2.7.RELEASE.

Please see my logback.xml file:

<configuration>
    <property name="LOGS" value="./logs" />

    <appender name="databaseLog"
        class="ch.qos.logback.core.FileAppender">
        <file>${LOGS}/spring-data-elasticsearch.log</file>
        <encoder
            class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <Pattern>%d %p %C{1.} [%t] %m%n</Pattern>
        </encoder>
    </appender>

    <logger
        name="org.springframework.data.elasticsearch.client.WIRE"
        level="trace">
        <appender-ref ref="databaseLog" />
    </logger>
</configuration>

edited Jun 02 '21 at 12:12

answered Jun 02 '21 at 11:12

pcsutar

1,715
2
9
14

thanks for your help. I tried this. Yes, it prints log on console but didn't write in my logFile as I have mentioned in the question. I can assure that my logback.xml other configurations are ok as I can write by the same log appendar- `databaseLog` – user404 Jun 02 '21 at 11:40
Hi @user404, I am able to write the logs to file. I have added my logback.xml file in the answer. Please check and let me know if you are still not able to write the logs to file. – pcsutar Jun 02 '21 at 12:03
can you please check my logback.xml file I have added now? – user404 Jun 03 '21 at 05:37
I have tested your logback.xml file on machine its working. I am able to see logs in file :). How did you created a bean of RestHighLevelClient? I have extended AbstractElasticsearchConfiguration class to create a bean of RestHighLevelClient as per my requirement. – pcsutar Jun 03 '21 at 06:01
using a constructor of` public RestHighLevelClient client() {...}` this way. But my es client is ok and it works fine in any operations. I can see the database.log file too. problem is, no elastic query of spring-data-elasticsearch is being log on that file. – user404 Jun 03 '21 at 06:06
I hope you have created a bean of RestHighLevelClient. – pcsutar Jun 03 '21 at 06:32
yes, I created bean for that. but don't know why facing this weird problem – user404 Jun 03 '21 at 06:33

Log Elasticsearch query to a logfile for spring data elasticsearch

1 Answers1