Configuring Logback with Spring Boot

Posted on 2024-03-07 Edited on 2024-04-16 In Java Web , Spring Boot

Logback is a popular logging framework for Java applications, designed as a successor to the well-known Apache Log4j framework. It’s known for its flexibility, performance, and configurability. Logback is extensively used in enterprise-level Java applications for logging events and messages.

In this post, I will cover various aspects of using Logback with Spring Boot.

Dependencies

Before we can use Logback in a Spring Boot application, we need to add its library dependencies to the project.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter</artifactId>
  <version>{LATEST_VERSION}</version>
</dependency>

It contains ch.qos.logback:logback-classic and org.slf4j:slf4j-api

Logback Configuration Files

Spring Boot projects use logback-spring.xml or logback.xml in the resources directory as the Logback configuration file by default.

Priority of the Logback default configuration file: logback.xml > logback-spring.xml.

If you want to use a custom filename. You can specify the log configuration file path in application.xml or application.yml. For example:

logging:
  config: classpath:my-logback.xml

Logback Basic Configurations

Property

You can define some properties that can be referenced in strings. Common properties: log message pattern, log file path, etc.

<configuration>
    <property name="console.log.pattern"
              value="%red(%d{yyyy-MM-dd HH:mm:ss}) %green([%thread]) %highlight(%-5level) %boldMagenta(%logger{36}:%line%n) - %msg%n"/>
    <property name="file.log.pattern" value="%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"/>
    <property name="file.log.dir" value="./logs"/>
    <property name="file.log.filename" value="mylogs"/>
  
</configuration>

Appender

Appenders define the destination and formatting (optional) for log messages.

Types of Logback Appenders:

ConsoleAppender: Writes log messages to the console window (standard output or error).
FileAppender: Writes log messages to a specified file.
RollingFileAppender: Similar to FileAppender, but it creates new log files based on size or time intervals, preventing a single file from growing too large.
SocketAppender: Sends log messages over a network socket to a remote logging server.
SMTPAppender: Sends log messages as email notifications.

ConsoleAppender

<configuration>
  
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${console.log.pattern}</pattern>
            <charset>utf8</charset>
        </encoder>
    </appender>
  
</configuration>

FileAppender

<configuration>
  
    <appender name="FILE" class="ch.qos.logback.core.FileAppender">
        <file>${file.log.dir}/${file.log.filename}.log</file>
        <encoder>
            <pattern>${file.log.pattern}</pattern>
        </encoder>
    </appender>
  
</configuration>

RollingFileAppender

<configuration>
  
    <appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${file.log.dir}/${file.log.filename}.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- Log file will roll over daily -->
            <fileNamePattern>${file.log.pathPattern}</fileNamePattern>
            <!-- Keep 30 days' worth of logs -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <!-- Log messages greater than or equal to the level -->
            <level>INFO</level>
        </filter>
        <encoder>
            <pattern>${file.log.pattern}</pattern>
        </encoder>
    </appender>

</configuration>

RollingPolicy

A rollingPolicy is a component attached to specific appenders that dictates how and when log files are automatically managed, primarily focusing on file size and archiving. Its primary function is to prevent log files from becoming excessively large, improving manageability and performance.

Purpose:

Prevents large log files: By periodically rolling over (rotating) log files, you avoid single files growing too large, which can be cumbersome to manage and slow down access.
Archiving logs: Rolling policies can archive rolled-over log files, allowing you to retain historical logs for analysis or auditing purposes.

Functionality:

Triggers rollover: Based on the defined policy, the rollingPolicy determines when to create a new log file and potentially archive the existing one. Common triggers include exceeding a certain file size or reaching a specific time interval (e.g., daily, weekly).
Defines archive format: The policy can specify how archived log files are named and organized. This helps maintain a clear structure for historical logs.

Benefits of using rollingPolicy:

Manageability: Keeps log files at a manageable size, making them easier to handle and access.
Performance: Prevents performance issues associated with excessively large files.
Archiving: Allows you to retain historical logs for later use.

Common types of rollingPolicy in Logback:

SizeBasedTriggeringPolicy: Rolls over the log file when it reaches a specific size limit (e.g., 10 MB).
TimeBasedRollingPolicy: Rolls over the log file based on a time interval (e.g., daily, weekly, monthly).
SizeAndTimeBasedRollingPolicy: Combines size and time-based triggers, offering more control over rolling behavior.

TimeBasedRollingPolicy

<configuration>
      <appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        ...
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- Log file will roll over daily -->
            <fileNamePattern>${file.log.dir}/${file.log.filename}-%d{yyyy-MM-dd}.log</fileNamePattern>
            <!-- Keep 30 days' worth of logs -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        ...
    </appender>
</configuration>

SizeAndTimeBasedRollingPolicy

<configuration>
      <appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        ...
        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <fileNamePattern>${file.log.dir}/${file.log.filename}-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
            <!-- each archived file's size will be max 10MB -->
            <maxFileSize>10MB</maxFileSize>
            <!-- 30 days to keep -->
            <maxHistory>30</maxHistory>
            <!-- total size of all archive files, if total size > 100GB, it will delete old archived file -->
            <totalSizeCap>100GB</totalSizeCap>
        </rollingPolicy>
        ...
    </appender>
</configuration>

Filter

A filter attached to an appender allows you to control which log events are ultimately written to the defined destination (file, console, etc.) by the appender.

Commonly used filters:

ThresholdFilter: This filter allows log events whose level is greater than or equal to the specified level to pass through. For example, if you set the threshold to INFO, then only log events with level INFO, WARN, ERROR, and FATAL will pass through.
LevelFilter: Similar to ThresholdFilter, but it allows more fine-grained control. You can specify both the level to match and whether to accept or deny log events at that level.

Filter only INFO level log messages.

<configuration>
      <appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        ...
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>INFO</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
        ...
    </appender>
</configuration>

Filter level greater than INFO

<configuration>
      <appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        ...
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
        ...
    </appender>
</configuration>

Logger

A logger in logback.xml represents a category or source for log messages within your application.

There are two types of logger tags in Logback: <root> and <logger>. They have hierarchical relationships. All <logger> are <root> child logger. Loggers can inherit their parent logger’s configurations. <root> represents the top level in the logger hierarchy which receives all package log messages. <logger> receives log messages from a specified package.

<configuration>
  
    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE"/>
    </root>
  
    <logger name="com.taogen" level="DEBUG" additivity="false">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE"/>
    </logger>

</configuration>

<root>: It receive all package log messages.
- level="INFO": define the default logger level to INFO for all loggers.
- <appender-ref>: Send messages to CONSOLE and ROLLING_FILE appender.
<logger>
- name="com.taogen: It receive the com.taogen package log messages.
- level="DEBUG": It overrides the logger level to DEBUG.
- additivity="false": If the message has been sent to a appender by its parent logger, current logger will not send the message to the same appender again.
- <appender-ref>: Send message to CONSOLE and ROLLING_FILE appender.

Using Logback

package com.taogen.commons.boot.mybatisplus;

import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;

@SpringBootTest(classes = AppTest.class)
@ExtendWith(SpringExtension.class)
@Slf4j
class LogTest {
    private static Logger logger = LoggerFactory.getLogger(LogTest.class);
	
    private static Logger customLogger = LoggerFactory.getLogger("my-custom-log");

    @Test
    void test1() {
        log.debug("This is a debug message");
        log.info("This is an info message");
        log.warn("This is a warn message");
        log.error("This is an error message");
        
        logger.debug("This is a debug message");
        
        customLogger.debug("This is a debug message");
        customLogger.info("This is an info message");
        customLogger.warn("This is a warn message");
        customLogger.error("This is an error message");
    }
}

@Slf4j is a Lombok annotation that automatically creates a private static final field named log of type org.slf4j.Logger. This log field is initialized with an instance of the SLF4J logger for the current class.

private static Logger log = LoggerFactory.getLogger(LogTest.class);

The commonly used Logback levels (in order of increasing severity):

TRACE: Captures the most detailed information.
DEBUG: general application events and progress.
INFO: general application events and progress.
WARN: potential problems that might not cause immediate failures.
ERROR: errors that prevent the program from functioning correctly.

Relationships between Logger object and <logger> in logback.xml

<logging> defined in logback.xml usually uses a package path as its name. Otherwise, use a custom name.
If you use Logback to print log messages in Java code, first, you need to pass a class or string to LoggerFactory.getLogger() method to get a Logger object, then call logger’s methods, such as debug().
If the Logger object is obtained through a class, Logback looks for <logger> from logback.xml using the object’s package or parent package path. If the Logger object is obtained through a string, Logback uses the string to find a custom <logger> from logback.xml.

More Configurations

Custom Loggers

You can create a custom logger by setting a name instead of using a package path as its name.

<configuration>
  
      <logger name="my-custom-log" level="DEBUG" additivity="false">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE_CUSTOM"/>
    </logger>
  
</configuration>

Output log messages:

2024-03-07 09:20:43 [main] INFO  my-custom-log - Hello!
2024-03-07 09:21:57 [main] INFO  my-custom-log - Hello!

my-custom-log: logger name.

Note that if you use a custom logger, you can’t get class information from the log message.

Configurations for Different Environments

Using `<springProfile>`

<configuration>
    ...
 .  <!-- springProfile: 1) name="dev | test". 2) name="!prod" -->
    <springProfile name="dev | test">
        <root level="INFO">
            <appender-ref ref="CONSOLE"/>
            <appender-ref ref="ROLLING_FILE"/>
        </root>
        <logger name="com.taogen.commons" level="DEBUG" additivity="false">
            <appender-ref ref="CONSOLE"/>
            <appender-ref ref="ROLLING_FILE"/>
        </logger>
    </springProfile>
    <springProfile name="prod">
        <root level="INFO">
            <appender-ref ref="CONSOLE"/>
            <appender-ref ref="ROLLING_FILE"/>
        </root>
    </springProfile>
</configuration>

Dynamically set the log configuration file path

You can dynamically set the log configuration file path in application.yml. Different spring boot environments use different log configuration files.

application.yml

logging:
  config: classpath:logback-${spring.profiles.active}.xml

logback-dev.xml

<configuration>
    ...
    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE"/>
    </root>
    <logger name="com.taogen.commons" level="DEBUG" additivity="false">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE"/>
    </logger>
</configuration>

logback-prod.xml

<configuration>
    ...
    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE"/>
    </root>
</configuration>

A Complete Example

Goals

Properties
- log patterns, log directory and log filename.
Appenders
- Colorful log pattern for console appender.
- Console and RollingFile appenders.
- Time based rolling policy. Roll over daily, Keep 30 days’ worth of logs.
- Filter log messages in appenders. Separate INFO, ERROR log messages.
Loggers
- Setting loggers of the root and the base package of project.
- Using custom loggers.
- Support multiple spring boot environments.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>
<configuration>
    <!-- Define properties. You can use these properties in appender configurations.-->
    <property name="console.log.pattern"
              value="%red(%d{yyyy-MM-dd HH:mm:ss}) %green([%thread]) %highlight(%-5level) %boldMagenta(%logger{36}:%line%n) - %msg%n"/>
    <property name="file.log.pattern" value="%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"/>
    <property name="file.log.dir" value="./logs"/>
    <property name="file.log.filename" value="mylogs"/>

    <!-- Define the CONSOLE appender. 1) log pattern. 2) log file path. -->
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${console.log.pattern}</pattern>
            <charset>utf8</charset>
        </encoder>
    </appender>

    <!-- RollingFileAppender: Adds the capability to perform log file rotation. You can define a rolling policy, specifying criteria such as time-based or size-based rollover. -->
    <appender name="ROLLING_FILE_INFO" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${file.log.dir}/${file.log.filename}-info.log</file>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- Log file will roll over daily -->
            <fileNamePattern>${file.log.dir}/${file.log.filename}-info-%d{yyyy-MM-dd}.log</fileNamePattern>
            <!-- Keep 30 days' worth of logs -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>${file.log.pattern}</pattern>
        </encoder>
    </appender>
  
    <appender name="ROLLING_FILE_ERROR" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${file.log.dir}/${file.log.filename}-error.log</file>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>ERROR</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- Log file will roll over daily -->
            <fileNamePattern>${file.log.dir}/${file.log.filename}-error-%d{yyyy-MM-dd}.log</fileNamePattern>
            <!-- Keep 30 days' worth of logs -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>${file.log.pattern}</pattern>
        </encoder>
    </appender>
  
    <appender name="ROLLING_FILE_CUSTOM" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${file.log.dir}/custom-logs.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- Log file will roll over daily -->
            <fileNamePattern>${file.log.dir}/custom-logs-%d{yyyy-MM-dd}.log</fileNamePattern>
            <!-- Keep 30 days' worth of logs -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>${file.log.pattern}</pattern>
        </encoder>
    </appender>

    <!-- Define root logger. 1) Set the default level for all loggers. 2) Set which appenders to use. -->
    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE_INFO"/>
        <appender-ref ref="ROLLING_FILE_ERROR"/>
    </root>
    <!-- custom logger -->
    <logger name="my-custom-log" level="INFO" additivity="false">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ROLLING_FILE_CUSTOM"/>
    </logger>
    <!-- springProfile: 1) name="dev | test". 2) name="!prod" -->
    <springProfile name="dev | test">
        <!-- Define loggers. Set log level for specific packages. -->
        <logger name="com.taogen.commons" level="DEBUG" additivity="false">
            <appender-ref ref="CONSOLE"/>
            <appender-ref ref="ROLLING_FILE_INFO"/>
            <appender-ref ref="ROLLING_FILE_ERROR"/>
        </logger>
    </springProfile>
</configuration>