Difference between revisions of "Jetty/Tutorial/Sifting Logs with Logback"

Revision as of 11:29, 30 September 2011

Introduction

This page describes how to create log files at the server level and name them based on an arbitrary context. You do this with Slf4j + Logback + Jetty Webapp Logging in the mix. Find example projects for this feature at github:
https://github.com/jetty-project/jetty-and-logback-example

Modules

/jetty-distro-with-logback-basic/

Configures the jetty distribution with logback enabled at the server level with an example logback configuration.

/jetty-distro-with-logback-sifting/

Configures the jetty distribution with logback, centralized webapp logging, an MDC handler, and a sample logback configuration that performs sifting based  on the incoming Host header on the requests.

/jetty-slf4j-mdc-handler/

Provides the Slf4J MDC key/value pairs that Jetty needs to perform the sample sifting.

/jetty-slf4j-test-webapp/

A sample webapp+servlet that accepts arbitrary values on a form POST and logs them via Slf4J, so that you can see the results of this example.

Details

Configuring Basic Logback for Jetty

Important
See the /jetty-distro-with-logback-basic/ for a Maven project that builds this configuration. Notice that the output directory /jetty-distro-with-logback-basic/target/jetty-distro/ is where Maven builds it.

To configure basic logback for Jetty:

1. Unpack your Jetty 7.x Distribution Zip of choice.
   The example uses the latest stable release
   (7.4.5.v20110725 at the time of writing).
2. Install the slf4j and logback jars into ${jetty.home}/lib/logging/.
   • slf4j-api-1.6.1.jar
   • logback-classic-0.9.29.jar
   • logback-core-0.9.29.jar
3. Configure ${jetty.home}/start.ini to add the lib/logging directory into the server classpath.

===========================================================
 # Start classpath OPTIONS.
 # These control what classes are on the classpath
 # for a full listing do
 #   java -jar start.jar --list-options
 #-----------------------------------------------------------
 OPTIONS=Server,resources,logging,websocket,ext
 #-----------------------------------------------------------
 
 #===========================================================
 # Configuration files.
 # For a full list of available configuration files do
 #   java -jar start.jar --help
 #-----------------------------------------------------------
 etc/jetty.xml
 # etc/jetty-</nowiki>requestlog.xml
 etc/jetty-deploy.xml
 etc/jetty-webapps<nowiki>.xml
 etc/jetty-contexts.xml
 etc/jetty-testrealm.xml
 #===========================================================

4. Create a ${jetty.home}/resources/logback.xml file with the configuration you want.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${jetty.home}/logs/jetty.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>jetty_%d{yyyy-MM-dd}.log</fileNamePattern>

      <maxHistory>30</maxHistory>
    </rollingPolicy>

    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>

  <root level="info">
    <appender-ref ref="STDOUT" />
    <appender-ref ref="FILE" />
  </root>
</configuration>

That’s it, now you have (in the following order)

1. Jetty configured to use slf4j
   (via the existance slf4j-api.jar in the classpath on Jetty startup)
2. slf4j configured to use logback
   (via the existance of logback-core.jar in the classpath at Jetty startup)
3. logback configured to produce output to:
   • ${jetty.home}/logs/jetty.log (with daily rolling)
   • and STDOUT console

Pretty easy huh?

Go ahead and start Jetty.

$ java -jar start.jar

You’ll notice that the log events being produced by Jetty are being handled by Slf4j and Logback is doing the writing of those events to the STDOUT console and logs/jetty.log file

Now lets try something a bit more complex.

Sifting Logs produced by webapps via Hostname using Logback in Jetty

Lets say we have several virtual hosts, or a variety of DNS hostnames for the Jetty instance that is running.And you want to have the logging events being produced by the webapps captured into uniquely named log files by the hostname that the request came in on.

This too is possible with logback, albeit with a little help from slf4j and jettty WebappContextClassloader configuration.

See the /jetty-distro-with-logback-sifting/ project example from the github project above for a build-able configuration of the following instructions:

1. Unpack your Jetty 7.x Distribution Zip of choice.
   The example uses the latest stable release.
   (7.4.5.v20110725 at the time of writing this)
2. Install the slf4j and logback jars into ${jetty.home}/lib/logging/
   • slf4j-api-1.6.1.jar
   • logback-classic-0.9.29.jar
   • logback-core-0.9.29.jar
   • jetty-webapp-logging.jar (be sure you match your jetty version here)
   • jetty-slf4j-mdc-handler.jar (found in the examples project)
3. Configure ${jetty.home}/start.ini to add the lib/logging directory into the server classpath #=========================================================== # Start classpath OPTIONS. # These control what classes are on the classpath # for a full listing do # java -jar start.jar --list-options #----------------------------------------------------------- OPTIONS=Server,resources,logging,websocket,ext #----------------------------------------------------------- #=========================================================== # Configuration files. # For a full list of available configuration files do # java -jar start.jar --help #----------------------------------------------------------- etc/jetty.xml # etc/jetty-requestlog.xml

etc/jetty-mdc-handler.xml
etc/jetty-deploy.xml
etc/jetty-webapps.xml
etc/jetty-contexts.xml
etc/jetty-webapp-logging.xml
etc/jetty-testrealm.xml
 #===========================================================The key entries here are the addition of the “logging” OPTION to load the classes in ${jetty.home}/lib/logging into the jetty server classpath, and the 2 new configuration files:

1. etc/jetty-mdc-handler.xml

   This adds wraps the MDCHandler found in jetty-slf4j-mdc-handler around all of the handlers in Jetty Server.

   etc/jetty-webapp-logging.xml

   This adds a DeploymentManager lifecycle handler that configures the created Webapp’s Classloaders to deny      acccess to any webapp (war) file contained logger implementations in favor of using the ones that exist      on the server classpath.      This is a concept known as Centralized Webapp Logging.

2. Create a ${jetty.home}/resources/logback.xml file with the configuration you want. <?xml version="1.0" encoding="UTF-8"?>

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
    <discriminator>
      <key>host</key>
      <defaultValue>unknown</defaultValue>
    </discriminator>
    <sift>
      <appender name="FILE-${host}" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${jetty.home}/logs/jetty-${host}.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
          <fileNamePattern>jetty-${host}_%d{yyyy-MM-dd}.log</fileNamePattern>

          <maxHistory>30</maxHistory>
        </rollingPolicy>

        <encoder>
          <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
        </encoder>
      </appender>
    </sift>
  </appender>

  <root level="INFO">
    <appender-ref ref="STDOUT" />
    <appender-ref ref="SIFT" />
  </root>
</configuration>

That’s it, now you have (in the following order):

1. Jetty configured to use slf4j
   (via the existence slf4j-api.jar in the classpath on Jetty startup)
2. Jetty is configured to modify incoming Webapp’s classloaders to favor server logging classes   over the webapp’s own logging classes.
   (a.k.a. Centralized Webapp Logging)
3. slf4j configured to use logback
   (via the existence of logback-core.jar in the classpath at Jetty startup)
4. logback configured to produce output to:
   • ${jetty.home}/logs/jetty-${host}.log (with daily rolling)  and using “unknown” for log events that don’t originate from a request.
   • and STDOUT console

Not too bad huh?

Go ahead and start Jetty.

$ java -jar start.jar

]

If you have started the distribution produced by the example configuration, you can use the provided /slf4j-tests/ context to experiment with this.

Go ahead and use the default URL of http://localhost:8080/slf4j-tests/

]

Now try a few more URLs that are for the same Jetty instance.

• http://127.0.0.1:8080/slf4j-tests/
• http://127.0.1.1:8080/slf4j-tests/
• http://your development machine:8080/slf4j-tests/
• http://your development machine.local:8080/slf4j-tests/

You should now have a few different log files in your ${jetty.home}/logs/ directory.

]