Jump to: navigation, search

Difference between revisions of "Jetty/Tutorial/Sifting Logs with Logback"

m
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
{{Jetty Tutorial
 
{{Jetty Tutorial
 
| introduction =  
 
| introduction =  
 +
 +
{{Jetty Redirect|http://www.eclipse.org/jetty/documentation/current/configuring-logging.html}}
  
 
This page describes how to create log files at the server level and name them based on an arbitrary context. You do this with SLF4J + Logback + Jetty Webapp Logging in the mix. Find example projects for this feature at github:<br />https://github.com/jetty-project/jetty-and-logback-example
 
This page describes how to create log files at the server level and name them based on an arbitrary context. You do this with SLF4J + Logback + Jetty Webapp Logging in the mix. Find example projects for this feature at github:<br />https://github.com/jetty-project/jetty-and-logback-example
Line 18: Line 20:
 
===Configuring Basic Logback for Jetty===
 
===Configuring Basic Logback for Jetty===
  
{{tip|Important|See the /jetty-distro-with-logback-basic/ for a Maven project that builds this configuration. Notice that the output directory /jetty-distro-with-logback-basic/target/jetty-distro/ is where Maven builds it.}}
+
{{tip|Important|See the <code>/jetty-distro-with-logback-basic/</code> for a Maven project that builds this configuration. Notice that the output directory<code> /jetty-distro-with-logback-basic/target/jetty-distro/</code> is where Maven builds it.}}
  
 
To configure basic logback for Jetty:
 
To configure basic logback for Jetty:
Line 45: Line 47:
 
#-----------------------------------------------------------
 
#-----------------------------------------------------------
 
etc/jetty.xml
 
etc/jetty.xml
# etc/jetty-</nowiki>requestlog.xml
+
# etc/jetty-requestlog.xml
 
etc/jetty-deploy.xml
 
etc/jetty-deploy.xml
etc/jetty-webapps<nowiki>.xml
+
etc/jetty-webapps.xml
 
etc/jetty-contexts.xml
 
etc/jetty-contexts.xml
 
etc/jetty-testrealm.xml
 
etc/jetty-testrealm.xml
Line 102: Line 104:
 
#* and STDOUT console.
 
#* and STDOUT console.
  
<ol start="5"> <li> Start Jetty.
+
Now start Jetty.
  
 
<source lang="java">$ java -jar start.jar
 
<source lang="java">$ java -jar start.jar
Line 109: Line 111:
  
 
Notice that SLF4J handles the log events Jetty  produces, while Logback  writes those events to the STDOUT console and <code>logs/jetty.log</code> file.
 
Notice that SLF4J handles the log events Jetty  produces, while Logback  writes those events to the STDOUT console and <code>logs/jetty.log</code> file.
 
  
 
=== Using Logback to Sift Logs via Hostname===
 
=== Using Logback to Sift Logs via Hostname===
Line 128: Line 129:
 
#* [http://search.maven.org/#browse%7C-749446705 logback-core-0.9.29.jar]
 
#* [http://search.maven.org/#browse%7C-749446705 logback-core-0.9.29.jar]
 
#* [http://search.maven.org/#browse%7C1693692108 jetty-webapp-logging.jar] (be sure you match your jetty version)
 
#* [http://search.maven.org/#browse%7C1693692108 jetty-webapp-logging.jar] (be sure you match your jetty version)
#* [https://github.com/jetty-project/jetty-and-logback-example/tree/master/jetty-slf4j-mdc-handler jetty-slf4j-mdc-handler.jar] (found in the Examples project)
+
#* [https://github.com/jetty-project/jetty-and-logback-example/tree/master/jetty-slf4j-mdc-handler jetty-slf4j-mdc-handler.jar] -- from Github example project. Steps to compile to JAR
 +
#*# Clone or [https://github.com/jetty-project/jetty-and-logback-example/archive/master.zip download as ZIP] and unpack somewhere
 +
#*# Go to <tt>jetty-slf4j-mdc-handler/</tt> subfolder
 +
#*# Run <tt>mvn package</tt> (needs [http://maven.apache.org Maven], indeed)
 +
#*# Get compiled JAR <tt>jetty-slf4j-mdc-handler-1.0-SNAPSHOT</tt> from <tt>jetty-slf4j-mdc-handler/target/</tt>.
 
# Configure [https://github.com/jetty-project/jetty-and-logback-example/blob/master/jetty-distro-with-logback-sifting/src/main/config/start.ini ${jetty.home}/start.ini] to add the <code>lib/logging</code> directory into the server classpath.   
 
# Configure [https://github.com/jetty-project/jetty-and-logback-example/blob/master/jetty-distro-with-logback-sifting/src/main/config/start.ini ${jetty.home}/start.ini] to add the <code>lib/logging</code> directory into the server classpath.   
  
<<source lang="xml">
+
<source lang="XML">
 
#===========================================================
 
#===========================================================
 
# Start classpath OPTIONS.
 
# Start classpath OPTIONS.
Line 138: Line 143:
 
#  java -jar start.jar --list-options
 
#  java -jar start.jar --list-options
 
#-----------------------------------------------------------
 
#-----------------------------------------------------------
OPTIONS=Server,resources,logging,</nowiki>websocket<nowiki>,ext
+
OPTIONS=Server,resources,logging,websocket,ext
 
#-----------------------------------------------------------
 
#-----------------------------------------------------------
 
   
 
   
Line 147: Line 152:
 
#-----------------------------------------------------------
 
#-----------------------------------------------------------
 
etc/jetty.xml
 
etc/jetty.xml
# etc/jetty-</nowiki>requestlog.xml
+
# etc/jetty-requestlog.xml
 
etc/jetty-mdc-handler.xml
 
etc/jetty-mdc-handler.xml
 
etc/jetty-deploy.xml
 
etc/jetty-deploy.xml
Line 153: Line 158:
 
etc/jetty-contexts.xml
 
etc/jetty-contexts.xml
 
etc/jetty-webapp-logging.xml
 
etc/jetty-webapp-logging.xml
etc/jetty-testrealm<nowiki>.xml
+
etc/jetty-testrealm.xml
 
#===========================================================
 
#===========================================================
 
</source>
 
</source>
  
The key entries here are the addition of the <code>logging</code>OPTION to load the classes in <code>${jetty.home}/lib/logging</code> into the jetty server classpath, and the two new configuration files:
+
The key entries here are the addition of the <code>logging</code> OPTION to load the classes in <code>${jetty.home}/lib/logging</code> into the jetty server classpath, and the two new configuration files:
  
#; [https://github.com/jetty-project/jetty-and-logback-example/blob/master/jetty-distro-with-logback-sifting/src/main/config/etc/jetty-mdc-handler.xml etc/jetty-mdc-handler.xml]
+
; [https://github.com/jetty-project/jetty-and-logback-example/blob/master/jetty-distro-with-logback-sifting/src/main/config/etc/jetty-mdc-handler.xml etc/jetty-mdc-handler.xml]
#: Adds wraps the MDCHandler found in jetty-slf4j-mdc-handler around all of the handlers in Jetty Server.
+
: Adds wraps the MDCHandler found in jetty-slf4j-mdc-handler around all of the handlers in Jetty Server.
#; [http://git.codehaus.org/gitweb.cgi?p=jetty-project.git;a=blob;f=jetty-webapp-logging/src/main/config/etc/jetty-webapp-logging.xml;h=f69d9b540986b7c24a58de8c366844c717c0df20;hb=HEAD etc/jetty-webapp-logging.xml]
+
; [http://git.codehaus.org/gitweb.cgi?p=jetty-project.git;a=blob;f=jetty-webapp-logging/src/main/config/etc/jetty-webapp-logging.xml;h=f69d9b540986b7c24a58de8c366844c717c0df20;hb=HEAD etc/jetty-webapp-logging.xml]
#: Adds a DeploymentManager lifecycle handler that configures the created Webapp’s Classloaders to deny acccess to any webapp (WAR) file contained logger implementations in favor of using the ones that exist on the server classpath. This is a concept known as Centralized Webapp Logging.
+
: Adds a DeploymentManager lifecycle handler that configures the created Webapp’s Classloaders to deny acccess to any webapp (WAR) file contained logger implementations in favor of using the ones that exist on the server classpath. This is a concept known as Centralized Webapp Logging.
  
<ol start="4"> <li> Create a [https://github.com/jetty-project/jetty-and-logback-example/blob/master/jetty-distro-with-logback-sifting/src/main/config/resources/logback.xml ${jetty.home}/resources/logback.xml] file with the configuration you want.
+
<ol start="4"> <li> Create a [https://github.com/jetty-project/jetty-and-logback-example/blob/master/jetty-distro-with-logback-sifting/src/main/config/resources/logback.xml ${jetty.home}/resources/logback.xml] file with the configuration you want. </ol>
  
<source lang "xml">
+
<source lang ="XML">
 
<?xml version="1.0" encoding="UTF-8"?>
 
<?xml version="1.0" encoding="UTF-8"?>
 
<!--
 
<!--
Line 225: Line 230:
 
#* and STDOUT console.
 
#* and STDOUT console.
  
<ol start="5"> <li> Start Jetty.
+
Now start Jetty.
  
source lang = "java"
+
<source lang = "java">
  
 
$ java -jar start.jar
 
$ java -jar start.jar
  
 
</source>
 
</source>
 
+
<br/>
[[Image:example-start-jetty.png]]]
+
<br/>
 
+
[[Image:example-start-jetty.png]]
If you have started the distribution the example configuration produces, you can use the provided /slf4j-tests/ context to experiment with this.
+
<br/>
 
+
<br/>
<ol start="6."> <li>Use the default URL of [http://localhost:8080/slf4j-tests/ http://localhost<nowiki>:8080/slf4j-tests/</nowiki>].
+
<ol start="6."> <li> If you have started the distribution the example configuration produces, you can use the provided /slf4j-tests/ context to experiment with this.
 
+
Use the default URL of [http://localhost:8080/slf4j-tests/ http://localhost<nowiki>:8080/slf4j-tests/</nowiki>].</ol>
[[Image:example-form.png]]]
+
<br/>
 
+
<br/>
<ol start="7"> <li>Now try a few more URLs for the same Jetty instance.
+
[[Image:example-form.png]]
 +
<br/>
 +
<br/>
 +
<ol start="7"> <li> Now try a few more URLs for the same Jetty instance.</ol>
 +
<br/>
  
 
* http://127.0.0.1:8080/slf4j-tests/
 
* http://127.0.0.1:8080/slf4j-tests/
Line 249: Line 258:
  
 
You should now have a few different log files in your <code>${jetty.home}/logs/</code> directory.
 
You should now have a few different log files in your <code>${jetty.home}/logs/</code> directory.
 +
<br/>
 +
<br/>
 +
[[Image:example-logs-sifted.png]]
  
[[Image:example-logs-sifted.png]]]
 
 
}}
 
}}

Latest revision as of 15:57, 23 April 2013



Introduction


This page describes how to create log files at the server level and name them based on an arbitrary context. You do this with SLF4J + Logback + Jetty Webapp Logging in the mix. Find example projects for this feature at github:
https://github.com/jetty-project/jetty-and-logback-example

Modules

/jetty-distro-with-logback-basic/
Configures the jetty distribution with logback enabled at the server level with an example logback configuration.
/jetty-distro-with-logback-sifting/
Configures the jetty distribution with logback, centralized webapp logging, an MDC handler, and a sample logback configuration that performs sifting based  on the incoming Host header on the requests.
/jetty-slf4j-mdc-handler/
Provides the SLF4J MDC key/value pairs that Jetty needs to perform the sample sifting.
/jetty-slf4j-test-webapp/
A sample webapp+servlet that accepts arbitrary values on a form POST and logs them via SLF4J, so that you can see the results of this example.

Details

Configuring Basic Logback for Jetty

Idea.png
Important
See the /jetty-distro-with-logback-basic/ for a Maven project that builds this configuration. Notice that the output directory /jetty-distro-with-logback-basic/target/jetty-distro/ is where Maven builds it.


To configure basic logback for Jetty:

  1. Unpack your Jetty 7.x or 8.x distribution zip of choice.
    This example uses 7.4.5.v20110725.
  2. Install the slf4j and logback jars into ${jetty.home}/lib/logging/.
  3. Configure ${jetty.home}/start.ini to add the lib/logging directory into the server classpath.
===========================================================
# Start classpath OPTIONS.
# These control what classes are on the classpath
# for a full listing do
#   java -jar start.jar --list-options
#-----------------------------------------------------------
OPTIONS=Server,resources,logging,websocket,ext
#-----------------------------------------------------------
 
#===========================================================
# Configuration files.
# For a full list of available configuration files do
#   java -jar start.jar --help
#-----------------------------------------------------------
etc/jetty.xml
# etc/jetty-requestlog.xml
etc/jetty-deploy.xml
etc/jetty-webapps.xml
etc/jetty-contexts.xml
etc/jetty-testrealm.xml
#===========================================================
  1. Create a ${jetty.home}/resources/logback.xml file with the configuration you want.
<?xml version="1.0" encoding="UTF-8"?>
<!--
  Example LOGBACK Configuration File
 
http://logback.qos.ch/manual/configuration.html
 
  -->
<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
   <!-- encoders are assigned the type
         ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
 
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${jetty.home}/logs/jetty.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <!-- daily rollover -->
      <fileNamePattern>jetty_%d{yyyy-MM-dd}.log</fileNamePattern>
 
      <!-- keep 30 days' worth of history -->
      <maxHistory>30</maxHistory>
    </rollingPolicy>
 
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>
 
  <root level="info">
    <appender-ref ref="STDOUT" />
    <appender-ref ref="FILE" />
  </root>
</configuration>

That’s it, now you have (in the following order):

  1. Jetty configured to use SLF4J
    (via slf4j-api.jar in the classpath on Jetty startup).
  2. SLF4J configured to use Logback
    (via logback-core.jar in the classpath at Jetty startup).
  3. Logback configured to produce output to:
    • ${jetty.home}/logs/jetty.log (with daily rolling).
    • and STDOUT console.

Now start Jetty.

$ java -jar start.jar

Notice that SLF4J handles the log events Jetty produces, while Logback writes those events to the STDOUT console and logs/jetty.log file.

Using Logback to Sift Logs via Hostname

Here is a more complex example:

  • You have several virtual hosts, or a variety of DNS hostnames for the Jetty instance that is running.
  • You want Jetty to capture the logging events your webapps produce into uniquely named log files according to the hostname that the request came in on.

This is possible with Logback, SLF4J and jetty WebappContextClassloader.

See the /jetty-distro-with-logback-sifting/ project example from the github project listed above for a buildable configuration of the following:

  1. Unpack your Jetty 7.x or 8.x distribution zip of choice. This example uses 7.4.5.v20110725.
  2. Install the slf4j and logback jars into ${jetty.home}/lib/logging/.
  3. Configure ${jetty.home}/start.ini to add the lib/logging directory into the server classpath.
#===========================================================
# Start classpath OPTIONS.
# These control what classes are on the classpath
# for a full listing do
#   java -jar start.jar --list-options
#-----------------------------------------------------------
OPTIONS=Server,resources,logging,websocket,ext
#-----------------------------------------------------------
 
#===========================================================
# Configuration files.
# For a full list of available configuration files do
#   java -jar start.jar --help
#-----------------------------------------------------------
etc/jetty.xml
# etc/jetty-requestlog.xml
etc/jetty-mdc-handler.xml
etc/jetty-deploy.xml
etc/jetty-webapps.xml
etc/jetty-contexts.xml
etc/jetty-webapp-logging.xml
etc/jetty-testrealm.xml
#===========================================================

The key entries here are the addition of the logging OPTION to load the classes in ${jetty.home}/lib/logging into the jetty server classpath, and the two new configuration files:

etc/jetty-mdc-handler.xml
Adds wraps the MDCHandler found in jetty-slf4j-mdc-handler around all of the handlers in Jetty Server.
etc/jetty-webapp-logging.xml
Adds a DeploymentManager lifecycle handler that configures the created Webapp’s Classloaders to deny acccess to any webapp (WAR) file contained logger implementations in favor of using the ones that exist on the server classpath. This is a concept known as Centralized Webapp Logging.
  1. Create a ${jetty.home}/resources/logback.xml file with the configuration you want.
<?xml version="1.0" encoding="UTF-8"?>
<!--
    Example LOGBACK Configuration File
 
http://logback.qos.ch/manual/configuration.html
 
   -->
<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <!-- encoders are assigned the type
         ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
 
  <appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
    <!-- in the absence of the class attribute, it is assumed that the
         desired discriminator type is
         ch.qos.logback.classic.sift.MDCBasedDiscriminator -->
    <discriminator>
      <key>host</key>
      <defaultValue>unknown</defaultValue>
    </discriminator>
    <sift>
      <appender name="FILE-${host}" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${jetty.home}/logs/jetty-${host}.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
          <!-- daily rollover -->
          <fileNamePattern>jetty-${host}_%d{yyyy-MM-dd}.log</fileNamePattern>
 
          <!-- keep 30 days' worth of history -->
          <maxHistory>30</maxHistory>
         </rollingPolicy>
 
        <encoder>
          <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
        </encoder>
      </appender>
    </sift>
  </appender>
 
  <root level="INFO">
    <appender-ref ref="STDOUT" />
    <appender-ref ref="SIFT" />
  </root>
</configuration>

That’s it, now you have (in the following order):

  1. Jetty configured to use SLF4J
    (via slf4j-api.jar in the classpath on Jetty startup).
  2. Jetty is configured to modify incoming Webapp’s classloaders to favor server logging classes over the webapp’s own logging classes.
    (Centralized Webapp Logging).
  3. SLF4J configured to use Logback
    (via logback-core.jar in the classpath at Jetty startup).
  4. Logback configured to produce output to:
    • ${jetty.home}/logs/jetty-${host}.log (with daily rolling)  and using “unknown” for log events that don’t originate from a request.
    • and STDOUT console.

Now start Jetty.

$ java -jar start.jar



Example-start-jetty.png

  1. If you have started the distribution the example configuration produces, you can use the provided /slf4j-tests/ context to experiment with this. Use the default URL of http://localhost:8080/slf4j-tests/.



Example-form.png

  1. Now try a few more URLs for the same Jetty instance.


You should now have a few different log files in your ${jetty.home}/logs/ directory.

Example-logs-sifted.png