Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Linux Tools Project/LTTng2/User Guide

{{#eclipseproject:technology.linux-distros}}

Linux Tools
Website
Download
Community
Mailing ListForumsIRCmattermost
Issues
OpenHelp WantedBug Day
Contribute
Browse Source

Contents

Overview

LTTng (Linux Trace Toolkit, next generation) is a highly efficient tracing tool for Linux that can be used to track down kernel and application performance issues as well as troubleshoot problems involving multiple concurrent processes and threads. It consists of a set of kernel modules, daemons - to collect the raw tracing data - and a set of tools to control, visualize and analyze the generated data. It also provides support for user space application instrumentation.

For more information about LTTng, refer to the project site

Note: This User Guide covers the integration of the latest LTTng (v2.0) in Eclipse. The legacy version (v0.x) of both the tracer and the LTTng integration are no longer being maintained but are still available for download. The User Guide for the legacy LTTng integration is available here.

About Tracing

Tracing is a troubleshooting technique used to understand the behavior of an instrumented application by collecting information on its execution path. A tracer is the software used for tracing. Tracing can be used to troubleshoot a wide range of bugs that are otherwise extremely challenging. These include, for example, performance problems in complex parallel systems or real-time systems.

Tracing is similar to logging: it consists in recording events that happen in a system at selected execution locations. However, compared to logging, it is generally aimed at developers and it usually records low-level events at a high rate. Tracers can typically generate thousands of events per second. The generated traces can easily contain millions of events and have sizes from many megabytes to tens of gigabytes. Tracers must therefore be optimized to handle a lot of data while having a small impact on the system.

Traces may include events from the operating system kernel (IRQ handler entry/exit, system call entry/exit, scheduling activity, network activity, etc). They can also consists of application events (a.k.a UST - User Space Tracing) or a mix of the two.

For the maximum level of detail, tracing events may be viewed like a log file. However, trace analyzers and viewers are available to derive useful information from the raw data. These programs must be specially designed to handle quickly the enormous amount of data a trace may contain.

LTTng integration

The LTTng plug-in for Eclipse provides an Eclipse integration for the control of the LTTng tracer as well as fetching and visualization of the traces produced. It also provides the foundation for user-defined analysis tools.

The LTTng Eclipse plug-in provides the following views:

  • Project - an extension to the standard Eclipse Project view tailored for tracing projects
  • Control - to control the tracer and configure the tracepoints
  • Events - a versatile view that presents the raw events in tabular format with support for searching, filtering and bookmarking
  • Statistics - a view that that provides simple statistics on event occurrences by type
  • Histogram - a view that displays the event density with respect to time in traces

These views can be extended or tailored for specific trace types (e.g. kernel, HW, user app).

At present, the LTTng Eclipse plug-in for Eclipse supports the following kernel-oriented analysis:

  • Control Flow - to visualize processes state transitions
  • Resources - to visualize system resources state transitions

Although the control and fetching parts are targeted at the LTTng tracer, the underlying framework can also be used to process any trace that complies with the Common Trace Format (CTF). CTF specifies a very efficient and compact binary trace format that is meant to be application-, architecture-, and language-agnostic.

Features

The LTTng Eclipse plug-in has a number of features to allow efficient handling of very large traces (and sets of large traces):

  • Support for arbitrarily large traces (larger than available memory)
  • Support for correlating multiple time-ordered traces
  • Support for zooming down to the nanosecond on any part of a trace or set of traces
  • Views synchronization of currently selected event
  • Efficient searching and filtering of events
  • Support for trace bookmarks

There is also support for the integration of non-LTTng trace types:

  • Built-in CTF parser
  • Dynamic creation of customized parsers (for XML and text traces)

Installation

LTTng tracer

LTTng Eclipse Integration

              - Dependencies (RSE)

Features and Plug-ins organization

CTF Parser

TMF

LTTng

LTTng Kernel

LTTng

Tracing Perspective

The Tracing perspective is part of the Tracing and Monitoring Framework (TMF) andand groups the following views:

The views are synchronized i.e. selecting an event, a timestamp, a time range, etc will update the other views accordingly.

TracingPerspective.png

The perspective can be opened from the Eclipse Open Perspective dialog (Window > Open Perspective... > Other).

ShowTracingPerspective.png

On top to these views, the Tracing and Monitoring Framework (TMF) feature provides a set of generic tracing specific views, such as:

To open one of the above Tracing view, use the Eclipse Show View dialog (Window > Show View > Other.... Then select the relevant view from the Tracing.

ShowTracingViews.png

Additionally, the LTTng feature provides a LTTng Tracer Control. It comes with a dedicated Control View.

Project View

The project view is the standard Eclipse Project Explorer. Tracing projects are well integrated in the Eclipse's Common Navigator Framework. The Project Explorer shows Tracing project with a small "T" decorator in the upper right of the project folder icon.

Creating a Tracing Project

A new Tracing project can be created using the New Tracing Project wizard. To create a new Tracing select File > New > Project... from the main menu bar or alternatively form the context-sensitive menu (click with right mouse button in the Project Explorer.

The first page of project wizard will open.

NewTracingProjectPage1.png

In the list of project categories, expand category Tracing and select Tracing Project and the click on Next >. A second page of the wizard will show. Now enter the a name in the field Project Name, select a location if required and the press on Finish.

NewTracingProjectPage2.png

A new project will appear in the Project Explorer view.

NewProjectExplorer.png

Tracing projects have two sub-folders: Traces which holds the individual traces, and Experiments which holds sets of traces that we want to correlate.

Importing Traces in a Project

The Traces folder holds the set of traces available for experiments. To import a trace to the traces folder, select the Traces folder and click the right mouse button. Then select Import... menu item in the context-sensitive menu.

ProjectImportTraceAction.png

A new display will show for selecting traces to import. By default, it shows the correct destination directory where the traces will be imported to. Now, specify the location of the traces by entering the path directly in the Source Directory or by browsing the file system (click on button browse). Then select the traces to import in the list of files and folders. Optionally, select the Trace Type from the drop-down menu, select or deselect the checkboxes for Overwrite existing trace without warning and Create links into workspace. When all options are configured, click on Finish.

Note, that traces of certain types (e.g. LTTng Kernel) are actually a composite of multiple channel traces grouped under a folder. It is the folder that has to be imported.

ProjectImportTraceDialog.png

Upon successful importing the traces will be stored in the Traces folder. If a trace type was selected in the import dialog, then the corresponding icon will be displayed. Linked traces will have a little arrow as decorator on the right bottom corner.

Note that trace type is an extension point of the Tracing and Monitoring Framework (TMF). Depending on the which features are loaded, the list of trace types can vary.

Selecting a Trace Type

If no trace type was selected a trace type as to be associated to a trace before it can be opened. To select a trace type select the relevant trace and click the right mouse button. In the context-sensitive menu, select Select Trace Type... menu item. A sub-menu will show will all available trace type categories. From the relevant category select the required trace type. The examples, below show how to select the Common Trace Format types LTTng Kernel and Generic CTF trace.

SelectLTTngKernelTraceType.png

SelectGenericCTFTraceType.png

After selecting the trace type, the trace icon will be updated with the corresponding trace type icon.

ExplorerWithAssociatedTraceType.png

Creating a Experiment

An experiment consists in an arbitrary number of aggregated traces for purpose of correlation. In the degenerate case, an experiment can consist of a single trace. The experiment provides a unified, time-ordered stream of the individual trace events.

To create an experiment, select the folder Experiments and click the right mouse button. Then select New....

NewExperimentAction.png

A new display will open for entering the experiment name. Type the name of the experiment in the text field Experiment Name and the click on OK.

NewExperimentDialog.png

Selecting Traces for an Experiment

After creating an experiment, traces need to be added to the experiment. To select traces for an experiment select the newly create experiment and click the right mouse button. Select Select Traces... from the context sensitive menu.

SelectTracesAction.png

A new dialog box will open with a list of available traces. Select the traces to add from the list and then click on Finish.

SelectTracesDialog.png

Now the selected traces will be linked to the experiment and will be shown under the Experiments folder.

ExplorerWithExperiment.png

Alternatively, traces can be added to an experiment using Drag and Drop.

Removing Traces from an Experiment

To remove one or more traces for an experiment select the trace(s) to remove under the Experiment folder and click the right mouse button. Select Remove from the context sensitive menu.

RemoveTracesAction.png

After that the selected trace(s) are removed from the experiment. Note that the traces are still in the Traces folder.

Renaming a Trace or Experiment

Under construction (Bernd)

Copy a Trace

Deleting Trace

Deleting Supplementary Files

Opening a Trace

Opening an Experiment

Drag and Drop

Traces can be also be imported to a project by dragging from another tracing project and dropping to the project's trace folder. The trace will be copied and the trace type will be set.

Any resource can be dragged and dropped from a non-tracing project, and any file or folder can be dragged from an external tool, into a tracing project's trace folder. The resource will be copied or imported as a new trace, however the trace type will be unknown and need to be set manually by the user.

It is also possible to drop a trace, resource, file or folder into an existing experiment. If the item does not already exist as a trace in the project's trace folder, it will first be copied or imported, then the trace will be added to the experiment.

Events View

The Events view shows the basic trace data in chronological order in a tabular format.

When opened, the Events view displays the events of the currently selected trace or experiment.

The header displays the current trace (or experiment) name.

Being part of the Tracing and Monitoring Framework, the default table displays the following fields:

  • Timestamp: the event timestamp
  • Source: the source of the event
  • Type: the event type and localization
  • Reference the event reference
  • Content: the raw event content

The first row of the table is the header row a.k.a. the Search and Filter row.

DefaultEventsView.png

The highlighted event is the current event and is synchronized with the other views. If you select another event, the other views will be synchronized accordingly.

Searching and Filtering

Searching and filtering of events in the table can be performed by entering matching conditions in one or multiple columns in the header row (the first row below the column header).

To toggle between searching and filtering, click on the 'search' (TmfEventSearch.gif) or 'filter' (TmfEventFilter.gif) icon in the header row's left margin, or right-click on the header row and select Show Filter Bar or Show Search Bar in the context menu.

To apply a matching condition to a specific column, click on the column's header row cell, type in a regular expression and press the ENTER key. You can also enter a simple text string and it will be automatically be replaced with a 'contains' regular expression.

When matching conditions are applied to two or more columns, all conditions must be met for the event to match (i.e. 'and' behavior).

To clear all matching conditions in the header row, press the DEL key.

Searching

When a searching condition is applied to the header row, the table will select the next matching event starting from the top currently displayed event. Wrapping will occur if there is no match until the end of the trace.

All matching events will have a 'search match' icon in their left margin. Non-matching events will be dimmed.

DefaultTmfEvents-Search.png

Pressing the ENTER key will search and select the next matching event. Pressing the SHIFT-ENTER key will search and select the previous matching event. Wrapping will occur in both directions.

Press ESC to cancel an ongoing search.

Press DEL to clear the header row and reset all events to normal.

Filtering

When a filtering condition is entered in the head row, the table will clear all events and fill itself with matching events as they are found from the beginning of the trace.

A status row will be displayed before and after the matching events, dynamically showing how many matching events were found and how many events were processed so far. Once the filtering is completed, the status row icon in the left margin will change from a 'stop' to a 'filter' icon.

DefaultTmfEvents-Filter.png

Press ESC to stop an ongoing filtering. In this case the status row icon will remain as a 'stop' icon to indicate that not all events were processed.

Press DEL or right-click on the table and select Clear Filters from the context menu to clear the header row and remove the filtering. All trace events will be now shown in the table. Note that the currently selected event will remain selected even after the filter is removed.

You can also search on the subset of filtered events by toggling the header row to the Search Bar while a filter is applied. Searching and filtering conditions are independent of each other.

Bookmarking

Any event of interest can be tagged with a bookmark.

To add a bookmark, double-click the left margin next to an event, or right-click the margin and select Add bookmark.... Alternatively use the Edit > Add bookmark... menu. Edit the bookmark description as desired and press OK.

The bookmark will be displayed in the left margin, and hovering the mouse over the bookmark icon will display the description in a tooltip.

The bookmark will be added to the Bookmarks view. In this view the bookmark description can be edited, and the bookmark can be deleted. Double-clicking the bookmark or selecting Go to from its context menu will open the trace or experiment and go directly to the event that was bookmarked.

To remove a bookmark, double-click its icon, select Remove Bookmark from the left margin context menu, or select Delete from the Bookmarks view.

Bookmarks.png

Histogram View

The Histogram View displays the trace events distribution with respect to time. When streaming a trace, this view is dynamically updated as the events are received.


HistogramView.png


On the top left, there are two data controls:

  • Current Event (sec): Displays the timestamp of the currently selected event
  • Window Span (sec): Displays the current time range window size

Both control can be used to modify their respective value. After validation, the other controls and views will be synchronized and updated accordingly.


The large histogram, at the bottom, shows the event distribution over the whole trace or set of traces. It also has a smaller semi-transparent window, with a cross-hair, that shows the currently selected time range window. The time range window can be zoomed in/out by using the mouse wheel. It can also be selected by the mouse and dragged to another region of the trace.

The smaller histogram, on top right, corresponds to the currently selected time range window, a sub-range of the event set. Its size can also be zoomed in/out using the mouse wheel.

The x-axis of each histogram corresponds to the events timestamps. The timestamp of the first and the last event of the respective ranges is displayed. The y-axis of each histogram shows the minimum/maximum number of events in the corresponding histogram bars.

The dashed vertical magenta bar, on the right, shows the position of the last event. The dashed vertical red bar shows the relative position of the currently selected event. The current event can be changed by clicking on the histogram.

Hovering the mouse over an histogram bar pops up an information window that displays the start/end time of the corresponding bar as well as the number of events it represents.

In each histogram, the following keys are handled:

  • Left: Moves the current event to the previous non-empty bar
  • Right: Moves the current event to the next non-empty bar
  • Home: Displays the current event to the first histogram bar
  • End: Displays the current event to the last non-empty histogram bar

Statistics View

The Statistics View displays the various event counters that are collected when analyzing a trace. The data is organized per trace. To open the Statistics View, select Windows -> Show View -> Other... -> Tracing -> Statistics. A new view will open with the name Statistics. This view shows 2 columns: Level and Number of Events. After parsing a trace the view will display the number of events per event type. The cells where the number of events are printed also contain a colored bar that indicates the percentage of the event count in relation to the total number of events. The statistics is collected for the whole trace. This view is part of the Tracing and Monitoring Framework (TMF) and is generic. It will work for any trace type extensions. For the LTTng 2.0 integration the Statistics view will display statistics as shown below.:

LTTng2StatisticsView.png

Colors View

Filter View

Time Chart View

Environment Variables View

Custom Parser

LTTng Tracer Control

The LTTng Tracer Control in Eclipse for the LTTng Tracer toolchain version v2.0 (or later) is done using SSH and requires an SSH server to be running on the remote host. For the SSH connection the SSH implementation of RSE is used. For that a new System Type was defined using the corresponding RSE extension. The functions to control the LTTng tracer (e.g. start and stop), either locally or remotely, are available from a dedicated Control View.

In the following sections the LTTng 2.0 tracer control integration in Eclipse is described. Please refer to the LTTng 2.0 tracer control command line manual for more details and descriptions about all commands and their command line parameters References.

Control View

To open the Control View, select 'Window->Show View->Other...->LTTng->Control View.

LTTngControlView.png

Creating a New Connection to a Remote Host

To connect to a remote host, select the New Connection button in the Control View.

LTTngControlViewConnect.png

A new display will show for entering the remote host information. A drop down menu will filled with all existing host information which were used before. To enter the host information either select one of the hosts in the drop down menu or enter the host information manually.

LTTng2NewConnection.png

To use an existing connection definition, select the relevant entry in the drop-down menu and then select Ok.

LTTng2SelectConnection.png

To enter the host information manually select first the button Edit connection information. Then the text fields Connection Name and Host Name will enabled. Enter the relevant information and then select Ok.

LTTng2EditConnection.png

A new display will show for providing the user name and password. This display only opens if no password had been saved before. Enter user name and password in the Enter Password dialog box and select Ok.

The Host Name holds the IP address or DNS name of the remote system. The Connection Name is the alias name to be displayed in the Control View.

LTTng2EnterPassword.png

After pressing Ok the SSH connection will be established and after successful login the Control View implementation retrieves the LTTng Tracer Control information. This information will be displayed in the Control View in form of a tree structure.

LTTng2ControlViewFilled.png

The top level tree node is the representation of the remote connection (host). The connection name of the connection will be displayed. Depending on the connection state different icons are displayed. If the node is CONNECTED the icon is shown Target connected.gif, otherwise (states CONNECTING, DISCONNNECTING or DISCONNECTED the icon is Target disconnected.gif.

Under the host level two folder groups are located. The first one is the Provider group. The second one is the Sessions group.

Under the Provider group all trace providers are displayed. Trace providers are Kernel and any user space application that supports UST tracing. Under each provider a corresponding list of events are displayed.

Under the Sessions group all current sessions will be shown. The level under the sessions show the configured domains. Currently the LTTng 2.0 Tracer Toolchan supports domain Kernel and UST global. Under each domain the configured channels will be displayed. The last level is under the channels where the configured events are displayed.

Each session can be ACTIVE or INACTIVE. Active means that tracing has been started, inactive means that the tracing has been stopped. Depending on the state of a session a different icon is displayed. The icon for an active session is Session active.gif. The icon for an inactive session is Session inactive.gif.

Each channel can be ENABLED or DISABLED. An enabled channel means that all configured events of that channel will be traced and a disabled channel won't trace any of its configured events. Different icons are displayed depending on the state of the channel. The icon for an enabled channel is Channel.gif and the icon for a disabled channel is Channel disabled.gif.

Events within a channel can be in state ENABLED or DISABLED. Enabled events are stored in the trace when passed during program execution. Disabled events on the other hand won't be traced. Depending on the state of the event the icons for the event is different. An enabled event has the icon Event enabled.gif and a disabled event the icon Event disabled.gif.

Disconnecting from a Remote Host

To disconnect from a remote host, select the host in the Control View and press the Disconnect button. Alternatively, press the right mouse button. A context-sensitive menu will show. Select the Disconnect button.

LTTng2ControlViewDisconnect.png

Connecting to a Remote Host

To connect to a remote host, select the host in the Control View and press the Connect button. Alternatively, press the right mouse button. A context-sensitive menu will show. Select the Connect button. This will start the connection process as discribed in Creating a New Connection to a Remote Host.

LTTng2ControlViewConnect.png

Deleting to a Remote Host Connection

To delete a remote host connection, select the host in the Control View and press the Delete button. Alternatively, press the right mouse button. A context-sensitive menu will show. Select the Delete button. For that command to be active the connection state has to be DISCONNECTED.

LTTng2ControlViewDelete.png

Creating a Tracing Session

To create a tracing session, select the tree node Sessions and press the right mouse button. Then select the Create Session... button of the context-sensitive menu.

LTTng2CreateSessionAction.png

A dialog box will open for entering information about the session to be created.

LTTng2CreateSessionDialog.png

Fill in the Session Name and optionally the Session Path and press Ok. Upon successful operation a new session will be created and added under the tree node Sessions.

Enabling Channels - General

Enabling channels can be done using a session tree node when the domain hasn't be created in the session or, alternatively on a domain tree node of a session in case the domain is already available.

Enabling Channels On Session Level

To enable a channel, select the tree node of the relevant session and press the right mouse button. Then select the Enable Channel... button of the context-sensitive menu.

LTTng2CreateChannelAction.png

A dialog box will open for entering information about the channel to be created.

LTTng2CreateChannelDialog.png

By default the domain Kernel is selected and the corresponding default values are shown. To create a UST channel, select UST under the domain section. To get the default values of UST, then press button Default.

If required update the following channel information and then press Ok.

  • Channel Name: The name of the channel.
  • Number of Sub Buffers: The number of sub-buffers of the channel.
  • Overwrite Mode: The channel overwrite mode (true or false)
  • Read Timer Interval: The read timer interval.
  • Sub Buffer size: The size of the sub-buffers of the channel (in bytes).
  • Switch Timer Interval: The switch timer interval.

Upon successful operation, the requested domain will be created under the session tree node as well as the requested channel will be added under the domain. The channel will be ENABLED.

Enabling Channels On Domain Level

Once a domain is available, channels can be enabled directly using the domain. To enable a channel under an existing domain, select the tree node of the relevant domain and press the right mouse button. Then select the Enable Channel... button of the context-sensitive menu.

LTTng2CreateChannelOnDomainAction.png

The dialog box for enabling channel will open for entering information about the channel to be created. Note that the domain is pre-selected and cannot be changed.

LTTng2CreateChannelOnDomainDialog.png

Fill the relevant information and press Ok.

Enabling and Disabling Channels

To disable one or more enabled channels, select the tree nodes of the relevant channels and press the right mouse button. Then select the Disable Channel menu item of the context-sensitive menu.

LTTng2DisableChannelAction.png

Upon successful operation, the selected channels will be DISABLED and the icons for the channels will be updated.

To enable one or more disabled channels, select the tree nodes of the relevant channels and press the right mouse button. Then select the Enable Channel menu item of the context-sensitive menu.

LTTng2EnableChannelAction.png

Upon successful operation, the selected channels will be ENABLED and the icons for the channels will be updated.

Enabling Events - General

Enabling events can be done using different levels in the tree node. It can be done on the session, domain level and channel level. For the case of session or domain, i.e. when no specific channels is assigned then enabling of events is done on the default channel with the name channel0 which created, if not already exists, by the LTTng tracer control on the server side.

Enabling Kernel Events On Session Level

To enable events, select the tree node of the relevant session and press the right mouse button. Then select the Enable Event (default channel)... button of the context-sensitive menu.

LTTng2EventOnSessionAction.png

A dialog box will open for entering information about events to be enabled.

LTTng2EventOnSessionDialog.png

By default the domain Kernel is selected and the kernel specific data sections are created. From this dialog box kernel Tracepoint events, System calls (Syscall), a Dynamic Probe or a Dynamic Function entry/return probe can be enabled. Note that events of one of these types at a time can be enabled.

To enable Tracepoint events, first select the corresponding Select button, then select either all tracepoins (select All) or select selectively one or more tracepoints in the displayed tree of tracepoints and finally press Ok.

LTTng2TracepointEventsDialog.png

Upon successful operation, the domain Kernel will be created in the tree (if neccessary), the default channel with name "channel0" will be added under the domain (if necessary) as well as all requested events of type TRACEPOINT under the channel. The channel and events will be ENABLED.

LTTng2EnabledKernelTracepoints.png

To enable all Syscalls, select the corresponding Select button and press Ok.

LTTng2SyscallsDialog.png

Upon successful operation, the event with the name syscalls and event type SYSCALL will be added under the default channel (channel0). If necessary the domain Kernel and the channel channel0 will be created.

LTTng2EnabledKernelSyscalls.png

To enable a Dynamic Probe event, select the corresponding Select button, fill the Event Name and Probe fields and press Ok. Note that the probe can be an address, symbol or a symbol+offset where the address and offset can be octal (0NNN...), decimal (NNN...) or hexadecimal (0xNNN...).

LTTng2ProbeEventDialog.png

Upon successful operation, the dynamic probe event with the given name and event type PROBE will be added under the default channel (channel0). If necessary the domain Kernel and the channel channel0 will be created.

LTTng2EnabledKernelProbeEvent.png

To enable a Dynamic Function entry/return Probe event, select the corresponding Select button, fill the Event Name and Function fields and press Ok. Note that the funtion probe can be an address, symbol or a symbol+offset where the address and offset can be octal (0NNN...), decimal (NNN...) or hexadecimal (0xNNN...).

LTTng2FunctionEventDialog.png

Upon successful operation, the dynamic function probe event with the given name and event type PROBE will be added under the default channel (channel0). If necessary the domain Kernel and the channel channel0 will be created.

LTTng2EnabledFunctionProbeEvent.png

Enabling UST Events On Session Level

For enabling UST events, first open the enable events dialog as described in section Enabling Kernel Events On Session Level and select domain UST.

To enable Tracepoint events, first select the corresponding Select button, then select either all tracepoins (select All) or select selectively one or more tracepoints in the displayed tree of tracepoints and finally press Ok.

LTTng2UstTracepointEventsDialog.png

Upon successful operation, the domain UST global will be created in the tree (if neccessary), the default channel with name "channel0" will be added under the domain (if necessary) as well as all requested events under the channel. The channel and events will be ENABLED. Note that for the case that All tracepoints were selected the wildcard * is used which will be shown in the Control View as below.

LTTng2EnabledAllUstTracepoints.png

For UST it is possible to enable Tracepoint events using a wildcard. To enable Tracepoint events with a wildcard, select first the corresponding Select button, fill the Wildcard field and press Ok.

LTTng2UstWildcardEventsDialog.png

Upon successful operation, the event with the given wildcard and event type TRACEPOINT will be added under the default channel (channel0). If necessary the domain UST global and the channel channel0 will be created.

LTTng2EnabledUstWildcardEvents.png

For UST it is possible to enable Tracepoint events using log levels. To enable Tracepoint events using log levels, select first the corresponding Select button, select a log level from the drop down menu, fill in the relevant information (see below) and press Ok.

  • Event Name: Name to display
  • loglevel: To specify if a range of log levels (0 to selected log level) shall be configured
  • loglevel-only: To specify that only the specified log level shall be configured

LTTng2UstLoglevelEventsDialog.png

Upon successful operation, the event with the given event name and event type TRACEPOINT will be added under the default channel (channel0). If necessary the domain UST global and the channel channel0 will be created.

LTTng2EnabledUstLoglevelEvents.png

Enabling Events On Domain Level

Kernel events can also be enabled on the domain level. For that select the relevant domain tree node, click the right mouse button and the select Enable Event (default channel).... A new dialog box will open for providing information about the events to be enabled. Depending on the domain, Kernel or UST global, the domain specifc fields are shown and the domain selector is preselected and read-only.

LTTng2EventOnDomainAction.png

To enable events for domain Kernel follow the instructions in section Enabling Kernel Events On Session Level, for domain UST global Enabling UST Events On Session Level.

When enabling events on the domain level, the events will be add to the default channel channel0. This channel will be created by on the server side if neccessary.

Enabling Events On Channel Level

Kernel events can also be enabled on the channel level. If necessary, create a channel as described in sections Creating Channels On Session Level or Creating Channels On Domain Level.

Then select the relevant channel tree node, click the right mouse button and the select Enable Event.... A new dialog box will open for providing information about the events to be enabled. Depending on the domain, Kernel or UST global, the domain specifc fields are shown and the domain selector is preselected and read-only.

LTTng2EventOnChannelAction.png

To enable events for domain Kernel follow the instructions in section Enabling Kernel Events On Session Level, for domain UST global Enabling UST Events On Session Level.

When enabling events on the channel level, the events will be add to the selected channel.

Enabling and Disabling Events

To disable one or more enabled events, select the tree nodes of the relevant events and click the right mouse button. Then select Disable Event menu item in the context-sensitive menu.

LTTng2DisableEventAction.png

Upon successful operation, the selected events will be DISABLED and the icons for these events will be updated.

To enable one or more disabled events, select the tree nodes of the relevant events and press the right mouse button. Then select the Enable Event menu item of the context-sensitive menu.

LTTng2EnableEventAction.png

Upon successful operation, the selected events will be ENABLED and the icons for these events will be updated.

Note: There is currently a limitation for kernel event of type SYSCALL. This kernel event can not be disabled. An error will appear when trying to disable this type of event. A work-around for that is to have the syscall event in a separate channel and disable the channel instead of the event.

Enabling Tracepoint Events From Provider

It is possible to enable events of type Tracepoint directly from the providers and assign the enabled event to a session and channel. Before doing that a session has to be created as described in section Creating a Tracing Session. Also, if other than default channel channel0 is required, create a channel as described in sections Creating Channels On Session Level or Creating Channels On Domain Level.

To assign tracepoint events to a session and channel, select the events to be enabled under the provider (e.g. provider Kernel), click right mouse button and then select Enable Event... menu item from the context sensitive menu.

LTTng2AssignEventAction.png

A new display will open for defining the session and channel.

LTTng2AssignEventDialog.png

Select a session from the Session List drop-down menu, a channel from the Channel List drop-down menu and the press Ok. Upon successful operation, the selected events will be added to the selected session and channel of the domain that the selected provider belongs to. In case that there was no channel available, the domain and the default channel channel0 will be created for corresponding session. The newly added events will be ENABLED.

LTTng2AssignedEvents.png

Adding Contexts to Channels and Events of a Domain

It is possible to add contexts to channels and events. Adding contexts on channels and events from the domain level, will enable the specified contexts to all channels of the domain and all their events. To add contexts on the domain level, select a domain, click right mouse button on a domain tree node (e.g. provider Kernel) and select the menu item Add Context... from the context-sensitive menu.

LTTng2AddContextOnDomainAction.png

A new display will open for selecting one or more contexts to add.

LTTng2AddContextDialog.png

The tree shows all available context that can be added. Select one or more context and the press Ok. Upon successful operation, the selected context will be added to all channels and their events of the selected domain.

Note: The LTTng UST tracer only supports contexts procname, pthread_id, vpid vtid. Adding any other contexts in the UST domina will fail.

Adding Contexts to All Events of a Channel

Adding contexts on channels and events from the channel level, will enable the specified contexts to all events of the selected channel. To add contexts on the channel level, select a channel, click right mouse button on a channel tree node and select the menu item Add Context... from the context-sensitive menu.

LTTng2AddContextOnChannelAction.png

A new display will open for selecting one or more contexts to add. Select one or more contexts as described in chapter Adding Contexts to Channels and Events of a Domain. Upon successful operation, the selected context will be added to all channels and their events of the selected domain. Note that the LTTng 2.0 tracer control on the remote host doesn't provide a way to retrieve added contexts. Hence it's not possible to display the context information in the GUI.

Adding Contexts to a Event of a Specific Channel

Adding contexts to a event of a channel, select an event of a channel, click right mouse button on the corresponding event tree node and select the menu item Add Context... from the context-sensitive menu.

LTTng2AddContextToEventsAction.png

A new display will open for selecting one or more contexts to add. Select one or more contexts as described in chapter Adding Contexts to Channels and Events of a Domain. Upon successful operation, the selected context will be added to the selected event.

Start Tracing

To start tracing, select one or more sessions to start in the Control View and press the Start button. Alternatively, press the right mouse button on the session tree nodes. A context-sensitive menu will show. Then select the Start menu item.

LTTng2StartTracingAction.png

Upon successful operation, the tracing session will be ACTIVE and the icon of the session will be updated.

Stop Tracing

To stop tracing, select one or more sessions to stop in the Control View and press the Stop button. Alternatively, click the right mouse button on the session tree nodes. A context-sensitive menu will show. Then select the Stop menu item.

LTTng2StopTracingAction.png

Upon successful operation, the tracing session will be INACTIVE and the icon of the session will be updated.

Destroying a Tracing Session

To destroy a tracing session, select one or more sessions to destroy in the Control View and press the Destroy button. Alternatively, click the right mouse button on the session tree node. A context-sensitive menu will show. Then select the Destroy... menu item. Note that the session has to be INACTIVE for this operation.

LTTng2DestroySessionAction.png

A confirmation dialog box will open. Click on Ok to destroy the session otherwise click on Cancel.

LTTng2DestroyConfirmationDialog.png

Upon successful operation, the tracing session will be destroyed and removed from the tree.

Refreshing the Node Information

To refresh the remote host information, select any node in the tree of the Control View and press the Refresh button. Alternatively, click the right mouse button on any tree node. A context-sensitive menu will show. Then select the Refresh menu item.

LTTng2RefreshAction.png

Upon successful operation, the tree in the Control View will be refreshed with the remote host configuration.

Quantifing LTTng overhead (Calibrate)

The LTTng calibrate command can be used to find out the combined average overhead of the LTTng tracer and the instrumentation mechanisms used. For now, the only calibration implemented is that of the kernel function instrumentation (kretprobes). To run the calibrate command, select the a domain (e.g. Kernel), click the right mouse button on the domain tree node. A context-sensitive menu will show. Select the Calibrate menu item.

LTTng2CalibrateAction.png

Upon successful operation, the calibrate command is executed and relevant information is stored in the trace. Note: that the trace has to be active so that to command as any effect.

Importing Session Traces to a Tracing Project

To import traces from a tracing session, select the relevant session and click on the Import Button. Alternatively, click the right mouse button on the session tree node and select the menu item Import... from the context-sensitive menu.

LTTng2ImportAction.png

A new display will open for selecting the traces to import.

LTTng2ImportDialog.png

Select the trace to be imported by selecting the relevant traces in the tree viewer, select a tracing project from the Available Projects combo box and select the Overwrite button (Overwrite existing trace without warning) if required. Then press button Ok. Upon successful import operation the the selected traces will be stored in the Traces directory of the specified tracing project. From the Project Explorer view, the trace can be analyzed further.

Note: If the overwrite button (Overwrite existing trace without warning) was not selected and a trace with the same name of a trace to be imported already exists in the project, then a new confirmation dialog box will open.

LTTng2ImportOverwriteConfirmationDialog.png

To Overwrite select the Overwrite Button and press Ok.

If the existing trace should not be overwritten select, then select the Rename option of the confirmation dialog box above, enter a new name and then press Ok.

LTTng2ImportRenameDialog.png

Properties View

The Control View provides property information of selected tree component. Depending on the selected tree component different properties are displayed in the property view. For example, when selecting the node level the property view will be filled as followed:

LTTng2PropertyView.png

List of properties:

  • Host Properties
    • Connection Name: The alias name to be displayed in the Control View.
    • Host Name: The IP address or DNS name of the remote system.
    • State: The state of the connection (CONNECTED, CONNECTING, DISCONNNECTING or DISCONNECTED).
  • Kernel Provider Properties
    • Provider Name: The name of the provider.
  • UST Provider Properties
    • Provider Name: The name of the provider.
    • Process ID: The process ID of the provider.
  • Event Properties (Provider)
    • Event Name: The name of the event.
    • Event Type: The event type (TRACEPOINT only).
    • Log Level: The log level of the event.
  • Session Properties
    • Session Name: The name of the Session.
    • Session Path: The path on the remote host where the traces will be stored.
    • State: The state of the session (ACTIVE or INACTIVE)
  • Domain Properties
    • Domain Name: The name of the domain.
  • Channel Properties
    • Channel Name: The name of the channel.
    • Number of Sub Buffers: The number of sub-buffers of the channel.
    • Output type: The output type for the trace (e.g. splice() or mmap())
    • Overwrite Mode: The channel overwrite mode (true for overwrite mode, false for discard)
    • Read Timer Interval: The read timer interval.
    • State: The channel state (ENABLED or DISABLED)
    • Sub Buffer size: The size of the sub-buffers of the channel (in bytes).
    • Switch Timer Interval: The switch timer interval.
  • Event Properties (Channel)
    • Event Name: The name of the event.
    • Event Type: The event type (TRACEPOINT, SYSCALL or PROBE)..
    • Log Level: The log level of the event.
    • State: The Event state (ENABLED or DISABLED)

LTTng Trace Control Preferences

Serveral LTTng 2.0 tracer control preferences exists which can be configured. To configure these preferences, select Window->Preferences from the top level menu. The preference display will open. Then select Tracing->LTTng Tracer Control Preferences. This preferences page allows the user to specify the tracing group of the user and allows the user to configure the logging of LTTng 2.0 tracer control commands and results to a file.

LTTng2Preferences.png

To change the tracing group of the user which will be specified on each command line, enter the new group name in the Tracing Group text field and click ok. The default tracing group is tracing and can be restored by pressing the Restore Defaults button.

LTTng2PreferencesGroup.png

To configure logging of trace control commands and the corresponding command result to a file, selected the button Logging. To append to an existing log file, select the Append button. Deselect the Append button to overwrite any existing log file. It's possible to specify a verbose level. There are 3 levels with inceasing verbosity from Level 1 to Level 3. To change the verbosity level, select the relevant level or select None. If None is selected only commands and command results are logged. Then press on button Ok. The log file will be stored in the users home directory with the name lttng_tracer_control.log. The name and location cannot be changed. To reset to default preferences, click on the button Restore Defaults.

LTTng2PreferencesLogging.png

LTTng Kernel Analysis

LTTng Kernel Perspective

Control Flow View

Resources View

Events View*

Limitations

References

Back to the top