Skip to main content

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Gyrex/Administrator Guide

Gyrex
Website
Download
Community
Mailing ListForumsIRCmattermost
Issues
OpenHelp WantedBug Day
Contribute
Browse SourceProject Set File
Warning2.png
Draft Content
This page is currently under construction. Community members are encouraged to maintain the page, and make sure the information is accurate.


A guide to deployment and administration of Gyrex based servers.


Getting Started

Overview

Gyrex is a Java base application. For proper operation it requires a Java Virtual Machine installed on the system (at least Java 6). It comes with a native launcher that is capable for searching for installed Java VMs.

Quick Start

  1. Download Gyrex, unzip and run.
  2. Open Admin Console (by default running on http://localhost:3110/) and perform initial configuration


Concepts & Features

Modularity

Gyrex is based on OSGi/Equinox and completely modular. It consists of components implementing key features which can be mixed at runtime, i.e. components can be installed and activated individually. In addition, components can run in multiple versions.

Provisioning

Gyrex uses p2 for all provisioning operations. Whenever something needs to installed into the system a p2 operation must be performed. Typically, applications (or just individual application components) are published by their maintainers into a repository. An administrator can then cherry-pick at runtime the desired applications from a maintainers repository and install/update them into a server.

Cloud

With Gyrex it's possible to build your own private cloud platform. Gyrex integrates with ZooKeeper in order to allow easy discovery and management of nodes. You'll be able to appoint and retire nodes in your cloud quickly. With the help of tags it's possible to distribute applications and tasks among them.

Web

The scalable and performant Jetty web engine is also included with Gyrex. It allows for seamless development and deployment of pure OSGi based, modular and extensible web application. You can start with a single web application using the OSGi HttpService. Additionally, Gyrex provides an OSGi service for deploying and managing many application on different HTTP channels (secure, non-secure, different ports, etc.) on various nodes in your cloud. It's also possible to run multiple instances of the same application for multiple tenants including full separation of HTTP sessions, database connections, etc.

Operation Mode

Gyrex servers run using an operation mode. This allows to operate a server either in production or development mode. Typically, a fresh server installation initially runs in development mode and must be configured to not run in development mode. Depending on the selected mode, different default settings are applied. For example, a non-development system will user more strict defaults regarding security and may not expose as much information to end users as a development system. In contrast, a development system may run with more convenient defaults which does not require developers to perform a time-consuming system setup procedure.

For more information on roles, please have a look at the development guide section on roles.

Please be aware that in production mode no default server roles will be applied. For example, the Jetty web server will not start by default. Instead, nodes that should server web content should be tagged with the webserver tag. See Preparing for Production Deployment in the guide for details.

System Settings

A lot customization/configuration in Gyrex can be applied using system settings. System settings are usually set using a Java VM system property. Following the principles of the twelve factor apps, a process should allow to be configured using environment variables. Therefore, Gyrex uses the concept of System settings, which ensures that each system property can also be set using an environment variable.

If not specified otherwise, the environment variable name is the same as the system property but using underscores instead of dots.

Configuration

Logging

Gyrex provides a logging infrastructure based on SLF4J and Logback. This also includes support for capturing Apache Commons Logging and LOG4J log events. The default configuration logs all messages to the console in development mode. In product mode an rotating error log is created in the instance area logs location. However, it's possible to supply your own custom Logback configuration file in order to customize logging.

If you are new to Logback please have a look at the Logback manual.

Using Logback System Property

The first option is using the system property logback.configurationFile. This will instruct Logback to read the configuration from the specified file.

Example:

gyrex [...] -vmargs [...] -Dlogback.configurationFile=/path/to/config.xml

Note, this can also be set via gyrex.ini or configuration/config.ini.

Using Workspace Log Configuration

Another option is placing a configuration file into your runtime workspace location (aka. OSGi instance directory, Eclipse workspace). The configuration file must be <runtime-workspace>/etc/logback.xml and will be read at Gyrex start to configure Logback. The file is only read at start, though. However, once initialized you can use Logback capabilities to have Logback monitor the file for updates/changes to the configuration at runtime.

The benefit of using this method is that a substitution property gyrex.instance.area.logs will be provided which expands to the full path (includinding trailing slash) of the workspace logs directory. This allows to create configuration files without hard coded path names which can be reused across multiple workspaces.

Example:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

  <!-- console logger -->
  <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <!-- application log file -->
  <appender name="applog" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <!-- deny all events with a level below INFO, that is TRACE and DEBUG -->
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>INFO</level>
    </filter>
    <file>${gyrex.instance.area.logs:-logs/}application.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <!-- daily rollover -->
      <fileNamePattern>${gyrex.instance.area.logs:-logs/}application.%d{yyyy-MM-dd}.log</fileNamePattern>
      <!-- keep 30 days' worth of history -->
      <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <!-- query log file -->
  <appender name="querylog" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${gyrex.instance.area.logs:-logs/}queries.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <!-- daily rollover -->
      <fileNamePattern>${gyrex.instance.area.logs:-logs/}queries.%d{yyyy-MM-dd}.log</fileNamePattern>
      <!-- keep 30 days' worth of history -->
      <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <!-- enable logging for interested packages -->
  <logger name="org.eclipse.gyrex.examples.bugsearch" level="DEBUG">
    <!-- log to console -->
    <appender-ref ref="console" />
  </logger>

  <!-- configure query logging -->
  <logger name="bugsearch.querylog" level="INFO" additivity="false">
    <appender-ref ref="querylog" />
  </logger>

  <!-- configure default logging (children can override) -->
  <root level="WARN">
    <appender-ref ref="applog" />
  </root>

</configuration>

Cloud

The Gyrex cloud functionality is implemented based on ZooKeeper. When running in development mode, an embedded ZooKeeper server is started automatically so you don't have to configure anything at all. For production environments we highly recommend running a standalone ZooKeeper ensemble. Please have a look at the ZooKeeper Administrator's Guide for details on how to setup your own ensemble.

Connect to ZooKeeper

The first step is to connect each node with the ZooKeeper cluster. This can be done using the OSGi console or the Gyrex Admin UI.

Connect a node to ZooKeeper:

osgi> cloud setConnectString <zookeeper-connect-string>
...
osgi>

The zookeeper-connect-string can be any valid ZooKeeper connection string. In its simplest it's just a hostname or IP address.

To provide an zookeeper-connect-string without using the osgi console or the Gyrex Admin UI, add the following line to the configuration/config.ini

...
gyrex.zookeeper.connectString=<zookeeper-connect-string>

Node Management

An essential part of a cloud is node membership. Node membership in Gyrex is implemented by explicit approval of nodes. Each node has a unique identifier. Upon first start a new one is generated. It's also possible to manually set node ids. However, you need to ensure that node ids are unique. No two nodes with the same identifier are allows to join the cloud.

After a new node joins a cloud it will be registered with a pending state. In this state the node is not allowed to participate in any cloud activity other than node registration. An administrator need to explicitly approve the node first. Once the node has been approved it will become online.

Tags may be assigned to nodes. They will play an important role when distributing functionality and responsibilities within your cloud.

Console Commands

Two commands are available in the Equinox OSGi console for managing and analyzing the cloud. They both come with a small syntax help

osgi> help
[...]
---CloudConsoleCommands---
	cloud <cmd> [args]
		approve <NODEID> - approves a node
		ls pending|approved|online - list nodes
		retire <NODEID> - retires a node
		setConnectString <CONNECTSTRING> - sets the connect string for the local node
		setOffline - sets the current node offline
		setOnline - sets the current node on-line
		setTags <nodeId> [-t <tag> [-t <tag>]] - set tags for a node
---ZooKeeperConsoleCommands---
	zk <cmd> [args]
		create <PATH> - creates a path in ZK
		get <PATH> - prints the content of the node at the specified path as a string
		ls <PATH> [-r] - lists the specified path
		reconnect - reconnects the ZooKeeper gate
		rm <PATH> [-r] - removes the specified path
		set <PATH> - sets the content of the node at the specified path as a string
		stat <PATH> - prints ZK stats of the specified path
		sync <PATH> - performs ZK sync on the specified path
[...]
osgi>

List pending (unapproved) nodes:

osgi> cloud ls pending
[nodeId] (hostname)
...
osgi>

Approve a node:

osgi> cloud approve <nodeId>

Jetty

Jetty is an integral part of the Gyrex stack. It's responsible for serving OSGi HTTP applications. A key feature of Jetty administration in Gyrex is it's deep integration with the cloud. You no longer configure nodes individually. Instead the administration is cloud global. Filters are available to target a set of nodes (for example a specific region or a group of dedicated web nodes) or even an individual node.

Connectors

In Jetty connectors are responsible for accepting HTTP connections. Several connector types are available. The administration capabilities in Gyrex expose two types of Jetty connectors- one for unencrypted traffic and one for SSL encrypted traffic. Under the covers, Jetty's non-blocking IO connectors will be used. You can configure as many connectors as necessary on various different ports.

Certificates

Certificates are required by SSL connectors. Each certificate will be stored in its own encrypted key-store protected by passwords. It's possible to import an SSL certificate including the fill certificate chain and the private key from JKS or PKCS12 containers.

Using the Console

In the OSGi console a jetty command is available which allows to perform a basic administration of Jetty.

osgi> help
[...]
---Jetty Commands---
  jetty <cmd> [args]
    addConnector <connectorId> <port> [<secure> <certificateId>] - adds a connector
    importCertificate <certificateId> <keystorePath> <keystoreType> [<keystorePassword> [<keyPassword>]] - imports a certificate
    ls  connectors|certificates [filterString] 	 - list all channels
    removeCertificate <certificateId>	 - removes a certificate
    removeConnector <connectorId>	 - removes a connector
[...]
osgi>

Create a HTTP connector on port 8080:
This will create a non-secure connector which accepts connections on port 8080.

osgi> jetty addConnector http 8080
Connector http has been added!

osgi>

Import a SSL certificate from a PKCS12 file:
Jetty requires the private key and the signed certificate in a single container. Gyrex provides a convenient command for importing a PKCS12 file (as generated by OpenSSL or Windows tools) or JKS file (Java standard) which usually contains both. We recommend including the complete certificate chain in case some intermediate CAs were involved.

osgi> jetty importCertificate localhost d:\localhost.p12 PKCS12 password
Processing entry: localhost
Loading key for entry: localhost
Loading certificate chain for entry: localhost
Found certificate:
[.lot of keystore details..]
Imported certificate localhost!

osgi> 

It's possible to verify the import using the jetty ls command.

osgi> jetty ls certificates
localhost [localhost, valid till 2014-02-02]

osgi> 


Create a HTTPS connector on port 8443:
This will use the certificate imported above and create a connector which accepts secure connections.

osgi> jetty addConnector default-https 8443 true localhost
Connector default-https has been added!

osgi> 

When restarting the Jetty engine you can monitor the log output in order to varify the connectors are used correctly.

[...] INFO  org.eclipse.jetty.util.log - jetty-7.2.2.v20101205
[...] INFO  org.eclipse.jetty.util.log - Started SelectChannelConnector@0.0.0.0:8080
[...] INFO  org.eclipse.jetty.util.log - Started CertificateSslConnector@0.0.0.0:8443

Changing Ports in Gyrex

While the jetty ports for the provided applications can easily be configured in the Gyrex admin UI, there are some more ports, which are involved in a Gyrex runtime environment

Gyrex Admin UI port

The http port for the gyrex admin ui can be configured with the system/startup property gyrex.admin.http.port Without this setting the default port is 3110

Zookeeper port

The port for the embedded ZooKeeper can be configured with the system/startup property gyrex.zookeeper.port Without this setting the default port is 2181

Configuring a port offset

There are additional ports in a gyrex runtime environment for jmx connection and ssh console connection. In a development environment there is often the need to run several gyrex instances on one machine. To avoid port conflicts, gyrex provides a port offset property, which offsets (all) ports by the given offset The name of this property is gyrex.portOffset The ports effected by this offset are - admin ui port - zookeeper port - jmx port - ssh port - the default http port of jetty

Please note: the offset is only applied on a given port, if it still uses the default setting, e.g. if gyrex.zookeeper.port is set, then the offset property has no additional effect on the zookeeper port

Administration

Web Console

A web based administration console is available in Gyrex. By default it is started using a separate, embedded Jetty engine on an internal port. This allows for maximum flexibility and is independent of the main Jetty web server in Gyrex.

To open the web console please open a browser and enter the URL http://localhost:3110/.

Protecting Access to the Web Console

Access to the web console is not protected by default in development mode. The console runs on a different port which should have protected access by firewalls already. In addition it's also possible to secure access to the web console using a pluggable authentication scheme. Out of the box, BASIC authentication with encryption using SSL is possible.

In order to enable secure encryption and basic authentication the following two system properties must be set (either in config.ini or using command line arguments when launching the server.

  -Dgyrex.admin.secure=true
  -Dgyrex.admin.auth=BASIC/username/passwordhash

Please have a look at Secure Passwords for instructions on how to produce the password hash.

OSGi Console

Gyrex integrates the Equinox Console which is based on the Apache GoGo Shell. Countless console commands are provided in order to simplify administration of a Gyrex cloud. The Equinox Console is extensible and Gyrex also includes ARGS4J which allows developing additional commands using annotations.

Terminal

In order to activate the OSGi Console one simply needs to pass the "-console" command line argument when starting Gyrex using the gyrex executable. This is the default in all Gyrex server packages.

SSH

The OSGi Console is also available via SSH. This is the recommended way of connecting to production systems. However, please be careful with the close/exit commands. The SSH console listens on port 3122.

Authorization

Currently only key-based authorization is supported in the Gyrex bundled SSH console. Also, the OSGi SSH console does not support different roles/permissions. Thus, every logged in user can execute all commands. In order to be able to login an authorized_keys file need to be created in the instance data location at <runtime-workspace>/etc/.ssh/authorized_keys. That's a regular authorized_keys file as typically found on Linux systems (1, 2). When connecting to the SSH console using a key any username is sufficient.

Software Installation

Gyrex supports installation of software into a running cluster. The implementation is based in p2 - the standard for software installation of OSGi bundles at Eclipse. Please make yourself familiar with p2 concepts.

In order to properly install software the following steps must be performed:

  1. Create p2 repository with artifacts to install (eg., using Maven Tycho or PDE Build)
  2. Deploy repository to a webserver so that's accessible via an URL.
  3. Add repository information in Gyrex
  4. Create software package in Gyrex containing the artifacts to install.
  5. Roll-out the software package.

In the OSGi console a sw command is available which allows to perform a basic administration and handling of software installations.

osgi> help sw
---SoftwareConsoleCommands---
  sw <cmd> [args]
    addPkg <id> - adds a package
    addRepo <id> <uri> [filter] - adds a repository
    addToPkg <packageId> <installableUnitId> [<installUnitVersion>] - adds an installable unit to a package
    ls repos|packages [filterString] - list repos or packages
    revoke <ID> - revokes a package
    rmFromPkg <packageId> <installableUnitId> [<installUnitVersion>] - removes an installable unit from a package
    rmPkg <id> - removes a package
    rmRepo <id> - removes a repository
    rollout <ID> - marks a package for rollout
    status - prints status information about the sw process
    update - updates the local node
osgi> 

The examples below will use the OSGi commands in order to perform the tasks.

Add Repository Information

In order to install software, you need to produce a p2 repository containing the artifacts to install. Repositories are the source of installation. In Gyrex, everything is installed from a repository. However, Gyrex does not know about any repositories by default. Therefore you need to configure the list of available repositories.

The addRepo and rmRepo commands should be used to add or remove p2 repositories.

The following command will add an Orbit p2 repository.

osgi> sw addRepo orbit-R20130118183705 "http://download.eclipse.org/tools/orbit/downloads/drops/R20130118183705/repository/"
repository added
osgi> 

The following commands will look for available artifacts in all configured repositories

osgi> sw ls artifacts
Loading [...]
Done loading. Searching...
Found 392 artifacts:
[...]
osgi> 

Repository Node Filters

When adding repositories it's possible to specify a node filter. Such a filter can be used to limit the visibility of repositories to certain nodes. This is interesting in a multiple data center scenario when nodes in a specific region should use the repository located within their data center.

Another usage scenario is staging/testing/production repositories. Special "testing" nodes might see a testing repository which may contain newer information.

Create Software Package

Once repositories are available it's possible to define packages which can then be rolled out to nodes. After creating the package, artifacts must be added to it.

The following commands will create package using the id "myapplication" add the bundle 'org.apache.commons.pool' to it.

osgi> sw addPkg myapplication
Package added. You should now add installation artifacts to it.

osgi> sw addToPkg myapplication org.apache.commons.pool
package updated

osgi> 

Package Roll-Out

When the package is configured properly with all available artifacts it can be rolled out. Only packages marked for "rollout" will be installed on nodes. Packages marked as "revoke" will be de-installed.

The following command will roll-out the package "myapplication".

osgi> sw rollout myapplication
package marked for rollout

osgi> sw ls package myapplication
Package myapplication
  Is rolled out.
  Will be installed on all nodes.
  Contains the following artifacts:
     org.apache.commons.pool (no version constraint)
osgi> 

Background Task

Queues

Work can be scheduled among multiple queues in order to support distribution and prioritization. By default, Gyrex support a default queue and a priority queue. If you want to add additional queues, a (set of) worker must be configured at startup in order to process the queues.

The list of queues a worker processes can be set by setting the system property gyrex.jobs.workerEngine.queueIds=gyrex.jobs.queue.priority,gyrex.jobs.queue.default,....

Note, if you define the property, you must include the default and priority queue if you still want the worker to process jobs out of that queue. Multiple workers can process different queues.

Maintenance

Out of the box, the job history is persisted in ZooKeeper. But ZooKeeper is not the right place for long term storage of a growing data set. Therefore, an automatic cleanup job will run once in awhile to remove old data from ZooKeeper.

Automatic triggering of the cleanup job can be disabled by setting the system property gyrex.jobs.cleanup.autotrigger.disabled=true.

Preparing for Production Deployment

When deploying Gyrex to production (or even to a staging, testing or UAT environment), you should consider enabling the production mode. This will prevent Gyrex applying some configuration magic for simplifying application development.

Here is a list of bullet points to consider:

  • Setup ZooKeeper Cluster
    • It's possible to use a shared cluster for staging, tests and UAT environments with ZooKeeper's chroot feature
  • Configure start to use production mode
  • Configure ZooKeeper connection
  • Use tags to active server roles across your cluster
    • webserver for nodes that should server web apis and content
    • worker for nodes that should process background tasks
    • scheduler for nodes that act as scheduler for background work (consider defining at least two for redundancy reasons)

Back to the top