Jump to: navigation, search

Gyrex/Learning Material/Develop Background Processing

< Gyrex‎ | Learning Material
Revision as of 02:38, 1 August 2012 by Gunnar.wagenknecht.org (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Gyrex
Website
Download
Community
Mailing ListForumsIRC
Bugzilla
Open
Help Wanted
Bug Day
Contribute
Browse SourceProject Set File

This tutorial will guide you through the process of implementing background processing using Jobs in Gyrex. Gyrex has job scheduling and execution, which is based on the eclipse Job API. The eclipse standard Jobs are restricted to a single runtime, so a single node in a cluster. In a cluster it might be useful to let another node execute a job as the one who created it. Gyrex makes this feature available by providing new classes IJob, IJobManager and so on. The eclipse given Jobs will still run on every node in the class but they will get the necessary data from the ZooKeeper where every jobs parameters are stored. Every node can create a IJob and puts it in a job-queue maintained by the ZooKeeper. Every job is unique with an ID from the node which created the job and a UUID. Now the job is available for every node in the cluster. Preventing access of a job by two or more nodes at the same time the node taking the job first locks this job in the queue. If this was successful it executes the job and if necessary gives results back to the ZooKeeper where after that every other node can access this job again.

This tutorial shows how to implement the job and how to use the Gyrex features. We will reference the bundles sample.service and sample.jaxrs so use the bundles created with the last tutorials to test your implementation afterwards. The completed bundles with all the codes explained below you find here(zip-file).

Requirements

OSGi Declarative Services using Gyrex require a configured target platform as well as a Java 6 runtime environment.
It is recommend that you have at least the packages from the tutorial Develop JAX-RS Application.

This tutorial was created using eclipse Indigo Service Release 2.

Tutorial

Let's start with the implementation of the job. A new bundle is required where the job will be implemented. To do so create a new plug-in project name it like "sample.job". Don't create an Activator and don't use any templates. As in earlier tutorials you have to import some packages, too. A few packages like the eclipse job and the Gyrex job has to be marked as required. In the MANIFEST.MF you can perform this by adding following lines:
MANIFEST.MF bundle sample.job
Import-Package: sample.service;version="1.0.0",
 org.apache.commons.lang.exception;version="2.4.0",
 org.apache.commons.lang.math;version="2.4.0",
 org.eclipse.gyrex.cloud.environment;version="[1.0.0,2.0.0)",
 org.slf4j;version="[1.6.0,2.0.0)"
Bundle-ActivationPolicy: lazy
Require-Bundle: org.eclipse.gyrex.common;bundle-version="1.0.0",
 org.eclipse.gyrex.jobs;bundle-version="1.0.0",
 org.eclipse.core.jobs;bundle-version="3.5.200"
Export-Package: sample.job

The exported package we will use later to create a job and execute it. In Dependencies section Automated Management of Dependencies packages have to be added, too.

  • org.eclipse.osgi
  • org.eclipse.osgi.service
  • org.apache.commons.lang
greeting-job.xml
In a new folder "OSGI-INF" a component definition has to be created now. Properties are:
  • parent folder: sample.job/OSGI-INF
  • filename: greeting-job.xml
  • name: sample.job.component
  • class: sample.job.SampleJobProviderComponent
  • provided service: org.eclipse.gyrex.jobs.provider.JobProvider
  • referenced service: sample.service.GreetingService, bind="setGreetingService", cardinality="1..1"

After saving this file the line

Service-Component: OSGI-INF/greeting-job.xml

should be added in the MANIFEST.MF.

Now create a new package in folder source. The name will be something like "sample.job". At least 2 classes are needed. One class is the provider of the job. The name "SampleJobProviderComponent" shows that in this the provider and the component you know as 2 single classes from the tutorial Develop OSGi Declarative Services are combined. On the one hand it provides the job instances and on the other hand it delegates to the real implementation of the job. As a provider this component may only exist once. The implementation of the job will be achieved by creating a new instance of ProcessGreetingsJob. As it was said at the beginning this job is content of the org.eclipse.core plug-in. So you can use everything like methods, monitoring and so on from this class. Implementing the run method the function of this job is defined. In this sample it prints some text about the job and calls the processGreetings method from the interface GreetingService (bundle sample.jaxrs). To be able to call this method the job needs to know the GreetingService instance currently executed. In the greeting-job.xml a reference from the interface GreetingService to the method setGreetingService in SampleJobProviderComponent was created. Using now this method the job gets the GreetingService Implementation and is able to call the processGreeting method.
The Gyrex job is now implemented and is available as a DS. As you noticed no IJob or IJobManager was used yet. This wrapper API is used later, when the job is created and queued.

In this sample we will use console commands to create and process jobs. We will create this sample in another bundle. Again create a new plug-in project without Activator and templates. Its name could be something like "sample.console". A few further packages we need and to create Gyrex Jobs some packages marked as required are necessary, too. So add in MANIFEST.MF, register MANIFEST.MF:
MANIFEST.MF bundle sample.console
Bundle-ActivationPolicy: lazy
Import-Package: org.apache.commons.lang;version="2.4.0",
 org.eclipse.osgi.framework.console;version="1.1.0",
 org.kohsuke.args4j;version="2.0.12",
 org.osgi.framework;version="1.3.0",
 org.slf4j;version="1.6.4"
Require-Bundle: sample.job;bundle-version="1.0.0",
 org.eclipse.gyrex.common;bundle-version="1.0.0",
 org.eclipse.gyrex.context;bundle-version="1.0.0",
 org.eclipse.gyrex.jobs;bundle-version="1.0.0"

As Dependencies, Automated MAnagement of Dependencies, add this packages:

  • org.kohsuke.args4j
  • org.eclipse.osgi
  • org.apache.commons.lang
To create a new console command a CommandProvider interface implementation has to be declared. This will be done in a Component Defintion, placed in a new folder OSGI-INF. Creating it following parameters should be used:
console-commands-provider.xml
  • parent folder: sample.console/OSGI-INF
  • filename: console-commands-provider.xml
  • name: sample.console.component
  • class: sample.console.ConsoleCommandProviderComponent
  • provided service: org.eclipse.osgi.framework.console.CommandProvider
  • immedate="false"

Now check whether the line

Service-Component: OSG-INF/console-commands-provider.xml

is in MANIFEST.MF and add if necessary.

Ok, it's time for the implementations again. It was defined that the implementation class is ConsoleCommandProviderComponent, so just create this class in a new package (name like "sample.console"). In this class the commands will be defined with which the job is created. With the method public void _samplecommand() the main command is defined (samplecommand). Feel free to change this command but note the lower case and the underscore at the beginning. getCommandName() delivers the the command as a String to print in the console, so it's appropriate to set the same String as your command. Appending another command will perform the desired function. This other appending commands are defined in the constructor of the class. With registerCommand(command, commandImplementation) you can register the commands. In this sample there are processGreetings and echo. Both commands still need their implementation. That means two more classes are required either implementing the processGreetings or the echo command. There is one important method you need. In the "doExecute()" method you define the jobs task. processGreetings creates the previously implemented job and queues the job in the ZooKeeper, the classes name in this sample is "ProcessGreetingsComponent". In the method "doExecute()" of this class a jobmanager calls the method createJob(ID,ID,params). With this IDs you define the kind of the job you want to create, here it is SampleJobProviderComponent which delegates to the real implementation of the job. echo just echoes a text which can be entered right behind the command.

GreetingWindow-DevelopBackgroundProcessingConsoleCommand.png
With finishing this implementation now your are able to start the server, like described in other tutorials. For this sample it is useful to mount the application to an URL and enter some greetings. Now switch back to the eclipse console. If you only enter the command you defined in ConsoleCommandProvider, in this sample "samplecommand", you will response an error because a further command name is missing. Possible commands are shown. This sample provides "echo" and "processGreeting". If you enter
samplecommand echo Hello

and confirm the input "Hello" will be shown in the console. Typing in

samplecommand processGreeting

will do a little bit more. This command creates the processGreetings job and queues it in the Gyrex job queue (stored in ZooKeeper), but doesn't execute the job right now. This will be shown as a output like this:

Processing of greetings has been queue as job 'process.greetings.job_run_6d223c98-6669-4305-a2c4-7698b253e7e0'.
a few excerpts from console output
A little bit later new outputs show you that one node in your cluster, if you just run the application on your node it will be your own node, takes the job from the queue and locks this job in the Gyrex job queue. Then you will find somewhere in this text the processed greetings. That means in this sample that the greetings are shown again with the submitted node. After processing the job his lock is released. In the processGreetings implementation the greeting messages are appended with the node information of the node which ran the job, so when you again view all the greetings on the web page, you'll see that the processing messages which have been added.

That's how to create jobs and queue them in Gyrex using console commands. With this it's now possible to create jobs on one node but execute them on every other node registered in the cluster. Furthermore accessing an interface from another bundle has been shown.