Gyrex/Learning Material/Develop Background Processing
|Mailing List • Forums • IRC • mattermost|
|Open • Help Wanted • Bug Day|
|Browse Source • Project Set File|
This tutorial will guide you through the process of implementing background processing using Jobs. Jobs, provided in the org.eclipse.core.runtime plug-in, are units allowing clients to execute code in a separate thread. The problem is that they are restricted to a single node. In a cluster it might be useful to let another node execute a job as the one who created it. Gyrex makes this feature available by providing new classes IJob, IJobManager and so on. The eclipse given Jobs will still run on every node in the class but they will get the necessary data from the ZooKeeper where every jobs parameters are stored. Every node can create a IJob and puts it in a job-queue maintained by the ZooKeeper. With an ID from the node who created the job and a UUI every job is unique. Now the job is available for every node in the cluster. A node who doesn't execute a job at the moment can take and execute this job now. Preventing access of a job by two or more nodes at the same time the node taking the job first locks this job in the queue. If this was successful he executes the job and if necessary gives results back to the ZooKeeper where after that every other node can access this job again. This tutorial shows how to implement the job and how to use the Gyrex features. We will reference the bundles sample.service and sample.jaxrs so use the bundles created with the last tutorials to test your implementation afterwards. The completed bundles with all the codes explained below you find here(zip-file).
OSGi Declarative Services using Gyrex require a configured target platform as well as a Java 6 runtime environment.
It is recommend that you have at least the packages from the tutorial Develop JAX-RS Application.
This tutorial was created using eclipse Indigo Service Release 2.
Let's start with the implementation of the job. A new bundle is required where the job will be implemented. To do so create a new plug-in project name it like "sample.job". Don't create an Activator and don't use any templates. As in earlier tutorials you have to import some packages, too. A few packages like the eclipse job and the Gyrex job has to be marked as required. In the MANIFEST.MF you can perform this by adding following lines:
Import-Package: sample.service;version="1.0.0", org.apache.commons.lang.exception;version="2.4.0", org.apache.commons.lang.math;version="2.4.0", org.eclipse.gyrex.cloud.environment;version="[1.0.0,2.0.0)", org.slf4j;version="[1.6.0,2.0.0)" Bundle-ActivationPolicy: lazy Require-Bundle: org.eclipse.gyrex.common;bundle-version="1.0.0", org.eclipse.gyrex.jobs;bundle-version="1.0.0", org.eclipse.core.jobs;bundle-version="3.5.200" Export-Package: sample.job
The exported package we will use later to create a job and execute him. In Dependencies section Automated Management of Dependencies packages have to be added, too.
In a new folder "OSGI-INF" a component definition has to be created now. Properties are:
- parent folder: sample.job/OSGI-INF
- filename: greeting-job.xml
- name: sample.job.component
- class: sample.job.SampleJobProviderComponent
- provided service: org.eclipse.gyrex.jobs.provider.JobProvider
- referenced service: sample.service.GreetingService, bind="setGreetingService", cardinality="1..1"
After saving this file the line
should be added in the MANIFEST.MF.
Now create a new package in folder source. The name will be something like "sample.job". At least 2 classes are needed. One class is the provider of the job. The name "SampleJobProviderComponent" shows that in this the provider and the component you know as 2 single classes from the tutorial Develop OSGi Declarative Services are combined. On the one hand it provides the job instances and on the other hand it delegates to the real implementation of the job. As a provider this component may only exist once. The implementation of the job will be achieved by creating a new instance of ProcessGreetingsJob. As it was said at the beginning this job is content of the org.eclipse.core plug-in. So you can use everything like methods, monitoring and so on from this class. Implementing the run method the function of this job is defined. In this sample it prints some text about the job and calls the processGreetings method from the interface GreetingService (bundle sample.jaxrs). To be able to call this method the job needs to know the GreetingService instance currently executed. In the greeting-job.xml a reference from the interface GreetingService to the method setGreetingService in SampleJobProviderComponent was created. Using now this method the job gets the GreetingService and is able to call the processGreeting method.
With this the job is implemented. As you noticed no IJob or IJobManager was used yet. The connection to the Zookeper using Gyrex will be implented now.
In this sample we will use console commands to create and process jobs. They have to be defined in another bundle. Again create a new plug-in project without Activator and templates. Its name could be something like "sample.console". A few further packages we need and to create Gyrex Jobs some packages marked as required are necessary, too. So add in MANIFEST.MF, register MANIFEST.MF:
Bundle-ActivationPolicy: lazy Import-Package: org.apache.commons.lang;version="2.4.0", org.eclipse.osgi.framework.console;version="1.1.0", org.kohsuke.args4j;version="2.0.12", org.osgi.framework;version="1.3.0", org.slf4j;version="1.6.4" Require-Bundle: sample.job;bundle-version="1.0.0", org.eclipse.gyrex.common;bundle-version="1.0.0", org.eclipse.gyrex.context;bundle-version="1.0.0", org.eclipse.gyrex.jobs;bundle-version="1.0.0"
As Dependencies, Automated MAnagement of Dependencies, add this packages:
To listen on console command inputs a CommandProvider interface has to be set. This will be done in a Component Defintion, placed in a new folder OSGI-INF. Creating it following parameters should be used:
- parent folder: sample.console/OSGI-INF
- filename: console-commands-provider.xml
- name: sample.console.component
- class: sample.console.ConsoleCommandProviderComponent
- provided service: org.eclipse.osgi.framework.console.CommandProvider
Now check whether the line
is in MANIFEST.MF and add if necessary.
Ok, it's time for the implementions again. It was defined that the implementation class is ConsoleCommandProviderComponent, so just create this class in a new package (name like "sample.console"). In this class the commands will be defined with which the job is created. With the method public void _samplecommand() the main command is defined (samplecommand). Feel free to change this command but note the lower case and the underscore at the beginning. getCommandName() delivers the the command as a String to print in the console, so it's appropriate to set the same String as your command. Appending another command will perform the desired function. This other appending commands are defined in the constructor of the class. With registerCommand(command, commandImplementation) you can set the commands. In this sample there are processGreetings and echo. Both commands still need their implementation. That means two more classes are required either implementing the processGreetings or the echo command. There is one important method you need. In the "doExecute()" method you define the jobs task. processGreetings creates the previously implemented job and queues the job in the ZooKeeper, the classes name in this sample is "ProcessGreetingsComponent". In the method "doExecute()" of this class a jobmanager calls the method createJob(ID,ID,params). With this IDs you define the kind of the job you want to create, here it is SampleJobProviderComponent which delegates to the real implementation of the job. echo just echoes a text which can be entered right behind the command.
With finishing this implementation now your are able to start the server, described in other tutorials. For this sample it is useful to mount the application to an URL and enter some greetings. Now switch back to the eclipse console. If you only enter the command you defined in ConsoleCommandProvider, in this sample "samplecommand", you will response an error because a further command name is missing. Possible commands are shown. This sample provides "echo" and "processGreeting". If you enter
samplecommand echo Helloand confirm the input "Hello" will be shown in the console. Typing in
samplecommand processGreetingwill do a little bit more. This command creates the processGreetings job and queues it in the ZooKeeper, but doesn't execute the job right now. This will be shown as a output like this:
Processing of greetings has been queue as job 'process.greetings.job_run_6d223c98-6669-4305-a2c4-7698b253e7e0'.
A little bit later new outputs show you that one node in your cluster, if you just run the application on your node it will be your own node, takes the job from the queue and locks this job in the ZooKeeper. Then you will find somewhere in this text the processed greetings. That means in this sample that the greetings are shown again with the submitted node. After processing the job his lock is released.
That's how to create jobs and queue them in ZooKeeper using console commands. With this it's now possible to create jobs on one node but execute them on every other node registered in the cluster. Furthermore accessing an interface from another bundle has been shown.