Skip to main content
Jump to: navigation, search

Difference between revisions of "Gyrex/Learning Material/Develop Background Processing"

Line 1: Line 1:
 
{{Gyrex}}
 
{{Gyrex}}
  
This tutorial will guide you through the process of implementing background processing using ''Jobs''. ''Jobs'', provided in the ''org.eclipse.core.runtime'' plug-in, are units allowing clients to execute code in a separate thread. The problem is that they are restricted to a single node. In a cluster it might be useful to let another node execute a job as the one who created it. Gyrex makes this feature available by providing new classes ''IJob'', ''IJobManager'' and so on. The eclipse given ''Jobs'' will still run on every node in the class but they will get the necessary data from the ZooKeeper where every jobs parameters are stored. Every node can create a ''IJob'' and puts it in a job-queue maintained by the ZooKeeper. With an ID from the node who created the job and a UUI every job is unique. Now the job is available for every node in the cluster. A node who doesn't execute a job at the moment can take and execute this job now. Preventing access of a job by two or more nodes at the same time the node taking the job first locks this job in the queue. If this was successful he executes the job and if necessary gives results back to the ZooKeeper where after that every other node can access this job again.
+
This tutorial will guide you through the process of implementing background processing using ''Jobs'' in Gyrex. Gyrex has job scheduling and execution, which is based on the eclipse Job API. The eclipse standard Jobs are restricted to a single runtime, so a single node in a cluster. In a cluster it might be useful to let another node execute a job as the one who created it. Gyrex makes this feature available by providing new classes ''IJob'', ''IJobManager'' and so on. The eclipse given ''Jobs'' will still run on every node in the class but they will get the necessary data from the ZooKeeper where every jobs parameters are stored. Every node can create a ''IJob'' and puts it in a job-queue maintained by the ZooKeeper. With an ID from the node who created the job and a UUI every job is unique. Now the job is available for every node in the cluster. A node who doesn't execute a job at the moment can take and execute this job now. Preventing access of a job by two or more nodes at the same time the node taking the job first locks this job in the queue. If this was successful he executes the job and if necessary gives results back to the ZooKeeper where after that every other node can access this job again.
 
This tutorial shows how to implement the job and how to use the Gyrex features. We will reference the bundles ''sample.service'' and ''sample.jaxrs'' so use the bundles created with the last tutorials to test your implementation afterwards. The completed bundles with all the codes explained below you find [[Media:BackgroundProcessingBundles-DevelopBackgroundProcessingConsoleCommands.zip|here(zip-file)]].
 
This tutorial shows how to implement the job and how to use the Gyrex features. We will reference the bundles ''sample.service'' and ''sample.jaxrs'' so use the bundles created with the last tutorials to test your implementation afterwards. The completed bundles with all the codes explained below you find [[Media:BackgroundProcessingBundles-DevelopBackgroundProcessingConsoleCommands.zip|here(zip-file)]].
  

Revision as of 03:38, 6 July 2012

Gyrex
Website
Download
Community
Mailing ListForumsIRCmattermost
Bugzilla
OpenHelp WantedBug Day
Contribute
Browse SourceProject Set File

This tutorial will guide you through the process of implementing background processing using Jobs in Gyrex. Gyrex has job scheduling and execution, which is based on the eclipse Job API. The eclipse standard Jobs are restricted to a single runtime, so a single node in a cluster. In a cluster it might be useful to let another node execute a job as the one who created it. Gyrex makes this feature available by providing new classes IJob, IJobManager and so on. The eclipse given Jobs will still run on every node in the class but they will get the necessary data from the ZooKeeper where every jobs parameters are stored. Every node can create a IJob and puts it in a job-queue maintained by the ZooKeeper. With an ID from the node who created the job and a UUI every job is unique. Now the job is available for every node in the cluster. A node who doesn't execute a job at the moment can take and execute this job now. Preventing access of a job by two or more nodes at the same time the node taking the job first locks this job in the queue. If this was successful he executes the job and if necessary gives results back to the ZooKeeper where after that every other node can access this job again. This tutorial shows how to implement the job and how to use the Gyrex features. We will reference the bundles sample.service and sample.jaxrs so use the bundles created with the last tutorials to test your implementation afterwards. The completed bundles with all the codes explained below you find here(zip-file).

Requirements

OSGi Declarative Services using Gyrex require a configured target platform as well as a Java 6 runtime environment.
It is recommend that you have at least the packages from the tutorial Develop JAX-RS Application.

This tutorial was created using eclipse Indigo Service Release 2.

Tutorial

Let's start with the implementation of the job. A new bundle is required where the job will be implemented. To do so create a new plug-in project name it like "sample.job". Don't create an Activator and don't use any templates. As in earlier tutorials you have to import some packages, too. A few packages like the eclipse job and the Gyrex job has to be marked as required. In the MANIFEST.MF you can perform this by adding following lines:

MANIFEST.MF bundle sample.job
Import-Package: sample.service;version="1.0.0",
 org.apache.commons.lang.exception;version="2.4.0",
 org.apache.commons.lang.math;version="2.4.0",
 org.eclipse.gyrex.cloud.environment;version="[1.0.0,2.0.0)",
 org.slf4j;version="[1.6.0,2.0.0)"
Bundle-ActivationPolicy: lazy
Require-Bundle: org.eclipse.gyrex.common;bundle-version="1.0.0",
 org.eclipse.gyrex.jobs;bundle-version="1.0.0",
 org.eclipse.core.jobs;bundle-version="3.5.200"
Export-Package: sample.job

The exported package we will use later to create a job and execute him. In Dependencies section Automated Management of Dependencies packages have to be added, too.

  • org.eclipse.osgi
  • org.eclipse.osgi.service
  • org.apache.commons.lang
greeting-job.xml

In a new folder "OSGI-INF" a component definition has to be created now. Properties are:

  • parent folder: sample.job/OSGI-INF
  • filename: greeting-job.xml
  • name: sample.job.component
  • class: sample.job.SampleJobProviderComponent
  • provided service: org.eclipse.gyrex.jobs.provider.JobProvider
  • referenced service: sample.service.GreetingService, bind="setGreetingService", cardinality="1..1"

After saving this file the line

Service-Component: OSGI-INF/greeting-job.xml

should be added in the MANIFEST.MF.

Now create a new package in folder source. The name will be something like "sample.job". At least 2 classes are needed. One class is the provider of the job. The name "SampleJobProviderComponent" shows that in this the provider and the component you know as 2 single classes from the tutorial Develop OSGi Declarative Services are combined. On the one hand it provides the job instances and on the other hand it delegates to the real implementation of the job. As a provider this component may only exist once. The implementation of the job will be achieved by creating a new instance of ProcessGreetingsJob. As it was said at the beginning this job is content of the org.eclipse.core plug-in. So you can use everything like methods, monitoring and so on from this class. Implementing the run method the function of this job is defined. In this sample it prints some text about the job and calls the processGreetings method from the interface GreetingService (bundle sample.jaxrs). To be able to call this method the job needs to know the GreetingService instance currently executed. In the greeting-job.xml a reference from the interface GreetingService to the method setGreetingService in SampleJobProviderComponent was created. Using now this method the job gets the GreetingService and is able to call the processGreeting method.
With this the job is implemented. As you noticed no IJob or IJobManager was used yet. The connection to the Zookeeper using Gyrex will be implented now.

In this sample we will use console commands to create and process jobs. They have to be defined in another bundle. Again create a new plug-in project without Activator and templates. Its name could be something like "sample.console". A few further packages we need and to create Gyrex Jobs some packages marked as required are necessary, too. So add in MANIFEST.MF, register MANIFEST.MF:

MANIFEST.MF bundle sample.console
Bundle-ActivationPolicy: lazy
Import-Package: org.apache.commons.lang;version="2.4.0",
 org.eclipse.osgi.framework.console;version="1.1.0",
 org.kohsuke.args4j;version="2.0.12",
 org.osgi.framework;version="1.3.0",
 org.slf4j;version="1.6.4"
Require-Bundle: sample.job;bundle-version="1.0.0",
 org.eclipse.gyrex.common;bundle-version="1.0.0",
 org.eclipse.gyrex.context;bundle-version="1.0.0",
 org.eclipse.gyrex.jobs;bundle-version="1.0.0"

As Dependencies, Automated MAnagement of Dependencies, add this packages:

  • org.kohsuke.args4j
  • org.eclipse.osgi
  • org.apache.commons.lang

To listen on console command inputs a CommandProvider interface has to be set. This will be done in a Component Defintion, placed in a new folder OSGI-INF. Creating it following parameters should be used:

console-commands-provider.xml
  • parent folder: sample.console/OSGI-INF
  • filename: console-commands-provider.xml
  • name: sample.console.component
  • class: sample.console.ConsoleCommandProviderComponent
  • provided service: org.eclipse.osgi.framework.console.CommandProvider
  • immedate="false"

Now check whether the line

Service-Component: OSG-INF/console-commands-provider.xml

is in MANIFEST.MF and add if necessary.

Ok, it's time for the implementions again. It was defined that the implementation class is ConsoleCommandProviderComponent, so just create this class in a new package (name like "sample.console"). In this class the commands will be defined with which the job is created. With the method public void _samplecommand() the main command is defined (samplecommand). Feel free to change this command but note the lower case and the underscore at the beginning. getCommandName() delivers the the command as a String to print in the console, so it's appropriate to set the same String as your command. Appending another command will perform the desired function. This other appending commands are defined in the constructor of the class. With registerCommand(command, commandImplementation) you can set the commands. In this sample there are processGreetings and echo. Both commands still need their implementation. That means two more classes are required either implementing the processGreetings or the echo command. There is one important method you need. In the "doExecute()" method you define the jobs task. processGreetings creates the previously implemented job and queues the job in the ZooKeeper, the classes name in this sample is "ProcessGreetingsComponent". In the method "doExecute()" of this class a jobmanager calls the method createJob(ID,ID,params). With this IDs you define the kind of the job you want to create, here it is SampleJobProviderComponent which delegates to the real implementation of the job. echo just echoes a text which can be entered right behind the command.

GreetingWindow-DevelopBackgroundProcessingConsoleCommand.png

With finishing this implementation now your are able to start the server, described in other tutorials. For this sample it is useful to mount the application to an URL and enter some greetings. Now switch back to the eclipse console. If you only enter the command you defined in ConsoleCommandProvider, in this sample "samplecommand", you will response an error because a further command name is missing. Possible commands are shown. This sample provides "echo" and "processGreeting". If you enter

samplecommand echo Hello
and confirm the input "Hello" will be shown in the console. Typing in
samplecommand processGreeting
will do a little bit more. This command creates the processGreetings job and queues it in the ZooKeeper, but doesn't execute the job right now. This will be shown as a output like this:
Processing of greetings has been queue as job 'process.greetings.job_run_6d223c98-6669-4305-a2c4-7698b253e7e0'.
a few excerpts from console output

A little bit later new outputs show you that one node in your cluster, if you just run the application on your node it will be your own node, takes the job from the queue and locks this job in the ZooKeeper. Then you will find somewhere in this text the processed greetings. That means in this sample that the greetings are shown again with the submitted node. After processing the job his lock is released.

That's how to create jobs and queue them in ZooKeeper using console commands. With this it's now possible to create jobs on one node but execute them on every other node registered in the cluster. Furthermore accessing an interface from another bundle has been shown.

Back to the top