Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "G-Eclipse/AWS Cloud in headless mode"

(Prerequisites)
(The code)
 
(14 intermediate revisions by the same user not shown)
Line 13: Line 13:
 
* The name of a '''Amazon Machine Image''' (AMI) which can be started by the user. The AMI name can be a private one or a public one.  
 
* The name of a '''Amazon Machine Image''' (AMI) which can be started by the user. The AMI name can be a private one or a public one.  
 
* The name of a valid '''AWS Security Group'''. This security group can be seen as a firewall configuration for running Amazon Computing instances.  
 
* The name of a valid '''AWS Security Group'''. This security group can be seen as a firewall configuration for running Amazon Computing instances.  
*
+
* The name of a valid '''AWS Keypair''' in order to enable the login to the running AMI instance later.
  
== The workspace ==  
+
If you tried the g-Eclipse framework in UI mode, you can get parts of the required information with the help of the g-Eclipse Grid project view.
 +
 
 +
== The workspace ==
 +
 
 +
We start with a plain Eclipse installation and import the following plugin projects in the workspace.
 +
 
 +
From the g-Eclipse SVN repository we need:
 +
* eu.geclipse
 +
* eu.geclipse.core
 +
* eu.geclipse.core.filesystem
 +
* eu.geclipse.core.reporting
 +
* eu.geclipse.core.info
 +
* eu.geclipse.aws
 +
* eu.geclipse.aws.ec2
 +
* eu.geclipse.aws.s3
 +
* com.xerox.amazonws
 +
* org.jet3t
 +
 
 +
From the Orbit CVS repository we need
 +
* javax.xml.bind
 +
* javax.xml.stream
 +
 
 +
From the SMILA SVN repository we need from tag 0.5M1
 +
* com.sun.jaxb
 +
 
 +
Or you just download this [[Media:AWSCloudHeadless.psf‎]] project set to import the required projects.
  
 
== The code ==
 
== The code ==
 +
 +
IN order to demonstrate the use of the g-Eclipse core framework in headless mode, we create a simple plugin project and start an
 +
instance of EC2 with every start of the instance. So please do the following:
 +
 +
# Create an new plugin project and name it "eu.geclipse.demo.headless" and click "Next"
 +
# Deactivate the option "This plug-in will make contributions to the UI" because we don't want!
 +
# Click Finish!
 +
 +
Then open the Activator of the plugin "eu.geclipse.demo.headless.Activator" in the editor and add the following code in the <code>start<\code> method. First define the Access keys and use the your personnel ones.
 +
System.out.println(" Start AWS demo headless ");
 +
String aws_accessId = "'''Access Key ID'''";
 +
String aws_secretkey = "'''Secret Access Key'''";
 +
 +
Then create a Virtual Organisation (VO). A VO can be seen as a virtual data center composed by computing resources, storage resources and additional services. We define a AWS VO flavor and use the Access Key ID to define it. With the following code line the VO is created and put into the g-Eclipse core model.
 +
IVirtualOrganization vo = null;
 +
IVoManager manager = GridModel.getVoManager();
 +
AWSVoCreator creator = new AWSVoCreator();
 +
creator.setVoName("aws-vo");
 +
creator.setAwsAccessId(aws_accessId);
 +
vo = (IVirtualOrganization) manager.create(creator);
 +
System.out.println(" VO created");
 +
 +
The next step is the creation of an Authentication Token valid for the VO. This is done with the following code lines.
 +
PasswordManager.registerPassword(AWSAuthTokenDescription.SECURE_STORAGE_NODE + aws_accessId, aws_secretkey);
 +
IAuthenticationTokenDescription desc = new AWSAuthTokenDescription( (AWSVirtualOrganization) vo);
 +
AuthenticationTokenManager tokenManager = AuthenticationTokenManager.getManager();
 +
AWSAuthToken token = (AWSAuthToken) tokenManager.createToken(desc);
 +
 +
The next step is now the interaction with the Cloud model of the g-Eclipse framework. An important part is the InfoService of the Grid/Cloud model. With the help of the InfoService and the GridResourceCategory, the number of running instances and there name can be retrieved from the Cloud InfoService.
 +
 +
IGridInfoService infoService = vo.getInfoService();
 +
IGridResourceCategory category = GridResourceCategoryFactory.getCategory(IAWSCategories.CATEGORY_AWS_COMPUTING);
 +
IGridResource[] resources = infoService.fetchResources(vo, vo, category, null);
 +
for (IGridResource iRes : resources) {
 +
  System.out.println(iRes.getName() + "@" + iRes.getHostName());
 +
}
 +
 +
If you don't have started a EC2 instance before, you do not get any output here. But if there is an output shown, be aware, that this costs your money!!! Anyhow, it is time to start an instance. Do this with the following command.
 +
 +
AMILaunchConfiguration launchConf = new AMILaunchConfiguration("'''YOUR_AMI_NAME'''");
 +
launchConf.setKeyName("'''YOUR_AWS_KEYPAIR'''");
 +
List<String> securityGroup = new ArrayList<String>(5) ;
 +
securityGroup.add("'''YOUR_SECURITY_GROUP'''");
 +
launchConf.setSecurityGroup(securityGroup);
 +
launchConf.setInstanceType(InstanceType.DEFAULT);
 +
EC2 ec2 = new EC2((AWSVirtualOrganization) vo);
 +
ec2.runInstances(launchConf);
 +
 +
The instance starts and costs your money. Whenever your run this code, you start a new image. Do not forget to shutdown the AMI images! But at this moment it does not run due to the missing imports and unresolved bundle dependencies. Open the META-INF file of the eu.geclipse.demo.headless plugin project and add the following "Import-package" and "Require-Bundles".
 +
Import-Package: org.osgi.framework;version="1.3.0"
 +
Require-Bundle: eu.geclipse;bundle-version="1.0.0",
 +
  eu.geclipse.aws;bundle-version="1.0.0",
 +
  eu.geclipse.core;bundle-version="1.0.0",
 +
  eu.geclipse.core.filesystem;bundle-version="1.0.0",
 +
  eu.geclipse.core.reporting;bundle-version="1.0.0",
 +
  org.eclipse.core.runtime;bundle-version="3.5.0",
 +
  eu.geclipse.aws.s3;bundle-version="1.0.0",
 +
  eu.geclipse.aws.ec2;bundle-version="1.0.0",
 +
  com.xerox.amazonws;bundle-version="1.3.0"
 +
 +
Then use the "organize Import" feature of Eclipse for the Activator and import the following Classes.
 +
 +
import org.eclipse.core.runtime.Plugin;
 +
import org.osgi.framework.BundleContext;
 +
import com.xerox.amazonws.ec2.InstanceType;
 +
import eu.geclipse.aws.IAWSCategories;
 +
import eu.geclipse.aws.auth.AWSAuthToken;
 +
import eu.geclipse.aws.auth.AWSAuthTokenDescription;
 +
import eu.geclipse.aws.ec2.EC2;
 +
import eu.geclipse.aws.ec2.op.AMILaunchConfiguration;
 +
import eu.geclipse.aws.vo.AWSVirtualOrganization;
 +
import eu.geclipse.aws.vo.AWSVoCreator;
 +
import eu.geclipse.core.auth.AuthenticationTokenManager;
 +
import eu.geclipse.core.auth.IAuthenticationTokenDescription;
 +
import eu.geclipse.core.auth.PasswordManager;
 +
import eu.geclipse.core.model.GridModel;
 +
import eu.geclipse.core.model.IGridInfoService;
 +
import eu.geclipse.core.model.IGridResource;
 +
import eu.geclipse.core.model.IGridResourceCategory;
 +
import eu.geclipse.core.model.IVirtualOrganization;
 +
import eu.geclipse.core.model.IVoManager;
 +
import eu.geclipse.core.model.impl.GridResourceCategoryFactory;
 +
 +
You can then start the application as a OSGi application and should see the output in the console. Don't forget to change the values to your personnel keys from the AWS infrastructure. It is recommended to control the running instances with another tool. It is natural that we recommend the use of the g-Eclipse framework. See the download pages of the g-Eclipse project.
 +
 +
'''DON'T FORGET TO TERMINATE THE INSTANCES!!'''
 +
 +
 +
[[Category:g-Eclipse]]

Latest revision as of 10:48, 17 November 2009

API for the Cloud

The g-Eclipse project provides an integrated project view to Grid and Cloud infrastructures. With the help of the created "Grid-Project", the users is able to interact with Grid and Cloud infrastructures. The g-Eclipse framework contains the exemplary implementation for the Amazon Web Services, but the framework is generic to integrate other Cloud infrastructures too.

The architecture of the g-Eclipse framework is designed to be able to run the core of the g-Eclipse framework independent from the UI parts. With the feature, the core g-Eclipse framework provides a generic API for Cloud computing allowing to start and stop remote Cloud instances as well as management of data on the infrastructures.

In this tutorial, we demonstrate the usage of the g-Eclipse framework in headless mode.

Prerequisites

This tutorial shows the usage of the g-Eclipse core components together with the AWS web services. In order to perform the tutorial, the user requires a valid AWS account including the following features:

  • The AWS Access Credentials from AWS called "Access Key ID" and the "Secret Access Key"
  • The name of a Amazon Machine Image (AMI) which can be started by the user. The AMI name can be a private one or a public one.
  • The name of a valid AWS Security Group. This security group can be seen as a firewall configuration for running Amazon Computing instances.
  • The name of a valid AWS Keypair in order to enable the login to the running AMI instance later.

If you tried the g-Eclipse framework in UI mode, you can get parts of the required information with the help of the g-Eclipse Grid project view.

The workspace

We start with a plain Eclipse installation and import the following plugin projects in the workspace.

From the g-Eclipse SVN repository we need:

  • eu.geclipse
  • eu.geclipse.core
  • eu.geclipse.core.filesystem
  • eu.geclipse.core.reporting
  • eu.geclipse.core.info
  • eu.geclipse.aws
  • eu.geclipse.aws.ec2
  • eu.geclipse.aws.s3
  • com.xerox.amazonws
  • org.jet3t

From the Orbit CVS repository we need

  • javax.xml.bind
  • javax.xml.stream

From the SMILA SVN repository we need from tag 0.5M1

  • com.sun.jaxb

Or you just download this Media:AWSCloudHeadless.psf‎ project set to import the required projects.

The code

IN order to demonstrate the use of the g-Eclipse core framework in headless mode, we create a simple plugin project and start an instance of EC2 with every start of the instance. So please do the following:

  1. Create an new plugin project and name it "eu.geclipse.demo.headless" and click "Next"
  2. Deactivate the option "This plug-in will make contributions to the UI" because we don't want!
  3. Click Finish!

Then open the Activator of the plugin "eu.geclipse.demo.headless.Activator" in the editor and add the following code in the start<\code> method. First define the Access keys and use the your personnel ones.

System.out.println(" Start AWS demo headless ");
String aws_accessId = "Access Key ID";
String aws_secretkey = "Secret Access Key";

Then create a Virtual Organisation (VO). A VO can be seen as a virtual data center composed by computing resources, storage resources and additional services. We define a AWS VO flavor and use the Access Key ID to define it. With the following code line the VO is created and put into the g-Eclipse core model.

IVirtualOrganization vo = null;
IVoManager manager = GridModel.getVoManager();
AWSVoCreator creator = new AWSVoCreator();
creator.setVoName("aws-vo");
creator.setAwsAccessId(aws_accessId);
vo = (IVirtualOrganization) manager.create(creator);
System.out.println(" VO created");

The next step is the creation of an Authentication Token valid for the VO. This is done with the following code lines.

PasswordManager.registerPassword(AWSAuthTokenDescription.SECURE_STORAGE_NODE + aws_accessId, aws_secretkey);
IAuthenticationTokenDescription desc = new AWSAuthTokenDescription( (AWSVirtualOrganization) vo);
AuthenticationTokenManager tokenManager = AuthenticationTokenManager.getManager();
AWSAuthToken token = (AWSAuthToken) tokenManager.createToken(desc);

The next step is now the interaction with the Cloud model of the g-Eclipse framework. An important part is the InfoService of the Grid/Cloud model. With the help of the InfoService and the GridResourceCategory, the number of running instances and there name can be retrieved from the Cloud InfoService.

IGridInfoService infoService = vo.getInfoService();
IGridResourceCategory category = GridResourceCategoryFactory.getCategory(IAWSCategories.CATEGORY_AWS_COMPUTING);
IGridResource[] resources = infoService.fetchResources(vo, vo, category, null);
for (IGridResource iRes : resources) {
  System.out.println(iRes.getName() + "@" + iRes.getHostName());
}

If you don't have started a EC2 instance before, you do not get any output here. But if there is an output shown, be aware, that this costs your money!!! Anyhow, it is time to start an instance. Do this with the following command.

AMILaunchConfiguration launchConf = new AMILaunchConfiguration("YOUR_AMI_NAME");
launchConf.setKeyName("YOUR_AWS_KEYPAIR"); 
List<String> securityGroup = new ArrayList<String>(5) ; 
securityGroup.add("YOUR_SECURITY_GROUP"); 
launchConf.setSecurityGroup(securityGroup); 
launchConf.setInstanceType(InstanceType.DEFAULT);
EC2 ec2 = new EC2((AWSVirtualOrganization) vo); 
ec2.runInstances(launchConf); 

The instance starts and costs your money. Whenever your run this code, you start a new image. Do not forget to shutdown the AMI images! But at this moment it does not run due to the missing imports and unresolved bundle dependencies. Open the META-INF file of the eu.geclipse.demo.headless plugin project and add the following "Import-package" and "Require-Bundles".

Import-Package: org.osgi.framework;version="1.3.0"
Require-Bundle: eu.geclipse;bundle-version="1.0.0",
 eu.geclipse.aws;bundle-version="1.0.0",
 eu.geclipse.core;bundle-version="1.0.0",
 eu.geclipse.core.filesystem;bundle-version="1.0.0",
 eu.geclipse.core.reporting;bundle-version="1.0.0",
 org.eclipse.core.runtime;bundle-version="3.5.0",
 eu.geclipse.aws.s3;bundle-version="1.0.0",
 eu.geclipse.aws.ec2;bundle-version="1.0.0",
 com.xerox.amazonws;bundle-version="1.3.0"

Then use the "organize Import" feature of Eclipse for the Activator and import the following Classes.

import org.eclipse.core.runtime.Plugin;
import org.osgi.framework.BundleContext;
import com.xerox.amazonws.ec2.InstanceType;
import eu.geclipse.aws.IAWSCategories;
import eu.geclipse.aws.auth.AWSAuthToken;
import eu.geclipse.aws.auth.AWSAuthTokenDescription;
import eu.geclipse.aws.ec2.EC2;
import eu.geclipse.aws.ec2.op.AMILaunchConfiguration;
import eu.geclipse.aws.vo.AWSVirtualOrganization;
import eu.geclipse.aws.vo.AWSVoCreator;
import eu.geclipse.core.auth.AuthenticationTokenManager;
import eu.geclipse.core.auth.IAuthenticationTokenDescription;
import eu.geclipse.core.auth.PasswordManager;
import eu.geclipse.core.model.GridModel;
import eu.geclipse.core.model.IGridInfoService;
import eu.geclipse.core.model.IGridResource;
import eu.geclipse.core.model.IGridResourceCategory;
import eu.geclipse.core.model.IVirtualOrganization;
import eu.geclipse.core.model.IVoManager;
import eu.geclipse.core.model.impl.GridResourceCategoryFactory;

You can then start the application as a OSGi application and should see the output in the console. Don't forget to change the values to your personnel keys from the AWS infrastructure. It is recommended to control the running instances with another tool. It is natural that we recommend the use of the g-Eclipse framework. See the download pages of the g-Eclipse project.

DON'T FORGET TO TERMINATE THE INSTANCES!!

Back to the top