Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Tutorial: ECF Remote Services for Accessing Existing REST Services

Revision as of 14:56, 8 December 2014 by Slewis.composent.com (Talk | contribs) (Asynchronous Access to the Service)


Introduction

Remote Services are usually defined with both a 'host' component (aka 'server') and a 'consumer' component of the service (aka 'client'). The host implements and exports a service for remote access, and the consumer discovers, imports, and then calls the remote service. ECF has a previous tutorial showing how to create a custom RESTful Remote Service Provider with both these host and consumer components, and there are also [[ ECF#Introductory Materials | a number of other tutorials and examples] that show how to use existing ECF providers to expose, discover, import, and use a remote service.

There are many situations, however, where it would be desirable to expose an existing (e.g. web-based REST) service as a Remote Service. There are of course many public and private web services already in place. Obviously, these services are frequently not running within an OSGi framework, and therefore not exposed as OSGi Remote Services. For many such existing services, it would be desirable to have only the consumer of the service be running OSGi and using Remote Services, and not require that the host be running within an OSGi framework, or be implemented in Java. Such a 'consumer/client-only' approach has several important advantages

  1. Existing server/services don't change, and may be implemented in any language, using any framework
  2. Existing clients don't change, and may be implemented in any language, using any framework
  3. This new Remote Service consumer can benefit from advantages coming from Remote Services
    1. Clear separation between the remote service interface/API and the remote service implementation
    2. Support for rich versioning (e.g. of the service interface)
    3. Full dynamics of the remote service
    4. Remote service discovery
    5. Fine-grained security and a standardized management agent provided by Remote Service Admin (RSA)
    6. Synchronous and/or asynchronous/non-blocking access to the remote service without any implementation changes
    7. Integration with/use of a variety of injection frameworks (e.g. Spring/Blueprint, Declarative Services, others)
    8. Use of open standards (OSGi Remote Services/RSA) and open, community-developed implementations (ECF Remote Services) to avoid vendor lock-in

The tutorial below will focus on the creation of a remote service client provider that exposes an existing REST service as an OSGi Remote Service. The example remote service used in this tutorial is the Geonames Timezone service, which is a publicly available service to provide timezone information for a given point on Earth. However, the steps shown below, however, may be applied to creating a consumer/client for any remote service, whether exposed via REST/xml, REST/json, via http/https, or any other protocol. This allows a very powerful approach to integration of remote services into applications: The use of all of the advantages of ECF Remote Services listed above, without any required changes existing services or existing clients of those services.

The complete code for this tutorial is available at the ECF GitHub GeoNames repository.

Declaration of the Service Interface

The OSGi Services model is based upon a clear separation between the service interface (one or more java interface classes) and the service implementation (a POJO instance that implements those interfaces. The first step in making the Geonames Timeservice available as an ECF Remote Service is therefore to declare a new service interface class that exposes the appropriate semantics of the Geonames Timezone service. Here is a declaration that captures the documented semantics of the Geonames Timezone service

package org.geonames.timezone;
/**
 * Service interface to represent consumer access to the Geonames timezone service. 
 * This web-service is documented here:
 * http://www.geonames.org/export/web-services.html#timezone
 *
 */
public interface ITimezoneService {
	/**
	 * Get a timezone instance, given latitude and longitude values.
	 * 
	 * @param latitude the latitude of the location to get the timezone for
	 * @param longitude the longitude of the location to get the timezone for
	 * @return the Timezone information for the given latitude and longitude.  Should
	 * return <code>null</code> if the latitude or longitude are nonsensical (e.g. negative values)
	 */
	Timezone getTimezone(double latitude, double longitude);
}

The getTimezone method returns an instance of the Timezone class. The Timezone class exposes the data returned by the Geonames Timezone service after converting the JSON to a Java Timezone instance. Here is the full source to both the ITimezoneService and the Timezone classes.

Asynchronous Access to the Service

ECF Remote Services can automatically and dynamically create an asynchronous proxy to the remote service. This allows the consumer application (the code that calls the ITimezoneService.getTimezone method) to choose whether to make a synchronous remote call that will block the calling thread until the remote call is complete, or to make an asynchronous remote call that allows the application thread to continue while the remote call is made. As described in ECF/Asynchronous Remote Services Asynchronous Remote Services all that's necessary to have ECF create an asynchronous proxy is to declare an asynchronous version of the ITimezoneService interface. Here is such a declaration for the ITimezoneService interface

package org.geonames.timezone;
 
import java.util.concurrent.CompletableFuture;
/**
 * Asynchronous version of ITimezoneService
 */
public interface ITimezoneServiceAsync {
	/**
	 * Immediately (without blocking) return a CompletableFuture to later retrieve a
	 * returned Timezone instance
	 * 
	 * @param latitude the latitude of the location to get the timezone for
	 * @param longitude the longitude of the location to get the timezone for
	 * @return the CompletableFuture<Timezone> to provide later access to an
	 * instance of Timezone information for the given latitude and longitude.  Should
	 * not return <code>null</code>.
	 */
	CompletableFuture<Timezone> getTimezoneAsync(double latitude, double longitude);	
}

Note the 'Async' suffix in both the interface class name (ITimezoneServiceAsync) and the method name (getTimezoneAsync). This is as described in ECF/Asynchronous Remote Services Asynchronous Remote Services and is all that is necessary to get ECF's Remote Services implementation to dynamically construct a proxy exposing the ITimezoneServiceAsync method for use by consumer/application code.

These three classes: ITimezoneService, Timezone, and ITimezoneServiceAsync are all that are needed in this service API bundle. The manifest.mf and other meta-data to complete this bundle is available in our Github repository here. Note that there are no references in this API to ECF classes, or any other dependencies for that matter. As it should be, this API is completely independent of the distribution system, and only depends upon the semantics of the Geonames Timezone service.

Provider Step 1: Implement New Provider Namespace

ECF allows providers to create their own Namespace types, so that the endpoint id of the remote service can be properly interpreted. Here is a TimeServiceRestNamespace class

public class TimeServiceRestNamespace extends RestNamespace {
 
	public static final String NAME = "com.mycorp.examples.timeservice.provider.rest.namespace";
 
	public ID createInstance(Object[] parameters) throws IDCreateException {
		return new TimeServiceRestID(this, URI.create((String) parameters[0]));
	}
 
	public static class TimeServiceRestID extends RestID {
 
		public TimeServiceRestID(Namespace namespace, URI uri) {
			super(namespace, uri);
		}
	}
}

When the createInstance call is made a new instance of TimeServiceRestID is created from the single String parameter[0] provided by the createInstance caller. In addition, we may define an extension for the org.eclipse.ecf.identity.namespace ECF extension point to allow this TimeServiceRestNamespace to be used when needed by the ECF Remote Service implementation.

   <extension
         point="org.eclipse.ecf.identity.namespace">
      <namespace
            class="com.mycorp.examples.timeservice.provider.rest.common.TimeServiceRestNamespace"
            description="Timeservice Rest Namespace"
            name="com.mycorp.examples.timeservice.provider.rest.namespace">
      </namespace>
   </extension>

Since this Namespace/ID is needed for both the host and the consumer providers, it should be placed in a 'common' bundle that can be depended upon by both the host and consumer provider bundles (below). The complete common bundle is available via the ECF git repo via the com.mycorp.examples.timeservice.provider.rest.common bundle project.

Note that the package with this new Namespace/ID class must be exported by having this in the necessary Export-Package in the common bundle's manifest:

Export-Package: com.mycorp.examples.timeservice.provider.rest.common

Provider Step 2: Creating the Provider Host Implementation

Since we would like to access this service via http+rest+json, we will create a Servlet to actually handle the request...and register it dynamically via the OSGi standard HttpService. That way we may implement the remote ITimeService.getCurrentTime() method by doing the appropriate http GET request which will be handled by our HttpServlet.doGet' implementation. Here is the complete implementation of the TimeRemoteServiceHttpServlet as well as the required ECF remote service container code

public class TimeServiceServerContainer extends ServletServerContainer {
 
	public static final String TIMESERVICE_HOST_CONFIG_NAME = "com.mycorp.examples.timeservice.rest.host";
	public static final String TIMESERVICE_SERVLET_NAME = "/" + ITimeService.class.getName();
 
	private final HttpService httpService;
 
	TimeServiceServerContainer(String id, HttpService httpService) throws ContainerCreateException {
		super(IDFactory.getDefault()
				.createID(TimeServiceRestNamespace.NAME, id));
		this.httpService = httpService;
		// Register our servlet with the given httpService with the TIMESERVICE_SERVLET_NAME
		// which is "/com.mycorp.examples.timeservice.ITimeService"
		try {
			this.httpService.registerServlet(TIMESERVICE_SERVLET_NAME,
					new TimeRemoteServiceHttpServlet(), null, null);
		} catch (Exception e) {
			throw new ContainerCreateException("Could not create Time Service Server Container",e);
		}
	}
 
	public void dispose() {
		httpService.unregister(TIMESERVICE_SERVLET_NAME);
		super.dispose();
	}
 
	public Namespace getConnectNamespace() {
		return IDFactory.getDefault().getNamespaceByName(
				TimeServiceRestNamespace.NAME);
	}
 
	class TimeRemoteServiceHttpServlet extends RemoteServiceHttpServlet {
 
		// Handle get call right here.
		protected void doGet(HttpServletRequest req, HttpServletResponse resp)
				throws ServletException, IOException {
 
		    // Get local OSGi ITimeService
		    ITimeService timeService = HttpServiceComponent.getDefault()
					.getService(ITimeService.class);
		    // Call local service to get the time
		    Long currentTime = timeService.getCurrentTime();
 
		    // Serialize response
		    try {
				resp.getOutputStream().print(new JSONObject().put("time", currentTime).toString());
			} catch (JSONException e) {
				throw new ServletException("json response object could not be created for time service", e);
			}
		}
	}
 
}

In the TimeServiceServerContainer constructor a new instance of the TimeRemoteServiceHttpServlet is created and registered as a servlet with the given HttpService

this.httpService.registerServlet(TIMESERVICE_SERVLET_NAME,
				new TimeRemoteServiceHttpServlet(), null, null);

Note that the TIMESERVICE_SERVLET_NAME is defined in this example to be /com.mycorp.examples.timeservice.ITimeService. Since this is the path associated with the Servlet this means that the remote service may be accessed via a GET request to the following URL:

http://localhost:8080/com.mycorp.examples.timeservice.ITimeService

Alternative paths may be specified as desired by changing the value of TIMESERVICE_SERVLET_NAME.

Also present are overrides of two ECF remote service container lifecycle methods

public Namespace getConnectNamespace() {
	return IDFactory.getDefault().getNamespaceByName(
			TimeServiceRestNamespace.NAME);
}

which references the TimeServiceRestNamespace.NAME to associate this remote service container with the Namespace we created above.

public void dispose() {
	httpService.unregister(TIMESERVICE_SERVLET_NAME);
	super.dispose();
}

which dynamically unregisters the servlet when the host container instance is disposed.

All of the actual behavior is given by the doGet method implementation in the TimeRemoteServiceHttpServlet

protected void doGet(HttpServletRequest req, HttpServletResponse resp)
		throws ServletException, IOException {
 
	// 1. Get local OSGi ITimeService
	ITimeService timeService = HttpServiceComponent.getDefault()
			.getService(ITimeService.class);
 
	// 2. Call local service to get the time
	Long currentTime = timeService.getCurrentTime();
 
	// 3. Serialize response
        try {
		resp.getOutputStream().print(new JSONObject().put("time", 
                                              currentTime).toString());
	} catch (JSONException e) {
		throw new ServletException("json response object could not be created for                    
                                            time service", e);
	}
}

What's happening here

  1. The local ITimeService implementation is retrieved
  2. The ITimeService.getCurrentTime() method is called to get the local time (Long)
  3. The getCurrentTime() result is serialized to json (in this case using the json.org json implementation) and printed to the HttpServletResponse output stream.

The next thing to do is to implement a ECF remote service container instantiator, which is responsible for creating and using instances of TimeServiceServerContainer under the appropriate conditions. Here is the container instantiator class:

public class TimeServiceServerContainerInstantiator extends
		ServletServerContainerInstantiator {
 
	public IContainer createInstance(ContainerTypeDescription description,
			Object[] parameters) throws ContainerCreateException {
		try {
			// Get HttpServices
			Collection<HttpService> httpServices = TimeServiceHttpServiceComponent.getDefault().getHttpServices();
			HttpService httpService = httpServices.iterator().next();
			Map<String, Object> map = (Map<String, Object>) parameters[0];
			String id = (String) map.get("id");
			return new TimeServiceServerContainer(IDFactory.getDefault()
					.createID(TimeServiceRestNamespace.NAME, id), httpService);
		} catch (Exception e) {
			throw new ContainerCreateException(
					"Could not create TimeServiceServerContainer", e);
		}
	}
 
	public String[] getSupportedConfigs(ContainerTypeDescription description) {
		return new String[] { TimeServiceServerContainer.NAME };
	}
}

The createInstance method selects an HttpService to use in the TimeServiceServerContainer constructor, and the "id" parameter is used to define the container's ID. The host container ID therefore corresponds to the http://localhost:8080 in the remote service URL given above.

Note also the container instantiator's getSupportedConfigs method. This method allows providers to define a String[] of config types that they support. This is used by the ECF RSA implementation to determine at export time exactly which remote service containers to create and use to export the remote service.

This container instantiator class can then be declared as an extension for the ECF org.eclipse.ecf.containerFactory extension point:

   <extension
         point="org.eclipse.ecf.containerFactory">
      <containerFactory
            class="com.mycorp.examples.timeservice.internal.provider.rest.host.TimeServiceServerContainerInstantiator"
            hidden="false"
            name="com.mycorp.examples.timeservice.rest.host"
            server="true">
      </containerFactory>
   </extension>

Note that the name attribute...with value com.mycorp.examples.timeservice.rest.host...must match the value defined in the code for TimeServiceServerContainer.NAME.

Finally, the HttpService is injected into the TimeServiceHttpServiceComponent via OSGi Declarative Services. See the standard DS xml for that component here. Here is the entire host provider implementation in com.mycorp.examples.timeservice.rest.host bundle project.

Provider Step 3: Creating the Consumer Provider Implementation

Here is the consumer remote service container provider implementation

public class TimeServiceRestClientContainer extends RestClientContainer {
 
	public static final String TIMESERVICE_CONSUMER_CONFIG_NAME = "com.mycorp.examples.timeservice.rest.consumer";
 
	private IRemoteServiceRegistration reg;
 
	TimeServiceRestClientContainer() {
		// Create a random ID for the client container
		super((RestID) IDFactory.getDefault().createID(
				TimeServiceRestNamespace.NAME, "uuid:"
						+ java.util.UUID.randomUUID().toString()));
		// This sets up the JSON deserialization of the server's response.
		// See below for implementation of TimeServiceRestResponseDeserializer
		setResponseDeserializer(new TimeServiceRestResponseDeserializer());
	}
 
	public void connect(ID targetID, IConnectContext connectContext1)
			throws ContainerConnectException {
		super.connect(targetID, connectContext1);
		// Create the IRemoteCallable to represent
		// access to the ITimeService method.  
		IRemoteCallable callable = RestCallableFactory.createCallable(
				"getCurrentTime", ITimeService.class.getName(), null,
				new HttpGetRequestType(), 30000);
		// Register the callable and associate it with the ITimeService class
		// name
		reg = registerCallables(new String[] { ITimeService.class.getName() },
				new IRemoteCallable[][] { { callable } }, null);
	}
 
	public void disconnect() {
		super.disconnect();
		if (reg != null) {
			reg.unregister();
			reg = null;
		}
	}
 
	class TimeServiceRestResponseDeserializer implements
			IRemoteResponseDeserializer {
		public Object deserializeResponse(String endpoint, IRemoteCall call,
				IRemoteCallable callable,
				@SuppressWarnings("rawtypes") Map responseHeaders,
				byte[] responseBody) throws NotSerializableException {
			// We simply need to read the response body (json String),
			// And return the value of the "time" field
			try {
				return new JSONObject(new String(responseBody)).get("time");
			} catch (JSONException e1) {
				throw new NotSerializableException(
						TimeServiceRestResponseDeserializer.class.getName());
			}
		}
 
	}
 
	public Namespace getConnectNamespace() {
		return IDFactory.getDefault().getNamespaceByName(
				TimeServiceRestNamespace.NAME);
	}
}

First, as with the host, the getConnectNamespace override uses/refers to the TimeServiceRestNamespace.NAME definition. This associates this consumer provider with the host provider.

The TimeServiceRestClientContainer constructor first creates a unique id for itself and then it sets up a json response deserializer (for handling the json result of getCurrentTime()) with this code:

	// This sets up the JSON deserialization of the server's response.
	// See below for implementation of TimeServiceRestResponseDeserializer
	setResponseDeserializer(new TimeServiceRestResponseDeserializer());

When the response is received, the TimeServiceRestResponseDeserializer.deserializeResponse method will be called, and this code then parses the json from the host

	return new JSONObject(new String(responseBody)).get("time");

The connect method implementation creates and registers an IRemoteCallable instance that associates the proxy's method name getCurrentTime with the URL of the time service (consisting of the hosts's container id...i.e. http://localhost:8080/ with the ITimeService servlet path /com.mycorp.examples.timeservice.ITimeService.

The disconnect method simply unregisters the IRemoteCallable.

As for the host remote service container, a container instantiator must be created for the TimeServiceRestClientContainer

public class TimeServiceRestClientContainerInstantiator extends
		RestClientContainerInstantiator {
 
	private static final String TIMESERVICE_HOST_CONFIG_NAME = "com.mycorp.examples.timeservice.rest.host";
 
	@Override
	public IContainer createInstance(ContainerTypeDescription description,
			Object[] parameters) throws ContainerCreateException {
		// Create new container instance
		return new TimeServiceRestClientContainer();
	}
 
	public String[] getImportedConfigs(ContainerTypeDescription description,
			String[] exporterSupportedConfigs) {
		@SuppressWarnings("rawtypes")
		List supportedConfigs = Arrays.asList(exporterSupportedConfigs);
		// If the supportedConfigs contains the timeservice host config,
		// then we are the client to handle it!
		if (supportedConfigs.contains(TIMESERVICE_HOST_CONFIG_NAME))
			return new String[] { TimeServiceRestClientContainer.TIMESERVICE_CONSUMER_CONFIG_NAME };
		else return null;
	}
 
}

Note the getImportedConfigs method, which is automatically called by the ECF Remote Service implementation in order to allow the provider to convey that the TimeServiceRestClientContainer should be used for importing when the TIMESERVICE_HOST_CONFIG_NAME i.e. com.mycorp.examples.timeservice.rest.host.

This remote service container instantiator is then declared using an extension point in the plugin.xml

   <extension
         point="org.eclipse.ecf.containerFactory">
      <containerFactory
            class="com.mycorp.examples.timeservice.internal.provider.rest.consumer.TimeServiceRestClientContainerInstantiator"
            hidden="false"
            name="com.mycorp.examples.timeservice.rest.consumer"
            server="false">
      </containerFactory>
   </extension>

And that completes the consumer provider. The source for the complete bundle is in the com.mycorp.examples.timeservice.provider.rest.consumer bundle project.

Step 4: Using the New Provider

Now we have a completed host provider, and a completed consumer provider for the restful timeservice. These two providers are entirely represented by the three bundles

  1. com.mycorp.examples.timeservice.provider.rest.common
  2. com.mycorp.examples.timeservice.provider.rest.host
  3. com.mycorp.examples.timeservice.provider.rest.consumer

With these three bundles and their dependencies present in a runtime, the following may now be used to export a remote service using the host provider

Dictionary<String, String> props = new Hashtable<String, String>();
props.put("service.exported.interfaces", "*");
// Specify the newly created com.mycorp.examples.timeservice.rest.host provider
props.put("service.exported.configs","com.mycorp.examples.timeservice.rest.host");
// Specify the 'id' parameter for the ID creation of the host (see 
// the TimeServiceServerContainerInstantiator.createInstance method 
props.put("com.mycorp.examples.timeservice.rest.host.id","http://localhost:8181");
// Register a new TimeServiceImpl with the above props
bundleContext.registerService(ITimeService.class, new TimeServiceImpl(), props);

During registerService, the ECF RSA implementation does the following:

  1. Detects that the "service.exported.interfaces" property is set and so the service is to be exported
  2. Detects that the "service.exported.configs" property is set, and selects the container instantiator that returns a matching value from a call to getSupportedConfigs
  3. Creates a new instance of TimeServiceServerContainer by calling the approprate container instantiator's createInstance method, and passes in a Map of appropriate service properties (i.e. com.mycorp.examples.timeservice.rest.host.id).
  4. Uses the created remote service container to export the remote service
  5. Publishes the EndpointDescription resulting from the export for consumer discovery

Note that after host registration as above that this restful provider can be tested simply by using a browser and going to

http://localhost:8181/com.mycorp.examples.timeservice.ITimeService

In the browser this will return the following json

{"time":1386738084894}

On the OSGi Remote Service consumer, upon discovery of the EndpointDescription (through network discovery protocol, or EDEF) the ECF RSA implementation does the following

  1. Select a remote service consumer container by calling all container instantiator's getImportedConfigs method...with the value of exporterSupportedConfigs from the discovered EndpointDescription
  2. Create a new container via the selected container instantiator's createInstance method
  3. Call IContainer.connect(ID,IConnectContext) on the newly created container
  4. Create an ITimeService proxy
  5. Registers this ITimeService proxy in the consumer's local OSGi service registry...along with the standardized service property values

If using DS, the last step above will result in the ITimeService proxy being injected into client code and the client code may then call the ITimeService.getCurrentTime() method. Calling this method will result in a http GET to the URL:

http://localhost:8181/com.mycorp.examples.timeservice.ITimeService

The TimeRemoteServiceHttpServlet.doGet method will be called by the HttpService and then the resulting json will be deserialized via the TimeServiceRestResponseDeserializer in the consumer...resulting in a Long value returned from by the proxy.

As with the Tutorial:_Building_your_first_OSGi_Remote_Service the consumer code is simply

package com.mycorp.examples.timeservice.consumer.ds;
 
import com.mycorp.examples.timeservice.ITimeService;
 
public class TimeServiceComponent {
 
	void bindTimeService(ITimeService timeService) {
		System.out.println("Discovered ITimeService via DS");
		// Call the service and print out result!
		System.out.println("Current time is: " + timeService.getCurrentTime());  // Call the ITimeService remote service
	}
}

Note that both the consumer code and the host code (except for the service property values on the host) are exactly the same. This makes it possible to develop, test, and deploy remote services independent of the underlying providers being used.

Background and Related Articles

Getting Started with ECF's OSGi Remote Services Implementation

OSGi Remote Services and ECF

Asynchronous Proxies for Remote Services

Static File-based Discovery of Remote Service Endpoints

Download ECF Remote Services/RSA Implementation

How to Add Remote Services/RSA to Your Target Platform

Customization of ECF Remote Services

Back to the top