Difference between revisions of "Tutorial: ECF Remote Services for Accessing Existing REST Services"

Revision as of 16:40, 8 December 2014

Introduction

Remote Services are usually defined with both a 'host' component (aka 'server') and a 'consumer' component of the service (aka 'client'). The host implements and exports a service for remote access, and the consumer discovers, imports, and then calls the remote service. ECF has a previous tutorial showing how to create a custom RESTful Remote Service Provider with both these host and consumer components, and there are also [[ ECF#Introductory Materials | a number of other tutorials and examples] that show how to use existing ECF providers to expose, discover, import, and use a remote service.

There are many situations, however, where it would be desirable to expose an existing (e.g. web-based REST) service as a Remote Service. There are of course many public and private web services already in place. Obviously, these services are frequently not running within an OSGi framework, and therefore not exposed as OSGi Remote Services. For many such existing services, it would be desirable to have only the consumer of the service be running OSGi and using Remote Services, and not require that the host be running within an OSGi framework, or be implemented in Java. Such a 'consumer/client-only' approach has several important advantages

1. Existing server/services don't change, and may be implemented in any language, using any framework
2. Existing clients don't change, and may be implemented in any language, using any framework
3. This new Remote Service consumer can benefit from advantages coming from Remote Services
   1. Clear separation between the remote service interface/API and the remote service implementation
   2. Support for rich versioning (e.g. of the service interface)
   3. Full dynamics of the remote service
   4. Remote service discovery
   5. Fine-grained security and a standardized management agent provided by Remote Service Admin (RSA)
   6. Synchronous and/or asynchronous/non-blocking access to the remote service without any implementation changes
   7. Integration with/use of a variety of injection frameworks (e.g. Spring/Blueprint, Declarative Services, others)
   8. Use of open standards (OSGi Remote Services/RSA) and open, community-developed implementations (ECF Remote Services) to avoid vendor lock-in

The tutorial below will focus on the creation of a remote service client provider that exposes an existing REST service as an OSGi Remote Service. The example remote service used in this tutorial is the Geonames Timezone service, which is a publicly available service to provide timezone information for a given point on Earth. However, the steps shown below, however, may be applied to creating a consumer/client for any remote service, whether exposed via REST/xml, REST/json, via http/https, or any other protocol. This allows a very powerful approach to integration of remote services into applications: The use of all of the advantages of ECF Remote Services listed above, without any required changes existing services or existing clients of those services.

The complete code for this tutorial is available at the ECF GitHub GeoNames repository.

Declaration of the Timezone Service Interface

The OSGi Services model is based upon a clear separation between the service interface (one or more java interface classes) and the service implementation (a POJO instance that implements those interfaces. The first step in making the Geonames Timeservice available as an ECF Remote Service is therefore to declare a new service interface class that exposes the appropriate semantics of the Geonames Timezone service. Here is a declaration that captures the documented semantics of the Geonames Timezone service

package org.geonames.timezone;
/**
 * Service interface to represent consumer access to the Geonames timezone service. 
 * This web-service is documented here:
 * http://www.geonames.org/export/web-services.html#timezone
 *
 */
public interface ITimezoneService {
	/**
	 * Get a timezone instance, given latitude and longitude values.
	 * 
	 * @param latitude the latitude of the location to get the timezone for
	 * @param longitude the longitude of the location to get the timezone for
	 * @return the Timezone information for the given latitude and longitude.  Should
	 * return <code>null</code> if the latitude or longitude are nonsensical (e.g. negative values)
	 */
	Timezone getTimezone(double latitude, double longitude);
}

The getTimezone method returns an instance of the Timezone class. The Timezone class exposes the data returned by the Geonames Timezone service after converting the JSON to a Java Timezone instance. Here is the full source to both the ITimezoneService and the Timezone classes.

Asynchronous Access to the Remote Service

ECF Remote Services can automatically and dynamically create an asynchronous proxy to the remote service. This allows the consumer application (the code that calls the ITimezoneService.getTimezone method) to choose whether to make a synchronous remote call that will block the calling thread until the remote call is complete, or to make an asynchronous remote call that allows the application thread to continue while the remote call is made. As described in Asynchronous Remote Services all that's necessary to have ECF create an asynchronous proxy is to declare an asynchronous version of the ITimezoneService interface. Here is such a declaration for the ITimezoneService interface

package org.geonames.timezone;
 
import java.util.concurrent.CompletableFuture;
/**
 * Asynchronous version of ITimezoneService
 */
public interface ITimezoneServiceAsync {
	/**
	 * Immediately (without blocking) return a CompletableFuture to later retrieve a
	 * returned Timezone instance
	 * 
	 * @param latitude the latitude of the location to get the timezone for
	 * @param longitude the longitude of the location to get the timezone for
	 * @return the CompletableFuture<Timezone> to provide later access to an
	 * instance of Timezone information for the given latitude and longitude.  Should
	 * not return <code>null</code>.
	 */
	CompletableFuture<Timezone> getTimezoneAsync(double latitude, double longitude);	
}

Note the 'Async' suffix in both the interface class name (ITimezoneServiceAsync) and the method name (getTimezoneAsync). This is as described in Asynchronous Remote Services and is all that is necessary to get ECF's Remote Services implementation to dynamically construct a proxy exposing the ITimezoneServiceAsync method for use by consumer/application code.

These three classes: ITimezoneService, Timezone, and ITimezoneServiceAsync are all that are needed in this service API bundle. The manifest.mf and other meta-data to complete this bundle is available in our Github repository here. Note that there are no references in this API to ECF classes, or any other dependencies for that matter. As it should be, this API is completely independent of the distribution system, and only depends upon the semantics of the Geonames Timezone service.

Implementing a Client Provider

ECF's implementation of Remote Services exposes open, community-created APIs to make it easy to create and use a custom Client Provider. What is a Client Provider? It's a custom implementation of the consumer/client-side of the remote services distribution system. An ECF client provider is able to substitute custom protocols/transports (e.g. http/https, MQTT, JMS, private protocols) and custom serialization formats (e.g. xml, JSON, object serialization, others), without having to implement the entire OSGi Remote Services and Remote Service Admin specification. In an open, modular fashion, an ECF Client Provider can reuse any/all existing ECF or third party remote service provider functionality. A Client Provider automatically is fully compliant with the OSGi Remote Services/RSA standards, while providing the flexibility to use new, custom, or even private transports. Here is an architecture diagram showing the relationship between distribution providers, the Remote Services standards and ECF's extensible and open implementation.

Integration with ECF Remote Service Admin

The Remote Service Admin specification standardizes an API and mechanism for dynamic discovery of remote services. ECF's implementation of this standard provides a way for new Client Providers to dynamically insert themselves into this discovery process. The way this is done is to implement a new container class along with a container instantiator, and to register a container type description instance as a local OSGi service using the whiteboard pattern.

For the example Timezone remote service the container class is called TimezoneClientContainer. The full source for this class is available here.

The role of the container instantiator is to create an appropriately typed instance of the container when needed by RSA. When a remote service is discovered and then imported, the ECF RSA implementation takes the OSGi-define remote service properties and passes them to every available container instantiator, essentially asking: 'a container of given type has been discovered, and a new instance is needed, can you provide such an instance?'. The implementation of this query is via the TimezoneClientContainer.Instantiator.getImportedConfigs method (labelled in comment with '1').

	public static class Instantiator extends RestClientContainerInstantiator {
 
		/**
		 * 1. This method is called by the ECF RSA implementation when a remote
		 * service is to be imported. The exporterSupportedConfigs parameter
		 * contains the exported config types associated with the remote
		 * service. The implementation of this method decides whether we are
		 * interested in this remote service config type. If we are
		 * (exporterSupportedConfigs contains CONTAINER_TYPE_NAME, then we
		 * return an array of strings containing our CONTAINER_TYPE_NAME
		 */
		@Override
		public String[] getImportedConfigs(
				ContainerTypeDescription description,
				String[] exporterSupportedConfigs) {
			/**
			 * If the exporterSupportedConfigs contains CONTAINER_TYPE_NAME,
			 * then return that CONTAINER_TYPE_NAME to trigger RSA usage of this
			 * container instantiator
			 */
			if (Arrays.asList(exporterSupportedConfigs).contains(
					CONTAINER_TYPE_NAME))
				return new String[] { CONTAINER_TYPE_NAME };
			return null;
		}
 
		@Override
		/**
		 * 2. This method is called by the ECF RSA to create a new instance of
		 * the appropriate
		 * container type (aka OSGi config type)
		 */
		public IContainer createInstance(ContainerTypeDescription description,
				Object[] parameters) throws ContainerCreateException {
			return new TimezoneClientContainer();
		}
	}

. The getImportedConfigs' method logic is: 'if my CONTAINER_TYPE_NAME matches one of the exporterSupportedConfigs then return a String[] including my CONTAINER_TYPE_NAME. This approach provides the flexibility to support multiple exporterSupportedConfigs with a single container type. The ECF RSA implementation then turns around and calls the TimezoneClientContainer.'Instantiator.createContainer method, which in the implementation above (2) returns a new instance of the TimezoneClientContainer.

Once the TimezoneClientContainer instance is created, ECF RSA calls it's connect method with the targetID defined in the standardized meta-data for the remote service. Here are the lines of the TimezoneClientContainer connect method explained

super.connect(targetID, connectContext1);

Passes the targetID to the RestClientContainer super class, which sets a member variable to the value of targetID.

		setAlwaysSendDefaultParameters(true);

This line configures the container instance to always send the default parameters on the remote call. This is necessary for the Geonames Timezone service because one of the required http request parameters is the username parameter. Setting this to 'true' means that the default parameter values (username) are always sent (as required by the Timezone service).

To create the Remote Service proxy (the ECF-constructed proxy instance of the ITimezoneService and ITimezoneServiceAsync) it's necessary to define the association between the getTimezone method and the Geonames Timezone request. As documented here, the Timzeone service request consists of

1. A complete URL for the request (protocol/hostname/port + path)
2. The required parameters. In the case of the Geonames Timezone service the parameters are
   1. latitude (aka 'lat')
   2. longitude ('lng')
   3. username (for authentication)

First we construct the necessary remote call parameters using a RemoteCallParameter.Builder

RemoteCallParameter.Builder parameterBuilder = new RemoteCallParameter.Builder()
		.addParameter("lat").addParameter("lng")
		.addParameter("username", USERNAME);

Then we construct an IRemoteCallable, defining the association between the getTimezone method and the /timezoneJSON remote service path, including the above-created parameters, and defining that this service is based upon an http GET request.

RemoteCallable.Builder callableBuilder = new RemoteCallable.Builder(
		"getTimezone", "/timezoneJSON").setDefaultParameters(
		parameterBuilder.build()).setRequestType(
		new HttpGetRequestType());

Finally, we register this IRemoteCallable and associate it with the ITimezoneService.class service interface

tzServiceRegistration = registerCallables(ITimezoneService.class,
		new IRemoteCallable[] { callableBuilder.build() }, null);

The above registration is all that's needed for the ECF remote service code to construct a proxy implementing ITimezoneService and ITimezoneServiceAsync. When a consumer application actually calls either the getTimezone or getTimezoneAsync methods on the proxy, the following will occur automatically

1. Construct the appropriate URL for calling the Geonames Timezone service: http://api.geonames.org/timezoneJSON
2. Serialize the parameters passed to the getTimezone method (i.e. 'lat' and 'lng') and add them as URL request parameters, resulting in the URL: http://api.geonames.org/timezoneJSON?lat=<lat param value>&lng=<lng param value>&username=<username default value>
3. Use the resulting URL to issue an HTTP GET request to the Geonames service

When the Geonames service responds to the GET request, it returns a single instance of a JSON Object container several fields with various information associated with the timezone...e.g. countryCode, countryName, lat, lng, timezoneId, dstOffset, etc. The following code sets up a response deserializer, that parses the returned String using a common JSON parser. Then given the data, it creates an instance of the Timezone class and returns it. At remote method call time, when the Geonames service responds this method will be called to deserialize the HTTP response and return a proper Timezone instance to the caller of getTimezone.

		setResponseDeserializer(new IRemoteResponseDeserializer() {
			@Override
			public Object deserializeResponse(String endpoint,
					IRemoteCall call, IRemoteCallable callable,
					@SuppressWarnings("rawtypes") Map responseHeaders,
					byte[] responseBody) throws NotSerializableException {
				try {
					// Convert responseBody to String and parse using org.json
					// lib
					JSONObject jo = new JSONObject(new String(responseBody));
					// Check status for failure. Throws exception if
					// error status
					if (jo.has("status")) {
						JSONObject status = jo.getJSONObject("status");
						throw new JSONException(status.getString("message")
								+ ";code=" + status.getInt("value"));
					}
					// No exception, so get each of the fields from the
					// json object
					String countryCode = jo.getString("countryCode");
					String countryName = jo.getString("countryName");
					double lat = jo.getDouble("lat");
					double lng = jo.getDouble("lng");
					String timezoneId = jo.getString("timezoneId");
					double dstOffset = jo.getDouble("dstOffset");
					double gmtOffset = jo.getDouble("gmtOffset");
					double rawOffset = jo.getDouble("rawOffset");
					String time = jo.getString("time");
					String sunrise = jo.getString("sunrise");
					String sunset = jo.getString("sunset");
					// Now create and return Timezone instance with all the
					// appropriate
					// values of the fields
					return new Timezone(countryCode, countryName, lat, lng,
							timezoneId, dstOffset, gmtOffset, rawOffset,
							dateFormat.parse(time), dateFormat.parse(sunrise),
							dateFormat.parse(sunset));
					// If some json parsing exception (badly formatted json and
					// so on,
					// throw an appropriate exception
				} catch (Exception e) {
					NotSerializableException ex = new NotSerializableException(
							"Problem in response from timezone service endpoint="
									+ endpoint + " status message: "
									+ e.getMessage());
					ex.setStackTrace(e.getStackTrace());
					throw ex;
				}
			}
		});
	}

Note that this code may use any JSON parser, or in fact may do whatever is appropriate (e.g. parse xml, deserialize java Objects, use some other serialization technique like protocol buffers, etc to convert from the remote service response to the proper return type for the service interface (Timezone).

The connect implementation above is sufficient for the TimezoneClientContainer to create and execute the Geonames Timezone service request, and handle/deserialize the JSON response, and associate this implementation with the ITimezoneService.getTimezone method and provided parameters. Note that it's also possible to reuse other codebases (other than the RestClientContainer superclass) if the remote service is not REST-based, or requires more control and customization of the remote invocation process.

Registering the Container and Namespace

In the Activator for this Client Provider, there are two OSGi service registrations, which are necessary to hook into the ECF RSA implementation as described above

	public void start(BundleContext bundleContext) throws Exception {
		Activator.context = bundleContext;
		// Register an instance of TimezoneNamespace
		bundleContext.registerService(Namespace.class, new TimezoneNamespace(),
				null);
		// Register an instance of TimezoneContainerTypeDescription (see class
		// below)
		bundleContext.registerService(ContainerTypeDescription.class,
				new TimezoneContainerTypeDescription(), null);
	}
 
	class TimezoneContainerTypeDescription extends ContainerTypeDescription {
		public TimezoneContainerTypeDescription() {
			super(TimezoneClientContainer.CONTAINER_TYPE_NAME,
					new TimezoneClientContainer.Instantiator(),
					"Geonames Timezone Remote Service Client Container");
		}
	}

As described in comments, these two service registrations (Namespace and ContainerTypeDescription) setup the necessary structures for RSA to create the TimezoneClientContainer, call connect and other methods on the container at the appropriate times during remote service discovery, import, proxy creation, and remote service call/invocation.

Background and Related Articles

Getting Started with ECF's OSGi Remote Services Implementation

OSGi Remote Services and ECF

Asynchronous Proxies for Remote Services

Static File-based Discovery of Remote Service Endpoints

Download ECF Remote Services/RSA Implementation

How to Add Remote Services/RSA to Your Target Platform

Customization of ECF Remote Services