View source for Tutorial: ECF Remote Services for Accessing Existing REST Services

<!-- This CSS wraps code blocks in pretty boxes -->
<css>
.mw-code { 
  background-color: #fafafa; 
  padding: 20px; 
  border-color: #ddd; 
  border-width: 3px; 
  border-style: solid; 
  border-radius: 5px; 
  margin-left: 10px; 
  margin-right: 10px; 
  overflow-x: auto; 
}

.mw-headline {
  font-family: Helvetica Neue,Helvetica,Arial,sans-serif;
  color: rgb(51, 51, 51);
}

p {
  font-family: Helvetica Neue,Helvetica,Arial,sans-serif;
  text-align: justify;
}

h1 {
  border-bottom-color: rgb(238, 238, 238);
  font-weight: bold;
  padding-bottom: 17px;
  font-size: 30px;
  line-height: 36px;
}

h2 {
  border-bottom-color: rgb(238, 238, 238);
  padding-bottom: 12px;
  font-weight: bold;
  font-size: 24px;
  line-height: 36px;
}

h3 {
  border-bottom-width: 1px;
  border-bottom-style: solid;
  border-bottom-color: rgb(238, 238, 238);
  padding-bottom: 8px;
  font-size: 18px;
  line-height: 27px;
}

h4 {
  font-size: 14px;
}

h5 {
  font-size: 12px;
}

h6 {
  font-size: 11px;
  color: #EBEBEB;
  text-transform: uppercase;
}

a {
  color: rgb(0, 105, 214);
  font-family: Helvetica Neue,Helvetica,Arial,sans-serif;
}

a:visited {
  color: rgb(0, 105, 214);
}
</css>

==Introduction==

Remote Services are usually defined with both a 'host' component (aka 'server') and a 'consumer' component (aka 'client').   The host typically implements and exports a service for remote access, and the consumer discovers, imports, and then uses the remote service by calling methods on the service interface.   ECF has a previous tutorial showing how to [[ Tutorial: Creating a RESTful Remote Service Provider  | create a custom RESTful Remote Service Provider]] with both these host and consumer components, and there are also [[ ECF#Introductory Materials | a number of other tutorials and examples]] showing how to use existing ECF providers to expose, discover, import, and use a remote service.

There are many situations, however, where it would be desirable to expose an '''existing''' (e.g. web-based REST) service as a Remote Service.  There are of course many public and private web services already in place.  These services are frequently not running within an OSGi framework, nor are implemented in Java, and therefore not exposed as a Remote Service host.    For many such existing services, it would be valuable to have only the '''consumer''' of the service be running OSGi and using Remote Services, and not require that the host be running within an OSGi framework, nor even necessarily be implemented in Java.  

With any Remote Service, what the consumer '''sees''' at the time of use is a dynamically-constructed proxy implementing one or more service interfaces.  ECF's RSA implementation provides open, community-developed APIs allowing full customization of the both the creation and runtime behavior of the proxy, further enabling the creation of a 'consumer/client-only' remote service provider capable of interacting with any Internet-based service.

The use of a 'consumer/client-only' provider has several important advantages for consumer application development

#Existing server/services don't change, and may be implemented in any language, using any framework or service infrastructure
#Existing clients don't change, and may continue to be implemented in any language, using any framework
#This Remote Service consumer can automatically benefit from advantages of using Remote Services
##Clear separation between the '''remote service interface/API''' and the '''remote service implementation'''
##Support for rich versioning (e.g. of the service interface)
##Full dynamics of the remote service
##Remote service discovery
##Fine-grained service-level type-safe security
##Standardized management agent via [[Remote_Services_Admin | Remote Service Admin (RSA)]]
##Synchronous and/or asynchronous/non-blocking access to the remote service without any implementation changes
##Automatic integration with/use of a variety of injection frameworks (e.g. Spring/Blueprint, Declarative Services, others)
##Automatic integration with any framework that uses OSGi Services
##Use of open standards (OSGi Remote Services/RSA) and open, community-developed implementations (ECF Remote Services) to avoid vendor lock-in and provide easy customization

The tutorial below will focus on the creation of a remote service client provider that exposes an existing REST service as an OSGi Remote Service.   The example remote service used in this tutorial is the [http://www.geonames.org/ Geonames] [http://www.geonames.org/export/web-services.html#timezone Timezone service], which is a publicly available service to provide timezone information for a given point on Earth.  However, the steps shown below, however, may be applied to creating a consumer/client for any remote service, whether exposed via REST/xml, REST/json, via http/https, or any other protocol.  This allows a very powerful approach to integration of remote services into applications:  The use of all of the advantages of ECF Remote Services listed above, without '''any''' required changes existing services or existing clients of those services.

The complete code for this tutorial is available at the [https://github.com/ECF/Geonames ECF GitHub GeoNames repository].

==Declaring the Timezone Remote Service==

The OSGi Services model is based upon a clear separation between the service interface (one or more java interface classes) and the service implementation (a POJO instance that implements those interfaces.   The first step in making the Geonames Timeservice available as an ECF Remote Service is therefore to declare a new service interface class that exposes the appropriate semantics of the [http://www.geonames.org/export/web-services.html#timezone Geonames Timezone service].   

<source lang="java">
package org.geonames.timezone;
/**
 * Service interface to represent consumer access to the Geonames timezone service. 
 * This web-service is documented here:
 * http://www.geonames.org/export/web-services.html#timezone
 *
 */
public interface ITimezoneService {
	/**
	 * Get a timezone instance, given latitude and longitude values.
	 * 
	 * @param latitude the latitude of the location to get the timezone for
	 * @param longitude the longitude of the location to get the timezone for
	 * @return the Timezone information for the given latitude and longitude.  Should
	 * return <code>null</code> if the latitude or longitude are nonsensical (e.g. negative values)
	 */
	Timezone getTimezone(double latitude, double longitude);
}
</source>

The '''getTimezone''' method returns an instance of the Timezone class.   The Timezone class exposes the data returned by the [http://www.geonames.org/export/web-services.html#timezone Geonames Timezone service] after converting the JSON to a Java Timezone instance.   [https://github.com/ECF/Geonames/tree/master/bundles/org.geonames.timezone/src/org/geonames/timezone Here is the full source to both the ITimezoneService and the Timezone classes].

===Asynchronous Access to the Remote Service===

[[ECF#OSGi Remote Services  | ECF Remote Services]] can automatically and dynamically create an [[ECF/Asynchronous Remote Services | asynchronous proxy]] to the remote service.   This allows the consumer application (the code that calls the '''ITimezoneService.getTimezone''' method) to '''choose''' whether to make a synchronous remote call that will block the calling thread until the remote call is complete, or to make an asynchronous remote call that allows the application thread to continue while the remote call is made.   As described in [[ECF/Asynchronous Remote Services | Asynchronous Remote Services]] all that's necessary to have ECF create an asynchronous proxy is to declare an asynchronous version of the '''ITimezoneService''' interface.  Here is such a declaration for the '''ITimezoneService''' interface

<source lang="java">
package org.geonames.timezone;

import java.util.concurrent.CompletableFuture;
/**
 * Asynchronous version of ITimezoneService
 */
public interface ITimezoneServiceAsync {
	/**
	 * Immediately (without blocking) return a CompletableFuture to later retrieve a
	 * returned Timezone instance
	 * 
	 * @param latitude the latitude of the location to get the timezone for
	 * @param longitude the longitude of the location to get the timezone for
	 * @return the CompletableFuture<Timezone> to provide later access to an
	 * instance of Timezone information for the given latitude and longitude.  Should
	 * not return <code>null</code>.
	 */
	CompletableFuture<Timezone> getTimezoneAsync(double latitude, double longitude);	
}
</source>

Note the 'Async' suffix in both the interface class name ('''ITimezoneServiceAsync''') and the method name ('''getTimezoneAsync''').   This is as described in [[ECF/Asynchronous Remote Services | Asynchronous Remote Services]] and is all that is necessary to get ECF's Remote Services implementation to dynamically
construct a proxy exposing the '''ITimezoneServiceAsync''' method for use by consumer/application code.

These three classes:  '''ITimezoneService''', '''Timezone''', and '''ITimezoneServiceAsync''' are all that are needed in this service API bundle.   The manifest.mf and other meta-data to complete this bundle is available in our Github repository [https://github.com/ECF/Geonames/tree/master/bundles/org.geonames.timezone  here].   Note that there are no references in this API to ECF classes, or any other dependencies for that matter.  As it should be, this API is completely independent of the distribution system, and only depends upon the '''semantics''' of the [https://github.com/ECF/Geonames/tree/master/bundles/org.geonames.timezone  Geonames Timezone service].

==Implementing a Client Provider==

ECF's implementation of Remote Services exposes open, community-created APIs to make it easy to create and use a custom Client Provider.   What is a Client Provider?   It's a custom implementation of the consumer/client-side of the remote services distribution system.   An ECF client provider is able to substitute custom protocols/transports (e.g. http/https, MQTT, JMS, private protocols) and custom serialization formats (e.g. xml, JSON, object serialization, others), without having to implement the entire OSGi Remote Services and Remote Service Admin specification.   In an open, modular fashion, an ECF Client Provider can reuse any/all existing ECF or third party remote service provider functionality.   A Client Provider automatically is fully compliant with the OSGi Remote Services/RSA standards, while providing the flexibility to use new, custom, or even private transports.   [[OSGi Remote Services and ECF | Here is an architecture diagram]] showing the relationship between distribution providers, the Remote Services standards and ECF's extensible and open implementation.

===Integration with ECF Remote Service Admin===

The Remote Service Admin specification standardizes an API and mechanism for dynamic discovery of remote services.   ECF's implementation of this standard provides a way for new Client Providers to dynamically insert themselves into this discovery process.  The way this is done is to implement a new '''container''' class along with a '''container instantiator''', and to register a '''container type description''' instance as a local OSGi service using the [http://wiki.osgi.org/wiki/Whiteboard_Pattern whiteboard pattern].

For the example Timezone remote service the container class is called '''TimezoneClientContainer'''.  The full source for this class is available [https://github.com/ECF/Geonames/blob/master/bundles/org.eclipse.ecf.provider.geonames.timezone.client/src/org/eclipse/ecf/internal/provider/geonames/timezone/client/TimezoneClientContainer.java  here].

The role of the container instantiator is to create an appropriately typed instance of the container when needed by RSA.   When a remote service is discovered and then imported, the ECF RSA implementation takes the OSGi-define remote service properties and passes them to every available container instantiator, essentially asking:  'a container of given type has been discovered, and a new instance is needed, can you provide such an instance?'.   The implementation of this query is via the '''TimezoneClientContainer.Instantiator.getImportedConfigs''' method  (labelled in comment with '1').

<source lang="java">
	public static class Instantiator extends RestClientContainerInstantiator {
		/**
		 * 1. This method is called by the ECF RSA implementation when a remote
		 * service is to be imported. The exporterSupportedConfigs parameter
		 * contains the exported config types associated with the remote
		 * service. The implementation of this method decides whether we are
		 * interested in this remote service config type. If we are
		 * (exporterSupportedConfigs contains CONTAINER_TYPE_NAME, then we
		 * return an array of strings containing our CONTAINER_TYPE_NAME
		 */
		public String[] getImportedConfigs(
				ContainerTypeDescription description,
				String[] exporterSupportedConfigs) {
			/**
			 * If the exporterSupportedConfigs contains CONTAINER_TYPE_NAME,
			 * then return that CONTAINER_TYPE_NAME to trigger RSA usage of this
			 * container instantiator
			 */
			if (Arrays.asList(exporterSupportedConfigs).contains(
					CONTAINER_TYPE_NAME))
				return new String[] { CONTAINER_TYPE_NAME };
			return null;
		}
		/**
		 * 2. This method is called by the ECF RSA to create a new instance of
		 * the appropriate
		 * container type (aka OSGi config type)
		 */
		public IContainer createInstance(ContainerTypeDescription description,
				Object[] parameters) throws ContainerCreateException {
			return new TimezoneClientContainer();
		}
	}
</source>
.
The '''getImportedConfigs''' method logic is summarized as:  if my CONTAINER_TYPE_NAME matches one of the '''exporterSupportedConfigs''' then return a String[] including my CONTAINER_TYPE_NAME.   This approach provides the flexibility to support multiple '''exporterSupportedConfigs''' with a single container type. The ECF RSA implementation then turns around and calls the '''TimezoneClientContainer.'Instantiator.createContainer''' method, which in the implementation above (2) returns a new instance of the '''TimezoneClientContainer'''.

Once the TimezoneClientContainer instance is created, ECF RSA calls it's '''connect''' method with the '''targetID''' defined in the standardized meta-data for the remote service.   Here are the lines of the TimezoneClientContainer connect method explained

<source lang="java">
super.connect(targetID, connectContext1);
</source>

This line passes the RSA-created '''targetID''' to the RestClientContainer super class, which simply sets a member variable to the value of '''targetID'''.

<source lang="java">
setAlwaysSendDefaultParameters(true);
</source>

This line configures the container instance to always send the default parameters on the remote call.   This is necessary for the Geonames Timezone service because one of the required http request parameters is the '''username''' parameter.  Setting this to 'true' means that the default parameter values (username) are always sent as required by the Timezone service.

To automatically construct the proxy instance of the '''ITimezoneService''' and '''ITimezoneServiceAsync''' it's necessary to define the association between the '''getTimezone''' method and the Geonames Timezone request.  As documented [http://www.geonames.org/export/web-services.html#timezone  here], the Timzeone request consists of

#A complete URL for the request (protocol/hostname/port + path)
#The required parameters.  In the case of the Geonames Timezone service the parameters are
##latitude (aka 'lat')
##longitude ('lng')
##username (for authentication)

First we construct the necessary remote call parameters using a '''RemoteCallParameter.Builder'''

<source lang="java">
RemoteCallParameter.Builder parameterBuilder = new RemoteCallParameter.Builder()
		.addParameter("lat").addParameter("lng")
		.addParameter("username", USERNAME);
</source>

Then we construct an IRemoteCallable, defining the association between the '''getTimezone''' method and the '''/timezoneJSON''' remote service path, including the above-created parameters, and setting the request to use HTTP GET.

<source lang="java">
RemoteCallable.Builder callableBuilder = new RemoteCallable.Builder(
		"getTimezone", "/timezoneJSON").setDefaultParameters(
		parameterBuilder.build()).setRequestType(
		new HttpGetRequestType());
</source>

Finally, we register this IRemoteCallable and associate it with the '''ITimezoneService.class''' service interface

<source lang="java">
tzServiceRegistration = registerCallables(ITimezoneService.class,
		new IRemoteCallable[] { callableBuilder.build() }, null);
</source>

The above registration is all that's needed for the ECF remote service code to construct a proxy implementing '''ITimezoneService''' and '''ITimezoneServiceAsync'''.  When a consumer application actually calls either the '''getTimezone''' or '''getTimezoneAsync''' methods on the proxy, the following will occur automatically

#Construct the appropriate URL for calling the Geonames Timezone service:  '''http://api.geonames.org/timezoneJSON'''
#Serialize the parameters passed to the '''getTimezone''' method (i.e. 'lat' and 'lng') and add them as URL request parameters, resulting in the URL:  '''http://api.geonames.org/timezoneJSON?lat=<lat param value>&lng=<lng param value>&username=<username default value>'''
#Use the resulting URL to issue an HTTP GET request to the Geonames service

When the Geonames service responds to the GET request, it returns a single instance of a JSON Object container several fields with various information associated with the timezone...e.g. countryCode, countryName, lat, lng, timezoneId, dstOffset, etc.  The following code sets up a response deserializer, that parses the returned String using a common JSON parser.  Then given the data, it creates an instance of the '''Timezone''' class and returns it.   At remote method call time, when the Geonames service responds this method will be called to deserialize the HTTP response and return a proper Timezone instance to the caller of '''getTimezone'''.

<source lang="java">
		setResponseDeserializer(new IRemoteResponseDeserializer() {
			@Override
			public Object deserializeResponse(String endpoint,
					IRemoteCall call, IRemoteCallable callable,
					@SuppressWarnings("rawtypes") Map responseHeaders,
					byte[] responseBody) throws NotSerializableException {
				try {
					// Convert responseBody to String and parse using org.json
					// lib
					JSONObject jo = new JSONObject(new String(responseBody));
					// Check status for failure. Throws exception if
					// error status
					if (jo.has("status")) {
						JSONObject status = jo.getJSONObject("status");
						throw new JSONException(status.getString("message")
								+ ";code=" + status.getInt("value"));
					}
					// No exception, so get each of the fields from the
					// json object
					String countryCode = jo.getString("countryCode");
					String countryName = jo.getString("countryName");
					double lat = jo.getDouble("lat");
					double lng = jo.getDouble("lng");
					String timezoneId = jo.getString("timezoneId");
					double dstOffset = jo.getDouble("dstOffset");
					double gmtOffset = jo.getDouble("gmtOffset");
					double rawOffset = jo.getDouble("rawOffset");
					String time = jo.getString("time");
					String sunrise = jo.getString("sunrise");
					String sunset = jo.getString("sunset");
					// Now create and return Timezone instance with all the
					// appropriate
					// values of the fields
					return new Timezone(countryCode, countryName, lat, lng,
							timezoneId, dstOffset, gmtOffset, rawOffset,
							dateFormat.parse(time), dateFormat.parse(sunrise),
							dateFormat.parse(sunset));
					// If some json parsing exception (badly formatted json and
					// so on,
					// throw an appropriate exception
				} catch (Exception e) {
					NotSerializableException ex = new NotSerializableException(
							"Problem in response from timezone service endpoint="
									+ endpoint + " status message: "
									+ e.getMessage());
					ex.setStackTrace(e.getStackTrace());
					throw ex;
				}
			}
		});
	}
</source>

Note that deserialization code may do whatever is appropriate (e.g. parse xml, parse JSON, deserialize java Objects, use some other serialization technique like protocol buffers, etc) to convert from appropriate remote response (JSON) to an instance of the service interface ('''Timezone''').

The '''connect''' implementation above is sufficient for the TimezoneClientContainer to create and execute the Geonames Timezone service request, and handle/deserialize the JSON response, and associate this implementation with the '''ITimezoneService.getTimezone''' method and provided parameters.   Note that it's also possible to reuse other codebases (other than the RestClientContainer superclass) if the remote service is not REST-based, or requires more control and customization of the remote invocation process.

===Registering the Container and Namespace===

In the Activator for this Client Provider, there are two OSGi service registrations, which are necessary to hook into the ECF RSA implementation as described above

<source lang="java">
	public void start(BundleContext bundleContext) throws Exception {
		Activator.context = bundleContext;
		// Register an instance of TimezoneNamespace
		bundleContext.registerService(Namespace.class, new TimezoneNamespace(),
				null);
		// Register an instance of TimezoneContainerTypeDescription (see class
		// below)
		bundleContext.registerService(ContainerTypeDescription.class,
				new TimezoneContainerTypeDescription(), null);
	}

	class TimezoneContainerTypeDescription extends ContainerTypeDescription {
		public TimezoneContainerTypeDescription() {
			super(TimezoneClientContainer.CONTAINER_TYPE_NAME,
					new TimezoneClientContainer.Instantiator(),
					"Geonames Timezone Remote Service Client Container");
		}
	}
</source>

As described in comments, these two service registrations (Namespace and ContainerTypeDescription) setup the necessary structures for RSA to create the TimezoneClientContainer, call '''connect''' and other methods on the container at the appropriate times during remote service discovery, import, proxy creation, and remote service call/invocation.

The entire/complete provider, along with all meta-data are available in the ECF Github repository [https://github.com/ECF/Geonames/tree/master/bundles/org.eclipse.ecf.provider.geonames.timezone.client here].

==Testing the Timezone Remote Service==

To test the above defined Timezone Client Provider, we can use an OSGi ServiceTracker to create some code that uses the '''ITimezoneService'''.   The Remote Service consumer (application) code is appropriately in a separate bundle from both the Timezone service bundle (which contains the '''ITimezoneService''' and '''ITimezoneServiceAsync''' interface classes), and the Client Provider bundle defined above.    Here is a ServiceTracker.addingService method implementation, where the proxy will be made available when the proxy is created

<source lang="java">
public ITimezoneServiceAsync addingService(
		ServiceReference<ITimezoneServiceAsync> reference) {
	ITimezoneServiceAsync service = getContext().getService(reference);
	System.out.println("Got ITimezoneServiceAsync");
	// Get completable future and when complete
	service.getTimezoneAsync(47.01, 10.2).whenComplete(
			(result, exception) -> {
				// Check for exception and print out
				if (exception != null) {
					System.out.println(exception.getMessage());
					exception.printStackTrace();
				} else
					// Success!
					System.out.println("Received response:  timezone="+ result);
			});
	// Report
	System.out.println("Returning ITimezoneServiceAsync");
	return service;
}
</source>

Once the '''ITimezoneServiceAsync''' proxy has been retrieved, the '''getTimezoneAsync''' method is called with parameters latitude=47.01 and longitude=10.2.   With the whenComplete method and Java8 lambda, we provide the block of code that is executed asynchronously when the remote call completes

<source lang="java">
	(result, exception) -> {
		// Check for exception and print out
		if (exception != null) {
			System.out.println(exception.getMessage());
			exception.printStackTrace();
		} else
			// Success!
			System.out.println("Received response:  timezone="+ result);
	}
</source>

As from the above code, if an exception occurs then the message and stack trace are printed out.  If successful, the returned Timezone instance is printed out.

==Background and Related Articles==

[[Tutorial: Creating a RESTful Remote Service Provider | Creating a RESTful Remote Service Provider ]]

[[Getting Started with ECF's OSGi Remote Services Implementation]]

[[OSGi Remote Services and ECF]]

[[Asynchronous Proxies for Remote Services]]

[[File-based Discovery | Static File-based Discovery of Remote Service Endpoints]]

[[EIG:Download|Download ECF Remote Services/RSA Implementation]]

[[EIG:Add to Target Platform|How to Add Remote Services/RSA to Your Target Platform]]

[[ECF#Customization_of_ECF_Remote_Services_.28new_Discovery_and.2For_Distribution_providers.29 | Customization of ECF Remote Services]]