SMILA/Documentation/Using The ReST API

From Eclipsepedia

Jump to: navigation, search

Most functionality of SMILA is already acessible via an HTTP ReST API using JSON to represent data. This means basically that you can control and monitor a running SMILA server using your web browser (at least with some small plugins). Also, it is relatively simple to talk to SMILA programmatically or using scripts. On this page we will recommend some tools and best practices to work with SMILA.

Feel free to extend this page if you find interesting tools that you want to recommend to other SMILA users.

Contents

Basics

A HTTP ReST API consists of a set of URLs that can be invoked using standard HTTP requests. One URL refers to a "resource" in the system, which may be a service or some element managed by a service. Each resource may support several HTTP methods for different operations, not each resource will support every method. In SMILA we use the following methods, usually with one the described semantics:

  • GET: get the content, definition or description of the resource; get statistic data or state information
  • POST: execute the main functionality of the resource (e.g. execute a pipeline); add or update a sub element of the resource
  • PUT: set the content, definition or description of the resource
  • DELETE: clear the contents of the resource; remove an element from the resource.

Most resources should at least support the GET method which is the method used by the web browser when you enter the URL, and the result should contain links to resources associated to this resource. So you can explore the system state with your web browser.

GET and DELETE requests never have a request body, parameters must be passed by using ...?param1=value1&param2=value2 in the URL. With POST and PUT requests the data is passed in the request body as a JSON object, i.e. a JSON string like this:

{
  "param1": "value1",
  "param2": "value2
}

In SMILA this corresponds to the metadata part of a record. In POST requests it is also possible to add binary attachments, see Documentation of the SMILA HTTP server for details. Finally, some resources accept or produce so-called "JSON bulks". This means that it is possible to send or get a bunch of records in a single request. The JSON of the record metadata must then be printed on a single line, and the records are separated by newlines. However, it is not possible to add attachments to such records.

See SMILA/REST_API_Reference for a list of available resources and links to documentation.

Interactive Tools

Web Browser Add-Ons

For most web browsers exist free plugins or add-ons that support pretty-printed viewing of JSON documents and interactive invocation of ReST commands.

  • JSON viewer: By default browsers do not know how to handle JSON results and will suggest to download them to a file. There are add-ons that display the JSON right in the browser, nicely formatted and highlighted and make contained URLs clickable, which makes it easy to explore the system state
  • ReST Client: ReST Client allow to compose other requests than GET requests with the browser. They allow to enter the resource URL, select the request method, and add the request body content, and then you can execute the request and view the result. There are lots of these add-ons out there, we can recommend the following:
    • Firefox: REST Client
    • Chrome:
      • cREST Client: Easy to use and keeps a nice history of the executed requests so that it is easy to repeat sequences of commands for test purposes.
      • Advanced REST client application: A bit harder to use, but the only one we currently know of that lets you create "multipart" requests containing record attachments: In the following screenshot we have created a request to submit a record (first file field contains name of a JSON file) with an attachment (second file field) to the "indexUpdate" job:

SMILA-SendAttachmentWithAdvancedRestClient.png

Shell scripting

Basically, it is possible to work with the ReST API using generic HTTP tools like cURL. However, for improved usability you might want to check out resty. This is a simple wrapper (for bash and zsh) for cURL that allows you to write ReST requests like shell commands:

# initialize
> resty http://localhost:8080 
# get entry page
> GET /smila                   
# define a job
> POST /smila/jobmanager/jobs '{"name":"myJob","workflow":"myWorkflow","parameters":{ ... }}' 
# etc.

See the resty web site for more information.

Programmatical Access

It's quite easy to use the SMILA API from external programs. In Java, basically everything you need is available in packages java.net. But it's easier using the SMILA data model classes and JSON utilities and a HTTP library like the Apache HttpClient. Below we provide some simple examples for doing requests with it.

There are also free Java libraries available to make working with JSON ReST APIs easier, e.g. Resty (yes, same name, but another tool ;-). However, we do not have experience with them.

Apache HttpClient 3.1

Using HttpClient 3.1 (included in SMILA), invoking a pipeline could be done like this:

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.StringRequestEntity;
import org.eclipse.smila.datamodel.ipc.IpcAnyReader;
import org.eclipse.smila.datamodel.ipc.IpcAnyWriter;
import org.eclipse.smila.datamodel.AnyMap;
 
HttpClient client = new HttpClient();
IpcAnyReader jsonReader = new IpcAnyReader();
IpcAnyWriter jsonWriter = new IpcAnyWriter();
 
AnyMap request = ...; // create request
String jsonString = jsonWriter.writeJsonObject(request);
PostMethod method = new PostMethod("http://localhost:8080/smila/pipelines/MyPipeline/process");
method.setRequestEntity(new StringRequestEntity(jsonString, "application/json", "utf-8"));
try {
  client.executeMethod(postMethod);
  if (method.getStatusCode() == HttpStatus.SC_OK) {
    AnyMap result = (AnyMap) jsonReader.readJsonStream(postMethod.getResponseBodyAsStream());
    ... // use result.
  } else {
    ... // handle error.
  }
} finally {
  method.releaseConnection();
}

Apache HttpClient 4.x

Using HttpClient 4.x (currently NOT included in SMILA), invoking a pipeline could be done like this:

import org.apache.http.HttpResponse;
import org.apache.http.HttpStatus;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
 
HttpClient client = new DefaultHttpClient();
IpcAnyReader jsonReader = new IpcAnyReader();
IpcAnyWriter jsonWriter = new IpcAnyWriter();
 
AnyMap request = ...; // create request
String jsonString = jsonWriter.writeJsonObject(request);
HttpPost method = new HttpPost("http://localhost:8080/smila/pipelines/MyPipeline/process");
method.setEntity(new StringEntityjsonString, "application/json", "utf-8"));
try {
  HttpResponse response = client.execute(method);
  if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
    AnyMap result = (AnyMap) jsonReader.readJsonStream(response.getEntity().getContent());
    ... // use result.
  } else if (response.getEntity() != null) {
    EntityUtils.consume(response.getEntity()); // be sure to read the response in error cases, too.
    ... // handle error.
  }
} catch (Exception ex) {
  request.abort();
}