IoT/IoTConnectorProposal

This is a proposal to start a project under the Eclipse IoT umbrella for creating the IoT Connector component. After several weeks of review by the eclipse IoT community in this wiki we have formally proposed the IoT Connector project at eclipse on Dec 23rd, 2015. Once the proposal has been accepted (which we expect to happen during January 2016) you should be able to find it on the Eclipse Project Proposals page. The proposal will be available for public review for two to three weeks there. Feel free to add comments or add your name as an Interested Party if you would like to join the project or collaborate in any other way.

PLEASE DO NOT EDIT THIS PAGE ANYMORE! USE THE OFFICIAL ECLIPSE PROJECT PROPOSAL PAGE TO ADD COMMENTS.

---

Proposal

The following sections describe the intent and scope of the IoT Connector project we would like to start at Eclipse. The layout follows the Eclipse Create Project page.

Background

The open source community has produced a lot of excellent technology, frameworks and products that help with implementing IoT applications. A developer usually selects an appropriate set of technology and components and incorporates them into an application. The chosen components need to suport the implementation of all relevant aspects of an IoT solution including device connectivity, management, monitoring, business logic and last but not least security enforcement at all levels. In an enterprise context a typical solution is about connecting a given number of homogenous devices to a particular application hosted on dedicated server infrastructure. Over time this results in a set of independent silo applications each managing their own (limited) set of devices. As of today, most of the technology created in Eclipse IoT projects specifically supports the development of IoT applications in this way.

Bosch is a major manufacturer of all kinds of electronic devices, most of which (if not all) will have connectivity built into them in the near future. Integrating these devices with individual IoT applications as descibed above has several drawbacks:

• The repetitive implementation of common functionality like device communication and management is error prone and inefficient regarding costs, development time and runtime resources.
• The tight integration of devices with a specific application de facto partitions the set of things into classes determined by the application the devices have been intially integrated with. This makes it hard to create new cross-domain solutions and business models leveraging devices coming from different application domains.
• The applications implemented this way are often designed to interact with a limited number of devices in an enterprise environment only. Scaling out such applications with an increasing number of devices therefore often requires massive refactoring (if not re-architecting) of the device integration layer in order to support horizontal scalability as required in most cloud-based use cases.

Description

Connectivity is at the heart of IoT solutions. Devices (things) need to be connected to a back end component where the data and functionality of the devices is leveraged to provide some higher level business value. IoT solution developers can pick from a wide array of existing (open source) technology to implement a device connectivity & management layer for the particular type of devices at hand. While this is often fun for the developers to do, the resulting solutions are often silo applications lacking the ability to scale horizontally with the number of devices connected and the number of back end components consuming the device data and functionality.

The Eclipse IoT Working Group has therefore discussed a more generic, cloud-based IoT platform architecture which better supports the implementation of IoT solutions without requiring developers to solve some of the recurring (technical) challenges over and over again. The diagram below provides an overview of the IoT Server Platform as discussed in the working group.

The diagram shows how devices in the field are connected to a cloud-based back end either via a Field Gateway (e.g. something like Eclipse Kura) or directly to so-called Protocol Adapters. The Protocol Adapters' responsibility is abstracting communication protocols as well as providing location transparency of devices to the other back end components. The devices upload (sensor) data to the back end while the functions/services they expose can be invoked from the back end. These two directions of information flow can be characterized as follows:

• Telemetry
  Data flowing upstream (left to right) from devices to the back end to a consumer like a Business Application or the Device Management component usually consists of a small set of discrete values like sensor readings or status property values. In most cases these messages are one-way only, i.e. devices sending this kind of data usually do not expect a reply from the back end.
• Command & Control
  Messages flowing downstream (right to left) from back end components like Business Applications often represent invocations of services or functionality provided by connected devices, e.g. instructions to download and apply a firmware update, setting configuration parameters or querying the current reading of a sensor. In most cases a reply to the sent message is expected by the back end component.

It seems reasonable to assume that the number of messages flowing upstream (Telemetry) will be orders of magnitude larger than the number of messages flowing downstream (Command & Control). The aggregated overall number of messages flowing upstream is expected to be in the range of several hundred thousand to millions per second. Note that in this architecture the same (cloud-based) infrastructure is shared by multiple solutions.

The IoT Connector provides the central link between the device-facing Protocol Adapters, additional re-usable back end components, e.g. Device Management or Software Provisioning, and last but not least the IoT solutions leveraging the devices' data and services. Solution developers can use the IoT Connector to uniformly and transparently interact with all kinds of devices without the need for caring about the particular communication protocol(s) the devices use. Multiple solutions can use the same IoT Connector instance running in a shared cloud environment in order to share the data and functionality of all connected devices. The IoT Connector ensures that only those components can consume data and control devices that have been granted authorization by the device owner. In this regard the IoT Connector can be considered an IoT specific message broker targeted at cloud deployment scenarios.

Features at a glance

• Secure message dispatching
• Support for different message exchange patterns
• Used for cloud service federation
• Provides interfaces to support implementation of protocol adaptors which allow:
  • Sending telemetry data
  • Receiving device control messages (from applications/solutions)
  • Registering authorized consumers of telemetry data received from connected devices

Scope

The IoT Connector is at the heart of the IoT Server Platform as illustrated in the Description section. It provides the link between the downstream Protocol Adapters and the upstream IoT solutions and other re-usable components leveraging the devices' data and services.

The IoT Connector needs to fulfill a set of non-functional requirements, in particular regarding horizontal scalability, that are specific to both the deployment environment (cloud) and the intended architectural platform characteristics (as opposed to embedding a connectivity layer into applications individually). However, these requirements are not specific to any particular application domain. From a technical point of view it makes no difference if a sensor reading received via a LWM2M protocol adapter represents a temperature or the relative humidity. In both cases the IoT Connector's responsibility is to forward the messages containing the values to (potentially multiple) authorized consumers without introducing too much latency.

Acknowledging the fact that there already exist many message brokers both in the open source world as well as by means of commercial offerings either to be deployed on premises or provided as cloud services, it is not the project's intention to create an additional message broker implementation. Instead, the goal is to create a uniform (remote) service interface that supports both the Telemetry as well as Command & Control message exchange pattern requirements. In order to do so, the IoT Connector also introduces a standard service interface for managing the identity and access restrictions.

The IoT Connector project should provide an initial set of implementations of the service interfaces described above, leveraging existing messaging infrastructure components. As a starting point we would like to provide an implementation based on RabbitMQ because of its easy availability both as a service in existing Cloud Foundry based environments as well as in the form of pre-built Docker images. In order to also provide an implementation supporting horizontal scale-out, we would also like to create an implementation based on Apache Kafka. Future versions may also support using other cloud-based offerings (e.g. Microsoft's Azure Message Bus or Amazon's Simple Queue Service).

People

Project Lead(s)

•  ???

Committers

•  ???

Mentors

•  ???

Interested Parties

• Bosch Software Innovations GmbH
• Red Hat, Inc.
• innoQ Deutschland GmbH (Thomas Eichstädt-Engelen)

Why Here?

The Eclipse IoT Working Group already serves as an incubator for projects and technology that helps with the development of IoT solutions. The IoT Connector will leverage some of this technology, in particular for the implementation of protocol adapters, while also adding missing but fundamental pieces to an open source cloud-based IoT Server Platform.

Some of the IoT Working Group's current member companies already have expressed an interest in collaborating on the IoT Connector and we have also started discussions with other prospect companies regarding their involvement in both the project as well as the IoT Working Group.

The following list includes Eclipse IoT projects that provide technology relevant for the implementation of the IoT Connector:

• leshan
• Californium
• Mosquitto
• Paho

Licenses

Eclipse Public License 1.0 [1]

Initial Contribution

The initial contribution will contain a ready-to-run messaging component and a java client implementation to interact with this service.

The java client includes the following functionality:

• establish a connection to a RabbitMQ broker
• managing authorization information per topics
• sending messages
• register topic based handlers for receiving messages

The message component includes the following functionality:

• store all authorization information
• accept all incoming messages and
• dispatch messages to all authorized back end components.

All contributed components are structured into multiple maven modules.

The messaging service is proven to run stand alone (executable jar) or as a Docker container.

The java client implementation can be used as a java library.

The messaging component depends internally on RabbitMQ as broker and the java client on the AMQP 0.9 protocol.

Naming Proposals

Please feel free to add your proposal for a proper name (IoT Connector sucks big time ;-)).

• Mycelium