IoT/IoTServerPlatform

Introduction

At EclipseCon NA 2015 a number of IoT WG members meet to discuss the definition of an open source IoT Server Platform. The general view was that a lot of the open source technology has been focused on the devices and edge but not the server and enterprise middleware. The long-term goal of this discussion would be to determine what open source technology need to be created to support an open source IoT server platform.

Out of this meeting, the main feature/components of an IoT Server platform would include:

• Communication/Protocol adapters
• Device Management
• Data storage (in raw form)
• Event Management/Rules Engine/BPM
• User Management/Identity/Security
• API

The next steps for this discussion needs to be to

1. add detail to each feature/component,
2. map how this integrates and extends the existing open source middleware that exists in many server solutions, and
3. map this to cloud computing solutions.

Context

Following the Arc42 approach, we start with some simple context views to illustrate the surrounding actors and components an IoT server platform needs to deal with.

Business Context

The diagram shows how multiple (potentially millions of) Devices are connected to the Iot Server Platform. The devices upload (sensor) data to the back end while the functions/services they expose can be invoked from the back end. These two directions of information flow can be characterized as follows:

1. Telemetry
   Data flowing upstream (left to right) from devices to the back end to a consumer like a Business Application or the Device Management component usually consists of a small set of discrete values like sensor readings or status property values. In most cases these messages are one-way only, i.e. devices sending this kind of data usually do not expect a reply from the back end.
2. Command & Control
   Messages flowing downstream (right to left) from back end components like Business Applications often represent invocations of services or functionality provided by connected devices, e.g. instructions to download and apply a firmware update, setting configuration parameters or querying the current reading of a sensor. In most cases a reply to the sent message is expected by the back end component.

It seems reasonable to assume that the number of messages flowing upstream (Telemetry) will be orders of magnitude larger than the number of messages flowing downstream (Command & Control).

Business Applications use the Iot Sever Platform as a proxy for accessing the devices' data and functionality. Applications can subscribe to particular events like notifications about data changes or device life-cycle events. This way applications do not need to deal with the technical and semantical specifics of the devices and their particular data models and communication protocols. Both of these aspects are abstracted away by the IoT Server Platform.

The data gathered from the devices can (optionally) be persisted in third party Data Stores. Arbitrary persistence strategies are possible, e.g. flat file, relational database, document store etc. The IoT Server Platform does not provide any means to store data out-of-the-box because it is expected that Business Applications will usually implement a their own, domain specific persistence layer anyways.

The IoT Server Platform uses an Identity Provider to authenticate incoming requests from Business Applications. (Note: The Identity Provider should probably also be used to get a user's authorities as well).

Technical Context

TBD

Building Blocks

The top level building block view provides a white box view of the overall IoT server platform. All of its components are shown as black boxes only, i.e. at this level their internal structure is not relevant (yet) but instead the focus lies on responsibilities and relationships. If necessary, the internal structure of each of the components (the white box view) can be documented in subsequent sections, thus forming an alternating row of black box/white box views providing a drill down into the inner workings of the system.

Current Draft Building Block View

The draft diagram below provides an overview of the (logical) components of the IoT Server Platform. As a main difference to the previous building blocks view, the former IoT Hub has been renamed to IoT Connector while at the same time its relevance for the overall architecture has been stressed out by placing it at the very center of the system, serving as the central messaging infrastructure for connecting all of the other components with each other.

Please note that the implementation of the Iot Connector is not subject of the current discussion. Instead, the focus currently should be on the IoT Connector's role, purpose and responsibilities. It might very well turn out that we end up with multiple implementations for the IoT Connector's exposed interfaces, specifically targeted at the requirements of the use case at hand, e.g. we might want to have a lightweight implementation for use cases where only a limited number of devices will be connected and only a moderate number of messages need to be exchanged whereas we would also want to have a horizontally scalable implementation targeted at cloud scenarios as well.

TODO: Update diagram to follow UML Component Diagram notation.

Blackbox LWM2M Protocol Adapter

Purpose	Provides transparent access to LWM2M based devices. Maps data structures exchanged with LWM2M devices (based on LWM2M Object definitions) to/from normalized messages based on Vorto Information Models. Serves as a gateway to data and services provided by connected Devices for Business Applications.
Provided Interfaces	LWM2M Server for managing Devices based on the LWM2M protocol LWM2M Bootstrap Server for provisioning LWM2M based Devices with configuration data and security credentials Command & Control (downstream) for invoking services represented by LWM2M Object resources on connected Devices
Required Interfaces	Vorto Repository for retrieving Information Model definitions and their mapping to/from LWM2M Object definitions. This information is used to map messages exchanged with LWM2M Clients (devices/sensors) to/from normalized messages exchanged with back end components like Software Provisioning and Business Applications. Telemetry Ingestion (upstream) used to forward normalized messages based on Vorto Information Models to upstream subscribers
Quality Attributes	Supports horizontal scale-out Asserts client identity and authorization to connect

Blackbox IoT Connector

IMPORTANT
We are currently creating a proposal for starting a project to create the IoT Connector under the Eclipse IoT umbrella. You are more than welcome to contribute your thoughts & ideas to improve the proposal. We would also like to attract more people as interested parties and/or collaborators. Feel free to get in touch on the IoT Working Group mailing list or simply add your name to the proposal directly :-)

Purpose	Provides access to upstream telemetry data produced by Devices Provides means to invoke services provided by connected Devices De-couples data producers (Devices) from data consumers (e.g. Business Applications)
Provided Interfaces	Command & Control (downstream) for transparently invoking services on devices Telemetry Ingestion (upstream) for receiving telemetry data from connected Devices Subscription Management for registering authorized upstream consumers of telemetry data received from connected Devices
Required Interfaces	Identity Provider for authorizing back end components' access to Device data (both consuming telemetry data as well as service invocations) Command & Control (downstream) for invoking services on Devices Telemetry Ingestion (upstream) for forwarding telemetry data to interested upstream consumers like Business Applications
Quality Attributes	Data from Devices is only forwarded to authorized consumers High throughput Support for dozens (hundreds?) of parallel consumers Scales out horizontally Consumers do not need to know about the specifics of how a Device is connected and where it is located

Black Box Device Management

Purpose	Provides uniform access to data and services of all types of devices regardless of communication protocol and location Keeps a registry of all device meta-data like manufacturer, model, serial number, location, Information Model etc.
Provided Interfaces	Device Access Device Administration Access Control Telemetry Ingestion (upstream) for receiving telemetry data from connected Devices
Required Interfaces	Vorto Repository for retrieving Information Model definitions. This information is used to encode and decode normalized messages exchanged with other back end components like Protocol Adapters. Identity Provider for authenticating and authorizing requests Command & Control (downstream) for transparently invoking services on devices Subscription Management for subscribing to telemetry from Devices
Quality Attributes	Enforces device data privacy

Black Box Software Provisioning

Purpose	Provides means to plan, perform and monitor large scale roll-out of software packages to connected Devices
Provided Interfaces	Telemetry Ingestion (upstream) for receiving life-cycle events for connected Devices TBD
Required Interfaces	Command & Control (downstream) for orchestrating the software provisioning process on connected Devices which usually requires setting configuration properties and invoking functionality on Devices Subscription Management for subscribing to life-cycle events for connected Devices TBD
Quality Attributes	TBD

Previous Draft Building Block View

Blackbox LWM2M Connector

Purpose	Provides transparent access to LWM2M based devices. Maps data structures exchanged with LWM2M devices (based on LWM2M Object definitions) to/from Vorto Information Model instances. Serves as a gateway to data and services provided by connected devices for Business Applications.
Provided Interfaces	LWM2M Server (as defined by the LWM2M spec) LWM2M Bootstrap Server (as defined by the LWM2M spec) Device Access
Required Interfaces	InfoModel Repository Event Consumer (for notifying subscribers)
Quality Attributes	Supports horizontal scale-out

Blackbox Event Hub

Purpose	Provides means for Business Applications to subscribe to data from devices and thus getting notified about events produced by devices De-couples data producers (Devices) from data consumers (e.g. Business Applications)
Provided Interfaces	Subscription Management Event Consumer (for receiving notifications from Protocol Adaptors)
Required Interfaces	Event Consumer (representing the event consumers, e.g. Business Applications or the Device Management component)
Quality Attributes	Data from Devices is only forwarded to authorized consumers High throughput Support for dozens (hundreds?) of parallel consumers Scales out horizontally

Black Box Device Management

Purpose	Provides uniform access to data and services of all types of devices regardless of communication protocol and location Keeps a registry of all device meta-data like manufacturer, model, serial number, location, Information Model etc.
Provided Interfaces	Device Access Device Administration Event Consumer (for receiving notifications from devices) Access Control
Required Interfaces	InfoModel Repository Identity Provider (for authenticating and authorizing requests) Device Access (for accessing a Device's data and services)
Quality Attributes	Enforces device data privacy

Communication/Protocol adapters

• Which protocols should be supported?
• Authentication? authorization?
• Encryption?

white box view (TBD)

Event Hub

white box view (TBD)

Device Management

• Which protocols should be supported?
• Provisioning
• Bootstrapping

white box view (TBD)

Data storage (in raw form)

• Pure key-value store vs. time-series database?
• What querying capabilities are needed?

white box view (TBD)

Event Hub

• TBD

white box view (TBD)

Identity Provider

• TBD

white box view (TBD)

External Interfaces

• Existing (standard) APIs to leverage?

Meetings

The IoT Working Group members will be having regular meetings to discuss the requirements and action items to establish an open source IoT server platform. Participation to these meetings is for Working Group members only, but minutes are public.

Relevant Technology Components

• Communication, Device Management, API
  • ECF's implementation of OSGi Remote Services.
    • As demonstrated by commercial adoption/usage of Remote Services, by OSGi Alliance efforts in IoT and by various tutorials, Open implementations of OSGi standards are already being heavily used in both server development and in IoT.
      • Communication - ECF's RS impl protocol-independence is ideal for dealing with a heterogeneous IoT networking/communication/integration environment. Further ECF's impl already has MQTT/Paho-based and websockets-based Remote Service providers and we or others can easily add others based upon other IoT protocols.
      • Device Management - Remote Services provides a standardized, open, elegant, and simple solution (persistence is another orthogonal aspect to device management).
      • API - The use of OSGi Services and Remote Services provides an open, elegant, time-and-use-tested, dynamic, version-enabled approach to IoT API definition.