Introduction to the Span service

Span is a service that sits between your IoT devices and your backend system. The primary purpose of Span is to manage and coordinate the communication between the backend and your devices. It is primarily designed for low-power devices. A backend that communicates with low-power devices requires both strict timing requirements since devices will be online only for very short periods of time when they are sending and receiving. The availability requirements will also be quite high since the backend must be available whenever a device sends data.

In the majority of the cases the biggest draw on power will be the connectivity itself and a single retransmit of data or DNS lookup will be quite expensive for the device. Span remedies this by offloading as much as possible of the device configuration from the device. A typical embedded low-power device will use message- oriented protocols as opposed to typical Internet devices (and backends) which will be online all the time and uses protocols with both more state and bigger payloads.

Implementing a backend provides its own set of challenges, mostly by handling the device state and metadata for the device. Rather than having a raw stream of payloads Span augments the messages with additional metadata that the backend can use to process the data.

Span takes care of message retention for shorter or longer periods of time so if your backend is down for a short or longer period of time you can replay the messages that are sent while the backend is down. At the same time the latency for messages is in the single digit millisecond range for the backend itself so you won’t introduce massive delays in data delivery.

Concepts

A device in Span is - very simply - “something connected to the outside world”. A device can send and receive messages to a backend somewhere else. A device can be anything from a sensor somewhere in the world to an entire building, it’s all a matter of definition. The backend is a service somewhere on the Internet that processes data from the devices.

Devices are grouped in collections. A collection is just a logical grouping of devices. A typical grouping would be “production”, “test” and “development”. Each collection operates independently. A device can only be a member of one collection.

Both collections and devices can have additional metadata. This is called tags in the provide custom metadata. The metadata can be useful if you want to associate additional information with the device or collection like external system IDs or sensor calibration data.

Devices sends messages to and from the backend and the payload of the messages are just bytes. They are opaque for the service. Each collection can have one or more Data Routers that will forward messages to the backend.

Backend Interfaces

Span presents different interfaces for the devices and backends. The backends are mostly online all the time while the devices will spend most of their time disconnected.

Management API

The management API lets you manage devices, collections, data routers and messages going to and from the devices. There are clients for several languages including Java, C#, Go, Rust and TypeScript. The API itself is available as an OpenAPI spec if you need a different client.

MQTT Broker

By default all messages from the devices will be available in a built-in MQTT broker. If you require a custom structure for the MQTT topics or payload you can customise this with a data router configuration.

Data Routers

The data routers allows you to link up one or more backend systems to Span. Each backend system operates independently from each other so if you are migrating from one system or adding a new one you can do this without changing the configuration for the devices.

Device Interface

The device interface is quite a bit different from the backend interface. The priority of these interfaces are simplicity for the device.

If you use a cellular IoT device with NB-IoT or LTE-M the endpoints the device use is the same across all operator networks. If you change the SIM card for the device from one operator to another the device won’t need any changes when sending or receiving data.

Similarly if the device jumps from a cellular network to WiFi the existing backends won’t need any additional configuration.

The cellular IoT integration in Span is done on the APN level and the Span service manages the connectivity on your behalf via the APN. Since Span is responsible for the address allocations on the mobile network the device does not need any form of authentication or authorisation when sending packets. In the simplest configuration the device can send a single UDP packet with a minimum payload and the payload will show up in Span on the correct collection and device.

Currently Span offers two alternatives for devices: An UDP endpoint or a CoAP endpoint. The CoAP endpoint is also used for Firmware Updates Over the Air (FOTA). Span handles the firmware images automatically and a firmware version can be assigned to a single device or for the entire collection.

Project Life Cycle Support

Another goal for Span is to provide appropriate APIs and features for the entire project life cycle.

Initial Prototype

When prototyping a new device or implementing a technology spike you typically create a device with the bare minimum of features. This might be a quite crude prototype and you are mostly concerned with the hardware itself. The prototype is commonly implemented on a breadboard and its primary purpose is to determine if it is a viable solution or not. At most this is maybe just a single device and maybe a quickly hacked together backend that might not be much more than a simple charting application or web page that displays the received data. The data won’t be stored or processed, just displayed in real time.

The firmware is still in active development and any updates are flashed to the device there and then.

For this purpose you can build an entire prototype that consists of a web page uploaded somewhere and a web socket to read the data from the device. There’s no need for a dedicated host or backend running at this point.

Field Test

The next step might be a small scale field test, typically with a handful of devices. The devices might be deployed relatively close to the office or locally (if you are lucky) and the firmware is a bit more solid and the hardware might be soldered breadboards or a simple PCB and the firmware. Work on the backend have started and you have to worry about data retention, reliability and updates. Firmware updates can still be done with an USB cable but it might involve some ladders and inaccessible locations.

At this point you can use one of the data routers in Span or the MQTT broker for data from the devices to your backend. Since Span handles message retention you won’t have to worry about backend updates. The backend gets bigger and more complex and the start of the first integrations might happen at this point.

Towards the end of the field test the firmware is approaching its final state and the FOTA implementation is finalised.

Pre-Production

The next step might be a bigger field test or a limited pre-production run. The hardware is mostly done, the firmware has most of the quirks done and focus is more on the backend and systems that integrate with this.

This might include anything from a few hundred to a few thousand devices in the field and the firmware might see a new revision or two.

Production

Finally - the system is in production. From here on the bugs are probably mostly in the user-facing systems but in the long run you might need some firmware updates for the entire fleet of devices if some kind of critical bug appears.

Later on in the production phase you might release a second version of the hardware as part of the normal evolution but these can be tested independently from the existing devices in a new collection.

Some of the integrations might change but they can be replaced without configuring the devices or other backend systems.