Streaming API

The Trackunit Streaming API is a near-realtime, low latency API, that enables customers to create their own data lake and services, leveraging Trackunit as Telematics Data Service Provider. This document describes the overall design and how to connect to the Streaming API.

📘

Available by request

The Streaming API is only available by request as an add-on service. Customers need to be on an "Insight" subscription level and set-up costs/API license pricing will be handled by the account manager. Once commercials are agreed upon Trackunit support can be contacted to set up the Streaming API functionality.

Design

The Trackunit Streaming API is implemented as a Kafka-based solution, backed by Confluent Cloud and AWS infrastructure, in order to be highly available and performant. When a customer signs up for access to the Streaming API, a Kafka topic is created and a Flink job starts forwarding data, related to the customer's machines, to that topic. Data is then available for the customer to consume. The data is kept on the topic for 14 days.
Only the data related to the customer's machines will be made available for the customer.

Data is available in the Streaming API with low latency after it enters the Trackunit Iris platform. There can be delays between the telematic devices (or 3rd party integrations such as ISO feeds) and the Iris platform due to network connectivity or similar. The common case is however that data is available with near-realtime delays.

Data Formats and Compatibility

Data is available in two formats:

• Machine Insights-based: High-level semantic information about machines, normalized and cleansed. Immediately ready to ingest into a data-lake, or for use in AI-based analytics.
• RAW data: As a direct replacement for the classic Unit APIs, this format is a direct migration path from those.

We recommend the Machine Insights based topic for any future integrations.

Technically, the data is provided in the binary Avro dataformat which is very efficient, while at the same time allowing us to extend the dataformat in the future with additional datapoints. A Schema Registry is used to allow producers and consumers to evolve while maintaining compatibility. We will only do compatible changes to the dataformats and stay compatible with the previous version of the schemas, and announce any changes. We recommend to stay up to date, and implement these schema changes in a timely fashion in order to have access to all the available datapoints.

How to connect

The technical solution is a Kafka cluster with Avro messages using a Schema Registry. Confluent provides a number of examples in various programming languages. We provide a customized and complete example of how to connect through Java.

As part of the introductory package we provide the following parameters to get started: