Eclipse Ditto 2.1.0

2.1.0

Description

Eclipse Ditto 2.1.0 includes the following topics/enhancements:

  • Support consuming messages from Apache Kafka -> completing the Apache Kafka integration as fully supported Ditto managed connection type
  • Conditional requests (updates + retrievals)
  • Enrichment of extra fields for ThingDeleted events
  • Support for using (HTTP) URLs in Thing and Feature "definition" fields, e.g. linking to WoT (Web of Things) Thing Models
  • HMAC based authentication for Ditto managed connections, see Blogpost
  • SASL authentication for Azure IoT Hub
  • Publishing of connection opened/closed announcements
  • Addition of new "misconfigured" status category for managed connections indicating that e.g. credentials are wrong or connection to endpoint could not be established to to configuration problems
  • Support "at least once" delivery for policy subject expiry announcements

The following notable fixes are included:

  • Fix "search-persited" acknowledgement not working for thing deletion
  • Fix reconnect loop to MQTT brokers when using separate MQTT publisher client 

The following non-functional work is also included:

  • Support for tracing reporting traces to an "Open Telemetry" endpoint
  • Improving cluster failover and coordinated shutdown + rolling updates
  • Logging improvements, e.g. configuring a logstash server to send logs to or more options to configure a logging file appender
  • Improving background deletion of dangling DB journal entries / snapshots based on the current MongoDB load
  • Improving search update by applying "delta updates" saving lots of bandwith to MongoDB
  • Reducing cluster communication for search updates using a smart cache
Architectural Issues

Ditto's codebase is implemented in Java. By default Java objects are mutable which does not work well in highly scalable, message driven architectures, as possible side-effects during runtime may occur when concurrently modifying objects. Because of that the Ditto team decided to use immutable objects wherever and whenever possible. Immutablity of the objects is ensured in unit tests.

Ditto's model modules and also the Ditto Java client are OSGi bundles so that they may be used in OSGi environments without much effort. The model modules are configured to be checked by a "binary compatibility checker" in the Maven build so that APIs are not broken unintentionally.

On a level higher than the model, Ditto uses a microservice based architecture. As Ditto's microservices are interacting via an event driven approach, Ditto provides a very modular setup on the microservice level meaning that single services must be not started at all if their functionality is not needed. Another benefit from this architecture is that the services may be scaled horizontally if more resources are required.

Ditto utilizes the CQRS and EventSourcing pattern and mainly uses "inserts" (append only) into the database in favor to "updates" in order to get a better performance when doing database writes. The only exception is the "search" microservice which uses traditional CRUD in order to update its search index.

Security Issues

There are no security issues known at the time of this writing.

Non-Code Aspects

Ditto comes with:

Conforms To UI/UX Guidelines
Not applicable (project doesn't provide UI)
Usability Details

Ditto does not provide a UI yet.

End of Life

None.

Standards

Ditto is able to process AMQP 1.0, AMQP 0.9.1, MQTT 3.1.1 and MQTT 5 messages which are all standardized (IoT) protocols.

Ditto can use JSON Web Tokens specified by RFC 7519 in order to extract "subjects" (e.g. a user-id) to use for the access control to twins.

Ditto's API documentation is defined using the OpenAPI specification 3.0.0 allowing both creation of an interactive HTML-based documentation and creation of skelettons for various programming languages based on that specification.

Communities

Users of Ditto ask their questions via these channels:

  • The Gitter chatroom of Ditto
  • Directly via GitHub issues (the Ditto teams labels with "question")
  • A Stackoverflow tag "eclipse-ditto" on which community quersions are answered by our committers
  • The mailing-list (ditto-dev@eclipse.org) contains mainly administrational topics regarding the Ditto project

Ditto seeks for a good integration with Eclipse Hono in order to being able to create digital twins for all devices connected via Hono. The Ditto team is also in regular discussion with the Hono team.

Some of the Ditto team are committers in the Eclipse IoT Packages project aiming to simplify deployment and integration between Eclipse Hono and Ditto.