The software behind complex applications, such as robots and cars, is best described as the composition of many, independent, interconnected smaller blocks. Developing an application following this model yields many benefits: (1) separation of concerns — also known as the "single responsibility" principle, (2) parallel developments — developers can work on different blocks at the same time, (3) code reuse — the same building block can be reused in several applications with similar context, (4) evolvability — the entire software does not have to be updated at once but can be replaced block by block. This model is more formally known a data flow programming.
There already exist successful open-source projects that leverages this approach. Perhaps the most famous in the robotics field is ROS, the Robot Operating System.
However, to the best of our knowledge, none of the frameworks we have studied provide (a) structural guarantees, i.e. the flow of data described (either in the code or in some configuration) matches exactly what is deployed; and (b) the capabilities needed to address safety-critical requirements, i.e. fine-grained control over: the flow of data between blocks, the execution of each block — be it on which device and, within the device, on which core/thread — or temporal constraints with, for instance, deadlines.
Eclipse Zenoh-Flow provides a data flow programming framework that simplifies and structures (i) the declaration, (ii) the deployment and (iii) the writing of complex and, potentially, safety-critical applications that can span from the Cloud to the Microcontroller.
This Eclipse Zenoh-flow data flow programming framework exposes semantics to:
- define reusable data-flow operators, sources and sinks,
- declaratively express data-flow computations,
- distribute your application across your infrastructure,
- control the execution of the blocks that compose it, specifically:
- which block(s) should execute next and/or in parallel,
- optionally prioritise the execution of block(s),
- where each block should execute (at the device level and within each device, at the CPU level),
- assign time constraints to each block's execution.
In addition to these semantics, this project will take care of enforcing the requirements expressed by developers, leveraging (when applicable) the capabilities of the hardware on which the application will run.
It is in this project focus to provide an easy to use Application Programming Interface and sane defaults. It is impossible to abstract away all in the inherent complexity of safety-critical applications but we believe having both properties is key to user adoption.
Eclipse Zenoh-Flow aims at simplifying and structuring (i) the declaration, (ii) the deployment and (iii) the writing of complex and, potentially, safety-critical applications that can span from the Cloud to the Microcontroller.
To these ends, Eclipse Zenoh-Flow leverages the data flow programming model — where applications are viewed as a directed graph of computing units, and Eclipse Zenoh — an Edge-native, data-centric, location transparent, communication middleware. This makes for a powerful combination as Zenoh offers flexibility and extensibility while data flow programming structures computations. The main benefit of this approach is that this allows us to separate the applications from the underlying infrastructure: data are published and subscribed to (automatically with Zenoh-Flow) without the need to know where they are actually located.
Eclipse Zenoh-Flow further separates the business logic contained within each computing unit from the constraints of the overall application. An application is described in a descriptor file that groups together the different blocks that compose it as well as (i) how these blocks are connected, (ii) how they should be deployed on the infrastructure and (iii) possible execution constraints (e.g. time requirements or execution order).
As Eclipse Zenoh-Flow is also targeting safety-critical applications, it leverages the Rust programming language for its core. It also offers bindings in other languages to facilitate integration with existing software.
We have not identified any potential legal issues.
We believe that a data flow programming framework that is specifically designed to express the requirements of safety-critical applications is key to successfully answer tomorrow's use-cases, such as autonomous driving.
By developing this project within the Eclipse Foundation, we expect to reach a broader audience, facilitate interaction with other projects in relevant Working Groups. As a consequence, we then expect to gain more visibility from the industry and strive to establish our project as a viable open-source solution.
We are already planning to release minor versions of Eclipse Zenoh-Flow in the upcoming months that add the following features:
- 0.6.1-rc that will introduce the "guided-execution" feature to control the execution order and parallelism of a data flow.
- 0.6.2-rc that will introduce the "custom scheduling" feature to allow users to control, through a custom node (the "scheduler"), which node(s) should execute next at runtime.
We are publishing blog posts on Zenoh's blog and intend to keep publishing them at regular intervals:
- https://zenoh.io/blog/2023-02-10-zenoh-flow/
- https://zenoh.io/blog/2024-01-31-zenoh-flow-getting-started/
Our marketing team is also involved on social media and already promotes Zenoh and Zenoh-Flow.
Initial contributions were already made and an alpha version is available.
We are planning to release new versions at every quarter.
Eclipse Zenoh
The code pertaining to this project is already available publicly under the Eclipse-Zenoh GitHub organisation:
- https://github.com/eclipse-zenoh/zenoh-flow: the core project
- https://github.com/eclipse-zenoh/zenoh-flow-python: Python bindings.
The copyright holder is ZettaScale Technology.
At the time of this writing, the project has reached the 0.6.0-rc version and provides the following:
- zenoh-flow-nodes: an API to create blocks, "nodes" in the Zenoh-Flow terminology, in the Rust programming languages.
- zenoh-flow-standalone-runtime: an executable that allows running a single data flow on a single host.
- zenoh-flow-standalone-daemon: an executable encapsulating a Zenoh-Flow runtime that can process requests, made through Zenoh, to create / start / abort / delete a data flow that can spawn on multiple Zenoh-Flow runtimes. The requests can be made using our other executable zfctl.
- zfctl: the Zenoh-Flow Command Line Tool through which one can manage data flows and obtain information regarding the different Zenoh-Flow runtimes running on the same Zenoh network.
- zenoh-plugin-zenoh-flow: a shared library to load a Zenoh-Flow daemon as a plugin on a Zenoh router. This plugin offers the exact same capabilities as the standalone daemon.
Through the Zenoh community we are attempting to raise awareness about Zenoh-Flow and have already received some positive feedback. Our objective by separating from Eclipse-Zenoh and falling under the SDV WG is to reach an even broader audience.
- Log in to post comments