This proposal has been approved and the Eclipse fog05 project has been created.
Visit the project page for the latest information and development.

Eclipse fog05

Sunday, April 8, 2018 - 07:32 by Angelo Corsaro
This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process) and is written to declare its intent and scope. We solicit additional participation and input from the community. Please login and add your feedback in the comments section.
Project
Parent Project
Working Group
Proposal State
Created
Background

Fog computing aims at providing horizontal, system-level, abstractions to distribute computing, storage, control and networking functions closer to the user along a cloud-to-thing continuum. Whilst fog computing is increasingly recognized as the key paradigm at the foundation of Consumer and Industrial Internet of Things (IoT), most of the initiatives on fog computing focus on extending cloud infrastructure. As a consequence, these infrastructure fall short in addressing heterogeneity and resource constraints characteristics of fog computing environments.  We decided to start working on fog05 to address these short comings.

fog05 has been designed from first principle to address the requirements characteristic of fog and multi-access edge computing (MEC). fog05 provides a decentralised  infrastructure that unifies computing, networking and storage fabrics end-to-end, while addressing the challenges imposed by resource heterogeneity.

Finally, it is worth point out that fog05 has leveraged of the experience  we have gained through our experience with fog computing which dates back to its inception.  As an example, we were part of the team that in 2015 demonstrated the first fog computing platform operting smart cities services in Barcelona.

Scope

Eclipse fog05 aims at providing a fog computing platform that provides the equivalent of cloud IaaS  along with some foundational PaaS services.

In other terms, the main focus of fog05 is in providing a scalable infrastructure to:

  1. Virtualise compute, storage and communication end-to-end -- from cloud to thing.
  2. Provide a unified abstraction to provision, manage and monitor applications that may be deployed in the cloud to things continuum.
  3. Provide data virtualisation, as this is a foundational service upon which fog-based analytics, monitoring, etc. will be built.

Whilst monitoring and management is in the scope of fog05, and dynamic resource management is one of the main area in which we plan to contribute, tools for visualisation are out of scope. Third party tools or other projects will leverage fog05 API to provide 

Description

Early IoT applications, especially those addressing the  consumer market, have been embracing cloud-centric architectures in which data is pushed up to the cloud. It is within the cloud the everything takes place before eventually pushing some data or action back to the edge. This architectural approach leverages the availability and operational maturity of the cloud but it is not generally applicable in  IoT /  Cyber Physical Systems (CPS).  

Figure1– Cloud centric architecture.

 

The cloud-centric paradigm quickly shows  its limitations in the context of Industrial IoT (IIoT) and in general with Cyber Physical Systems (CPS). More specifically, the following assumptions, at the foundation of cloud-centric architectures, are generally violated in IIoT applications:

  • Connectivity. Cloud-centric architectures assume that things are sufficiently often connected. While this is mostly true for IoT applications, it is far from being the common case in IIoT applications. As an example, autonomous agricultural vehicles, or robots in a smart farm are often deployed in locations with very poor connectivity.
  • Latency. Cloud-centric architectures assume applications can tolerate the latency associated with pushing data from things to the cloud, processing information on the cloud and eventually sending back some control information. This latency, is orders of magnitude higher of the reaction times required by several IIoT applications, such as, autonomous vehicles, smart factories and smart grids.
  • Throughput. Cloud-centric architectures assume that the throughput required to push data from things to the cloud may be massive when looking at the aggregate traffic, but it is generally composed by limited individual flows. In IIoT the situation is quite different as, often, there are data flows with high individual throughput and in several applications the aggregate volume is incredibly high. This makes it unfeasible or not very effective to stream these massive volumes of data to a data centre.
  • Cost of Connectivity. Consumer IoT (CIoT) commonly assumes that the cost of connectivity is negligible. This stems from the fact that consumer pays for connectivity – either via their mobile data plan or their home internet connection. In IIoT the situation is completely different, it is the owner of the system that pays for connectivity. This cost is non-negligible in applications with large number of data streams, such as Smart Grids as well as for applications deployed in remote areas such as oil exploitation rigs that can only rely on expensive satellite communication where 1MByte of data can cost as much as $8!
  • Security. Cloud-centric architectures operate under the assumption that end-users are comfortable in giving away their data. While this may be true for CIoT applications, the situation is completely different in IIoT. Information is a strategic asset which the vast majority of companies, operating in an industrial, do not want to leave their premises.

Fog computing has emerged as an architectural approach to deal with the limitations exposed by cloud-centric architectures in the context of CPS and IIoT applications. fog05 is an infrastructure designed from first principle to address the needs of  fog computing.

Eclipse fog05 provides  a virtualised infrastructure that allows to distribute computing, storage, control and networking functions closer to the users along a cloud-to-thing continuum. From this description it should be clear that  fog05 can leverage cloud infrastructure or equally function without it. 

Figure2– fog05 end-to-end resource virtualisation.

fog05 makes available a set of abstractions to unify the compute, storage and communication fabric end-to-end, thus allowing applications to be managed, monitored and orchestrated across the cloud to thing continuum. 

In other terms, fog05 provides the same abstraction of an elastic compute, storage and communication fabric but in a decentralised fashion. The main advantages of this architectural approach, when compared with cloud computing, is the ability to keep the data and the processing close to where it makes the most sense, which in many cases is close to where the data is produced. 

Finally, it is worth mentioning that the "5" in fog05 beside representing the letter "S" in hackers jargon, it refers to 5G and specifically to Multi-access  Edge Computing. fog05 addresses the needs of MEC and is compatible with ETSI MEC manifests. 

Why Here?

ADLINK has taken the corporate decision of supporting Eclipse and creating an ecosystem of technologies for edge/fog computing in the context of Eclipse IoT. Eclipse Cyclone was the first step in that direction, fog05 is the logical consequence. 

When evaluating the different open source foundations, we selected Eclipse and specifically Eclipse IoT because of its focus on IoT, its transparent community approach and its extremely active marketing.  

Future Work

Our main areas of focus are the following:

- fog05 Agent: We are rewriting the core of the agent in OCaml in order to improve performance, security and to allow for deployments on the  MirageOS (http://mirage.io)  unikernel. This kind of deployment (depicted below), will allow us to have an extremely small surface of attack. For those of you that are not aware of the MirageOS they have had a bit-coin challenge for a few years and nobody has managed to hack into this unikernel yet. 

-  Plugin API: Currently fog05 plugins are written in python. We are working to make it possible for user to write a plugin in any programming language they wish to.

- Distributed Store (DStore): We are also re-writing the distributed store implementation in OCaml to improve performance, security and safety compared to the current version written in python. 

- Dynamic Resource Management: fog05 currently uses a first-fit algorithm to place deployable entities on available resources. We plan to work on optimisation algorithms to improve the placement strategy and potentially dynamically adapt it.

Project Scheduling

As we are already running and Open Source project we are ready to do the initial contribution immediately.

Committers
Angelo Corsaro (This committer does not have an Eclipse Account)
Gabriele Baldoni (This committer does not have an Eclipse Account)
Julien Enoch (This committer does not have an Eclipse Account)
olivier.hecart (This committer does not have an Eclipse Account)
Erik Boasson (This committer does not have an Eclipse Account)
Interested Parties

OpenFog Consortium (https://www.openfogconsortium.org)

The Open Fog Consortium is currently considering to adopt fog05 as the reference implementation of the reference architecture and as one of the fog  plarforms to use in fog testbeds.  

ITRI (https://www.itri.org.tw/eng/)

ITRI is a the biggest Taiwanese Governmental R&D centre, whose mission is to accelerate innovation and technology adoption. ITRI has adopted fog05 for all fog computing, MEC and 5G initiatives.

UC3M (https://www.uc3m.es/Home)

The networking and edge computing group at the Universidad Carlos III de Madrid, leader by professors Arturo Azcorra and Antonio Oliva, are actively collaborating with us and contributing to fog05.

5G Coral EU Project

The 5G Coral EU project has adopted fog05 as the infrastructure for MEC and its actively contributing to its R&D. 

Huawei

fog05 along and some of the services provided such as its decentralised key/value store are of extreme interest to Huawei Corporate R&D. Huawei has been actively discussing requirements and directions of these technologies.

 

Initial Contribution

fog05 is already a usable fog infrastructure. Its code base is avilable at https://github.com/atolab/fog05

fog05 has also dependencies on other Open Source projects that we run. Some of these projects are not yet part for eclipse and we plan to contribute them as  fog05 sub-projects, other may naturally become part of Eclipse Cyclone. These are:

- Eclipse Cyclone

- Eclipse Cyclone Python API http://github.com/atolab/python-cdds/

- Eclipse Cyclone OCaml API http://github.com/atolab/ocaml-cdds

- DStore http://github.com/atolab/python-dstore/

fog05 core is written on OCaml and Python. 

Source Repository Type