This proposal has been approved and the Eclipse Dataspace Connector project has been created. For information about the project and access to the source code, builds, and more, click here.
The data economy is a strong driver for new business models and creativity. Portability, interoperability, sovereignty, and transparency are all issues that must be addressed. Various projects, such as GAIA-X and the International Data Space (IDS), have taken on the task of establishing a uniform standard for data sharing (IDS) and infrastructure (GAIA-X) in the form of architecture models and anchoring the issue of data sovereignty in response to this need. Data Spaces is a concept that defines the interaction of different technological components in order to promote cross-company data sharing while adhering to data sovereignty principles.
One of the most important components is the connector, which links the individual participants in such a data space and provides the endpoint for the actual exchange of data according to existing standards.
- International Data Spaces: https://www.internationaldataspaces.org
- GAIA-X: https://www.data-infrastructure.eu
Eclipse Dataspace Connector provides a connector framework for sovereign, inter-organizational data exchange. The framework contains modules for performing data query, data exchange, policy enforcement, monitoring, and auditing. Specifically, it integrates with existing identity, data catalogue, and transfer technologies to provide compliance, policy, and control capabilities across organizations.
As such, the framework will be fully extensible and architected to integrate with other technology solutions in the following areas:
- Data transfer wire protocols for a variety of use cases including data streams, IoT data, and large data sets.
- Identity providers, including OAuth2-based implementations as well as distributed identity systems.
- Data storage, cataloguing, and taxonomy systems.
- Host environments from on-premise installations to multiple cloud platforms.
The connector will provide an open and publicly accessible solution for data providers and consumers, allowing them to concentrate on the use case at hand.
Existing open-source projects address the technical challenges of cataloguing and transferring data for a wide range of use cases. However, there is no open-source effort aimed at providing an interoperable, cross-organization framework for data sharing that is built on a common identity model and uniform policy enforcement. This project will integrate with existing data exchange technologies and provide these missing pieces to create a system for data sharing where each organization is able to exert control over how its shared data is used.
A data-sharing system requires a protocol implementation for policy enforcement among participants. The Eclipse Dataspace Connector will implement the International Data Spaces standard (IDS) as well as relevant protocols associated with the GAIA-X project. However, the connector will be extensible so that it can support alternative protocols.
This project will provide implementation and use case feedback to IDS and GAIA-X.
One possible issue in regard to legal topics could be the naming of the project. We would like to use the project name of “Eclipse Dataspace Connector (EDC)”, which might have already been used and requires a background check.
Another topic to address will be the licensing of dependencies used by the components of Data Appliance GX and Dataspace Connector as the project's basis. We already ran a check for the first level dependencies but need to further analyze possible usage of LGPL licensing within these transitive dependencies to comply with Eclipse Foundations’ guidelines. As an example, the Dataspace Connector uses Spring JPA, which itself makes use of Hibernate (which is under LGPL).
The Eclipse Foundation offers a technical environment for Eclipse Dataspace Connector's future growth (governance, licensing, and intellectual property management). As a European foundation, it fits the value proposition of Eclipse Dataspace Connector and related initiatives of GAIA-X and IDS as well as it is a well-known trusted partner to all partners initially involved in the project.
These benefits make it easier to develop an open source, extensible ecosystem for sovereign data sharing, which has the potential to be the place for more projects around IDS and GAIA-X in the future.
The companies involved have already settled on critical technology foundations for project implementation with the International Data Spaces (IDS) standard for data sovereignty, which also lays the groundwork for the European cloud data infrastructure GAIA-X. As a result, all of the parties involved intend to contribute to other related FOSS initiatives that help to exploit Eclipse Dataspace Connector and endorse the goal of making data sovereignty a built-in part of (cloud-driven) information systems, thus supporting any already formed or upcoming data space.
There is currently no other FOSS tool suite available that offers a similar collection of functionalities and a comparable maturity level.
The project Eclipse Dataspace Connector will be seeded by two initial code contributions, one providing core infrastructure capabilities (Data Appliance GX) and another which includes support for IDS data exchange patterns (Dataspace Connector). The committers have agreed to merge the codebases by integrating the IDS data exchange support into the core infrastructure framework. This will be done by having the Data Appliance GX codebase served as the initial commit. The Dataspace Connector codebase will then be iteratively merged into the former. A technical plan with milestones has already been developed for this integration effort and can start immediately upon project approval.
- Amazon AWS
- BMW AG
- Daimler TSS GmbH
- Deutsche Telekom AG
- Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
- GAIA-X AISBL
- International Data Spaces Association e.V.
- Microsoft
- SAP SE
- ZF Friedrichshafen AG
The initial contribution will be the commit of Data Appliance GX that is a project developed entirely by Microsoft. The GitHub address is: https://github.com/microsoft/Data-Appliance-GX. Microsoft Corporation holds the copyright on the code contained in that repository. The repository is currently private but can be made accessible upon request for review purposes.
With the initial check, we will also directly contribute the source code of the Dataspace Connector as well as the IDS Framework. Both projects are licensed under the Apache 2.0 and IPR held by involved parties of this project https://github.com/International-Data-Spaces-Association/DataspaceConnector and https://github.com/International-Data-Spaces-Association/IDS-Connector-Framework. There is a project website available at www.dataspace-connector.io and basic community processes like a bi-weekly JF and support mailing lists.
After the contribution of these three repositories, the Eclipse Dataspace Connector will be developed by a milestone-based merge plan:
- Log in to post comments
Broken Link?
Submitted by Kai Hudalla on Tue, 05/25/2021 - 08:20
Hi, I was trying to open https://github.com/microsoft/Data-Appliance-GX but this resulted in GitHub's 404 page ...
Re: Broken Link?
Submitted by James Marino on Thu, 05/27/2021 - 02:17
In reply to Broken Link? by Kai Hudalla
Hi,
Sorry about that. We're working on making the repository public. I'll let you know as soon as that is done, which should be completed shortly.
Re: Broken Link?
Submitted by James Marino on Fri, 05/28/2021 - 09:45
In reply to Broken Link? by Kai Hudalla
Hi Kai,
Sorry, the repository did not have the correct permissions. Can you please try now:
https://github.com/microsoft/Data-Appliance-GX
Jim
Re: Broken Link?
Submitted by James Marino on Fri, 05/28/2021 - 09:46
In reply to Broken Link? by Kai Hudalla
Hi Kai,
Sorry, the repository did not have the correct permissions. Can you please retry?
Jim
Public access to Data Appliance GX
Submitted by Ed Merks on Fri, 05/28/2021 - 00:52
This looks very interesting! The proprosal mentions that the https://github.com/microsoft/Data-Appliance-GX repository is currently private but can be made accessible upon request for review purposes. Like Kai, I'd like to request access to that.