Typical arguments for critical systems treat software differently from hardware. Whereas hardware is expected to suffer both random and systematic failures, only systematic failures are considered for software.
In practice this has led safety engineers to focus on processes and practices which have not kept pace with industry norms, and are arguably no longer fit for purpose in our rapidly evolving environment. Whereas most software organisations claim to be "Agile" and rely heavily on open source, critical systems practitioners are still advocating "Waterfall" and relying on techniques (and in some cases even tools and software) that fell out of mainstream use decades ago.
The world has changed significantly, but the standards (and indeed the techniques used to devise and maintain the standards) are not keeping up.
We consider that delivery of software for critical systems must involve identification and management of the risks associated with the development, integration, release and maintenance of the software.
Further we consider that delivery is not complete without appropriate documentation and systems in place to review and mitigate those risks.
The Trustable Software Framework (TSF) provides a basis to help us to manage these risks as we understand them. Broadly the approach is to consider supply chain and tooling risks as well as the risks inherent in pre-existing or newly developed software, and to apply statistical methods to measure confidence of the whole solution. We believe that this is most usefully applied at the integration level, which is where the problems will usually be noticed.
The Eclipse Trustable Software project focuses on the development of the Trustable Software Framework (TSF) methodology and supporting tools, and their application to open source software projects. Guidance for applying the same approach for proprietary software and commercial projects is also in scope, but will not be the project's main focus.
The scope of the project is expected to include examples of software development and integration methodologies providing more detailed applications of TSF to projects, provide implementers with a 'toolkit' of processes, and criteria to evaluate their own existing processes. Supporting open source tools will either be provided or recommended, and their use documented.
The Eclipse Trustable Software project's focus is practical approaches to understanding risks in software engineering.
Describing and quantifying these risks requires a scalable framework that can be applicable for systems of varying degrees of complexity. Complex systems involving software, hardware, safety, and security properties can greatly benefit from continuous quantitative analysis to inform decision-making.
A single generalised system cannot address all domains effectively, so Eclipse Trustable Software aims to evolve into an ecosystem built around a unified methodology in the longer term. The first step in this journey is the Eclipse Trustable Software Framework (TSF) and its set of Tenets and Assertions: short statements (propositions that could be either true or false) identifying what objectives software projects must consider in order to be considered Trustable.
TSF specifies how metadata about a software project is stored and managed in a git repository, alongside the software's source code and documentation. This involves systematically tracking a set of statements that specify the software project, which form a directed acyclic graph (DAG) and provide metrics for continuous analysis and iteration.
These statements document claims about the software or the project, identify evidence that supports these, or specify requests that must be satisfied by another claim - or by something or someone external to the project. The graph describes how high-level or external goals for the software project (Expectations) are supported by more specific objectives (Assertions) and ultimately Evidence. The latter must necessarily reference artifacts: files in a git repository, or the result of automated processes operating on such files.
This approach is intended to replace traditional techniques that rely on office tools like MS Word or Excel to author documentation, and specialised commercial tooling to manage system specifications and requirements, and trace these to implemented software and tests. Such approaches frequently fail to maintain integrity in complex, continuously evolving systems, because they are not closely integrated with the software that they describe.
There are no legal issues that we are aware of. We hold the Trustable trademark and logo and will contribute these to the project. All non-code assets are licensed under CC BY-SA 4 and all code assets are licensed GPL v2. As part of moving the project to Eclipse, we will adopt EPL v2 w/ secondary license GPL v2.
We chose to establish the Eclipse Trustable Software project with the Eclipse Foundation to leverage its well-established governance framework, strong industry reputation, and extensive developer community. The Eclipse Foundation provides a structured, vendor-neutral environment that ensures transparency and long-term sustainability, both of which are needed for broad adoption of the project.
As a European company we are also drawn to the fact that Eclipse is now headquartered in Brussels.
Planned work includes:
- Documentation of example software development and integration methodologies based on the Trustable Software Framework and linked to the Trustable Assertions;
- Improved and extended tooling, including the replacement of Doorstop with a redesign of the underlying graph structure and an extended user CLI;
- Enhancement of tooling to support integration with TSF graphs hosted in a different project;
- Improved instructions and examples of how to apply TSF to a project; and
- Documentation explaining the confidence scoring approach.
Initial contributions are ready now. Our provisional roadmap for the project has deliverables identified on a regular cadence throughout February and May 2025, with longer term plans in progress.
The initial contribution will consist of:
- Documentation in markdown describing:
- the background and rationale behind the Trustable Software Framework
- a set of Tenets and Assertions that form the basis for evaluating the 'trustability' of a software project
- the methodology of the TSF
- the model that defines a TSF graph
- how to use the provided tooling
- Python source code, implementing tooling to:
- manage and verify the integrity of a TSF graph (currently using Doorstop)
- publish the contents of the graph in markdown (optimised for rendering via mkdocs)
- plot a graphical representation of the graph (using Graphviz)
- examples of validators (routines to process data associated with evidence)
- calculate confidence scores using data stored in the graph or updated by validators
- A GitLab CI pipeline providing:
- build, test and release of the Python tooling
- report generation for a TSF graph
- plotting a visual representation of the graph
- generation of web-based documentation
- deployment of the generated docs to GitLab Pages
- Log in to post comments