Eclipse Scenario Architect provides a lightweight graphical user interface that allows a straightforward realization and manipulation of concrete driving testing scenarios. These can then be used to validate an online verification framework, capable of safeguarding AI driving functions.
Machine learning (ML)-based functions – especially methods that continue the learning process online – cannot be approved by current methods and standards (e.g. ISO 26262). One approach to address this issue is to introduce an online verification module, that constantly checks and guarantees the function of interest to stay within safe bounds. That way, safety guarantees can be provided for an ML-based function.
In order to validate sufficiently safe behaviour of the online verification module, real-world no-fire tests as well as scenario-based tests have to be passed. The focus of this project is a lightweight tool, that supports the direct and straightforward realization of concrete testing scenarios. Within this project, the focus is approval of motion planning algorithms. Therefore, the tool (Eclipse Scenario Architect) allows for track boundary specification, as well as the spatiotemporal arrangement of the vehicle under test (VUT) as well as other dynamic vehicles.
Further details about the underlying functionality and possible use-cases can be found in the paper pre-print on arXiv (https://arxiv.org/abs/2006.09731).
This is a project that includes code development, which is aimed to be publicly available via GitHub.
Scenario-based testing is necessary to validate online verification modules. Since online verification is one of the most promising approaches tackling online ML, a scenario realization tool supports the development process towards safe AI and should, therefore, be hosted within the openGENESIS WG.
The Eclipse Scenario Architect was developed as part of research cooperation between the Technical University of Munich and the TÜV Süd Auto Service GmbH with the focus on online verification of AI-based methods.
none known
Project finished – open-source publication via GitHub pending.
The following extensions of the Eclipse Scenario Architect are planned and may be introduced in the future:
- introduction of surrogate safety metrics for every timestamp
- extensions for import / export of other common data-formats (e.g. OpenSCENARIO)
- support of multi-lane roadwork
Furthermore, future work targets the development and validation of the online verification module itself.