Notice: Some of the services that support the smooth operation of our websites are still in the process of being restored. As a result, certain features—such as images and committer paperwork—may be temporarily unavailable. Our team is actively working to resolve these issues and restore full functionality as soon as possible.

Thank you for your patience and understanding.

Eclipse Scenario Architect

Wednesday, July 29, 2020 - 07:35 by Tim Stahl
Basics
This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process) and is written to declare its intent and scope. We solicit additional participation and input from the community. Please login and add your feedback in the comments section.
Parent Project
Eclipse Automotive
Proposal State
Withdrawn
Scope

Eclipse Scenario Architect provides a lightweight graphical user interface that allows a straightforward realization and manipulation of concrete driving testing scenarios. These can then be used to validate an online verification framework, capable of safeguarding AI driving functions.

Description

Machine learning (ML)-based functions – especially methods that continue the learning process online – cannot be approved by current methods and standards (e.g. ISO 26262). One approach to address this issue is to introduce an online verification module, that constantly checks and guarantees the function of interest to stay within safe bounds. That way, safety guarantees can be provided for an ML-based function.

In order to validate sufficiently safe behaviour of the online verification module, real-world no-fire tests as well as scenario-based tests have to be passed. The focus of this project is a lightweight tool, that supports the direct and straightforward realization of concrete testing scenarios. Within this project, the focus is approval of motion planning algorithms. Therefore, the tool (Eclipse Scenario Architect) allows for track boundary specification, as well as the spatiotemporal arrangement of the vehicle under test (VUT) as well as other dynamic vehicles.

Further details about the underlying functionality and possible use-cases can be found in the paper pre-print on arXiv (https://arxiv.org/abs/2006.09731).

This is a project that includes code development, which is aimed to be publicly available via GitHub.

Licenses
Eclipse Public License 2.0
Legal Issues

none known

Why Here?

Scenario-based testing is necessary to validate online verification modules. Since online verification is one of the most promising approaches tackling online ML, a scenario realization tool supports the development process towards safe AI and should, therefore, be hosted within the openGENESIS WG.

The Eclipse Scenario Architect was developed as part of research cooperation between the Technical University of Munich and the TÜV Süd Auto Service GmbH with the focus on online verification of AI-based methods.

Future Work

The following extensions of the Eclipse Scenario Architect are planned and may be introduced in the future:

  • introduction of surrogate safety metrics for every timestamp
  • extensions for import / export of other common data-formats (e.g. OpenSCENARIO)
  • support of multi-lane roadwork

Furthermore, future work targets the development and validation of the online verification module itself.

Project Scheduling

Project finished – open-source publication via GitHub pending.

 

People
Project Leads
Tim Stahl
Committers
Tim Stahl
Frank Diermeyer
Interested Parties

n/a

Source Code
Initial Contribution

The following python libraries are used:

  • numpy
  • matplotlib
  • scikit-learn
Source Repository Type
GitHub