A Sensor Twin Enhances the Safety of Self-Driving Vehicles

Steinbeis experts develop optimized scenarios for creating robust phenomenological models

For the automotive industry, alongside electric vehicles, automated vehicles are one of the most important topics of the 21st century. To ensure the control units used in driver assistance systems are safe, there are two approaches to logging millions of test kilometers. One is to record driving situations in a real setting and then replay them using re-simulation methods; the other is to create an artificial environment capable of providing synthetic, but realistic data in a testing environment. With the synthetic approach, the performed simulations might deviate from reality and this can impair the smooth functioning of the control unit. This is where the experts at Steinbeis Interagierende Systeme come in with their SensorTwin project, which is being funded by the Baden-Wuerttemberg Ministry of Economics as part of a competition called AI for SMEs. The goal of the project is to enhance the realisticness (fidelity) of virtual vehicle testing and thus reduce the resources required to conduct real testing on the road.

The relationship between realism and complexity in actual, recorded situations


One problem with synthetic testing approaches is that they are based on replicated models that are “too ideal” and take no or very few real phenomena into account. As a result, they may not deliver suitable data for testing units in every simulated situation. To solve this problem, models are needed of sensors that take in their surroundings. Depending on the actual situation that needs to be simulated, these would deliver the required fidelity and the complexity needed by the simulation.

A distinction is made between ideal (ground truth) replication, which offers ideal simulations of models without phenomena, probabilistic models (mathematical models), which also take random variables and the distribution of probability into account, and phenomenological models, which include the effects experienced with real sensors. There is also the possibility to construct physical models to calculate physical properties in simulations.

Systematic and automated

With simulated environments, complexity is taken to mean the totality of all dependent sensor properties, and corresponding elements, creating comprehensive sets of relationships within the system. It follows that the better the ability of models to include real phenomena, the more difficult it is to replicate those models as digital twins. On the other hand, in actual, recorded situations, there is a direct dependency between fidelity and complexity. With regard to recordings, this means that data have to offer maximum fidelity. But then, only those exact situations are available for re-simulation.

For the SensorTwin project, the Steinbeis team is focusing on an approach aimed at systematically identifying model inaccuracies and applying them to phenomenological models. To do this, the challenges presented by methods applied to future cars will be replicated on an experimental setup with the aim of creating suitable measurement maneuvers. To carry out the testing, an existing 1:8-scale vehicle is being used featuring environmental detection sensor technology, also allowing automated vehicle testing to be performed via scenario-based commands. This experimental setup will first be used to systematically gather labeled data sets. The next step will be to examine how artificial intelligence can be used to support the simulated testing of future vehicle software.

Sponsored by the


Benjamin Baur (author)
SensorTwin project manager
Steinbeis Interagierende Systeme GmbH (Herrenberg)