An interview with Professor Dr.-Ing. Frank Deinzer, Steinbeis Entrepreneur at the Steinbeis Transfer Center for New Media and Data Science
Have you ever stood in an unfamiliar building with time running out before your next appointment, and you haven’t got a clue how to get to the room you’re looking for? Here’s the answer, complete with directions: SimpleLoc, a new system co-developed by Steinbeis expert Professor Dr.-Ing. Frank Deinzer. TRANSFER magazine met up with the Steinbeis Entrepreneur, who is also a professor of media informatics and multimedia technologies at the Würzburg-Schweinfurt University of Applied Sciences, to talk about navigation inside buildings, augmented reality, and bringing artworks to life in museums.
Hello Professor Deinzer. You played a central role in the development of SimpleLoc, an indoor navigation system. Could you tell us what the project was about?
The idea behind SimpleLoc came about many years ago. A colleague at the time told me about a big government agency moving into a new building, and the architects were having difficulty locating mistakes left behind by tradesmen. They had photos, but they couldn’t work out which photo came from which office because they all looked the same and they weren’t labeled. Basically they needed some sort of technology to work out the locations inside the building, a bit like a satellite system and GPS technology used for outdoor navigation. This gave me the idea of so-called sensor data fusion, which I’d already been looking into for my doctoral thesis. It allows you to merge data from different sources and come up with a result. The question was, how could it be used to locate places inside a building?
Before long we realized we needed a component similar to GPS that would give us a position inside buildings, that was inexpensive, and would allow us to attach it to a camera. So we came up with the idea of using WiFi and Bluetooth, and measuring signal strength. This is probably something you’ve experienced at home: In one room you’ve got a good WiFi signal but it gets worse if you go into another room. That was exactly what we wanted to make use of. You take an access point and based on the signal strength in a specific location, you calculate the distance to the router. If you use several transmitters, you can roughly work out where you’re standing by looking at the distance to each element. There was a crucial drawback, of course – the level of inaccuracy – so it quickly became clear that as a potential solution it wouldn’t be accurate enough. But of course there was another source of information – the architect moving around the building. We could pick out when he was moving around, work out the direction, and see if he changed direction. We could then overlay this information on construction plans and gain a more accurate bearing on his position.
Well, so much for the theory. In 2009 and 2010, we tested if it would work as part of a small-scale research project at Würzburg-Schweinfurt University of Applied Sciences, but we pretty quickly realized it wasn’t the success we’d hoped for. But from a scientific standpoint, it was extremely exciting, so we continued with our research. My Steinbeis Transfer Center had the setup I needed to try out the technology in a real environment. We were already in contact with RothenburgMuseum in Rothenburg ob der Tauber at the time. It’s a former convent from the twelfth century, with complicated and winding thick walls. So it was an ideal building to test our idea in. We installed our system there, did some dummy runs, tracked the visitors as they moved around inside the museum and discovered that it worked really well. We knew with a pretty high degree of certainty where everyone was at any given time, the system was stable, and it was just about as accurate as it would be outdoors.
We then looked for funding so we could offer the system as a product through our Steinbeis Transfer Center. We found what we were looking for at the Bavarian Sparkassenstiftung [cooperative bank foundation]. SimpleLoc is now mainly being used in museums, which after all is what it was made for. Museums need an app that responds to locations, offers interactive content, and thinks intuitively.
What do you think was the biggest challenge when it came to implementing SimpleLoc?
The hardest part was the sensor fusion component and how to pool the different sources of information to get a reliable fix on the current location. This was where the math came in and provided us with a way to determine the probability of someone being in a specific place – for any person, in any location in the building, at any time.
Are there any other places you could imagine using SimpleLoc, besides museums?
We’re working on a project with the City of Würzburg involving a navigation system for the city hall. It’s a complex comprising three separate buildings, and visitors have difficulties finding their way around. They need a solution that works like Google Maps and guides people to their destination. The system is ready now and is currently undergoing beta testing. The main issue is user engagement and what goes down well with the user. For example, we could expand it into a kind of Würzburg app that allows users to carry out certain actions directly via the app, such as making appointments or contacting a customer advisor via video call.
What would also be really interesting is a supermarket system that guides users through stores based on their shopping lists. We’ve also been thinking about systems for big libraries; the app could show visitors where to find a specific book. And then there could also be applications and different scenarios for industry, such as walking routes for employees, or tracking packages and workpieces. The aim would be to improve throughput.
Another interesting project you’re working on is an augmented art app that allows art lovers to engage with a work of art in their own time anywhere they want to. It could also be used by museums to bring exhibits to life by providing visitors with digital content such as videos, audio, or 3D objects. What was the particular appeal about this project for you?
This was another project that started in RothenburgMuseum. When we first met up with the director we were taken on a wonderful tour of the museum. We were standing in front of a series of paintings on the passion of Christ and the director said that especially with those kinds of works, visitors need a guided tour to understand the full meaning. But as is so often the case, it wasn’t possible for money reasons. So how could they convey this kind of information to visitors, in a contemporary way?
This problem gave rise to the augmented art app, because everyone has a smartphone these days. All you have to do is hold your camera up to a picture and it recognizes the image and where you’re standing. The system then overlays the image with more information. When we started, it just showed dots with additional information. We tested it as part of a student’s degree project and it went down really well. Then we turned it into a professional solution at the Steinbeis Transfer Center and this helped with the aim of museums in offering guided tours: storytelling, i.e. explaining the stories behind works of art. We equipped the app with a content management system to allow the museum to manage content on its own.
In 2022, we used the app in Würzburg for the 100th anniversary of Mozartfest. Among other things, there was a special exhibition at the Museum im Kulturspeicher, which we also organized the app for. We succeeded in inspiring an art historian to work with us on the project and within two weeks, all of the artworks had been entered into the system along with content.
What kinds of topics are the experts at your Steinbeis Enterprise working on at the moment?
If you think about augmented art from a broader angle, you could say we’re designing augmented reality. For example, there was the project we worked on for the Porsche Museum in Stuttgart. The development center was marking its 50th anniversary, so a special exhibition was set up at the Porsche Museum. Among other things, this involved building a model of the development center. We overlaid it from top to bottom with an environment we developed based on a virtual world of augmented reality. It offers lots of different interaction options.
Of course, artificial intelligence is also a big issue for us at the moment. An example of this is project Fördercafé [funding cafe], which we moved into productive use around six months ago. Fördercafé is about people, but also companies looking for funding opportunities for their projects. The problem is that there are so many different funding options now – offered by the government, the state, the EU, or foundations. If it’s not something you’re involved in on a regular basis, it’s difficult to work out the best offer. We were approached by a management consultancy from Cologne, which already had a similar system, but it hadn’t been received very well because it wasn’t particularly easy to use. So based on that, we were asked to develop an AI solution for funding programs that would offer users specific funding opportunities based on their needs. It does this by querying different parameters, and based on those, it provides users with a list of suitable programs.
That was just an excerpt of the kinds of things we work on in our daily business, but I think it gives you a good insight into the topics we’re dealing with.
SIMPLELOC
For more information, go to www.simpleloc.de.
Interested in finding out more about a specific project? Take a look at this: https://t.ly/1Hrt
AUGMENTED ART
For further information, go to www.augmented-art.de
AUGMENTED REALITY AT THE PORSCHE MUSEUM
More information is available by going to https://tinyurl.com/4b48urep
Contact
Prof. Dr.-Ing. Frank Deinzer (interviewee)
Steinbeis Entrepreneur
Steinbeis Transfer Center New Media and Data Science (Dettelbach)
www.newmedia-datascience.de