Sensor fusion: Enabling devices to be aware of their surroundings

By Serena Brischetto

SEMI spoke with Christian Mandl, Senior Director for Human Machine Interface (HMI), Infineon Technologies AG, ahead of the European MEMS & Sensors Summit. Mandl discussed how the sensing capabilities of machines are getting ever closer to the five human senses, allowing machines to comprehend the environment before acting.

SEMI: What’s it like to lead the Human Machine Interface (HMI) group at Infineon?

Mandl: This example of contextually aware smart devices describes our challenge very well. Devices need to be aware of their surroundings to better adapt their configurations to each specific user. In other words, provide consumers with a more personalized experience. If machines understand the context around them better, their decision-making capabilities are improved, just like humans! Sensor fusion is the key enabler to contextual awareness. Thanks to sensor fusion, machines can provide more reliable feedback based on data from different sensors taken in the same situation, thus making the system more robust. Compared to traditional devices, false positives and false negatives are reduced to make the whole solution smarter.

The challenge we are addressing within the HMI group at Infineon is to enable systems that are aware of their surroundings by combining our best-in-class sensors with sophisticated machine learning algorithms. We create solutions that can better sense the environment around the device, to then trigger user-specific reactions. This is what we call intuitive sensing.

SEMI: Will you elaborate on this challenge? What are the greatest difficulties in combining existing technologies and devices with sensors?

Mandl:

The traditional approach to add sensors to technology has been very simplistic. For example, radar sensors for presence detection typically provide you with the distance to the closest object and trigger a specific action. This approach works but is limited in the amount of use cases it can address since it is not customizable. By using sensor fusion with the sophisticated machine learning techniques, the solutions are becoming robust and stable. When equipping smart speakers with our microphones and radar sensors, they can detect a user’s presence and track location and motion. When adding advanced algorithms such as beam-forming, the audio reception beam can be steered towards the user and filter out noises for more clear understanding of commands.

The market is demanding more of these innovative ready-to-use solutions. Delivering these solutions requires a thorough evaluation based on very strong knowledge of the sensing elements and the raw data they provide. Infineon has a leading edge here, with more than 40 years’ experience in sensing solutions and a deep-rooted system understanding, to create the ready-to-use sensor solutions demanded by the market.

SEMI: You mentioned that data is key to technological development. Re-innovating our world depends on the quality of valuable and secured data about the environment, and what is done with it. How do you make this possible?

Mandl: Indeed, collecting valuable and trustworthy information is critical for any application, as mislabeled or incorrect data reduces the accuracy of any solution. Using reliable and secured sensors is the first critical step towards high quality data. This is where Infineon´s XENSIV^TMsensor portfolio plays a crucial role. Our sensors are exceptionally reliable thanks to our industry-leading technologies, and they are the perfect fit for various applications in automotive, industrial and consumer markets. With clean-labeled data in hand and a good understanding of each use-case, we can drastically improve the probability of detecting an event.

SEMI: Can you further explain the sensor fusion concepts that you are working on to connect the real world with the digital world by sensors?

Mandl: A good example is the integration of radar sensors into smart speakers, which improves tremendously the capabilities of current devices to understand the real world and enables numerous use cases that were not possible before.

Starting with keyword-less interactions with technology, the next generation of IoT devices with capabilities to locate and track users will be capable of adjusting the intelligent actions to your position. For example, when we ask our smart speaker in the living room to “turn on the lights” or “play music,” only the lights and speakers in the user´s surroundings should be activated, and not the ones in the kitchen. When walking into another room, the music and light should be capable of following the user´s position and shift flawlessly into the new room. Precise presence detection and tracking by radar will enable optimal interaction with consumers for a more clear understanding of commands and a flawless user experience. It should also create power savings for the smart home by switching off lights and other devices when no one is around.

SEMI: Machines sensing capabilities are getting closer to the five human senses as they understand the environment before acting. What will the new wave of applications include with regard to consumer markets?

Mandl: The potential of sensor fusion to enhance the sensing capabilities of machines cannot yet be imagined. There are innumerable use cases that can be enabled with the right combination of sensors, data processing algorithms and machine learning tools. Smart devices will be more aware of the situation and anticipate their actions to user commands, leading to the era of intuitive sensing. Imagine a world where you can communicate with your smart device like you talk to another human being!

Thanks to the advanced intelligence that we bring with our HMI group, devices will have a sensor brain for use case-specific matching of multiple sensor fusion data with the customer needs for each application. Not only the smart speaker market will experience this transformation, but also other IoT devices in areas such as home security or user authentication, or wearables for optimized wellbeing tracking and monitoring. Devices will be capable of achieving more if provided with the right technology combination. Sensor fusion will enable technology to take better and smarter decisions in complex situations, in some cases even better than humans would.

SEMI: What do you expect from European MEMS & Sensors Summit 2018 and why do you recommend attending in Grenoble?

Mandl: This event is a great opportunity not only to stay informed and see what is happening in the MEMS and sensors industry, but also to meet current and new partners and customers. Attending is important to observe how industry leaders are working towards the latest market trends, and discuss what else can be done to make life easier, safer and greener for everyone.

Serena Brischetto is a marketing and communications manager at SEMI Europe.

Originally published on the SEMI blog.