Situation-aware dialog shell for the semantic access to media and services

Before the Internet of Services can act effectively on behalf of a user, it must first understand the user’s intent. Work Package 4 (WP4) “Situation-aware Dialog Shell” of the THESEUS Core Technology Cluster develops technologies that help transform natural human communication to information that a computer can process.

Natural human-computer interaction
In the future, users should be able to communicate intuitively with the Internet of Services using their PC, notebook or smartphone, and integrate it into their daily lives. However, traditional keyboard and mouse input is limited in how it can allow users to interface with a system, and is being combined with alternative modalities, such as speech, head/body movement or facial expressions. These “multimodal user interfaces” are the product of innovative algorithms that transform natural language queries of the user to semantically appropriate data records which the computer can process.

Situation-aware dialog shell
The system also identifies inaccuracies or word sense ambiguity when processing the request. Using a dialog system, the user can clarify these issues through step-by-step communication with the computer. In this way, even incomplete entries can be understood in the context of the given situation, incorporating every available communication channel (keyboard, mouse, microphone, touchscreen etc.).

Developing comprehensive solutions
These individual components can then be used to develop overall solutions that combine new semantic services with intuitive multimodal user interfaces. In this way, users can find the material or services they need and generate new information on them. The results are prepared in such a way that they are available the next time users interact with the system.

A holistic research approach
WP4 spans the entire spectrum of research in the field of situation-aware dialog processing. Apart from the creation of guidelines for designing user-friendly interfaces, this also includes the development of technologies for processing input, controlling interactions and outputting data, as well as solutions for dialog system integration.