Interactive Machine Learning (IML) is the design and implementation of algorithms and intelligent user interface (IUI) frameworks that facilitate machine learning (ML) with the help of human interaction.
With the convergence of Artificial intelligence and Machine Learning, IML is where the Human-Computer Interaction (HCI) and IUI community meets the ML community. We also combine contributions from related fields such as data science, cognitive science, computer graphics, design or the arts, and natural language processing, data mining/data analytics, and knowledge representation and reasoning. Our focus is to improve the interaction between humans and machines to update ML models, by leveraging both state-of-the-art HCI and IUI approaches, as well as solutions that involve state-of-the-art ML techniques.
We can “assist” AI systems in becoming self-sustaining, “lifelong” learners. This includes to (1) develop insights into the importance of the social and cultural contexts of machine learning; (2) create ML systems that actively seek information; (3) realise the need to pay attention to the incomplete context understandings and naive generalisations that ML systems, in particular end-to-end systems, bring with them to a given subject. We focus on multimodal multisensor interfaces for ML.
Mixed initiative interaction:
The ML system and the domain expert engage in a two-way dialogue to facilitate more accurate learning from less data compared to the classical approach of passively observing labeled data.
IML tools for enhancing human cognitive capabilities:
Transparency and human interpretability in ML:
Explainable artificial intelligence applications:
General AI for ML:
Motivation The usage of interactive public displays has increased including the number of sensitive applications and, hence, the demand for user authentication methods. We implement a calibration-free authentication method for situated displays based on saccadic Read more…
Visual Search Target Inference in Natural Interaction Settings with Machine Learning Visual search is a perceptual task in which humans aim at identifying a search target object such as a traffic sign among other objects. Read more…
Digital pen signals were shown to be predictive for cognitive states, cognitive load and emotion in educational settings. We investigate whether low-level pen-based features can predict the difficulty of tasks in a cognitive test and Read more…
In this present day and age of smart phones, ipads and other smart devices a digital camera has become an integral part of our daily experience. This combined with our desire to capture moments in Read more…
We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. Read more…
In this project, we consider scenarios which include one or more expert users that interact with a multimodal-multisensor interface. In particular, we focus on applications of eye tracking and speech-based input for semantic search applications, Read more…
Embodied conversational agents will be used for exploring multimodal approaches of active input in virtual reality (combination of speech and pen input) to determine the cognitive status of the user. Contact Fabrizio Nunnari
Cognitive assessments have been the subject of recent debate because there are limitations when they are conducted using pen and paper. For example, the collected material is monomodal (written form) and there is no direct Read more…
Gaze estimation error can severely hamper usability and performance of mobile gaze-based interfaces given that the error varies constantly for different interaction positions. In this work, we explore error-aware gaze-based interfaces that estimate and adapt to Read more…
We present a virtual reality (VR) application that enables interactive image clustering and fine-tuning of a deep neural network by using natural hand gestures. Using a pre-trained AlexNet, we apply PCA and t-SNE dimensionality reduction Read more…
Recently there has been a number of research questions regarding user interaction in virtual reality, but only a few are concerned with the interaction using real world objects. For complex data entry we use a Read more…
We explore clustering of objects in a 3D virtual environment. As traditional clustering is done in 2D, we investigate how an additional dimension influences the machine learning process when clustering is done interactively in virtual reality. Read more…
Visual Attention and the Artificial Episodic Memory Recent advances in mobile eye tracking technologies opened the way to design novel attention-aware intelligent user interaction. In particular, the gaze signal can be used to create or Read more…
Traditionally neurocognitive testing is done using pen and paper, which is both expensive and time consuming and often leads to a biased outcome. As part of the Intera-KT project select and digitize existing cognitive tests Read more…
Crowdsourcing is recently used to automate complex tasks when computational systems alone fail. In this project, we investigate how humans can effectively contribute to automate natural language translation. The envisioned goal is a hybrid machine translation Read more…
In the medical domain physicians are often forced to decide quickly whether a further examination is appropriate or which of several treatments is best suited for a patient.A first preliminary ruling can sometimes be made Read more…
As medical records may cover a very long history of diseases (up to 30 years) and include a vast number of diagnoses, symptoms, results, medications, and laboratory values, we could highly benefit from advanced search Read more…
In comparison to typing, handwriting stimulates different parts of the human brain and brings into play a very different cognitive process. Many of today’s tasks benefit from digitalization and digitization of handwriting input and with Read more…
Handwriting provides a natural and intuitive way to input text in virtual reality. We explore different approaches on how sketched user input can be captured, here we use the Oculus Rift controller as a substitute Read more…
As part of the Clinical Data Intelligence project we present a speech dialogue system that facilitates medical decision support for doctors in a virtual reality (VR) application. The therapy prediction is based on a recurrent Read more…
Funding period 04/2017 – 06/2020 The goal of the Interakt (interactive cognitive assessment tool) project is to improve the diagnostic process of dementia and other forms of cognitive impairments by digitising and digitalising standardised cognitive Read more…
Learning through Human-Robot Dialogues Today, robots are present in our lives and they will be even more ubiquitous in the future. This advance of robotic technologies also poses the question how humans can guide the Read more…
Many of todays processes benefits from digitization and digitalization of written content when it comes to data aquisition for machine learning tasks. Through the use of state of the art handwriting and gesture recognition we Read more…
Funding period 04/2014 – 04/2017 In the KDI project, funded by the Federal Ministry for Economic Affairs and Energy (BMWi), we assume that clinical big data analytics needs to focus on the clinical decision processes Read more…
In Kognit (2014–2015), we enter the mixed reality realm for helping dementia patients. Dementia is a general term for a decline in mental ability severe enough to interfere with daily life. Memory loss is an Read more…
Daniel Sonntag (DFKI, project lead)
E-mail: Daniel.Sonntag@dfki.de
Fax: +49(0)681-85775-5341
Deutsches Forschungszentrum für
Künstliche Intelligenz GmbH (DFKI)
Standort Niedersachsen
Interaktives Maschinelles Lernen
Marie-Curie-Straße 1, 26129 Oldenburg, Germany
Deutsches Forschungszentrum für
Künstliche Intelligenz GmbH (DFKI)
Interaktives Maschinelles Lernen
Stuhlsatzenhausweg 3
Saarland Informatics Campus, Geb. D3 2
66123 Saarbrücken, Germany