Interactive Machine Learning (IML) is the design and implementation of algorithms and intelligent user interface (IUI) frameworks that facilitate machine learning (ML) with the help of human interaction.
With the convergence of Artificial intelligence and Machine Learning, IML is where the Human-Computer Interaction (HCI) and IUI community meets the ML community. We also combine contributions from related fields such as data science, cognitive science, computer graphics, design or the arts, and natural language processing, data mining/data analytics, and knowledge representation and reasoning. Our focus is to improve the interaction between humans and machines to update ML models, by leveraging both state-of-the-art HCI and IUI approaches, as well as solutions that involve state-of-the-art ML techniques.
We can “assist” AI systems in becoming self-sustaining, “lifelong” learners. This includes to (1) develop insights into the importance of the social and cultural contexts of machine learning; (2) create ML systems that actively seek information; (3) realise the need to pay attention to the incomplete context understandings and naive generalisations that ML systems, in particular end-to-end systems, bring with them to a given subject. We focus on multimodal multisensor interfaces for ML.
Mixed initiative interaction:
The ML system and the domain expert engage in a two-way dialogue to facilitate more accurate learning from less data compared to the classical approach of passively observing labeled data.
IML tools for enhancing human cognitive capabilities:
Transparency and human interpretability in ML:
Explainable artificial intelligence applications:
General AI for ML:
The biodiversity crisis is maybe the most pressing issue of our time, which makes measuring the biodiversity an important precursor for improvement. Biodiversity assessment is limited by data management and analysis tools. Prior work has Read more…
Ophthalmo-AI Age-related macular degeneration (AMD) is the primary cause of severe vision loss in the western world. One cause is the retinal accumulation of harmful byproducts of metabolism processes occurring in the eye. Treatment aims Read more…
Medical AI is about to become more powerful and versatile thanks to the fast creation of reusable AI models that can learn from different types of data. We call this new approach a foundation model Read more…
Detecting animal sounds is an essential aspect of scientific research and conservation efforts. It provides valuable insights into biodiversity, species behavior, and ecosystem health. Monitoring of terrestrial and marine wildlife is a way to decrease Read more…
Passive acoustic monitoring (PAM), the recording of sounds using microphones (e.g. in biosphere reserves), is an increasingly popular method for continuous, reproducible, scalable and cost-effective monitoring of wildlife [Sugai et al., 2018]. It is widely Read more…
Figure 1. An overview of the transfer learning framework with BERT-SNER. Information extraction from clinical text has the potential for clinical research and personalized care, but annotating large data for customized requirements is prohibitive. We Read more…
Explainable AI (XAI) has demonstrated its potential in deciphering discriminatory features in machine learning (ML) decision-making processes. Specifically, XAI’s feature attribution methods shed light on individual decisions made by ML models. However, despite their visual Read more…
Funding period: 01/2023 – 06/2026 Many industries are transitioning to Industry 4.0 production models by adopting robots in their processes. In parallel, Extended Reality (XR) technologies have reached sufficient maturity to enter the industrial applications Read more…
We present an interactive image captioning pipeline. Image captioning models take an image as input, for which they generate one or more captions, or descriptions in natural language. State-of-the-art image captioning systems are trained in Read more…
Contract cheating, an academic dishonesty when students ask another person to take an exam or to do an assignment instead of them, turns out to be a significant and growing problem, receiving a large amount Read more…
Medical image analysis encompasses two crucial research areas: disease grading and fine-grained lesion segmentation. Although disease grading often relies on fine-grained lesion segmentation, they are usually studied separately. Disease severity grading can be approached as Read more…
Figure 1: The interface guides annotators in human evaluation. Snippet A displays the original oncology report with highlighted sentences extracted automatically. Snippet C lets annotators choose the model for generation, showing its architecture. The middle Read more…
Motivation The usage of interactive public displays has increased including the number of sensitive applications and, hence, the demand for user authentication methods. We implement a calibration-free authentication method for situated displays based on saccadic Read more…
Funding period 03/2021 – 03/2024 Ophthalmo-AI focuses on the two most common causes of blindness in people aged 50 and older: Age-related macula degeneration (AMD) and diabetic retinopathy (DR). An intelligent assistance system is under Read more…
Funding Period: 10/2020 – 9/2023 To broaden the use of AI methods in medicine, networked, digital patient data is required, for example for use in disease diagnosis, personalized treatment, and faster drug discovery. Our project Read more…
Visual Search Target Inference in Natural Interaction Settings with Machine Learning Visual search is a perceptual task in which humans aim at identifying a search target object such as a traffic sign among other objects. Read more…
Digital pen signals were shown to be predictive for cognitive states, cognitive load and emotion in educational settings. We investigate whether low-level pen-based features can predict the difficulty of tasks in a cognitive test and Read more…
In this present day and age of smart phones, ipads and other smart devices a digital camera has become an integral part of our daily experience. This combined with our desire to capture moments in Read more…
We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. Read more…
In this project, we consider scenarios which include one or more expert users that interact with a multimodal-multisensor interface. In particular, we focus on applications of eye tracking and speech-based input for semantic search applications, Read more…
Embodied conversational agents will be used for exploring multimodal approaches of active input in virtual reality (combination of speech and pen input) to determine the cognitive status of the user. Contact Fabrizio Nunnari
Cognitive assessments have been the subject of recent debate because there are limitations when they are conducted using pen and paper. For example, the collected material is monomodal (written form) and there is no direct Read more…
Gaze estimation error can severely hamper usability and performance of mobile gaze-based interfaces given that the error varies constantly for different interaction positions. In this work, we explore error-aware gaze-based interfaces that estimate and adapt to Read more…
We present a virtual reality (VR) application that enables interactive image clustering and fine-tuning of a deep neural network by using natural hand gestures. Using a pre-trained AlexNet, we apply PCA and t-SNE dimensionality reduction Read more…
Recently there has been a number of research questions regarding user interaction in virtual reality, but only a few are concerned with the interaction using real world objects. For complex data entry we use a Read more…
We explore clustering of objects in a 3D virtual environment. As traditional clustering is done in 2D, we investigate how an additional dimension influences the machine learning process when clustering is done interactively in virtual reality. Read more…
Visual Attention and the Artificial Episodic Memory Recent advances in mobile eye tracking technologies opened the way to design novel attention-aware intelligent user interaction. In particular, the gaze signal can be used to create or Read more…
Traditionally neurocognitive testing is done using pen and paper, which is both expensive and time consuming and often leads to a biased outcome. As part of the Intera-KT project select and digitize existing cognitive tests Read more…
Crowdsourcing is recently used to automate complex tasks when computational systems alone fail. In this project, we investigate how humans can effectively contribute to automate natural language translation. The envisioned goal is a hybrid machine translation Read more…
In the medical domain physicians are often forced to decide quickly whether a further examination is appropriate or which of several treatments is best suited for a patient.A first preliminary ruling can sometimes be made Read more…
Daniel Sonntag (DFKI, project lead)
E-mail: Daniel.Sonntag@dfki.de
Fax: +49(0)681-85775-5341
Deutsches Forschungszentrum für
Künstliche Intelligenz GmbH (DFKI)
Standort Niedersachsen
Interaktives Maschinelles Lernen
Marie-Curie-Straße 1, 26129 Oldenburg, Germany
Deutsches Forschungszentrum für
Künstliche Intelligenz GmbH (DFKI)
Interaktives Maschinelles Lernen
Stuhlsatzenhausweg 3
Saarland Informatics Campus, Geb. D3 2
66123 Saarbrücken, Germany