Interactive Machine Learning (IML) is the design and implementation of algorithms and intelligent user interface (IUI) frameworks that facilitate machine learning (ML) with the help of human interaction.
With the convergence of Artificial intelligence and Machine Learning, IML is where the Human-Computer Interaction (HCI) and IUI community meets the ML community. We also combine contributions from related fields such as data science, cognitive science, computer graphics, design or the arts, and natural language processing, data mining/data analytics, and knowledge representation and reasoning. Our focus is to improve the interaction between humans and machines to update ML models, by leveraging both state-of-the-art HCI and IUI approaches, as well as solutions that involve state-of-the-art ML techniques.
We can “assist” AI systems in becoming self-sustaining, “lifelong” learners. This includes to (1) develop insights into the importance of the social and cultural contexts of machine learning; (2) create ML systems that actively seek information; (3) realise the need to pay attention to the incomplete context understandings and naive generalisations that ML systems, in particular end-to-end systems, bring with them to a given subject. We focus on multimodal multisensor interfaces for ML.
Mixed initiative interaction:
The ML system and the domain expert engage in a two-way dialogue to facilitate more accurate learning from less data compared to the classical approach of passively observing labeled data.
IML tools for enhancing human cognitive capabilities:
Transparency and human interpretability in ML:
Explainable artificial intelligence applications:
General AI for ML:
Machine learning models are powerful, yet they often sideline the domain experts who understand the data best. In conventional active learning pipelines, the model drives the process while the user simply responds, leaving domain knowledge Read more
Passive Acoustic Monitoring (PAM) enables continuous and non-invasive biodiversity monitoring, but analysing large acoustic datasets remains difficult because sound event detectors usually require temporally precise annotations. Creating such instance-level labels is expensive and requires expert Read more
Visual-Language-Action (VLA) models are typically trained through imitation learning, which teaches policies to reproduce demonstrated actions but provides limited supervision about the conditions that define task success. We propose a framework that automatically extracts executable Read more
Passive acoustic monitoring (PAM) has become a powerful tool for studying wildlife by continuously recording environmental soundscapes. However, analysing large acoustic datasets remains highly time-consuming, as recordings are often annotated manually by domain experts. In Read more
Building reliable AI models depends not only on how much data is annotated, but on the quality and meaning of the labels used during annotation. In many workflows, labels are flat, task-specific class names. They Read more
Image annotation remains a significant bottleneck in image analysis pipelines. In research settings especially, annotating large image corpora demands substantial effort, often forcing teams to compromise on quality by resorting to coarser methods like point Read more
Large language models (LLMs) and large multimodal models (LMMs) have significantly impacted the AI community, industry, and various economic sectors. In journalism, integrating AI poses unique challenges and opportunities, particularly in enhancing the quality and Read more
Image captioning has seen immense progress in the last few years. However, general-purpose systems often fail to provide personalised, context-aware captions tailored to individual users or domains. In this work, we investigate the task of Read more
In the Autoprompt project funded by a grant from Accenture, one of the world’s leading consulting, technology and outsourcing companies, we focus on developing automated biomedical claim verification systems designed to assist clinicians and researchers Read more
Left: Examples from HealthVer [1]; Right: Example of a claim that is supported and refuted by different evidence [2] With the rapid growth of biomedical research and the concurrent rise in misinformation, ensuring the accuracy Read more
Example from n2c2 of relation extraction [1] This work explores the effectiveness of employing Clinical BERT for Relation Extraction (RE) tasks in medical texts within an Active Learning (AL) framework. Our main objective is to Read more
Clinical Named Entity Recognition (NER) is essential for extracting important medical insights from clinical narratives. Given the challenges in obtaining expert training datasets for real-world clinical applications related to data protection regulations and the lack Read more
The biodiversity crisis is maybe the most pressing issue of our time, which makes measuring the biodiversity an important precursor for improvement. Biodiversity assessment is limited by data management and analysis tools. Prior work has Read more
Ophthalmo-AI Age-related macular degeneration (AMD) is the primary cause of severe vision loss in the western world. One cause is the retinal accumulation of harmful byproducts of metabolism processes occurring in the eye. Treatment aims Read more
Medical AI is about to become more powerful and versatile thanks to the fast creation of reusable AI models that can learn from different types of data. We call this new approach a foundation model Read more
Detecting animal sounds is an essential aspect of scientific research and conservation efforts. It provides valuable insights into biodiversity, species behavior, and ecosystem health. Monitoring of terrestrial and marine wildlife is a way to decrease Read more
Passive acoustic monitoring (PAM), the recording of sounds using microphones (e.g. in biosphere reserves), is an increasingly popular method for continuous, reproducible, scalable and cost-effective monitoring of wildlife [Sugai et al., 2018]. It is widely Read more
Figure 1. An overview of the transfer learning framework with BERT-SNER. Information extraction from clinical text has the potential for clinical research and personalized care, but annotating large data for customized requirements is prohibitive. We Read more
Explainable AI (XAI) has demonstrated its potential in deciphering discriminatory features in machine learning (ML) decision-making processes. Specifically, XAI’s feature attribution methods shed light on individual decisions made by ML models. However, despite their visual Read more
Funding period: 01/2023 – 06/2026 Many industries are transitioning to Industry 4.0 production models by adopting robots in their processes. In parallel, Extended Reality (XR) technologies have reached sufficient maturity to enter the industrial applications Read more
We present an interactive image captioning pipeline. Image captioning models take an image as input, for which they generate one or more captions, or descriptions in natural language. State-of-the-art image captioning systems are trained in Read more
Contract cheating, an academic dishonesty when students ask another person to take an exam or to do an assignment instead of them, turns out to be a significant and growing problem, receiving a large amount Read more
Medical image analysis encompasses two crucial research areas: disease grading and fine-grained lesion segmentation. Although disease grading often relies on fine-grained lesion segmentation, they are usually studied separately. Disease severity grading can be approached as Read more
Figure 1: The interface guides annotators in human evaluation. Snippet A displays the original oncology report with highlighted sentences extracted automatically. Snippet C lets annotators choose the model for generation, showing its architecture. The middle Read more
Motivation The usage of interactive public displays has increased including the number of sensitive applications and, hence, the demand for user authentication methods. We implement a calibration-free authentication method for situated displays based on saccadic Read more
Funding period 03/2021 – 03/2024 Ophthalmo-AI focuses on the two most common causes of blindness in people aged 50 and older: Age-related macula degeneration (AMD) and diabetic retinopathy (DR). An intelligent assistance system is under Read more
Funding Period: 10/2020 – 9/2023 To broaden the use of AI methods in medicine, networked, digital patient data is required, for example for use in disease diagnosis, personalized treatment, and faster drug discovery. Our project Read more
Visual Search Target Inference in Natural Interaction Settings with Machine Learning Visual search is a perceptual task in which humans aim at identifying a search target object such as a traffic sign among other objects. Read more
Digital pen signals were shown to be predictive for cognitive states, cognitive load and emotion in educational settings. We investigate whether low-level pen-based features can predict the difficulty of tasks in a cognitive test and Read more
In this present day and age of smart phones, ipads and other smart devices a digital camera has become an integral part of our daily experience. This combined with our desire to capture moments in Read more
Daniel Sonntag (DFKI, project lead)
E-mail: Daniel.Sonntag@dfki.de
Phone: +49 441 99833 7440 (Secretariat Oldenburg)
+49 681 85775 5320 (Secretariat Saarbrücken)
Deutsches Forschungszentrum für
Künstliche Intelligenz GmbH (DFKI)
Standort Niedersachsen
Interaktives Maschinelles Lernen
Marie-Curie-Straße 1, 26129 Oldenburg, Germany
Deutsches Forschungszentrum für
Künstliche Intelligenz GmbH (DFKI)
Interaktives Maschinelles Lernen
Stuhlsatzenhausweg 3
Saarland Informatics Campus, Geb. D3 2
66123 Saarbrücken, Germany