We present a virtual reality (VR) application that enables interactive image clustering and fine-tuning of a deep neural network by using natural hand gestures. Using a pre-trained AlexNet, we apply PCA and t-SNE dimensionality reduction techniques to project image data sets into the 3D VR space. Resulting spatial clusters of Read more…
Recently there has been a number of research questions regarding user interaction in virtual reality, but only a few are concerned with the interaction using real world objects. For complex data entry we use a tablet with integrated stylus, that is represented in virtual reality through real-time tracking of markers. Read more…
We explore clustering of objects in a 3D virtual environment. As traditional clustering is done in 2D, we investigate how an additional dimension influences the machine learning process when clustering is done interactively in virtual reality. Through the use of intuitive hand gestures performed using the Oculus Rift controllers, the user Read more…
Handwriting provides a natural and intuitive way to input text in virtual reality. We explore different approaches on how sketched user input can be captured, here we use the Oculus Rift controller as a substitute for a real pen. The user “grabs” the virtual pen using the index finger and Read more…
As part of the Clinical Data Intelligence project we present a speech dialogue system that facilitates medical decision support for doctors in a virtual reality (VR) application. The therapy prediction is based on a recurrent neural network model that incorporates the examination history of patients. A central supervised patient database Read more…