In comparison to typing, handwriting stimulates different parts of the human brain and brings into play a very different cognitive process. Many of today’s tasks benefit from digitalization and digitization of handwriting input and with the availability of low-cost consumer grade capturing devices, such as digital pens, graphic tablets and smartphones, it becomes increasingly popular in many fields. In this project we use state of the art handwriting recognition in combination with machine learning approaches for automated spell checking, where users can interactively train both the gesture recognition and the spell checking models. We use a Samsung Tab S3 with integrated SPen stylus for writing directly on the screen of the mobile device.
The screenshot above shows the user writing an email message and while writing the system provides instant feedback about the handwriting recognition process as well as the spell checking. In this case the word handwritten word “projekt” is not recognized by the spell checker and therefore marked as incorrect (red). The user may now either correct the mistake or adjust the spell checker by performing natural gestures with the pen. Starting with a pretrained model, the user can refine the underlying model interactively by simply using the app for daily tasks, such as email correspondance or shopping lists.
