Towards an IMU-based Pen Online Handwriting Recognizer

Most online handwriting recognition systems require the use of specific writing surfaces to extract positional data. In this paper we present a system for word which is based on inertial measurement units (IMUs) digitizing text written paper. This obtained by means sensor-equipped pen that provides acceleration, angular velocity, and magnetic forces streamed via Bluetooth. Our model combines convolutional bidirectional LSTM networks, trained with Connectionist Temporal Classification loss allows interpretation raw sensor data into words without need sequence segmentation. We dataset collected using multiple sensor-enhanced pens evaluate our distinct test sets seen unseen achieving character error rate 17.97% 17.08%, respectively, dictionary or language model.