Deep Learning and NLP based Side Channel Attack for Text Inference in Smartphones

Abstract

Over the past years, smartphones have witnessed an alarming rise in embedded sensors which enhance their support for applications. However, they can be regarded as loopholes as seemingly innocuous information can be obtained without any user permissions in Android thus invading the user’s privacy. Our work establishes a side channel attack by illegitimately inferring the information being typed by the user on a smartphone using the readings from ‘zero-permission’ sensors like accelerometer and gyroscope. This serves as a proof of concept to prevent such attacks on mobile devices in the future. While previous research has been conducted in this space, our narrative involves a predictive model using Recurrent Neural Networks that can predict the letters being typed in the keyboard solely based on the motion sensor readings, thus inferring the text. Our research was able to identify 37.5% of the unseen words typed by the user using a very small volume of training data. Our tap detection method has shown 92% accuracy which plays a critical role in the text inference. This research lays the foundation to further progress in this area, thus helping to strengthen the mobile security.