ReLU to enhance MDLSTM for offline arabic handwriting recognition

Abstract

Multi-dimensional Long Short-Term Memory networks (MDLSTMs) are now a state-of-the-art technology that provide a very good performance on different machine learning tasks due to their ability to model any n dimensional pattern using n recurrent connections with n forget gates. For this reason, we are going to focus on the handwritten Arabic word recognition, in which we will need to use only two dimensional MDLSTM for 2D input images then try to improve the accuracy of this baseline recognition system. Such several deep neural networks, the vanishing gradient problem can affect the performance of this MDLSTM-based recognition system. To solve this problem, Rectified Linear Units (ReLUs) are added with different modes, to draw out the best MDLSTM topology for the offline Arabic handwriting recognition system. Proposed systems are evaluated on a large database IFN/ENIT. According to the experimental results and compared to the baseline system, the best tested architecture gives a 5.57% reduction in the label error rate.