{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T22:28:52Z","timestamp":1775860132203,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2022,8,5]],"date-time":"2022-08-05T00:00:00Z","timestamp":1659657600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Funds for the National Key R&amp;D Program of China","award":["2021YFF0307603"],"award-info":[{"award-number":["2021YFF0307603"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Gesture recognition based on wearable devices is one of the vital components of human\u2013computer interaction systems. Compared with skeleton-based recognition in computer vision, gesture recognition using wearable sensors has attracted wide attention for its robustness and convenience. Recently, many studies have proposed deep learning methods based on surface electromyography (sEMG) signals for gesture classification; however, most of the existing datasets are built for surface EMG signals, and there is a lack of datasets for multi-category gestures. Due to model limitations and inadequate classification data, the recognition accuracy of these methods cannot satisfy multi-gesture interaction scenarios. In this paper, a multi-category dataset containing 20 gestures is recorded with the help of a wearable device that can acquire surface electromyographic and inertial (IMU) signals. Various two-stream deep learning models are established and improved further. The basic convolutional neural network (CNN), recurrent neural network (RNN), and Transformer models are experimented on with our dataset as the classifier. The CNN and the RNN models\u2019 test accuracy is over 95%; however, the Transformer model has a lower test accuracy of 71.68%. After further improvements, the CNN model is introduced into the residual network and augmented to the CNN-Res model, achieving 98.24% accuracy; moreover, it has the shortest training and testing time. Then, after combining the RNN model and the CNN-Res model, the long short term memory (LSTM)-Res model and gate recurrent unit (GRU)-Res model achieve the highest classification accuracy of 99.67% and 99.49%, respectively. Finally, the fusion of the Transformer model and the CNN model enables the Transformer-CNN model to be constructed. Such improvement dramatically boosts the performance of the Transformer module, increasing the recognition accuracy from 71.86% to 98.96%.<\/jats:p>","DOI":"10.3390\/s22155855","type":"journal-article","created":{"date-parts":[[2022,8,9]],"date-time":"2022-08-09T04:16:55Z","timestamp":1660018615000},"page":"5855","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3308-6157","authenticated-orcid":false,"given":"Yujian","family":"Jiang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China"},{"name":"Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Communication University of China, Beijing 100024, China"},{"name":"Beijing Key Laboratory of Modern Entertainment Technology, Communication University of China, Beijing 100024, China"},{"name":"School of Information and Communication Engineering, Communication University of China, Beijing 100024, China"}]},{"given":"Lin","family":"Song","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China"},{"name":"Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Communication University of China, Beijing 100024, China"},{"name":"Beijing Key Laboratory of Modern Entertainment Technology, Communication University of China, Beijing 100024, China"},{"name":"School of Information and Communication Engineering, Communication University of China, Beijing 100024, China"}]},{"given":"Junming","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China"},{"name":"Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Communication University of China, Beijing 100024, China"},{"name":"Beijing Key Laboratory of Modern Entertainment Technology, Communication University of China, Beijing 100024, China"},{"name":"School of Information and Communication Engineering, Communication University of China, Beijing 100024, China"}]},{"given":"Yang","family":"Song","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China"},{"name":"Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Communication University of China, Beijing 100024, China"},{"name":"Beijing Key Laboratory of Modern Entertainment Technology, Communication University of China, Beijing 100024, China"},{"name":"School of Information and Communication Engineering, Communication University of China, Beijing 100024, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8979-8490","authenticated-orcid":false,"given":"Ming","family":"Yan","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China"},{"name":"Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Communication University of China, Beijing 100024, China"},{"name":"Beijing Key Laboratory of Modern Entertainment Technology, Communication University of China, Beijing 100024, China"},{"name":"School of Information and Communication Engineering, Communication University of China, Beijing 100024, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Said, S., Boulkaibet, I., Sheikh, M., Karar, A.S., Kork, S., and Nait-Ali, A. (2020). Machine-learning-based muscle control of a 3D-printed bionic arm. Sensors, 20.","DOI":"10.3390\/s20113144"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Colli Alfaro, J.G., and Trejos, A.L. (2022). User-Independent Hand Gesture Recognition Classification Models Using Sensor Fusion. Sensors, 22.","DOI":"10.3390\/s22041321"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Tang, Y., Zhao, S., and Zhang, X. (2019, January 6\u20138). Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on Support Vector Machine. Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China.","DOI":"10.1109\/ROBIO49542.2019.8961436"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"L\u00f3pez, L.B., Caraguay, V., Vimos, V., Zea, J., V\u00e1sconez, J., \u00c1lvarez, M., and Benalc\u00e1zar, M. (2020). An energy-based method for orientation correction of EMG bracelet sensors in hand gesture recognition systems. Sensors, 20.","DOI":"10.3390\/s20216327"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3522","DOI":"10.1007\/s12555-019-1058-5","article-title":"EMG based control of transhumeral prosthesis using machine learning algorithms","volume":"19","author":"Sattar","year":"2021","journal-title":"Int. J. Control. Autom. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bisi, S., De Luca, L., Shrestha, B., Yang, Z., and Gandhi, V. (2018). Development of an EMG-controlled mobile robot. Robotics, 7.","DOI":"10.3390\/robotics7030036"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.jocs.2018.04.019","article-title":"Subject-independent hand gesture recognition using normalization and machine learning algorithms","volume":"27","author":"Wahid","year":"2018","journal-title":"J. Comput. Sci."},{"key":"ref_8","first-page":"1069","article-title":"Muscle activation and inertial motion data for noninvasive classification of activities of daily living","volume":"65","author":"Totty","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"985","DOI":"10.1080\/01691864.2020.1713886","article-title":"Depth vision guided hand gesture recognition using electromyographic signals","volume":"34","author":"Su","year":"2020","journal-title":"Adv. Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"103368","DOI":"10.1016\/j.bspc.2021.103368","article-title":"Multi-sensory assessment for hand pattern recognition","volume":"72","author":"Amrani","year":"2022","journal-title":"Biomed. Signal Processing Control."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1155\/2021\/1434347","article-title":"Channel noise optimization of polar codes decoding based on a convolutional neural network","volume":"2021","author":"Yan","year":"2021","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zia ur Rehman, M., Waris, A., Gilani, S.O., Jochumsen, M., Niazi, I.K., Jamil, M., Farina, D., and Kamavuako, E.N. (2018). Multiday EMG-based classification of hand motions with deep learning techniques. Sensors, 18.","DOI":"10.3390\/s18082497"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1109\/TNSRE.2019.2896269","article-title":"Deep learning for electromyographic hand gesture signal classification using transfer learning","volume":"27","author":"Fall","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100353","DOI":"10.1016\/j.sbsr.2020.100353","article-title":"Percentage estimation of muscular activity of the forearm by means of EMG signals based on the gesture recognized using CNN","volume":"29","author":"Rubiano","year":"2020","journal-title":"Sens. Bio-Sens. Res."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/TBCAS.2020.3005148","article-title":"A study of personal recognition method based on EMG signal","volume":"14","author":"Lu","year":"2020","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"C\u00f4t\u00e9-Allard, U., Gagnon-Turcotte, G., Laviolette, F., and Gosselin, B. (2019). A low-cost, wireless, 3-D-printed custom armband for sEMG hand gesture recognition. Sensors, 19.","DOI":"10.3390\/s19122811"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1747","DOI":"10.1109\/TNNLS.2019.2927224","article-title":"A semisupervised recurrent convolutional attention model for human activity recognition","volume":"31","author":"Chen","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nasri, N., Orts-Escolano, S., Gomez-Donoso, F., and Cazorla, M. (2019). Inferring static hand poses from a low-cost non-intrusive sEMG sensor. Sensors, 19.","DOI":"10.3390\/s19020371"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Guo, H., and Sung, Y. (2020). Movement estimation using soft sensors based on Bi-LSTM and two-layer LSTM for human motion capture. Sensors, 20.","DOI":"10.3390\/s20061801"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, Z., He, C., and Yang, K. (2020). A novel surface electromyographic signal-based hand gesture prediction using a recurrent neural network. Sensors, 20.","DOI":"10.3390\/s20143994"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Nasri, N., Orts-Escolano, S., and Cazorla, M. (2020). A semg-controlled 3D game for rehabilitation therapies: Real-time time hand gesture recognition using deep learning techniques. Sensors, 20.","DOI":"10.3390\/s20226451"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Nasri, N., Gomez-Donoso, F., Orts-Escolano, S., and Cazorla, M. (2019). Using Inferred Gestures from sEMG Signal to Teleoperate a Domestic Robot for the Disabled. International Work-Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/978-3-030-20518-8_17"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5775","DOI":"10.1109\/JSEN.2019.2904595","article-title":"Cooperative sensing and wearable computing for sequential hand gesture recognition","volume":"19","author":"Zhang","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2243","DOI":"10.1109\/TBME.2022.3140269","article-title":"Recurrent Convolutional Neural Networks as an Approach to Position-Aware Myoelectric Prosthesis Control","volume":"69","author":"Williams","year":"2022","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1186\/s12938-018-0539-8","article-title":"PCA and deep learning based myoelectric grasping control of a prosthetic hand","volume":"17","author":"Li","year":"2018","journal-title":"Biomed. Eng. Online"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4235","DOI":"10.1109\/ACCESS.2019.2942108","article-title":"Real-time human intention recognition of multi-joints based on MYO","volume":"8","author":"Sun","year":"2019","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Cascarano, G.D., Loconsole, C., Brunetti, A., Lattarulo, A., Buongiorno, D., Losavio, G., di Sciascio, E., and Bevilacqua, V. (2019). Biometric handwriting analysis to support Parkinson\u2019s Disease assessment and grading. BMC Med. Inform. Decis. Mak., 19.","DOI":"10.1186\/s12911-019-0989-3"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Motoche, C., and Benalc\u00e1zar, M.E. (2018). Real-Time Hand Gesture Recognition Based on Electromyographic Signals and Artificial Neural Networks. International Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/978-3-030-01418-6_35"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1007\/s10514-020-09928-7","article-title":"Disturbance observer enhanced variable gain controller for robot teleoperation with motion capture using wearable armbands","volume":"44","author":"Huang","year":"2020","journal-title":"Auton. Robot."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"103588","DOI":"10.1016\/j.bspc.2022.103588","article-title":"Classification of surface electromyography and gyroscopic signals of finger gestures acquired by Myo armband using machine learning methods","volume":"75","author":"Tepe","year":"2022","journal-title":"Biomed. Signal Processing Control"},{"key":"ref_31","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is All You Need. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mendes Junior, J.J.A., Freitas, M.L.B., Campos, D.P., Farinelli, F.A., Stevan, S.L., and Pichorim, S.F. (2020). Analysis of influence of segmentation, features, and classification in sEMG processing: A case study of recognition of brazilian sign language alphabet. Sensors, 20.","DOI":"10.3390\/s20164359"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lai, S., Xu, L., Liu, K., and Zhao, J. (2015, January 25\u201330). Recurrent Convolutional Neural Networks for Text Classification. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9513"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Chiu, C., Shih, T.K., Lin, C., Hui, L., Utaminingrum, F., and Yang, T. (2019, January 5\u20138). Application of Hand Recognition System Based on Electromyography and Gyroscope Using Deep Learning. Proceedings of the 2019 Twelfth International Conference on Ubi-Media Computing (Ubi-Media), Bali, Indonesia.","DOI":"10.1109\/Ubi-Media.2019.00027"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Romero, R., Cruz, P.J., V\u00e1sconez, J.P., Benalc\u00e1zar, M., \u00c1lvarez, R., Barona, L., and Valdivieso, L. (2022). Hand Gesture and Arm Movement Recognition for Multimodal Control of a 3-DOF Helicopter. International Conference on Robot Intelligence Technology and Applications, Springer.","DOI":"10.1007\/978-3-030-97672-9_32"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Cho, K., van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/15\/5855\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:04:38Z","timestamp":1760141078000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/15\/5855"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,5]]},"references-count":38,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22155855"],"URL":"https:\/\/doi.org\/10.3390\/s22155855","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,5]]}}}