Al-Qadisiyah Journal for Engineering Sciences

Design and Implementation of Upper Limb Controlled by Human Voice and Artificial Intelligence

Document Type : Research Paper

Authors

Ahmed Mohee Aldeen Ahmed 1
Mithaq Nama Raheema 2
Jabbar Salman Hussein 2

1 College of Engineering, Department of Electrical and Electronic Engineering, University of Kerbala, Kerbala, Iraq.

2 College of Engineering, Department of Prosthetics and Orthotics Engineering, University of Kerbala, Kerbala, Iraq.

10.30772/qjes.2024.149892.1241
Abstract
Given the restrictions and complexity of EMG-based hand prostheses, the present work chose to investigate the possibility of a much simpler method based on voice-operated. Voice commands are simpler to understand than EMG signals to regulate the prosthetic arm. Therefore, the first and most important benefit of this method is that the related prosthesis is simple to use. Speech recognition is a vital focus in artificial intelligence, serving as a prominent mode of human interaction. Researchers have developed speech-driven prosthetic hand systems using traditional speech recognition frameworks and neural network models. This work intends to employ the Raspberry Pi 4 Model B – 4GB RAM embedded inside the prosthetic limb instead of Arduino without requiring a computer to lower hand weight; this offers simplicity of usage. The proposed system captures and transforms speech input into features resembling spectrograms. It processes them using the MLFFNN to categorize speech as signals (words) and forward it to the prosthetic hand, fingers, and wrist control system involving six servo motors. 3D printers created a light and sturdy prosthetic locally. This work stands out for the freedom of usage amputees of
this limb have, with the ability of the researcher to increase the movements by adding more signals (words). The entire system is implemented in Python using Keras and a deep learning framework with a TensorFlow backend. The simulation results demonstrate an accuracy of 99.4%, real-time test accuracy of 96.05%, and operational validation efficiency of 97.6%. These findings indicate that the MLFFNN can be effectively utilized for real-time control of prosthetic hands.

Keywords

Automatic Speech Recognition
Multi-Layer Feed Forward Neural Network
Mel-frequency cepstral coefficients
Root Mean Square
Standard Deviation
  • S. Varshney, A. Suman, G. Varshney, and P. Suman, “Applications of ai in prosthetics: A critical analysis,” Computology: Journal of Applied Computer Science and Intelligent Technologies, vol. 2, pp. 23–32, 2022. [Online]. Available: https://doi.org/10.17492/computology.v2i1.2203
  •  J. P. A´ ngel-Lo´pez and N. Arzola de la Pen˜a, “Voice controlled prosthetic hand with predefined grasps and movements,” in VII Latin American Congress on Biomedical Engineering CLAIB 2016, Bucaramanga, Santander, Colombia, October 26th -28th, 2016, I. Torres, J. Bustamante, and D. A. Sierra, Eds. Singapore: Springer Singapore, 2017, pp. 520–523. [Online]. Available: https://doi.org/10.1007/978-981-10-4086-3 131
  • M. Cutajar, E. Gatt, I. Grech, O. Casha, and J. Micallef, “Comparative study of automatic speech recognition techniques,” IET Signal Processing, vol. 7, no. 1, pp. 25–46, 2013. [Online]. Available: https://doi.org/10.1049/iet-spr.2012.0151
  • J. Hussein, “Speech recognition based on statistical features,” Computational Semantics, 2022. [Online]. Available: https: //doi.org/10.5772/intechopen.104671
  • L. F. Tshimanga, F. Del Pup, M. Corbetta, and M. Atzori, “An overview of open source deep learning-based libraries for neuroscience,” Applied Sciences, vol. 13, no. 9, 2023. [Online]. Available: https://doi.org/10.3390/app13095472
  • M. Jafarzadeh and Y. Tadesse, “Convolutional neural networks for speech controlled prosthetic hands,” 2019 First International Conference on Transdisciplinary AI (TransAI), pp. 35–42, 2019. [Online]. Available: https://doi.org/10.1109/TransAI46475.2019.00014
  • T. Triwiyanto, S. D. Musvika, S. Luthfiyah, E. Yulianto, A. M. Maghfiroh, L. Lusiana, and I. D. M. Wirayuda, “Upper limb exoskeleton using voice control based on embedded machine learning on raspberry pi,” in Proceedings of the 2nd International Conference on Electronics, Biomedical Engineering, and Health Informatics, T. Triwiyanto, A. Rizal, and W. Caesarendra, Eds. Singapore: Springer Nature Singapore, 2022, pp. 557–569. [Online]. Available: https://doi.org/10.1007/978-981-19-1804-9 42
  • D. W. Thiang, “Limited speech recognition for controlling movement of mobile robot implemented on atmega162 microcontroller,” in 2009 International Conference on Computer and Automation Engineering, 2009, pp. 347–350. [Online]. Available: https://doi.org/10.1109/ICCAE.2009.26
  • A. Hussein Shatti Alisawi and A. Qasim Jumaah Althahab, “Improve the performance of voice-excited lpc vocoder in speech coding application,” Al-Qadisiyah Journal for Engineering Sciences, vol. 8, no. 4, pp. 558–568, 2015. [Online]. Available: https://qjes.qu.edu.iq/article 107872.html
  • H. Herath, N. Nishshanka, P. Madhumali, and S. Gunawardena, “Voice control system for upper limb rehabilitation robots using machine learning,” in 2021 IEEE 7th World Forum on Internet of Things (WF-IoT), 2021, pp. 729–734. [Online]. Available: https://doi.org/10.1109/WF-IoT51360.2021.9595827
  • M. H. Asyali, M. Yilmaz, and M. Tokmakc¸i, “Design and implementation of a voice-controlled prosthetic hand,” Turkish Journal of Electrical Engineering and Computer Sciences, 2011. [Online]. Available: https://api.semanticscholar.org/CorpusID:16860496
  • W. Ghai and N. Singh, “Literature review on automatic speech recognition,” International Journal of Computer Applications, vol. 41, no. 8, pp. 42–50, March 2012. [Online]. Available: https://doi.org/10.5120/5565-7646
  • A. Kuzdeuov, R. Gilmullin, B. Khakimov, and H. A. Varol, “An open-source tatar speech commands dataset for iot and robotics applications,” in IECON 2024 - 50th Annual Conference of the IEEE
    Industrial Electronics Society, 2024, pp. 1–5. [Online]. Available: https://doi.org/10.1109/IECON55916.2024.10905876
  • L. Yuan and Z. Chen, “A novel statistical model for speech recognition and pos tagging,” in 2006 IEEE International Conference on Video and Signal Based Surveillance, 2006, pp. 61–61. [Online]. Available:
    https://doi.org/10.1109/AVSS.2006.9
  • H. Ibrahim and A. Varol, “A study on automatic speech recognition systems,” in 2020 8th International Symposium on Digital Forensics and Security (ISDFS), 2020, pp. 1–5. [Online]. Available: https://doi.org/10.1109/ISDFS49300.2020.9116286
  • A. Ahmed Abed and A. A. Jasim, “Design and implementation of a wireless voice-controlled mobile robot,” Al-Qadisiyah Journal for Engineering Sciences, vol. 9, no. 2, pp. 135–147, 2016. [Online].
    Available: https://qjes.qu.edu.iq/article 179828.html
  • H. A. Abdulmohsin, H. B. Abdul wahab, and A. M. J. Abdul hossen, “A new proposed statistical feature extraction method in speech emotion recognition,” Computers Electrical Engineering, vol. 93, p. 107172, 2021. [Online]. Available: https://doi.org/10.1016/j.compeleceng.2021.107172
  • C. Panagiotakis and G. Tziritas, “A speech/music discriminator based on rms and zero-crossings,” IEEE Transactions on Multimedia, vol. 7, no. 1, pp. 155–166, 2005. [Online]. Available: https://doi.org/10.1109/TMM.2004.840604
  • D. Nagajyothi and P. Siddaiah, “Speech recognition using convolutional neural networks,” International Journal of Engineering and Technology, vol. 7, no. 4.6, p. 133–137, Sep. 2018. [Online]. Available: https://www.sciencepubco.com/index.php/IJET/article/view/20449
  • “Ieee signal processing society,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 25, no. 4, pp. C3–C3, 2017. [Online]. Available: https://doi.org/10.1109/TASLP.2017.2684725
  • N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, p. 1929–1958, Jan. 2014.
  • R. Jozefowicz, O. Vinyals, M. Schuster, N. Shazeer, and Y. Wu, “Exploring the limits of language modeling,” Computation and Language, 2016. [Online]. Available: https://doi.org/10.48550/arXiv.1602.0241
  •  J. Kacprzyk, O. Kaynak, W. Pedrycz, M. M. Polycarpou, and J. Wang, Lecture Notes in Networks and Systems Volume 430. Springer Cham, 2016. [Online]. Available: https://link.springer.com/bookseries/15179
  • K. Gundogdu, S. Bayrakdar, and I. Yucedag, “Developing and modeling of voice control system for prosthetic robot arm in medical systems,” Journal of King Saud University - Computer and Information
    Sciences, vol. 30, no. 2, pp. 198–205, 2018. [Online]. Available: https://doi.org/10.1016/j.jksuci.2017.04.005
  • J. Salman, T. R. Saeed, and A. H. Ali, “Design and implementation of a spoken letter recognition system for controlling the upper prosthetics,” in 2018 Third Scientific Conference of Electrical Engineering (SCEE), 2018, pp. 174–179. [Online]. Available: https://doi.org/10.1109/SCEE.2018.8684098
Al-Qadisiyah Journal for Engineering Sciences
Volume 18, Issue 3
Summer 2025
Pages 320-326

  • Receive Date 16 May 2024
  • Revise Date 05 August 2024
  • Accept Date 20 August 2025

Submit Manuscript

Guide for Authors

Article Processing Charges (APC)

Reviewers

Call for Reviewers

Contact Us