ERUSLR: A new Turkish sign language dataset and its recognition using hyperparameter optimization aided convolutional neural network


ÖZCAN T., BAŞTÜRK A.

JOURNAL OF THE FACULTY OF ENGINEERING AND ARCHITECTURE OF GAZI UNIVERSITY, cilt.36, sa.1, ss.527-542, 2021 (SCI-Expanded) identifier identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 36 Sayı: 1
  • Basım Tarihi: 2021
  • Doi Numarası: 10.17341/gazimmfd.746793
  • Dergi Adı: JOURNAL OF THE FACULTY OF ENGINEERING AND ARCHITECTURE OF GAZI UNIVERSITY
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Art Source, Compendex, TR DİZİN (ULAKBİM)
  • Sayfa Sayıları: ss.527-542
  • Anahtar Kelimeler: Sign language recognition, transfer learning, GoogLeNet, parameter optimization, global search, heuristic search
  • Erciyes Üniversitesi Adresli: Evet

Özet

Sign language is one of the most important tools of communication for deaf-and-dumb individuals who have lost their linguistic and auditory abilities. Significant problems may arise in situations where the sign language required to clearly understand deaf-and-dumb individuals is not known. More importantly, the failure to understand the disabled individuals who try to access emergency health services at a health institution may have fatal consequences. In this study, firstly, a new dataset was created with the frequently used words in the emergency department of hospitals. 25 words were repeated multiple times by 49 handicapped individuals where the videos were recorded from different angles. This dataset, named Erciyes University Sign Language Recognition (ERUSLR), contains 13186 samples. Classification experiments were performed by using the ERUSLR dataset. Sign language recognition can be realized by a convolutional neural network (CNN), which is frequently used for classification problems. Rather than developing a new CNN model, transfer learning, a more effective method, is preferred. Therefore, a GoogLeNet-based CNN model was created by transfer learning from the GoogLeNet pretrained model. Another factor that increases the performance of a CNN model is the optimization of its training parameters. Global and heuristic search methods are typically used in parameter optimization to save time. In this study, both grid search (GS), random search (RS), and genetic algorithm (GA) methods were used to optimize the training parameters of the GoogLeNet-based CNN model. According to the experimental results, the GA supported GoogLeNet-based CNN model is more successful (with a success rate of 93.93%) than the other methods.