Your Australian University in Dubai

News Article

It’s more than just a game: UOWD students use Xbox’s Kinect technology to create Sign Language teaching tool

Thursday, 7 May, 2015

Although many may associate Microsoft’s Kinect system with dance games, sports challenges and living-room fitness fads, final year Computer Science students from the University of Wollongong in Dubai have shown that the Xbox sensor-based technology has a capacity to change lives.

SAMA – a language learning program created by UOWD students Rohit Jaggi, Shuji Sheefeeqh and Hassan Belo, uses the technology to teach sign-language to both the hearing and non-hearing community, offering a dynamic system which gives feedback, picks up mistakes from the user and teaches progressively with online lessons and tests.

With an expandable database of signs, the software allows any organization or individual to use SAMA to teach and learn any type of sign language, be it Emirati Sign Language, American Sign Language or British Sign Language.

“Using clever pattern recognition, we have managed to develop an algorithm that can now compare two gestures and return feedback on the accuracy of the gestures,” said Rohit Jaggi, team member of SAMA project. “With this algorithm, we have developed the software to allow individuals to learn sign language on their own, without the help of another person.”

“From our research we realized that very few of the deaf community are employed” added team leader Shuji Shafiq. “This kind of technology would help them and companies to be more integrated into the community, helping them to work and develop. With around 8 million deaf people in the region alone, our work has the capacity to make a real impact on people’s lives – many of who are either unemployed or in low-paid jobs due to their disability.”

“The project started off as an ambitious attempt at making use of a common gaming technology for the greater community,” commented Dr Zeenath Reza Khan, lecturer and supervisor of the project at UOWD. “But the students have really taken on the challenge and been working hard to acquire and develop the algorithm needed to make Kinect read hands, motion and fingers in order to be able to enhance its application to include sign languages.”

The innovative program is already capturing the imagination of the media, speech and language professionals and the research community alike. Duo Rohit Jaggi and Shuji Shefeeqh recently captured the attention of the judges to win the Best Poster Presentation at the 7th Annual Undergraduate Research Conference hosted by Zayed University, Dubai Academic City.

Competing against more than 500 students who were representing 33 universities from across the region with 123 poster presentations, the team’s design was one of just eight of which were declared  top projects by a panel of expert judges.

Timothy Peck, the Program Executive at IBM MEA University Relations, who gave the keynote speech at the conference, was impressed with the project and hailed the students for their efforts. Dr Abdullah Mohammed Almuhaideb, Vice Dean of Academic Affairs, Chairman of Computer Science Department and Assistant Professor at King Faisal University commented, “this project has real potential for real applications, not just to teach sign language, but also to allow those with hearing impairment to communicate with ease… it can be developed to become a sign language translator to be used in airports, hotel industries and so on.”