enable talk

Gloves Enable Talk

Submitted to:Dr-Falah mohammad

Prepared by:Alaa mayyaleh Shuruoq abu salhieh

:إهداء

صغارا ربيانا من . إلى

والمعرفة العلم طريق لنا وأنار ، بيدنا وأخذ ، علمنا من كل .إلى

والنجاح التميز إلى رحلتنا في شجعنا من كل . إلى

بجانبا ووقف ، ساندنا من كل . إلى

تحفيزنا : في سببا فكان ، ال لنا قال من كل . إلى

سبيله والتميز ، هدفه والتفوق ، طريقه النجاح كان من كل . إلى

واإلحترام والتقدير الشكر جميعا . إليكم

1 | P a g e

Acknowledgment:

We would like to express our gratitude for everyone who helped us during the graduation project

(Gloves Enable Talk ) starting with endless thanks for Eng. Abdullah Hinnawi who didn’t keep any effort in encouraging us to do a great job, providing our group with valuable information and advice to be better in each time . Thanks for the continuous support and kind communication which had a great effect regarding to feel interesting about what we are working on .

Thanks are extended to Our supervisor Dr.Falah mohammed who gave us the Opportunity to work freely , which Support creativity in us , that was what we are looking for and which facilitated many things in our project .

Many thanks for Dr . Sufian samara for his a advice during programming .Thanks are extended to all instructors and engineers who helped us during the first stage of our graduation project .

Also, we would like to say many thanks for the our families and friends for their endless support

And their huge effort in contacting and providing us with all what we need, who we wish to enjoy the presentation and hopefully to be satisfied.

2 | P a g e

Table of Contents (TOC):

Tables of Contents

Acknowledgement......................................................................................................................2

list of figures..............................................................................................................................4

list of tabels................................................................................................................................4

Abstract......................................................................................................................................5

Chapter 1: Introduction..............................................................................................................6

Chapter 2: Constraints, Standards /codes and Earlier course work...........................................8

2.1 Constraints........................................................................................................................8

2.2 standards codes...............................................................................................................10

2.3Earlier coursework.......................................................................................................19

Chapter 3: Literature Review...................................................................................................20

Chapter 4: Methodology..........................................................................................................22

Chapter 5: Results and Analysis..............................................................................................44

Chapter 6: conclusion...............................................................................................................48

References................................................................................................................................52

3 | P a g e

List of Figures (LOF):

Figure 1 : charactrisitc of the project Figure 2 : the old sympol Figure 3 : the pic circuit Figure 4 : the gloves. Figure 5 : final gloves Figure 6 : basic circuit Figure 7 : RN blutooth Figure 8 :pic with RN Figure 9 : RN with Rduino Figure 10 : arduno result Figure 11 : andriod application Figure 12 : android application Figure 13 : final result Figure 14 : final result

List of Tables (LOT):

Table 1: blutooth modular properties ......................................................................................27

4 | P a g e

Abstract:

Enable Talk is a creative project, whose main idea is to translate hand signs into speech, the concept of the project considered of two sensory gloves and a mobile device which entailed recognition process. The whole idea is to give a voice to the voiceless. Giving a voice to the 70 million people who live every day with speech and hearing impairments this project aim is to create something that takes on one of “the world’s toughest problem”. Most of the projects focused on the environment or healthcare. But we tackled a much more basic problem: communication., A new way for easy life , now you can talk and send messages to your friends without looking to your mobile screen , your hand in your pocket ! your eyes is free ! . The Enable Talk gloves work by translating the gestures of the user’s hands through a text-to-talk engine connected to a Smartphone This project “Speaking hands” talk about a brave new world .it convert sign languages to voice in Arabic and English .

We use components which is the RN Bluetooth modular, Arduino microcontroller circuit and programming it to send the signs as characters through the Bluetooth technology to the mobile application. Glove hardware design to connect to sensors on the fingers and the android programming which convert the received character to voice and collecting all of these through interfaces , we have a good result tested on 5 words , and we can add words as much as sensors we have on the gloves so it’s a kind of an open source , and for the future work we want to develop our project to be more professional and ready for anybody to use it .

5 | P a g e

Chapter 1: Introduction

This chapter gives an overview of what was discussed in the Introduction of Graduation Project the past semester. This part of the project digs deep in the main core of the project which is Talk gloves.

According to the scope of the project, there are an estimated 70 million deaf, mute, or deaf-mute people in the world today. And while many of them can communicate with each other through sign language, there’s a linguistic wall between them and people who can speak and write but don’t know signs .from this point we are trying to bring down that barrier. So to design a system called Enable Talk.

Definition :

A simple definition for talk gloves project which is a hardware that includes two gloves that can sense the movements of the wearer’s hands and fingers and translate the signs into spoken words by a Smartphone.

6 | P a g e

http://enabletalk.com/abstract.html

Figure1: characteristic of the project

Why talk gloves ?

Here are some of its applications:

To solve deaf-mute people problem by give a voice to the voiceless. To help deaf families, companies and community to communicate with them by a simple

way regardless that they know sign languages or not. The talking gloves could be used by non-deaf people as a computer keyboard or mouse

in the near future. It will Support the educational process of Autism Childs. It is also used by interpreters spelling gloves which translate sign into speech, side by

side they do the same job but the gloves in wide way.

Motivation for Carrying out the Research:

We would like to establish a hard ware to solve 70 million person problems all over the world, especially in Palestine that Statistically in 2011 says that in Palestine the number of disability “deaf-mute” reached the 113000 person, 40000 from them in Gaza.

Organization of the report :

The main body of the report is preceded by detailed contents including lists of figures, tables, and annexes followed by units used in the report. This is followed the abstract which summary giving briefly the scope and objectives of the project, The first chapter is introduction to our project , the second one is about Constraints, Standards/ Codes and Earlier course , the third chapter Literature Review which Includes citation of relevant work and results , after that chapter four describe the Methodology , followed by Results and Analysis in chapter five , and discussion in chapter six and finally the Conclusions and Recommendation in chapter seven followed by References ,appendices and attachments .

7 | P a g e

Chapter 2: Constraints, Standards /codes and Earlier course work

2.1Constraints:

We had constraints while working on this project, that mention bellow:

1- Designing the hardware part wasn't that easy at all, to make two gloves with one pic circuit wasn't comfortable with the expecting movement of anybody hands through waking for example , tall wire was needed . then after many times trying to connected it, it wasn't a good idea to continue like this . so we thought about using two pic basic circuit to deal with this problem , every pic circuit is connected independently with a hand ,two circuit for two hands . but also that didn't work . because we have one Bluetooth to transmit data through , and it has one transmitter and one receiver which connected serially to pic circuit pins . and the idea of two independent circuits in all wrong because the two hands should make some signs with each other and this was another problem to consider . briefly , we choose to use one pic for one gloves just to test the codes on and to make sure to make our idea work "transmit signs signals to spoken words ",hoping after all to develop the project to take two hands in account and make our work more professional.

2- before , we had decided to use pic16f788 , So we connect the basic circuit of this pic and test it many times until it succeed , then we learned a lot about c++ programming and made a complete code to do the job "transmit the signs to numbers before transmit it through Bluetooth to mobile " , later on , after many times testing that on pic when we have connected the Bluetooth with pic circuit many errors happened , that the baud ratio between pic16f877 and Rn42 “Bluetooth device” , are not matched . testing these issues and trying to fix it take a lot of time “around month and a half ” , we will mention the details later , but after all , we forced to use arduino as Alternative.

3- another problem in hardware part was the gloves itself , to decide the proper gloves Material to use was difficult because we need something simple, easy to move , thin as possible for finger movements because of sensor between them , and thermally and electrically insulating. However we have many choices to see from like : Wool, leather, linen, polyester, nylon, velvet. Anyway we tried it all , then we decided to use leather because it's was comfortable more than the others , in size and movement and the other material have much problem like easy Rupture ,Heat ,Lack of electrical insulation ,Uncomfortable in use and Bad look .

8 | P a g e

4- we have a problem in programming languages, because we need to use c pic , c++ and android "java" programming programs , and after a while we forced to use arduino that has it's own programming way , and we are not familiar with any of them at all, so learning all of this in two months or less and trying to make your own codes to connect all of that together was very hard , Psychologically and physically tiring .

5- the signs of deaf and mute persons have two standards . unfortunately both are very complicated to be done by touch sensors "bush buttons " on the hands fingers , So after deciding to take one hand to accomplish our idea on it we avoided the standards signs , and decide to use some signs from our mind just to prove that the main idea of our project ( translating the signs to words) could be done .

6- our main constraint that our project is the first of its kind , yes some project lately tried to accomplish the idea but not the same way we take our decision on . so there is no background to take experience and feedback from . beginning from zero was really tough task in our main constraint that your project is the first of its kind , yes some project lately tried to accomplish the idea but not the same way we take our decision on . so there is no background to take experience and feedback from . beginning from zero was really tough task with limited time .

7-we faced a difficult problem that the arduino and pic in same place can’t transmit words through air so we need to transmit it as numbers (10010101) for example or characters .

9 | P a g e

2.2Standards/Codes :

We wrote several three languages (c++ , arduino, android)

I would mention the main codes that we used and some of the experimental

ones too .

first this is the c++ code for pic16f877 programming ; when we tried to make the fingers sensor enter the signs as Characters by connecting it to pins from A1-A5 of pic , any analog signal will determine the whole port binary number that would be transmitted through the Bluetooth :

#include "C:\Users\aala\Desktop\pic c\p.h"

int i;int A0;void main(){i=0;A0=i; setup_adc_ports(NO_ANALOGS); setup_adc(ADC_OFF); setup_psp(PSP_DISABLED); setup_timer_0(RTCC_INTERNAL|RTCC_DIV_1); setup_timer_1(T1_DISABLED); setup_comparator(NC_NC_NC_NC); setup_vref(FALSE);

// TODO: USER CODE

while(1){restart_wdt();i=0;while(input_A()== 0b00000000){delay_ms(100);restart_wdt();if(input_A()!=0b00000000){delay_ms(500);delay_ms(500);delay_ms(500);

10 | P a g e

if(input_A()==0b00000001) {output_high(PIN_B0);

printf("a");}if(input_A()==0b00000010) {output_high(PIN_B1);printf("b");}if(input_A()==0b00000011) {output_high(PIN_B2);printf("c");} if(input_A()==0b00000100) {output_high(PIN_B3);printf("d");}if(input_A()==0b00000101) {output_high(PIN_B4);printf("e");}if(input_A()==0b00000110) {output_high(PIN_B5);printf("f");} if(input_A()==0b00000111) {output_high(PIN_B0);printf("g");} if(input_A()==0b00001000) {output_high(PIN_B0);printf("h");} if(input_A()==0b00001001) {output_high(PIN_B0);printf("l");} if(input_A()==0b00001010) {output_high(PIN_B0);printf("j");} if(input_A()==0b00001011) {output_high(PIN_B0);printf("k");} if(input_A()==0b00001100) {

11 | P a g e

output_high(PIN_B0);printf("l");} if ((input_A()==0b00001101)) {output_high(PIN_B0);printf("m");} if ((input_A()==0b00001110)){output_high(PIN_B0);printf("n");} if ((input_A()==0b00001111)) {output_high(PIN_B0);printf("o");} if ((input_A()==0b00010000)) {output_high(PIN_B0);printf("p");} if ((input_A()==0b00010001)) {output_high(PIN_B0);printf("q");} if ((input_A()==0b00010010)) {output_high(PIN_B0);printf("r");} if ((input_A()==0b00010011)) {output_high(PIN_B0);printf("s");} if ((input_A()==0b00010100)) {output_high(PIN_B0);printf("t");} if ((input_A()==0b00010101)) {output_high(PIN_B0);printf("u");} if ((input_A()==0b00010110)) {output_high(PIN_B0);printf("v");} if ((input_A()==0b00010111)) {output_high(PIN_B0);printf("w");

12 | P a g e

} if ((input_A()==0b00011000)) {output_high(PIN_B0);printf("x");}if ((input_A()==0b00011001)) {output_high(PIN_B0);printf("y");} if ((input_A()==0b00011010)) {output_high(PIN_B0);printf("z");} if ((input_A()==0b00011011)) {output_high(PIN_B0);printf(" ");} if ((input_A()==0b00011100)) {output_high(PIN_B0);printf(".");

} else if ((input_A()==0b00011101)) {output_high(PIN_B0);printf("good");

} if ((input_A()==0b00011110)) {output_high(PIN_B0);printf("end");

} if ((input_A()==0b00011111)) {output_high(PIN_B0);printf("HI");}}}}}

13 | P a g e

The second one is the arduino code we tried to make it as simple as we can ; it’s an alternative for the pic because the pic has some problems with synchronizing with the bluetooth , as the baud ratio was fixed but some delays happens . so this is the our code that will make characters to transmit rather than numbers “zero ones” , in the receiver side this characters have in contrast words . character "a" in transmitter could mean for example the word "all" . that's because transmit the whole word was impossible , it's being received as cut one with missed characters .

This is the code :

#include <SoftwareSerial.h>

int bluetoothTx = 2;

int bluetoothRx = 3;

SoftwareSerial bluetooth(bluetoothTx, bluetoothRx);

const int sensor_1 = 5;




const int sensor_5= 9;


int firsttime=1;

void setup(){

Serial.begin(9600);

bluetooth.begin(9600);

}

void loop()

{

if(firsttime){

setup();

14 | P a g e

Serial.println("Checking");

firsttime=0;

}

if(digitalRead(sensor_1) == HIGH) // if the first sensor on => button ( connected with 5 volt)

{

Serial.println("a"); // this is for seeing "a" on arduino serial (hyper terminal)

bluetooth.print("a"); // this is for transmit "a" via bluetooth to android app

}

if(digitalRead(sensor_2) == HIGH) {

Serial.println("b");

bluetooth.print("b");

}

if(digitalRead(sensor_1) == HIGH && digitalRead(sensor_2) == HIGH) {

Serial.println("c");

bluetooth.print("c");

}if(digitalRead(sensor_3) == HIGH) {

Serial.println("d");


}if(digitalRead(sensor_1) == HIGH && digitalRead(sensor_3) == HIGH) {

Serial.println("e");

bluetooth.print("e");


Serial.println("f");

bluetooth.print("f");

15 | P a g e

}if(digitalRead(sensor_3) == HIGH&&digitalRead(sensor_1) == HIGH && digitalRead(sensor_2) == HIGH) {

Serial.println("g");

bluetooth.print("g");


Serial.println("h");

bluetooth.print("h");

}if(digitalRead(sensor_1) == HIGH&&digitalRead(sensor_4) == HIGH) {

Serial.println("i");

bluetooth.print("i");


Serial.println("j");

bluetooth.print("j");

}if(digitalRead(sensor_2) ==HIGH&&digitalRead(sensor_1) == HIGH&&digitalRead(sensor_4) == HIGH) {

Serial.println("k");

bluetooth.print("k");


Serial.println("l");

bluetooth.print("l");

}if(digitalRead(sensor_3) == HIGH&&digitalRead(sensor_1) == HIGH&&digitalRead(sensor_4) == HIGH) {

Serial.println("m");

bluetooth.print("m");


Serial.println("n");

16 | P a g e

bluetooth.print("n");

}if(digitalRead(sensor_2) == HIGH&&digitalRead(sensor_1) == HIGH&&digitalRead(sensor_4) == HIGH&&digitalRead(sensor_3) == HIGH) {

Serial.println("o");

bluetooth.print("o");


Serial.println("p");

bluetooth.print("p");


Serial.println("q");

bluetooth.print("q");

}

if(digitalRead(sensor_2) == HIGH&&digitalRead(sensor_5) == HIGH) {

Serial.println("r");

bluetooth.print("r");


Serial.println("s");

bluetooth.print("s");


Serial.println("t");

bluetooth.print("t");


Serial.println("u");

17 | P a g e

bluetooth.print("u");


Serial.println("v");

bluetooth.print("v");


Serial.println("w");

bluetooth.print("w");


Serial.println("x");

bluetooth.print("x");


Serial.println("y");

bluetooth.print("y");


Serial.println("z");

bluetooth.print("z");


Serial.println(" ");

bluetooth.print(" ");


Serial.println(",");

18 | P a g e

bluetooth.print(",");

}

}

And the Android Code will be mentioned later

2.3Earlier coursework :

This project is the first of his kind in all the details in its , but about the idea to translate the signs to words we read about some experience and motivate us to do the job in different way :

According to the World Federation of the Deaf SEPTEMBER 5, 2013there are “approximately 70 million deaf people worldwide”. This is a huge number when you think that 70 million people don’t have the ability to communicate with people who don’t know sign language. As a response to this communication issue a group of Ukrainian students invented a smart pair of gloves that can put sign language into vocalized speech using a Smartphone.

Inventors Anton Stepanov, Anton Posternikov and Maxin Osika created the Enable Talk gloves, a device equipped with 15 flex sensors that can sense bending degree of fingers, touch sensors, gyroscopes and accelerometers. These gloves store data sent via Bluetooth to a Smartphone. The position of fingers is translated into text and text into spoken words. All this technology can be applied to grant hearing impaired people a voice and the power of communication. The Enable Talk project got the first prize at Microsoft’s Imagine Cup 2012 in Sydney, Australia

19 | P a g e

Chapter 3: Literature Review

Our previous work in the last semester include some hardware test connections :

First of all , we bring one glove to make a test demo of the first step of the project, our thinking was to use a flex sensors that can sense bending degree of fingers ,but we have a problem in the local market that’s this kind of sensors doesn’t exist, so in the demo we solved that by using copper as touch sensor and we connected it as it show in the flowing figure .

Figure 2:the old symbol

After that we prepare the microcontroller circuit as shown in figure 5 , to connected it with the glove, to be program in project 2 to define the movement of the glove by the a code.

Figure 3: the pic circuit

20 | P a g e

the gloves connected with the microcontroller as shown in figure :

Figure4: gloves with the circuit

21 | P a g e

Chapter 4: Methodology

Now the important part of the project is the methodology in details:

previously we mention the first part in the first semester which is the pic circuit and designing of the gloves .

we designed the gloves that we have 6 sensors separating at the top of the 5 fingers and a common one on the middle of the hand

Figure4:the gloves

22 | P a g e

The final gloves is this

Figure5: the final gloves

In this semester we tried to make the pic circuit as small as possible so that we can put it on the hand gloves while anybody wearing it easily . as shown in figures bellow

23 | P a g e

Figure 6: the basic circuit

the main idea that we discard some component which is the serial ones . So we can reload codes through boot loader programming .

Overview of PIC 16F877 :

PIC 16F877 is one of the most advanced microcontroller from Microchip. This controller is widely used for experimental and modern applications because of its low price, wide range of applications, high quality, and ease of availability. It is ideal for applications such as machine control applications, measurement devices, study purpose, and so on.

After that we bring the Bluetooth device, we choose rn24 devise because it's easy in use and it's transmit binary numbers up to 8 digit , also it have compatibility to transmit characters too .

24 | P a g e

http://www.circuitstoday.com/wp-content/uploads/2011/01/PIC-16F877.gif

Rn42 Overview:

The RN42 is a small form factor, low power, Class 2 Bluetooth radio for designers who want to add wireless capability to their products. The RN42 supports multiple interface protocols, is simple to design in, and is fully certified, making it a complete embedded Bluetooth solution. The RN42 is functionally compatible with the RN41. With its high-performance PCB trace antenna and support for Bluetooth EDR, the RN42 delivers up to 3 Mbps data rate for distances up to 20 meters.

We will use the transmitter, receiver ,ground and 5 VDD pins in this microchip , the pins is so sensitive and so small that it's hard to connected it directly so we connect the pins to wires and this wires to a socket to make the operation easier on us while dealing with it.

Figure 7: the RN Bluetooth

The pins are RX , TX ,GND,VDD from right to left.

Here is some important data from the data sheet of RN42 microchip:

The supply voltage from 3.3 up to 3.6 volt

25 | P a g e

http://www.microchip.com/wwwproducts/Devices.aspx?dDocName=en558328

And the figure below describe some important details about the characteristic of pins

Table 1: the Bluetooth module properties

We mention something about the problem that faced us , that the pic16f877 baud ratio is 9600 by default and the Rn42 baud ratio =115200 . and this information came after many tries to connect between both of them and send data , according to the figure above the solution was in coding and another solution in pin 4 , if we connect it with 5 VCC it will make the baud ratio 9600 as default , so we take this solution and did it.

Then we should connect the Transmitter of pic 16f788 with the receiver of the Rn42 and the transmitter of the rn42 with the receiver of the pic taking in account to common the ground and the Vcc . as shown below

26 | P a g e

Figure8: te pic with RN

And here is some of picture that describe the trying process on HyperTerminal on pc .

Some problem was as the program make delays too much and there is problem as it print just ascii . so we decide after too long to use arduino because controlling baud ratio in it more easier than pic circuit and that time we were too late in time that we have less than 3 weeks to accomplish all the project .

27 | P a g e

What Arduino can do :

Arduino can sense the environment by receiving input from a variety of sensors and can affect its surroundings by controlling lights, motors, and other actuators. The microcontroller on the board is programmed using the Arduino programming language (based on Wiring) and the Arduino development environment (based on Processing). Arduino projects can be stand-alone or they can communicate with software running on a computer (e.g. Flash, Processing , Max MSP).

Figure9: the arduino

The Arduino Uno (that we used ) has a number of facilities for communicating with a computer, another Arduino, or other microcontrollers. The ATmega328 provides UART TTL (5V) serial communication, which is available on digital pins 0 (RX) and 1 (TX). An ATmega16U2 on the board channels this serial communication over USB and appears as a virtual com port to software on the computer. The '16U2 firmware uses the standard USB COM drivers, and no external driver is needed. However, on Windows, a .inf file is required. The Arduino software includes a serial monitor which allows simple textual data to be sent to and from the Arduino board. The RX and TX LEDs on the board will flash when data is being transmitted via the USB-to-serial chip and USB connection to the computer (but not for serial communication on pins 0 and 1).A SoftwareSerial library allows for serial communication on any of the Uno's digital pins.

28 | P a g e

http://www.processing.org/

http://wiring.org.co/

http://arduino.cc/en/Reference/HomePage

So we connect the Bluetooth with arduino as the data sheet of both supposed and try codes on the arduino to test that it’s work correctly with the Bluetooth .

Figure9: Rn with Bluetooth

29 | P a g e

Below there is a figure can describe the process . we write this code to print Aurd on the serial of the arduino “screen ” .

/*

AnalogReadSerial

Reads an analog input on pin 0, prints the result to the serial monitor.

Attach the center pin of a potentiometer to pin A0, and the outside pins to +5V and ground.

This example code is in the public domain.

*/

// the setup routine runs once when you press reset:

void setup() {

// initialize serial communication at 9600 bits per second:

Serial.begin(9600);

}

// the loop routine runs over and over again forever:

void loop() {

// print out the value you read:

Serial.println(Aurd);

delay(1); // delay in between reads for stability

}

30 | P a g e

And that success after trying as shown :

Then , we learned how to make codes to input sensor value on digital pins of arduino and to make some orders on each sensor as required .

previously we decide to use 5 pins on arduino to input 25 value that is the english alphabet as (0 ,1) for example ( 10110 ) that’s mean the second , third and five pin are connected with 5 volt . and the rest are zero voltage that the android application should collect the alphabet after (-) space or dash movement we decided before and another sign to speak the word on the mobile . but before that we make some tries to transmit a whole words in arabic and english .

31 | P a g e

and here is the first success code we made :







int firsttime=1;

void setup(){

Serial.begin(9600);

}

void loop()

{

if(firsttime){

setup();

Serial.println("Checking"); // it will print this in the first time of testing

firsttime=0;

}

if(digitalRead(sensor_1) == HIGH) // if pin 1 are on 5 Vcc it will print and transmit the word hello

{

Serial.println(" hello !");

Serial.println("مرحبا ");

}

}

32 | P a g e

delay (3000);

if(digitalRead(sensor_2) == HIGH) {// if pin 2 are on how are you Vcc it will print and transmit the word hello

Serial.println(" how are you ! ");

Serial.println(" ؟ حالك ;("كيف

}

}

}

The result was this after trying and success (it print the English only ) :

Figure10: Arduino result

33 | P a g e

then the second step is to establish the alphabet signs and prepare it to transmit since the whole word in bluetooth can't be send like ascii , but numbers or characters and we choose characters ..

and this is the final code we made and use .. 6 pins were used :


int bluetoothTx = 2;

int bluetoothRx = 3;

SoftwareSerial bluetooth(bluetoothTx, bluetoothRx);

const int sensor_1 = 5; //pin 3






int firsttime=1;

void setup(){

Serial.begin(9600);

bluetooth.begin(9600);

}

void loop()

{

if(firsttime){

setup();

Serial.println("Checking");

firsttime=0;

34 | P a g e

}

if(digitalRead(sensor_1) == HIGH) // if the first sensor on => button ( connected with 5 volt)

{

Serial.println("a"); // this is for seeing "a" on arduino serial (hyper terminal)

bluetooth.print("a"); // this is for transmit "a" via bluetooth to android app

}

if(digitalRead(sensor_2) == HIGH) {

Serial.println("b");

bluetooth.print("b");

}

if(digitalRead(sensor_1) == HIGH && digitalRead(sensor_2) == HIGH) {

Serial.println("c");



Serial.println("d");



Serial.println("e");

bluetooth.print("e");


Serial.println("f");

bluetooth.print("f");

}if(digitalRead(sensor_3) == HIGH&&digitalRead(sensor_1) == HIGH && digitalRead(sensor_2) == HIGH) {

Serial.println("g");

35 | P a g e

bluetooth.print("g");


Serial.println("h");

bluetooth.print("h");


Serial.println("i");

bluetooth.print("i");


Serial.println("j");

bluetooth.print("j");

}if(digitalRead(sensor_2) ==HIGH&&digitalRead(sensor_1) == HIGH&&digitalRead(sensor_4) == HIGH) {

Serial.println("k");

bluetooth.print("k");


Serial.println("l");

bluetooth.print("l");


Serial.println("m");

bluetooth.print("m");


Serial.println("n");

36 | P a g e

bluetooth.print("n");


Serial.println("o");

bluetooth.print("o");


Serial.println("p");

bluetooth.print("p");


Serial.println("q");

bluetooth.print("q");

}

if(digitalRead(sensor_2) == HIGH&&digitalRead(sensor_5) == HIGH) {

Serial.println("r");

bluetooth.print("r");


Serial.println("s");

bluetooth.print("s");


Serial.println("t");

bluetooth.print("t");


Serial.println("u");

bluetooth.print("u");

37 | P a g e


Serial.println("v");

bluetooth.print("v");


Serial.println("w");

bluetooth.print("w");


Serial.println("x");

bluetooth.print("x");


Serial.println("y");

bluetooth.print("y");


Serial.println("z");

bluetooth.print("z");


Serial.println(" ");

bluetooth.print(" ");


Serial.println(",");

bluetooth.print(",");

38 | P a g e

}

}

the final step is android programming that will make the application that will receive the characters and translate them to words and speak it.

In android we use eclipse program to do the job

We have 3 steps

1- to open the Bluetooth of the mobile and make scan on the devices that have open Bluetooth.

2- to connect and pairing with them .

3-to write an instruction in the Bluetooth code to receive data and to make the application using two languages Arabic and english.

The first step here is scanning and this is the code of this part :

bluetoothOnRun = new Runnable() {

//@SuppressLint("NewApi")

@Override

public void run() {

// TODO Auto-generated method stub

while(myBluetoothAdapter.isEnabled())

{

if(!isConnected)

{

find();

}

39 | P a g e

try {

Thread.sleep(7000);

} catch (InterruptedException e) {

// TODO Auto-generated catch block

e.printStackTrace();

}

}

on();

while(!myBluetoothAdapter.isEnabled())

{

try {

Thread.sleep(500);

} catch (InterruptedException e) {

// TODO Auto-generated catch block

e.printStackTrace();

}

}

handler.post(new Runnable() {

@Override

public void run() {


Toast.makeText(getApplicationContext(),"Bluetooth turned on" , Toast.LENGTH_LONG).show();

40 | P a g e

}

});

Thread turnBO = new Thread(bluetoothOnRun);

turnBO.start();

find();

}

};

Thread turnBO = new Thread(bluetoothOnRun);

turnBO.start();

} }

The second part in android is to connect with any Bluetooth that was found in scan process

And this is the code :

void connect_device(BluetoothDevice device)

{

BluetoothSocket tmp = null;

mmDevice = device;

//MY_UUID = device.getUuids()[0].getUuid();

// Get a BluetoothSocket to connect with the given BluetoothDevice

try {

// MY_UUID is the app's UUID string, also used by the server code

tmp = device.createRfcommSocketToServiceRecord(MY_UUID);

} catch (IOException e) { }

mmSocket = tmp;

41 | P a g e

Thread connectTh = new Thread(new Runnable() {

@Override

public void run() {


try {

// Connect the device through the socket. This will block

// until it succeeds or throws an exception

statusMessage = "Connecting with " + mmDevice.getName();

write_status();

mmSocket.connect();

statusMessage = "Listening to " + mmDevice.getName();

write_status();

handler.post(new Runnable() {

@Override

public void run() {


isConnected = true;

scanBut.setText("Close");

myListView.setVisibility(View.INVISIBLE);

receivedMessage.setVisibility(View.VISIBLE);

receivedMessage.setText("");

title.setText("Received messages:");

}

42 | P a g e

});

} catch (IOException connectException) {

// Unable to connect; close the socket and get out

statusMessage = "Con't connected";

write_status();

try {

mmSocket.close();

} catch (IOException closeException) { }

return;

}

// Do work to manage the connection (in a separate thread)

manageConnectedSocket(mmSocket);

}

});

connectTh.start();

}

43 | P a g e

Chapter 5: Results and Analysis:

The result here is pictures of the app screen and blutooth:

Figure11: android apps

44 | P a g e

Figure12: android apps

45 | P a g e

Figure13:the final result

46 | P a g e

Figure14:the final result

47 | P a g e

Chapter 6: conclusion

We made this project to make life easy through translating the signs to spoken words .

In near future we would make some development on our project to take in account the standard signs around the word especially for deaf and mute people .

We consider sample words around 10 to improve our idea and achieve your goal and we did it and we can add up to 100 words in Arabic and 100 in English .

We faced many problems with pic Bluetooth with learning the developing of android application and changing pic to arduino but finally we can do the idea of our project for some words in Arabic and English.

References:

48 | P a g e

https://www.smartairmedia.com/content/technology/item/java

http://www.ukessays.com/essays/computer-science/current-situation-of-operating- systems-in-smartphones-computer-science-essay.php

http://digimark.bg/en/programming-languages

http://cellphones.about.com/od/smartphonebasics/a/what_is_smart.htm

http://enabletalk.com/

http://newsfeed.time.com/2012/07/11/talk-with-your-hands-gloves-translate-sign- language-into-speech/

http://stab-iitb.org/wiki/Talk:Hand_Gesture_Input_Gloves

http://www.smartplanet.com/blog/rethinking-healthcare/gloves-turn-sign-language-into- speech/9489

http://blog.inventright.com/2013/09/05/talking-gloves/

http://thenewsconnections.com/2012/08/15/enable-talk-gloves-the-end-for-sign- language-interpreters.aspx

http://felesteen.ps/details/news/94421/

http://alhayat.com/Details/521099

http://www.nidcd.nih.gov/health/hearing/pages/asl.aspx

http://www.monkmakes.com/?page_id=55

http://thenewboston.org/list.php?cat=6

49 | P a g e

http://thenewboston.org/list.php?cat=6

http://www.monkmakes.com/?page_id=55

http://cellphones.about.com/od/smartphonebasics/a/what_is_smart.htm

http://digimark.bg/en/programming-languages

http://www.ukessays.com/essays/computer-science/current-situation-of-operating-systems-in-smartphones-computer-science-essay.php

http://www.ukessays.com/essays/computer-science/current-situation-of-operating-systems-in-smartphones-computer-science-essay.php

https://www.smartairmedia.com/content/technology/item/java

enable talk

Engineering