09-05-2011, 09:47 AM
[attachment=13438]
INTRODUCTION
Interpretation of human gestures by a computer is used for human-machine interaction in the area of computer vision. The main purpose of gesture recognition research is to identify a particular human gesture and convey information to the user pertaining to individual gesture. From the corpus of gestures, specific gesture of interest can be identified, and on the basis of that, specific command for execution of action can be given to the machine. Overall aim is to make the computer to understand human body language, thereby bridging the gap between machine and human. Hand gesture recognition can be used to enhance human– computer interaction without depending on traditional input devices such as keyboard and mouse. Hand gestures are extensively used for telerobotic control and applications. Robotic systems can be controlled naturally and intuitively with such telerobotic communication. A prominent benefit of such a system is that it presents a natural way to send geometrical information to the robot such as: left, right, etc. Robotic hand can be controlled remotely by hand gestures. Research is being carried out in this area for a long time. Several approaches have been developed for sensing hand movements and corresponding by controlling robotic hand.
Glove based technique is well-known means of recognizing hand gestures. It utilizes sensor-detached mechanical glove devices that directly measure hand and/or arm joint angles and spatial position. Although glove-based gestural interfaces give more precision, it limits freedom as it requires users to wear cumbersome patch of devices. Jae-Ho Shin used entropy analysis to extract hand region in complex background for hand gesture recognition system. Robot controlling is done by Fusion of Hand Positioning and Arm Gestures using data glove. Although it gives more precision, it limits freedom due to necessity of wearing gloves. For capturing hand gestures correctly, proper light and camera angle are required. The problem of visual hand recognition and tracking is quite challenging. Many early approaches used position markers or colored bands to make the problem of hand recognition easier, but due to their inconvenience, they cannot be considered as a natural interface for the robot control. We have proposed a fast as well as automatic hand gesture detection and recognition system. This approach of gesture identification On the basis of recognized hand gesture can be used in any robotic system or machines with a number of specific commands suitable to that system.
Chapter 2
GESTURE RECOGNITION
Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand. Current focuses in the field include emotion recognition from the face and hand gesture recognition. Many approaches have been made using cameras and computer vision algorithms to interpret sign language. However, the identification and recognition of posture, gait, human behaviors is also the subject of gesture recognition techniques.[1]
Gesture recognition can be seen as a way for computers to begin to understand human body language, thus building a richer bridge between machines and humans than primitive text user interfaces or even GUIs (graphical user interfaces), which still limit the majority of input to keyboard and mouse.
Gesture recognition enables humans to interface with the machine (HMI) and interact naturally without any mechanical devices. Using the concept of gesture recognition, it is possible to point a finger at the computer screen so that the cursor will move accordingly. This could potentially make conventional input devices such as mouse, keyboards and even touch-screens redundant.
Gesture recognition can be conducted with techniques from computer vision and image processing.
A gesture recognition system could be used in any of the following areas:
• Man-machine interface: using hand gestures to control the computer mouse and/or keyboard functions.
• 3D animation: Rapid and simple conversion of hand movements into 3D computer space for the purposes of computer animation.
• Visualisation: Just as objects can be visually examined by rotating them with the hand, so it would be advantageous if virtual 3D objects (displayed on the computer screen) could be manipulated by rotating the hand in space
• Computer games: Using the hand to interact with computer games would be more natural for many applications.
• Control of mechanical systems (such as robotics): Using the hand to remotely control a manipulator.
There are many challenges associated with the accuracy and usefulness of gesture recognition software. For image-based gesture recognition there are limitations on the equipment used and image noise. Images or video may not be under consistent lighting, or in the same location. Items in the background or distinct features of the users may make recognition more difficult.
The variety of implementations for image-based gesture recognition may also cause issue for viability of the technology to general usage. For example, an algorithm calibrated for one camera may not work for a different camera. The amount of background noise also causes tracking and recognition difficulties, especially when occlusions (partial and full) occur. Furthermore, the distance from the camera, and the camera's resolution and quality, also cause variations in recognition accuracy.
Chapter 3
METHODOLOGY
Proposed methodology is able to use live video camera for gesture identification. It sniffs frames of live video stream in some time interval. In our case frame capture rate for gesture search is 3 frames per second. Proposed technique to control robotic system using hand gesture display is divided into four subparts:
• Capture frame containing some gesture presentation.
• Extract hand gesture area from captured frame.
• Determine gesture by pattern matching using PCA algorithm
• Determine control instruction, corresponding to matched gesture, and give that instruction to specified robotic system.
The block diagram above shows the flow diagram of whole system, i.e. performing hand gesture identification and robot control. Gesture is captured by taking a snap shot from a continuous video. The captured image is searched for a valid hand gesture. The region showing the gesture is then cropped out and the image is resized to match with the gestures in the database. On the basis of gesture, identified by pattern matching, control instruction is determined from the stored instructions set. The selected instruction set, corresponding to recognized hand gesture is given to robot for carrying out the control action.
Hand Gesture Recognition
Human hand gestures are a set of movements of the hand and arm which range from the simple action of pointing at something to the complex ones used to communicate with other people. Understanding and interpreting these movements requires modeling them in both spatial and temporal domains. Static configuration of the human hand which is called hand posture and its dynamic activities are vital for human compute interaction.
Psychological studies show that a hands gesture consists of three phases. These phases are: Preparation, Nucleus, and Retraction. The preparatory phase is to bring the hand from its resting state to the starting posture of the gesture. This phase sometimes is very short and sometimes it is combined with the retraction phase of the previous gesture. The nucleus contains the main concept and has a definite form. The retraction phase shows the resting movement of the hand after completing the gesture. Retraction may be very short or not present if the gesture is succeeded by another gesture. The preparatory and retraction phases are generally short and the hand movements are faster compared to the nucleus phase.
Several classifications have been considered for hand gestures in the literature. One taxonomy which is more suitable for human computer interaction applications divides hand gestures into three groups. These groups are: communicative gestures, manipulative gestures, and controlling gestures. Communicative gestures are intended to express an idea or a concept. These gestures are either used together with speeches or are a substitute for verbal communications which on the other hand requires a high structured set of gestures such as those defined in sign languages. Manipulative gestures are used for interaction with objects in an environment. These gestures are mostly used for interaction in virtual environments such as tele operation or virtual assembly systems however; physical objects can be manipulated through gesture controlled robots. Controlling gestures are the group of gestures which are used to control a system or point and locate and object. Finger Mouse is a sample application which detects 2D finger movements and controls mouse movements on the computer desktop. Analyzing hand gestures is completely application dependant and involves analyzing the hand motion, modeling hand and arm, mapping the motion features to the model and interpreting the gesture in a time interval.
Hand gestures can be divided into two categories. Static gestures utilize only spatial information and dynamic gestures utilize both spatial and timed information. With static gestures, as number of predefined gestures is increased, the differences between gestures become harder to distinguish. In the case of dynamic gestures, they are easier and more comfortable to express and larger number of gestures can be predefined, but there are some difficulties with extracting proper data from load of meaningless information.
Glove based techniques and computer vision techniques are the two well-known means of recognizing hand gestures. The first utilizes sensor-detached mechanical glove devices that directly measure hand and/or arm joint angles and spatial position. But glove-based gestural interfaces require users to wear cumbersome patch of devices. The latter approach suggests using a set of video cameras and computer vision techniques to interpret gestures providing more natural way of interactions. However, since it is troublesome to analyze hand movements and recognize postures from complex images, methods such as putting certain colored marker on hands or wearing special types of gloves in restricted set of backgrounds are widely acknowledged limitations. In this paper, we propose a method of hand gestures recognition based on computer vision techniques but, without restricting backgrounds or using any markers.
INTRODUCTION
Interpretation of human gestures by a computer is used for human-machine interaction in the area of computer vision. The main purpose of gesture recognition research is to identify a particular human gesture and convey information to the user pertaining to individual gesture. From the corpus of gestures, specific gesture of interest can be identified, and on the basis of that, specific command for execution of action can be given to the machine. Overall aim is to make the computer to understand human body language, thereby bridging the gap between machine and human. Hand gesture recognition can be used to enhance human– computer interaction without depending on traditional input devices such as keyboard and mouse. Hand gestures are extensively used for telerobotic control and applications. Robotic systems can be controlled naturally and intuitively with such telerobotic communication. A prominent benefit of such a system is that it presents a natural way to send geometrical information to the robot such as: left, right, etc. Robotic hand can be controlled remotely by hand gestures. Research is being carried out in this area for a long time. Several approaches have been developed for sensing hand movements and corresponding by controlling robotic hand.
Glove based technique is well-known means of recognizing hand gestures. It utilizes sensor-detached mechanical glove devices that directly measure hand and/or arm joint angles and spatial position. Although glove-based gestural interfaces give more precision, it limits freedom as it requires users to wear cumbersome patch of devices. Jae-Ho Shin used entropy analysis to extract hand region in complex background for hand gesture recognition system. Robot controlling is done by Fusion of Hand Positioning and Arm Gestures using data glove. Although it gives more precision, it limits freedom due to necessity of wearing gloves. For capturing hand gestures correctly, proper light and camera angle are required. The problem of visual hand recognition and tracking is quite challenging. Many early approaches used position markers or colored bands to make the problem of hand recognition easier, but due to their inconvenience, they cannot be considered as a natural interface for the robot control. We have proposed a fast as well as automatic hand gesture detection and recognition system. This approach of gesture identification On the basis of recognized hand gesture can be used in any robotic system or machines with a number of specific commands suitable to that system.
Chapter 2
GESTURE RECOGNITION
Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand. Current focuses in the field include emotion recognition from the face and hand gesture recognition. Many approaches have been made using cameras and computer vision algorithms to interpret sign language. However, the identification and recognition of posture, gait, human behaviors is also the subject of gesture recognition techniques.[1]
Gesture recognition can be seen as a way for computers to begin to understand human body language, thus building a richer bridge between machines and humans than primitive text user interfaces or even GUIs (graphical user interfaces), which still limit the majority of input to keyboard and mouse.
Gesture recognition enables humans to interface with the machine (HMI) and interact naturally without any mechanical devices. Using the concept of gesture recognition, it is possible to point a finger at the computer screen so that the cursor will move accordingly. This could potentially make conventional input devices such as mouse, keyboards and even touch-screens redundant.
Gesture recognition can be conducted with techniques from computer vision and image processing.
A gesture recognition system could be used in any of the following areas:
• Man-machine interface: using hand gestures to control the computer mouse and/or keyboard functions.
• 3D animation: Rapid and simple conversion of hand movements into 3D computer space for the purposes of computer animation.
• Visualisation: Just as objects can be visually examined by rotating them with the hand, so it would be advantageous if virtual 3D objects (displayed on the computer screen) could be manipulated by rotating the hand in space
• Computer games: Using the hand to interact with computer games would be more natural for many applications.
• Control of mechanical systems (such as robotics): Using the hand to remotely control a manipulator.
There are many challenges associated with the accuracy and usefulness of gesture recognition software. For image-based gesture recognition there are limitations on the equipment used and image noise. Images or video may not be under consistent lighting, or in the same location. Items in the background or distinct features of the users may make recognition more difficult.
The variety of implementations for image-based gesture recognition may also cause issue for viability of the technology to general usage. For example, an algorithm calibrated for one camera may not work for a different camera. The amount of background noise also causes tracking and recognition difficulties, especially when occlusions (partial and full) occur. Furthermore, the distance from the camera, and the camera's resolution and quality, also cause variations in recognition accuracy.
Chapter 3
METHODOLOGY
Proposed methodology is able to use live video camera for gesture identification. It sniffs frames of live video stream in some time interval. In our case frame capture rate for gesture search is 3 frames per second. Proposed technique to control robotic system using hand gesture display is divided into four subparts:
• Capture frame containing some gesture presentation.
• Extract hand gesture area from captured frame.
• Determine gesture by pattern matching using PCA algorithm
• Determine control instruction, corresponding to matched gesture, and give that instruction to specified robotic system.
The block diagram above shows the flow diagram of whole system, i.e. performing hand gesture identification and robot control. Gesture is captured by taking a snap shot from a continuous video. The captured image is searched for a valid hand gesture. The region showing the gesture is then cropped out and the image is resized to match with the gestures in the database. On the basis of gesture, identified by pattern matching, control instruction is determined from the stored instructions set. The selected instruction set, corresponding to recognized hand gesture is given to robot for carrying out the control action.
Hand Gesture Recognition
Human hand gestures are a set of movements of the hand and arm which range from the simple action of pointing at something to the complex ones used to communicate with other people. Understanding and interpreting these movements requires modeling them in both spatial and temporal domains. Static configuration of the human hand which is called hand posture and its dynamic activities are vital for human compute interaction.
Psychological studies show that a hands gesture consists of three phases. These phases are: Preparation, Nucleus, and Retraction. The preparatory phase is to bring the hand from its resting state to the starting posture of the gesture. This phase sometimes is very short and sometimes it is combined with the retraction phase of the previous gesture. The nucleus contains the main concept and has a definite form. The retraction phase shows the resting movement of the hand after completing the gesture. Retraction may be very short or not present if the gesture is succeeded by another gesture. The preparatory and retraction phases are generally short and the hand movements are faster compared to the nucleus phase.
Several classifications have been considered for hand gestures in the literature. One taxonomy which is more suitable for human computer interaction applications divides hand gestures into three groups. These groups are: communicative gestures, manipulative gestures, and controlling gestures. Communicative gestures are intended to express an idea or a concept. These gestures are either used together with speeches or are a substitute for verbal communications which on the other hand requires a high structured set of gestures such as those defined in sign languages. Manipulative gestures are used for interaction with objects in an environment. These gestures are mostly used for interaction in virtual environments such as tele operation or virtual assembly systems however; physical objects can be manipulated through gesture controlled robots. Controlling gestures are the group of gestures which are used to control a system or point and locate and object. Finger Mouse is a sample application which detects 2D finger movements and controls mouse movements on the computer desktop. Analyzing hand gestures is completely application dependant and involves analyzing the hand motion, modeling hand and arm, mapping the motion features to the model and interpreting the gesture in a time interval.
Hand gestures can be divided into two categories. Static gestures utilize only spatial information and dynamic gestures utilize both spatial and timed information. With static gestures, as number of predefined gestures is increased, the differences between gestures become harder to distinguish. In the case of dynamic gestures, they are easier and more comfortable to express and larger number of gestures can be predefined, but there are some difficulties with extracting proper data from load of meaningless information.
Glove based techniques and computer vision techniques are the two well-known means of recognizing hand gestures. The first utilizes sensor-detached mechanical glove devices that directly measure hand and/or arm joint angles and spatial position. But glove-based gestural interfaces require users to wear cumbersome patch of devices. The latter approach suggests using a set of video cameras and computer vision techniques to interpret gestures providing more natural way of interactions. However, since it is troublesome to analyze hand movements and recognize postures from complex images, methods such as putting certain colored marker on hands or wearing special types of gloves in restricted set of backgrounds are widely acknowledged limitations. In this paper, we propose a method of hand gestures recognition based on computer vision techniques but, without restricting backgrounds or using any markers.
