19-12-2010, 06:29 PM
[attachment=7454]
OUTLINE
What is gesture
Need for gesture based system
Gesture recognition techniques
Data gloves
Vision based system
Sign langusge
Processing
execution
NEED FOR GESTURE BASED SYSTEM
Interactive human-machine interface.
Make computer to understand human language.
Human machine interaction without using traditional input devices.
Communicate at a distance
Interact in a 3-D world
HAND GESTURE RECOGNITION TECHNOLOGIES
glove-based method
Using special glove-based device to extract hand posture
Annoying
vision-based method
3D hand/arm modeling
Appearance modeling
Datagloves
Datagloves
Datagloves provide very accurate measurements of hand-shape
But are cumbersome to wear
Expensive
Connected by wires- restricts freedom of movement
Hand Gesture Recognition by Vision based system
INTRODUCTION
This is a new approach of controlling a robotic hand or a robot itself by merely showing hand gestures in front of the camera. A video camera captures a gesture shown in front of it for a time period and processes it to determine the command from the gesture. The command is then executed by generating proper control signals.
MODULES
Hand gesture formation monitor and gesture capture.
Feature extraction.
Pattern matching for gesture recognition.
Command determination.
Command execution
Sign Language
Sign Language
Two main elements:
Low and simple level signed alphabet, mimics the letters of the native spoken language
Higher level signed language, using actions to mimic the meaning or description of the sign
CSL
CSL is the abbreviation for Chinese Sign Language
30 letters in CSL alphabet
Pre-processing of Hand Gesture Recognition
Detection of Hand Gesture Regions
Aim to fix on the valid frames and locate the hand region from the rest of the image.
Low time consuming fast processing rate real time speed
Pre-processing of Hand Gesture Recognition
Detect skin region from the rest of the image by using color.
Each color has three components
hue, saturation, and value
chroma consists of hue and saturation is separated from value
Under different condition, chroma is invariant.
Pre-processing of Hand Gesture Recognition
Color is represented in RGB space, also in YUV and YIQ space.
Skins are between red and yellow
Extract the skin region using proper filtering
Convert to binary image with skin as white pixel and remaining as black pixels.
Pre-processing of Hand Gesture Recognition
Feature Extraction
Cropping of hand gesture from obtained frame.
Region with more number of white pixels are extracted for this.
Wrist and arm can be removed by locating position with minimum width of vertical histogram and cropping from there.
This image is resized to fit into the frame size of reference database
Pattern Matching
Principal Component Analysis is used for pattern matching.
mathematical procedure that transforms a number of possibly correlated variables into a smaller number of uncorrelated variables called principal components
The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible
Principal Components
PCA reduces the number of dimensions and so the memory requirement is much reduced.
The search time is also reduced
Two ways to apply PCA (1)
We could apply PCA to the whole training set.
Then each gesture is represented by a point in the PC space
We could then apply nearest neighbour to these points
Two ways to apply PCA (2)
Alternatively we could apply PCA to the set of gestures belonging to each gesture in the training set
Each class (gesture) is then reprented by a different ellipsoid and Mahalanobis distance can be used to classify a new unknown gesture
You need a lot of images of each gesture to do this
Execution
When a Gesture is matched with a meaningful gesture the action corresponding to that gesture will be retrieved from the database and executed.
CONCLUSION
Hand gesture based human computer interaction is a very user friendly.
It enables communication with computer without any hardware
Recognition of 2D and 3D gestures in real time
Fast detection process, allows the real time video application with low cost sensors, such as PC and USB camera
OUTLINE
What is gesture
Need for gesture based system
Gesture recognition techniques
Data gloves
Vision based system
Sign langusge
Processing
execution
NEED FOR GESTURE BASED SYSTEM
Interactive human-machine interface.
Make computer to understand human language.
Human machine interaction without using traditional input devices.
Communicate at a distance
Interact in a 3-D world
HAND GESTURE RECOGNITION TECHNOLOGIES
glove-based method
Using special glove-based device to extract hand posture
Annoying
vision-based method
3D hand/arm modeling
Appearance modeling
Datagloves
Datagloves
Datagloves provide very accurate measurements of hand-shape
But are cumbersome to wear
Expensive
Connected by wires- restricts freedom of movement
Hand Gesture Recognition by Vision based system
INTRODUCTION
This is a new approach of controlling a robotic hand or a robot itself by merely showing hand gestures in front of the camera. A video camera captures a gesture shown in front of it for a time period and processes it to determine the command from the gesture. The command is then executed by generating proper control signals.
MODULES
Hand gesture formation monitor and gesture capture.
Feature extraction.
Pattern matching for gesture recognition.
Command determination.
Command execution
Sign Language
Sign Language
Two main elements:
Low and simple level signed alphabet, mimics the letters of the native spoken language
Higher level signed language, using actions to mimic the meaning or description of the sign
CSL
CSL is the abbreviation for Chinese Sign Language
30 letters in CSL alphabet
Pre-processing of Hand Gesture Recognition
Detection of Hand Gesture Regions
Aim to fix on the valid frames and locate the hand region from the rest of the image.
Low time consuming fast processing rate real time speed
Pre-processing of Hand Gesture Recognition
Detect skin region from the rest of the image by using color.
Each color has three components
hue, saturation, and value
chroma consists of hue and saturation is separated from value
Under different condition, chroma is invariant.
Pre-processing of Hand Gesture Recognition
Color is represented in RGB space, also in YUV and YIQ space.
Skins are between red and yellow
Extract the skin region using proper filtering
Convert to binary image with skin as white pixel and remaining as black pixels.
Pre-processing of Hand Gesture Recognition
Feature Extraction
Cropping of hand gesture from obtained frame.
Region with more number of white pixels are extracted for this.
Wrist and arm can be removed by locating position with minimum width of vertical histogram and cropping from there.
This image is resized to fit into the frame size of reference database
Pattern Matching
Principal Component Analysis is used for pattern matching.
mathematical procedure that transforms a number of possibly correlated variables into a smaller number of uncorrelated variables called principal components
The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible
Principal Components
PCA reduces the number of dimensions and so the memory requirement is much reduced.
The search time is also reduced
Two ways to apply PCA (1)
We could apply PCA to the whole training set.
Then each gesture is represented by a point in the PC space
We could then apply nearest neighbour to these points
Two ways to apply PCA (2)
Alternatively we could apply PCA to the set of gestures belonging to each gesture in the training set
Each class (gesture) is then reprented by a different ellipsoid and Mahalanobis distance can be used to classify a new unknown gesture
You need a lot of images of each gesture to do this
Execution
When a Gesture is matched with a meaningful gesture the action corresponding to that gesture will be retrieved from the database and executed.
CONCLUSION
Hand gesture based human computer interaction is a very user friendly.
It enables communication with computer without any hardware
Recognition of 2D and 3D gestures in real time
Fast detection process, allows the real time video application with low cost sensors, such as PC and USB camera
