sixth sense technology

[summer project pal]

SixthSense
B.Tech Seminar report
by
Gokul Sudhakaran
Department of Computer Science And Engineering
Government Engineering College, Thrissur
December 2010

[attachment=8394]


Abstract
Information is everything in today's world. Yet the world of information is very small.
From details about the various stars or galaxies to the description of the tinniest piece
of junk that you can get in a supermarket, everything is present on the internet. Yet
it is conned, trapped in a screen on a desktop or on a mobile. Well SixthSense breaks
these connes and brings the information to the real world. Rather than adjusting
ourselves to the latest machine(gadgets), SixthSense adjusts the machine to us and
trains it to understand our natural hand gestures.

Chapter 1
Introduction
Steve Mann was the rst person to bring forth the idea of SixthSense in the form of
a device called Telepointer, it was originally referred to as 'Synthetic Synesthesia of
the Sixth Sense'. He is also considered as the father of SixthSense. This was later
developed by Pranav Mistry, a PhD student at MIT.
'SixthSense' is a wearable gestural interface that augments the physical world
around us with digital information and lets us use natural hand gestures to interact
with that information.
Hardware requirements for the device include :-
 Camera- recognizing and tracking hand gestures
 Pocket Projector- projects digital information on the wall or any other surface
 Cellphone- connects to the cloud and does the processing
 Mirror- helps in projecting the image on a horizontal surface by re
ection
 Coloured Caps- helps in keeping track of the hand and recognising gestures.
The technoligies used in this project are :-
 Hand Augmented Reality
 Gesture Recognition
 Image capturing, processing and manipulation
1.1 Organization Of the Report
1. Chapter 2 describes how gesture recognition is done and it's dierent phases2. Chapter 3 describes the applications of the device
3. Chapter 4 describes how TaPuMa works
4. Chapter 5 describes the merits and demerits of the device.
5. Chapter 6 discusses the future developments
6. Chapter 7 gives the conclusion of the paper.

Chapter 2
Gesture Recognition
Gesture recognition is an important aspect of this device. We would be tackling the
general gesture recognition technique(not specic to SixthSense). The entire process
of gesture recognition can be divided into the following parts :-
 Motion Segmentation
 Skin Colour Model
 Geometric Analysis
 Motion Trajectories
 Recognizing Motion Patterns Using Time Delay Neural Network(TDNN)
2.1 Motion Segmentation
Early methods can be divided into two types:-
1. Pixel based
Motion segmentation in this method is done on the basis of intensity variations
i.e. it regards intensity variations as a cause for motion and vice versa. This
method works well when we consider a scene with slow moving objects or in
which the number of objects are less. But as the number of objects or speed of
objects increases the performance falls.
2. Feature based
Feature based method matches image features like points dened by local inten-
sity, edges, corners, etc. Featuress are extracted using single scale segmentation.
These features are then matched across frames. Segmentation errors might make
it dicult to nd feature correspondence across frames.

even more information is in this thread:
http://studentbank.in/report-sixth-sense...186?page=3