Skip to main content

Sixth Sense Technology By Pranav Mistry


                   Sixth Sense' is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. it was invented by Pranav Mistry,a PhD candidate in the Fluid Interface Groups at the MIT MEDIA LAB.
By using a camera and a tiny projector mounted in a pendant like wearable device, 'Sixth Sense' sees what you see and visually augments any surfaces or objects we are interacting with. It projects information onto surfaces, walls, and physical objects around us, and lets us interact with the projected information through natural hand gestures, arm movements, or our interaction with the object itself. 'Sixth Sense' attempts to free information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.  All of us are aware of the five basic senses – seeing, feeling, smelling, tasting and hearing. But there is also another sense called the sixth sense. It is basically a connection to something greater than what their physical senses are able to perceive. To a layman, it would be something supernatural. Some might just consider it to be a superstition or something psychological. But the invention of sixth sense technology has completely shocked the world. Although it is not widely known as of now but the time is not far when this technology will change our perception of the world.
The Sixth Sense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant-like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The device projects visual information, enabling surfaces, walls and physical objects around the wearer to be used as interfaces; while the camera recognizes and tracks the user's hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus Sixth Sense also supports multi-touch and multi-user interaction.
 The device sees what we see but it lets out information that we want to know while viewing the object. It can project information on any surface, be it a wall, table or any other object and uses hand / arm movements to help us interact with the projected information. The device brings us closer to reality and assists us in making right decisions by providing the relevant information, thereby, making the entire world a computer.
                                     The world has shrunk. Distances have dissolved. Communication lines and interaction with countless systems have been rendered feasible. However this technological overhaul has been peripheral and not so much related to the human body; researchers and innovators have constantly grappled with the issue of bridging the gaps which limit the human-environment contact.  This device, tentatively name as the Sixth Sense, is a wearable machine that assists unexplored interactions between the real and the virtual sphere of data. It consists of certain commonly available components, which are intrinsic to its functioning. These include a camera, a portable battery-powered projection system coupled with a mirror and a cell phone. All these components communicate to the cell phone, which acts as the communication and computation device. The entire hardware apparatus is encompassed in a pendant-shaped mobile wearable device. Basically the camera recognizes individuals, images, pictures, gestures one makes with their hands and the projector assists in projecting any information on whatever type of surface is present in front of the person. The usage of the mirror is significant as the projector dangles pointing downwards from the neck. To bring out variations on a much higher plane, in the demo video which was broadcasted to showcase the prototype to the world, Mistry uses colored caps on his fingers so that it becomes simpler for the software to differentiate between the fingers, demanding various applications. The software program analyses the video data caught by the camera and also tracks down the locations of the colored markers by utilizing single computer vision techniques. One can have any number of hand gestures and movements as long as they are all reasonably identified and differentiated for the system to interpret it, preferably through unique and varied. This is possible only because the ‘Sixth Sense’ device supports multi-touch and multi-user interaction. There was once a clear divide between the virtual world and the real world, but that line is getting blurrier every day.
Source : http://sixthsenseblog.blogspot.in

figures shows components used in sixth sense technology

components used in sixth sense technology:

  • Camera
  • Color Markers
  • Mobile Component
  • Projector
  • Mirror

Camera:

                                                                          
It captures the image of the object in view and tracks the user’s hand gesture. The camera recognizes individuals, images, pictures, gestures that user makes with his hand. The camera then sends this data to a smart phone for processing. Basically the camera forms a digital eye which connects to the world of digital information.

Color Markers:

                                                               
There are color markers placed at the tip of users finger. Marking the user’s fingers with red, yellow green and blue coloured tape helps the webcam to recognize the hand gestures. The movements and arrangement of these markers are interpreted into gestures that act as a interaction instruction for the projected application interfaces.

Mobile Components:

                                                      
The Sixth Sense device consists of a web enabled smart phone which process the data send by the camera. The smart phone searches the web and interprets the hand gestures with help of the colored markers placed at the finger tips.

Projector:

                                                                    
                                                                 
The information that is interpreted through the smart phone can be projected into any surface. The projector projects the visual information enabling surfaces and physical objects to be used as interfaces. The projector itself consists of a battery which have 3 hours of battery life.
 A tiny LED projector displays the data sent from the smart phone on any surface in view- object, wall or person. The downward facing projector projects the image on to a mirror.

Mirror:

                                                      
The usage of a mirror is important as the projector dangles pointing downwards from the neck. The mirror reflects the image on to a desire surface. Thus finally the digital image is freed from its confines and placed in the physical world.

How its Works???



                                       
                          

The Sixth Sense technology works as follows:


1.      It captures the image of the object in view and track the user’s hand gesture.
2.      There are color  markers placed at the tip of users finger. Marking the user’s fingers with red, yellow green and blue color tape helps the webcam to recognize the hand gestures. The movements and arrangement of these markers are interpreted into gestures that act as a interaction instruction for the projected application interfaces.
3.      The smart phone searches the web and interprets the hand gestures with help of the coloured markers placed at the finger tips
4.      The information that is interpreted through the smart phone can be projected into any surface.
5.      The mirror reflects the image on to a desire surface. 

Application fields of sixth sense technology:


1. making hand as an mobile screen and dialing numbers displaying on hand through finger tips.

                                             
 2. taking pictures by angling yours finger tips to the picture you want to take.


                                               

     

   3. many others applications are reading newspaper,booking reservation tickets , reading facial expression locating place .                                                
                                                 



Comments

Anonymous said…
Good One
Anonymous said…
is it really working.....or is it a proposal only?
anup said…
Yes its working and is partially implemented. Do share it and thanks for liking it

Popular posts from this blog

CDAC CCAT Rank - Which Centre you Should go for...

Subscribe us for our YouTube channel and any kind of help Click here to ask questions regarding CDAC 1. C-DAC (Head Quarters) Pune    CDAC's Admission Booklet- Process of Admission to Post Graduate Diploma Courses of C-DAC                         click below to know about the CCAT's This batch allotment                click above to know about the CCAT's This batch allotment Rank 1-300 c-dac HQ has been the best from the start. 2.  Sunbeam Pune Rank 300-500 I got very positive feedback from my friends who are in c-dac banglure main campus,since there are many companies you may get more opportunities. 3.  C-DAC Knowledge Park Rank 400-700 It as very good faculty .Almost all the students get placed here every year. 4. C-DAC Hyderabad Rank 200-1000 (It depends on the course which you select) C-DAC hyderabad is very good for the course PG-DESD. PG-DESD course is in hyderbad is better than pune HQ as per the past feedback.It as v

CDAC COURSES AND PLACEMENTS, WHICH IS BETTER FOR YOU

Subscribe us for our YouTube channel and any kind of help   Click here to ask questions regarding CDAC Before going through this post, I would like to draw your attention towards the importance of this post. This page not only explains my experience in CDAC but also aims at answering the queries of you all who are going or looking to have a course from CDAC. Kindly post your queries at the bottom of this page and we will get back to you within 24 hours. Kindly do not post your queries as an Anonymous user and do not forget to subscribe via email so as to keep track of your query.                         click below to know about the CCAT's This batch allotment                 click below to know about the CCAT's This batch allotment ______________________________________________ NOW a day lot of the graduates and post graduates are wondering most of the times on which course they should go for. What are the pros & cons of joining a particular ce

Placement Statistics - 2011 & 2012

Subscribe us for our YouTube channel and any kind of help                          Post your queries below and we will get back to you in no more than 24 hours.  Click here to ask questions regarding CDAC                         click below to know about the CCAT's This batch allotment                     click above to know about the CCAT's This batch allotment CDAC's Admission Booklet- Process of Admission to Post Graduate Diploma Courses of C-DAC Click here to see placement statistics About C-CAT, Exam Pattern and Books No. of Seats Across Various Training Centres Important Dates - 2014 Tags: CDAC, CDAC scope in future, CDAC placements, CDAC training, CDAC recruitment, CDAC training centres, DSSD, DESD, DAC, PGDSSD, PGDESD, PGDAC, DABC, PGDABC, VLSI, PGDITISS, PGDIVESD, PGDESD, PGDWiMC, placement statistics