Controlling Mouse Movements Using hand Gesture And X box 360
1 like•56 views
This document describes a project to control computer mouse movements using hand gestures detected by an Xbox 360 camera and computer vision techniques. The system uses the Xbox 360 camera and OpenCV library to capture images of the user's hand, segment the hand from the background, identify finger positions including the index finger to act as the mouse cursor, and map hand and finger movements to mouse movements and clicks on the computer screen. The system was developed as an alternative to using a physical mouse to make computer interaction easier and more accessible. It achieves average recognition rates but has limitations in detecting gestures from long distances. Future work could improve hand recognition accuracy and range.
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072
© 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 986
often very costly. The motive was to create this technology
in the cheapest possible way and also to create it under a
standardized operating system.
REFERENCE
[1] Computer vision based mouse A. Erdem, E. Yardimci,
Y. Atalay V. Cetin, A. E. Acoustics, Speech and Signal
Processing, 2002. Proceedings. IEEE International
Conference
[2] Virtual mouse vision based interface, Robertson P.,
Laddaga Van Kleek M. 2004.
[3] Vision based Men-Machine Interaction
http://www.ceng.metu.edu.tr/~vbi/
[4] Chu-Feng Lie Portable Vision-Based HCI A Real-time
Hand Mouse System on Handheld Devices.
[5] Hailing Zhou, Lijun Xie, Xuliang Fang, Visual Mouse:
SIFT Detection & PCA Recognition 2007 International
Conference on Computational Intelligence and Security
Workshops CISW 2007
BIOGRAPHIES
Rajesh Jeevan Patil
BE( Computer Engineering )
Sangahvi CollegeOfEnginnering,
Nashik,Maharashtra
Siddharth Rajendra Pagar
BE( Computer Engineering )
Sangahvi College Of Enginnering,
Nashik ,Maharashtra](https://image.slidesharecdn.com/irjet-v4i2190-171117095215/85/Controlling-Mouse-Movements-Using-hand-Gesture-And-X-box-360-5-320.jpg)
Controlling Mouse Movements Using hand Gesture And X box 360
- 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 982 Controlling Mouse Movements Using hand Gesture And X box 360 Prof: N. Saindane , Rajesh Patil , Siddharth Pagar 1 Prof : N. Saindane Dept Of Computer Engineering Sanghavi College of Engineering Maharashtra ,India 2Rajesh Patil Student Department Of Computer Engineering Sanghavi College of Engineering 3Siddharth Pagar Student Department Of Computer Engineering Sanghavi College of Engineering ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract -Computers are developed today have increase in amount of power. Most of this used by software for processing data. Our project is gesture user interface implementation for human computer Interaction. The project presents a new approach for controlling the computer system and mouse movements using the X-BOX 360. The X-BOX 360 was developed by Microsoft especially for the home video game console. But in this project, we are trying to use this device for human interaction with computer system. This will make interaction with computer easy, interesting more users friendly. Our method is to use X-BOX 360 and Computer vision technology, such as image segmentation Gesture recognition to control mouse tasks and we are show how it can be perform everything on current mouse can. For the persons who are not able to interact with computer system physically, we are developing the voice recognition technology in our project. The X-BOX 360 supports the MP3 audio format forthe voice recognition. So, this project shows how to build this mouse control system. Key Words: Authentication, Image processing 1.INTRODUCTION This As the present computer vision technology is now growing up the scope for the human (HCI) computer interaction is increasing enormously . Nowadays mobile devices using a touch screen technology. this technology is still not cheap enough to used in desktop systems. Creating virtual human computer interaction device such as mouse , keyboard using web camera and computer vision technology can be an alternative mouse application has designed and implemented using a regular webcam. The motivation to create an object tracking to interact with computer and develop the virtual human computer interaction device. The goal of this paper is to provide an easier human- interaction routine. In this project, the hand movement of a user are mapped into mouse inputs. and A web camera is s to take the live video continuously and then this video images are being captured by using MATLAB. The user must have a particular color code or pointer in hishand so when the web camera takes an image it must visible This color is detected from the image pixel in MATLAB andobject detection is used to mapping pixel position intothe mouse input. Taking into the size of the image and resolution, scaling techniques needed to be used like image processing and segmentation. Sixth sensetechnologycanbe developed to include multimedia services videos and saving images using hand gestures. This work to provide usual mouse operations using x box 360. 2. Literature survey Previously noted that the hand gesture recognition would allow human computer interface such as cursor control and sign language recognition. There are two approaches for hand gesture recognition for human computer interface, which is hardware based and the vision based. One hardware based approach by 1990 uses a data glove for achieve the gesture recognition. The computer makes some gestures difficult to perform. this method gives high accuracy, it is not practical in user’s everyday life for CC. There were tremendous amount of research on vision based human computer interface techniques which is utilized camera and image processing techniques. Research in vision based human computer interface result in the introduction of many new interesting application such as the visual touchpad, proposed by augmented reality proposed by and a visual TV remote control proposed by Vision based hand gesture recognition can be Grouped into two categories, marker-based and marker-less approach. The marker-basedapproachrequires the user to wear color markers or gloves, while the marker- less approach doesn’t. The markerbasedapproachgenerally has better accuracy is much easier to implement but the requires the user to wear a colour glove. Thus thisapproach is not very practical for a CC system to replace the computer mouse. The marker less approach would be the better choice for the system since, there is no constraint on the user to wear any coloured or Data gloves.
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 983 3. System architecture 4. Hand recognition Hand Gesture Recognition 4.1 Image Resize First to recognize the hand gesture then we need to resize the input image in order to mapping the camera coordinates to Screen coordinates. There are two way to mapping from source data image to destination image. The first way is to compute the ratio of screen resolution to camera resolution. To determine the x & y on the screen of a given camera pixel, we use the following equation: X = ( x’ ∕ 640 )× CX , Y = ( y ’/480 ) × CY 4.2 Segmentation we need to separate the hand area from a complex background. Because It is difficult to detect skin color in natural environment because the variety of the illuminations and skin colors. So, we need to carefully pick a color range. To get better result then we converte from RGB color space to YCBCR color space, since YCbCr is insensitive to color variation. 4.3 Deleting noise Using this approach, we cannot get a good estimate of the hand gesture image because of the background noise. To getting a better estimate of the hand we need to delete the noisy pixels from the image. We use an image morphology algorithm that performs image deletion to eliminate noise Erosion the trim down image area where hand is not present and Dilation expands the area of the Image pixels which are not eroded. 4.4 Finding center and the size of the hand Where i x & i y are x and y coordinates of the i pixel in the hand region, and k denotes the number of pixels in the region. After we locate the center of hand we compute the radius of the palm region to getting hand size , To obtain the size of the hand, we draw a circle increasing the radius of the circle from the center coordinate until circle meets the first black pixel. When algorithm finds the first black pixel then it returns to the current radius value. This algorithm assumes that when the circle meets first black pixel after drawing a large and larger circle then the length from the center is the radius of the back of the hand.. Figure shows the results.
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 984 . 4.5 Finding finger tip To recognize finger is inside of the palm area or not we used a convex hull algorithm. The algorithm is used to solve the problem of finding the biggest polygons including the all vertices. Use this feature of algorithm, we can detect finger tips on hand. We are using algorithm to recognize if a finger is folded or not. To recognize those states, we multiplied 2 times we got this number through multiple trials to the hand radius value and check distance between the center and the pixel which is in convex hull set. If the distance is longer than the radius of the hand, then finger is spread In addition if two or more interesting points existed in the result then we regarded the longest vertex as t index finger and the hand gesture is click when the number of the result vertex two or more. The result of convex hull algorithm has a set of vertexes which includes all vertexes. Thus sometimes a vertex is placed near other vertexes. This case occurs corner of the finger tip . To solve this problem we are delete a vertex whose distance is less than 10 pixels when comparing with the next vertex. Finally we can get one interesting point on each finger. Figure shows the results. 4.6 Moving Mouse Cursor We are using the index finger as a cursor and controller to control mouse cursor. We using two different approaches for the moving mouse cursor. The first method is mapping cursor control.. In other words, the mouse cursor is placed on the desktop window along with index finger tips position displayed on the camera screen position cannot be accurate because when the camera resolution is converts to the desktop window resolution then we lose intermediate value. We expect the ratio of jumping pixel is up to 4 pixels b The second method is weight speed cursor control. it means that some machines which cannot achieve image processing 15 images per second do not work smoothly because computing the image center and the hand shape take more time. Thus this algorithm does not work properly. In this paper we use the first method which use absolute position of finger tips because it is more accurate than the second method. Fig.Pair-based Authentication Scheme which are place randomly and the interface change every time. The mechanisms involve in the pair-based authentication scheme is as follows: Firstly, the user has to consider the secret pass in terms of pair. The first letter in the pair is used to selected row and the second letter is used to selected the column in the 6X6 grid. The intersection letter of the selected row and column generate the character which is a part of the session password In this way the logic is reiterated for all other pairs in the secret pass . Thereafter, the password inputted
- 4. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 985 by the user i.e. the sessions passwords is now verified by the servers to authenticate the users. Kinect consists of Infra-red (IR) projector, IR camera and RGB camera Kinect as a 3D measuring device Kinect is a composite device consist of an IR projector of a pattern and IR camera, which is used to triangulate points in space. It works as a depth sense camera, and a color (RGB)camera, which can be used to recognize image content andtexture 3D points, As a measuring device, Kinectdelivers three outputs: IR image, RGB image, and (inverse) Depth image. IR image IR 1280×1024 pixels for 57×45 degrees FOV 6.1 mm focal length 5.2 μm pixel size camera is used to observe and decode the IR projection pattern to triangulate 3D scene If suitably illuminated by a halogen lamp and with the IR projector blocked, (c , d) it can be reliably calibrated by using the same checkerboard pattern used for the RGB camera. The camera exhibits non-negligible radial and tangential distortions. RGB image RGB 1280×1024 pixels for 63×50 degrees focal length, 2.8μm pixel size) camera delivers medium quality images. It will calibrated by and used to track the camera motion by SfM systems, Depth image The main output of Kinect is an image that corresponds to the depth in the scene. Rather than providing the actual depth z the Kinect returns “inverse depth” Taking account the depth resolution achievable with Kinect we adopted the model suggested in The Depth image is constructed by triangulation from the IR image and the projector How Kinect Works: Overview 5. CONCLUSIONS We are developing a system to control the mouse cursor using a real-time image processing through x-box 360. We implemented all mouse tasks such as left right click double click and scrolling. This system is based on computer vision algorithms and can do all mouse tasks there are many approaches for hand gesture recognition, and each approach has its strengths and weakness. The strength of the proposed method in this paper is the open CV library and the advanced processor The Proposed algorithm achieves average recognition rate. In this process we do not train a system with different hand images. Instead, we directly test the positive images using open CV. This is the major advantage of open CV. And The drawback of the system is that it cannot detect the actions performed by hand from long distances. Hence, in near future the system has to overcome this problem so that the actions can be perform from long distances and improve the hand recognition rate. Presently, the webcam, microphone and mouse are an integral part of the Computer system .Most of the applications require additional hardware which is
- 5. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 986 often very costly. The motive was to create this technology in the cheapest possible way and also to create it under a standardized operating system. REFERENCE [1] Computer vision based mouse A. Erdem, E. Yardimci, Y. Atalay V. Cetin, A. E. Acoustics, Speech and Signal Processing, 2002. Proceedings. IEEE International Conference [2] Virtual mouse vision based interface, Robertson P., Laddaga Van Kleek M. 2004. [3] Vision based Men-Machine Interaction http://www.ceng.metu.edu.tr/~vbi/ [4] Chu-Feng Lie Portable Vision-Based HCI A Real-time Hand Mouse System on Handheld Devices. [5] Hailing Zhou, Lijun Xie, Xuliang Fang, Visual Mouse: SIFT Detection & PCA Recognition 2007 International Conference on Computational Intelligence and Security Workshops CISW 2007 BIOGRAPHIES Rajesh Jeevan Patil BE( Computer Engineering ) Sangahvi CollegeOfEnginnering, Nashik,Maharashtra Siddharth Rajendra Pagar BE( Computer Engineering ) Sangahvi College Of Enginnering, Nashik ,Maharashtra