Project Prague & RealSense: il potere nelle mani!!
Download as PPTX, PDF1 like478 views
Project Prague is an SDK that enables developers to integrate gesture-based controls into applications using Intel RealSense cameras. The technology employs machine learning to detect hand motions and poses for gesture recognition, making it suitable for various interaction scenarios. Released in July 2017, it supports various applications, including Windows shell and Unity, and aims to enhance human-computer interaction through gesture recognition.
![Requirements
Category Recommended Minimal
CPU Intel® Core™ i7 series, 8 logical cores Intel® Core™ i5 series, 4 logical cores
Free RAM 2GB or more 1GB
Operating System Windows 10 with Creator Update installed Windows 10
Brand Gesture Detection Range Experience
Intel® RealSense™ SR300 camera 20-60 [cm] Best
Intel® RealSense™ F200 camera 20-60 [cm] Good
Kinect for Windows v2 60-110 [cm] Good
http://aka.ms/gestures/download](https://image.slidesharecdn.com/projectpraguerealsenseilpoterenellemani-171215113009/85/Project-Prague-RealSense-il-potere-nelle-mani-8-320.jpg)
![Project Prague Reference
Project Prague Web Site
https://labs.cognitive.microsoft.com/en-us/project-prague
Project Prague documentation
https://docs.microsoft.com/en-us/gestures/
Twitter/mail account
@ProjectPrague / gestures@Microsoft.com
Project Prague GitHub
https://github.com/Microsoft/Gestures-Samples
Toward Realistic Hands Gesture Interface: Keeping it Simple for Developers and Machines [PDF]
https://aka.ms/TRHGI
Sample GitHub
https://github.com/massimobonanni/ProjectPragueSamples](https://image.slidesharecdn.com/projectpraguerealsenseilpoterenellemani-171215113009/85/Project-Prague-RealSense-il-potere-nelle-mani-28-320.jpg)
Project Prague & RealSense: il potere nelle mani!!
- 1. #IoTDayAQ17
- 2. Project Prague & RealSense: il potere nelle mani!! Massimo Bonanni Paranormal Developer, with the head in the Cloud and all the REST in microservices! [email protected] @massimobonanni
- 4. “… the idea of gesture input will become mainstream – both when working far away from the screen (TV, big meeting room) and when working at your desk (PC).” Microsoft’s Focused Dreams to Survive and Thrive, Bill Gates, January 2016
- 6. What is Project Prague? Is an easy-to-use SDK that helps developers to incorporate gesture- based controls into their apps. Use a machine learning algorithm called DFE (Discriminative Ferns Ensemble) to obtain the hand skeleton. The gesture detection is implemented using a straight forward heuristic, with very few manually tuned parameters.
- 7. what do I need? Intel® RealSense™ SR300 https://aka.ms/IntelRS300 Intel® RealSense™ F200
- 8. Requirements Category Recommended Minimal CPU Intel® Core™ i7 series, 8 logical cores Intel® Core™ i5 series, 4 logical cores Free RAM 2GB or more 1GB Operating System Windows 10 with Creator Update installed Windows 10 Brand Gesture Detection Range Experience Intel® RealSense™ SR300 camera 20-60 [cm] Best Intel® RealSense™ F200 camera 20-60 [cm] Good Kinect for Windows v2 60-110 [cm] Good http://aka.ms/gestures/download
- 9. Hand pose, Hand Motion & Gestures
- 10. Palm Pose Finger Poses Fingertip Placement Relations + + Fingertip Distance Relations+ HandPose = Direction: {left, right, up, down, backward, forward} Flexion: {opened, openedStretched, folded, foldedTucked} RelativePlacement: {left, right, above, below, behind, inFornt} RelativeDistance: {notTouching, touching} Direction: {left, right, up, down, backward, forward} Orientation: {left, right, up, down, backward, forward} Hand Pose A hand pose refers to a snapshot of the hand at a given moment and contains a complete description of the state of the palm and the fingers in that snapshot
- 11. palm facing {forward}, orientation {up} {index, thumb} are {touching} {middle, ring, pinky} are {above} {index, thumb} {index} is {above} the {thumb} {left} hand {thumb, index, middle, ring, pinky} are {opened} Pinch pose Hand Pose
- 12. <svg> <path d="M 100 350 q 50 -50 100 0 q 50 -50 100 0 q 50 -50 100"/> </svg> HandMotion = MotionScript Upward, Downward, Right,Left, ClockwiseArcRightUpward, ClockwiseArcRightDownward, ClockwiseArcLeftUpward, ClockwiseArcLeftDownward, CounterClockwiseArcLeftDownward, CounterClockwiseArcLeftUpward, CounterClockwiseArcRightDownward, CounterClockwiseArcRightUpward, For example, the SVG curve above can be simply described by: ClockwiseArcRightUpward, ClockwiseArcRightDownward, ClockwiseArcRightUpward, ClockwiseArcRightDownward, ClockwiseArcRightUpward, ClockwiseArcRightDownward, DiagonalLeftUpward, DiagonalRightDownward, DiagonalRightUpward, DiagonalLeftDownward, Hand Motion As you move your hand, your fingertips trace a curve through space: this curve is a "hand motion"
- 13. A Gesture is a finite state machine that its states are either Pose objects, Motion objects or other Gesture object Basic templates: A gesture should be Un/Registered with respect to the application context. Only registered gestures trigger ( ). Pose Motion Pose Pose Pose Gesture = Gesture Gesture Pose Motion Gesture
- 14. hand facing {forward}, orientation {up} {index, thumb} are {touching} {index} is {above} {thumb} {left, right} hand {thumb, index} are {opened} Pinch Retract NotPinchingSlingshot = {index, thumb} are {opened} {left, right} hand {index, thumb} are {notTouching} {index, thumb} are {opened} {left, right} hand {index, thumb} are {notTouching} move {backward} {left, right} hand NotPinching Slingshot Example
- 16. Gestures Service The Gestures Service provides gesture detection as a local service for client applications. After you install Project Prague, the Gestures Service will be launched every time your machine starts.
- 17. Demo Project Prague SDK and Tools
- 18. Using Gesture Service The detection is performed in the Microsoft.Gestures.Service.exe process that acts as a service for gesture detection. You have to use the GesturesServiceEndpoint class in order to communicate with the service. Reference the following assemblies: • Microsoft.Gestures.dll • Microsoft.Gestures.Endpoint.dll • Microsoft.Gestures.Protocol.dll
- 19. Create a gesture
- 20. Detect a gesture You need to hook up the event indicating gesture detection to the appropriate handler: You can detect when the single pose is triggered by the user:
- 21. Declare a Gesture as XAML
- 22. Skeleton tracking Project Prague also provides API for the skeleton tracking. For each hand: Position of the fingertips Fingers direction Palm position Palm orientation
- 23. V1 Released July ‘17 Runtime • Supported sensors: Intel RealSense SR300 & F200, Kinect v2 • Gestures support injected into • Windows Shell, PowerPoint, Skype, YouTube and Photos App • Wolfly Unity Game • Camera Gesture-triggered overlays (can be used in skype) Docs • Great Overview + tutorials Samples @Github • .NET • Unity
- 24. Demo Using Project Prague SDK
- 25. Cognitive Service Labs https://labs.cognitive.microsoft.com/ Trials have both rate limit, in terms of transactions per second or minute, and a monthly usage cap.
- 26. Key takeaways Human Interaction is the new challenge. Project Prague may be one of the solution. In the roadmap will be different kind of camera (e.g. IR camera) Keep in mind: Project Prague is an experimental project. If we’ll use it, it will become a supported project.
- 27. Feedback & Questions Grazie!!!!! Massimo Bonanni [email protected] @massimobonanni https://www.linkedin.com/in/massimobonanni/ #ProjectPragueCT
- 28. Project Prague Reference Project Prague Web Site https://labs.cognitive.microsoft.com/en-us/project-prague Project Prague documentation https://docs.microsoft.com/en-us/gestures/ Twitter/mail account @ProjectPrague / [email protected] Project Prague GitHub https://github.com/Microsoft/Gestures-Samples Toward Realistic Hands Gesture Interface: Keeping it Simple for Developers and Machines [PDF] https://aka.ms/TRHGI Sample GitHub https://github.com/massimobonanni/ProjectPragueSamples