Lecture 2 Presence and Perception
10 likes•3,980 views
The document discusses virtual reality (VR) perception and presence, exploring aspects such as technology, user immersion, and human sensory experiences. It distinguishes between immersion and presence, highlighting their importance in creating realistic VR environments. Additionally, it examines various factors influencing presence, including sensory input and user interaction, while reviewing methods for measuring presence within VR contexts.
Lecture 2 Presence and Perception
- 1. LECTURE 2: VR PERCEPTION AND PRESENCE COMP 4010 - Virtual Reality Semester 5 - 2019 Mark Billinghurst, Bruce Thomas, Gun Lee University of South Australia August 6th 2019
- 2. Overview •Presence in VR •Perception and VR •Human Perception •VR Technology
- 4. The Incredible Disappearing Computer 1960-70’s Room 1970-80’s Desk 1980-90’s Lap 1990-2000’s Hand
- 5. Virtual Reality • Users immersed in Computer Generated environment • HMD, gloves, 3D graphics, body tracking
- 6. Augmented Reality •Virtual Images blended with the real world • See-through HMD, handheld display, viewpoint tracking, etc..
- 7. Milgram’s Mixed Reality (MR) Continuum Augmented Reality Virtual Reality Real World Virtual World Mixed Reality "...anywhere between the extrema of the virtuality continuum." P. Milgram and A. F. Kishino, (1994) A Taxonomy of Mixed Reality Visual Displays Internet of Things
- 10. Oculus Rift Sony Morpheus HTC/Valve Vive 2016 - Rise of Consumer HMDs
- 11. Projected HMD Sales • asdf
- 13. Conclusion • Virtual Reality has a long history • > 50 years of HMDs, simulators • Key elements for VR were in place by early 1990’s • Displays, tracking, input, graphics • Strong support from military, government, universities • First commercial wave failed in late 1990’s • Too expensive, bad user experience, poor technology, etc • We are now in second commercial wave • Better experience, Affordable hardware • Large commercial investment, Significant installed user base
- 14. PRESENCE
- 15. ‘Virtual Reality is a synthetic sensory experience which may one day be indistinguishable from the real physical world.” -Roy Kalawsky (1993) Today Tomorrow
- 16. Presence .. “The subjective experience of being in one place or environment even when physically situated in another” Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and virtual environments, 7(3), 225-240.
- 17. Presence Definition “Presence is a psychological state .. in which even though part or all of an individual’s current experience is generated by .. technology, part or all of the individual’s perception fails to .. acknowledge the role of the technology in the experience.” International Society for Presence Research, 2016 https://ispr.info/
- 18. Immersion vs. Presence • Immersion: the extent to which technology delivers a vivid illusion of reality to the senses of a human participant. • Presence: a state of consciousness, the (psychological) sense of being in the virtual environment. • So Immersion produces a sensation of Presence • Goal of VR: Create a high degree of Presence • Make people believe they are really in Virtual Environment Slater, M., & Wilbur, S. (1997). A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators and virtual environments, 6(6), 603-616.
- 19. Three Types of Presence • Personal Presence, the extent to which the person feels like he or she is part of the virtual environment; • Social Presence, the extent to which other beings (living or synthetic) also exist in the VE; • Environmental Presence, the extent to which the environment itself acknowledges and reacts to the person in the VE. Heeter, C. (1992). Being there: The subjective experience of presence. Presence: Teleoperators & Virtual Environments, 1(2), 262-271.
- 20. Benefits of High Presence • Leads to greater engagement, excitement and satisfaction • Increased reaction to actions in VR • People more likely to behave like in the real world • E.g. people scared of heights in real world will be scared in VR • More natural communication (Social Presence) • Use same cues as face to face conversation • Note: The relationship between Presence and Performance is unclear – still an active area of research
- 21. How to Create Strong Presence? • Use Multiple Dimensions of Presence • Create rich multi-sensory VR experiences • Include social actors/agents that interact with user • Have environment respond to user • What Influences Presence • Vividness – ability to provide rich experience (Steuer 1992) • Using Virtual Body – user can see themselves (Slater 1993) • Internal factors – individual user differences (Sadowski 2002) • Interactivity – how much users can interact (Steuer 1992) • Sensory, Realism factors (Witmer 1998)
- 22. Factors Contributing to Presence • From • Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence, 7(3), 225-240.
- 23. Presence Guidelines (Sadowski 2002)
- 24. Example: UNC Pit Room (2002) • Key Features • Training room and pit room • Physical walking • Fast, accurate, room scale tracking • Haptic feedback – feel edge of pit, walls • Strong visual and 3D audio cues • Task • Carry object across pit • Walk across or walk around • Dropping virtual balls at targets in pit • http://wwwx.cs.unc.edu/Research/eve/walk_exp/
- 25. Typical Subject Behaviour • Note – from another pit experiment • https://www.youtube.com/watch?v=VVAO0DkoD-8
- 27. Measuring Presence • Presence is very subjective so there is a lot of debate among researchers about how to measure it • Subjective Measures • Self report questionnaire • University College London Questionnaire (Slater 1999) • Witmer and Singer Presence Questionnaire (Witmer 1998) • ITC Sense Of Presence Inventory (Lessiter 2000) • Continuous measure • Person moves slider bar in VE depending on Presence felt • Objective Measures • Behavioural • reflex/flinch measure, startle response • Physiological measures • change in heart rate, skin conductance, skin temperature Presence Slider
- 28. Example: Witmer and Singer (1998) • 32 questions in 4 categories/factors • Control (CF), Sensory (SF), Realism (RF), Distraction factors (DF) • Answered on Likert scale from 1 to 7 ( 1 = low, 7 = high)
- 29. Example: Heartrate in Pit Experiment • Physiological measures can be used as a reliable measure of Presence - especially change in heart rate (HR) • Change in HR agreed with UCL subjective questionnaire • HR increased with passive haptics, and increase in fps Meehan, M., Insko, B., Whitton, M., & Brooks, F. P. (2001). Physiological measures of presence in virtual environments. In Proceedings of 4th International Workshop on Presence (pp. 21-23).
- 31. How do We Perceive Reality? • We understand the world through our senses: • Sight, Hearing, Touch, Taste, Smell (and others..) • Two basic processes: • Sensation – Gathering information • Perception – Interpreting information
- 33. Goal of Virtual Reality “.. to make it feel like you’re actually in a place that you are not.” Palmer Luckey Co-founder, Oculus
- 34. Creating the Illusion of Reality • Fooling human perception by using technology to generate artificial sensations • Computer generated sights, sounds, smell, etc.
- 35. Reality vs. Virtual Reality • In a VR system there are input and output devices between human perception and action • Goal is to create illusion of reality – high Presence
- 36. Example Birdly - http://www.somniacs.co/ • Create illusion of flying like a bird • Multisensory VR experience • Visual, audio, wind, haptic
- 38. HUMAN PERCEPTION
- 39. Motivation • Understand: In order to create a strong sense of Presence we need to understand the Human Perception system • Stimulate: We need to be able to use technology to provide real world sensory inputs, and create the VR illusion VR Hardware Human Senses
- 40. Senses • How an organism obtains information for perception: • Sensation part of Somatic Division of Peripheral Nervous System • Integration and perception requires the Central Nervous System • Five major senses (but there are more..): • Sight (Opthalamoception) • Hearing (Audioception) • Taste (Gustaoception) • Smell (Olfacaoception) • Touch (Tactioception)
- 41. Relative Importance of Each Sense • Percentage of neurons in brain devoted to each sense • Sight – 30% • Touch – 8% • Hearing – 2% • Smell - < 1% • Over 60% of brain involved with vision in some way
- 42. Other Lessor Known Senses.. • Proprioception = sense of body position • what is your body doing right now • Equilibrium = balance • Acceleration • Nociception = sense of pain • Temperature • Satiety (the quality or state of being fed or gratified to or beyond capacity) • Thirst • Micturition • Amount of CO2 and Na in blood
- 43. Sight
- 44. The Human Visual System • Purpose is to convert visual input to signals in the brain
- 45. The Human Eye • Light passes through cornea and lens onto retina • Photoreceptors in retina convert light into electrochemical signals
- 46. Photoreceptors – Rods and Cones • Retina photoreceptors come in two types, Rods and Cones • Rods – 125 million, periphery of retina, no colour detection, night vision • Cones – 4-6 million, center of retina, colour vision, day vision
- 47. Human Horizontal and Vertical FOV • Humans can see ~135 o vertical (60 o above, 75 o below) • See up to ~ 210 o horizontal FOV, ~ 115 o stereo overlap • Colour/stereo in centre, Black & White/mono in periphery
- 48. Vergence + Accommodation • saas
- 50. Vergence-Accommodation Conflict • Looking at real objects, vergence and focal distance match • In VR, vergence and accommodation can miss-match • Focusing on HMD screen, but accommodating for virtual object behind screen
- 51. Visual Acuity Visual Acuity Test Targets • Ability to resolve details • Several types of visual acuity • detection, separation, etc • Normal eyesight can see a 50 cent coin at 80m • Corresponds to 1 arc min (1/60th of a degree) • Max acuity = 0.4 arc min
- 52. Stereo Perception/Stereopsis • Eyes separated by IPD • Inter pupillary distance • 5 – 7.5cm (avge. 6.5cm) • Each eye sees diff. image • Separated by image parallax • Images fused to create 3D stereo view
- 54. Depth Perception • The visual system uses a range of different Stereoscopic and Monocular cues for depth perception Stereoscopic Monocular eye convergence angle disparity between left and right images diplopia eye accommodation perspective atmospheric artifacts (fog) relative sizes image blur occlusion motion parallax shadows texture Parallax can be more important for depth perception! Stereoscopy is important for size and distance evaluation
- 56. Depth Perception Distances • i.e. convergence/accommodation used for depth perception < 10m
- 57. Properties of the Human Visual System • visual acuity: 20/20 is ~1 arc min • field of view: ~200° monocular, ~120° binocular, ~135° vertical • resolution of eye: ~576 megapixels • temporal resolution: ~60 Hz (depends on contrast, luminance) • dynamic range: instantaneous 6.5 f-stops, adapt to 46.5 f-stops • colour: everything in CIE xy diagram • depth cues in 3D displays: vergence, focus, (dis)comfort • accommodation range: ~8cm to ∞, degrades with age
- 58. Comparison between Eyes and HMD Human Eyes HTC Vive FOV 200° x 135° 110° x 110° Stereo Overlap 120° 110° Resolution 30,000 x 20,000 2,160 x 1,200 Pixels/inch >2190 (100mm to screen) 456 Update 60 Hz 90 Hz See http://doc-ok.org/?p=1414 http://www.clarkvision.com/articles/eye-resolution.html http://wolfcrow.com/blog/notes-by-dr-optoglass-the-resolution-of-the-human-eye/
- 59. Hearing
- 60. Anatomy of the Ear
- 61. Auditory Thresholds • Humans hear frequencies from 20 – 22,000 Hz • Most everyday sounds from 80 – 90 dB
- 62. Sound Localization • Humans have two ears • localize sound in space • Sound can be localized using 3 coordinates • Azimuth, elevation, distance
- 64. Sound Localization (Azimuth Cues) Interaural Time Difference
- 65. HRTF (Elevation Cue) • Pinna and head shape affect frequency intensities • Sound intensities measured with microphones in ear and compared to intensities at sound source • Difference is HRTF, gives clue as to sound source location
- 66. Accuracy of Sound Localization • People can locate sound • Most accurately in front of them • 2-3° error in front of head • Least accurately to sides and behind head • Up to 20° error to side of head • Largest errors occur above/below elevations and behind head • Front/back confusion is an issue • Up to 10% of sounds presented in the front are perceived coming from behind and vice versa (more in headphones) BUTEAN, A., Bălan, O., NEGOI, I., Moldoveanu, F., & Moldoveanu, A. (2015). COMPARATIVE RESEARCH ON SOUND LOCALIZATION ACCURACY IN THE FREE-FIELD AND VIRTUAL AUDITORY DISPLAYS. InConference proceedings of» eLearning and Software for Education «(eLSE)(No. 01, pp. 540-548). Universitatea Nationala de Aparare Carol I.
- 67. Touch
- 68. Touch • Mechanical/Temp/Pain stimuli transduced into Action Potentials (AP) • Transducing structures are specialized nerves: • Mechanoreceptors: Detect pressure, vibrations & texture • Thermoreceptors: Detect hot/cold • Nocireceptors: Detect pain • Proprioreceptors: Detect spatial awareness • This triggers an AP which then travels to various locations in the brain via the somatosensory nerves
- 69. Haptic Sensation • Somatosensory System • complex system of nerve cells that responds to changes to the surface or internal state of the body • Skin is the largest organ • 1.3-1.7 square m in adults • Tactile: Surface properties • Receptors not evenly spread • Most densely populated area is the tongue • Kinesthetic: Muscles, Tendons, etc. • Also known as proprioception
- 70. Cutaneous System • Skin – heaviest organ in the body • Epidermis outer layer, dead skin cells • Dermis inner layer, with four kinds of mechanoreceptors
- 71. Mechanoreceptors • Cells that respond to pressure, stretching, and vibration • Slow Acting (SA), Rapidly Acting (RA) • Type I at surface – light discriminate touch • Type II deep in dermis – heavy and continuous touch Receptor Type Rate of Acting Stimulus Frequency Receptive Field Detection Function Merkel Discs SA-I 0 – 10 Hz Small, well defined Edges, intensity Ruffini corpuscles SA-II 0 – 10 Hz Large, indistinct Static force, skin stretch Meissner corpuscles RA-I 20 – 50 Hz Small, well defined Velocity, edges Pacinian corpuscles RA-II 100 – 300 Hz Large, indistinct Acceleration, vibration
- 72. Spatial Resolution • Sensitivity varies greatly • Two-point discrimination Body Site Threshold Distance Finger 2-3mm Cheek 6mm Nose 7mm Palm 10mm Forehead 15mm Foot 20mm Belly 30mm Forearm 35mm Upper Arm 39mm Back 39mm Shoulder 41mm Thigh 42mm Calf 45mm http://faculty.washington.edu/chudler/chsense.html
- 73. Proprioception/Kinaesthesia • Proprioception (joint position sense) • Awareness of movement and positions of body parts • Due to nerve endings and Pacinian and Ruffini corpuscles at joints • Enables us to touch nose with eyes closed • Joints closer to body more accurately sensed • Users know hand position accurate to 8cm without looking at them • Kinaesthesia (joint movement sense) • Sensing muscle contraction or stretching • Cutaneous mechanoreceptors measuring skin stretching • Helps with force sensation
- 74. Smell
- 75. Olfactory System • Human olfactory system. 1: Olfactory bulb 2: Mitral cells 3: Bone 4: Nasal epithelium 5: Glomerulus 6: Olfactory receptor neurons
- 76. How the Nose Works • https://www.youtube.com/watch?v=zaHR2MAxywg
- 77. Smell • Smells are sensed by olfactory sensory neurons in the olfactory epithelium • 10 cm2 with hundreds of different types of olfactory receptors • Human’s can detect at least 10,000 different odors • Some researchers say trillions of odors • Sense of smell closely related to taste • Both use chemo-receptors • Olfaction + taste contribute to flavour • The olfactory system is the only sense that bypasses the thalamus and connects directly to the forebrain
- 78. Taste
- 79. Sense of Taste • https://www.youtube.com/watch?v=FSHGucgnvLU
- 80. Basics of Taste • Sensation produced when a substance in the mouth reacts chemically with taste receptor cells • Taste receptors mostly on taste buds on the tongue • 2,000 – 5,000 taste buds on tongues/100+ receptors each • Five basic tastes: • sweetness, sourness, saltiness, bitterness, and umami • Flavour influenced by other senses • smell, texture, temperature, “coolness”, “hotness”
- 81. Taste Trivia
- 82. VR TECHNOLOGY
- 83. Using Technology to Stimulate Senses • Simulate output • E.g. simulate real scene • Map output to devices • Graphics to HMD • Use devices to stimulate the senses • HMD stimulates eyes Visual Simulation 3D Graphics HMD Vision System Brain Example: Visual Simulation Human-Machine Interface
- 84. Key Technologies for VR System • Visual Display • Stimulate visual sense • Audio/Tactile Display • Stimulate hearing/touch • Tracking • Changing viewpoint • User input • Input Devices • Supporting user interaction
- 85. What Happens When Senses Don’t Match? • 20-30% VR users experience motion sickness • Sensory Conflict Theory • Visual cues don’t match vestibular cues • Eyes – “I’m moving!”, Vestibular – “No, you’re not!”
- 86. Avoiding Motion Sickness • Better VR experience design • More natural movements • Improved VR system performance • Less tracking latency, better graphics frame rate • Provide a fixed frame of reference • Ground plane, vehicle window • Add a virtual nose • Provide peripheral cue • Eat ginger • Reduces upset stomach
- 87. 5 Key Technical Requirements for Presence • Persistence • > 90 Hz refresh, < 3 ms persistence, avoid retinal blur • Optics • Wide FOV > 90 degrees, comfortable eyebox, good calibration • Tracking • 6 DOF, 360 tracking, sub-mm accuracy, no jitter, good tracking volume • Resolution • Correct stereo, > 1K x 1K resolution, no visible pixels • Latency • < 20 ms latency, fuse optical tracking and IMU, minimize tracking loop http://www.roadtovr.com/oculus-shares-5-key-ingredients-for-presence-in-virtual-reality/
- 88. VISUAL DISPLAY
- 89. Creating an Immersive Experience •Head Mounted Display •Immerse the eyes •Projection/Large Screen •Immerse the head/body •Future Technologies •Neural implants •Contact lens displays, etc
- 90. HMD Basic Principles • Use display with optics to create illusion of virtual screen
- 91. Key Properties of HMDs • Lens • Focal length, Field of View • Occularity, Interpupillary distance • Eye relief, Eye box • Display • Resolution, contrast • Power, brightness • Refresh rate • Ergonomics • Size, weight • Wearability
- 92. Field of View Monocular FOV is the angular subtense of the displayed image as measured from the pupil of one eye. Total FOV is the total angular size of the displayed image visible to both eyes. Binocular(or stereoscopic) FOV refers to the part of the displayed image visible to both eyes. FOV may be measured horizontally, vertically or diagonally.
- 93. Ocularity • Monocular - HMD image to only one eye. • Bioccular - Same HMD image to both eyes. • Binocular (stereoscopic) - Different but matched images to each eye.
- 94. Interpupillary Distance (IPD) nIPD is the horizontal distance between a user's eyes. nIPD is the distance between the two optical axes in a binocular view system.
- 95. Distortion in Lens Optics A rectangle Maps to this HMD optics distort images shown in them
- 96. Example Distortion Oculus Rift DK2 HTC Vive
- 97. To Correct for Distortion • Must pre-distort image • This is a pixel-based distortion • Use shader programming
- 98. HMD Design Trade-offs • Resolution vs. field of view • As FOV increases, resolution decreases for fixed pixels • Eye box vs. field of view • Larger eye box limits field of view • Size, Weight and Power vs. everything else vs.
- 99. Oculus Rift • Cost: $399 USD • FOV: 110 o Horizontal • Refresh rate: 90 Hz • Resolution 1080x1200/eye • 3 DOF orientation tracking • 3 axis positional tracking
- 100. Inside an Oculus Rift
- 102. Computer Based vs. Mobile VR Displays
- 103. Google Cardboard • Released 2014 (Google 20% project) • >5 million shipped/given away • Easy to use developer tools + =
- 104. Multiple Mobile VR Viewers Available
- 105. Projection/Large Display Technologies • Room Scale Projection • CAVE, multi-wall environment • Dome projection • Hemisphere/spherical display • Head/body inside • Vehicle Simulator • Simulated visual display in windows
- 106. CAVE • Developed in 1992, EVL University of Illinois Chicago • Multi-walled stereo projection environment • Head tracked active stereo Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., & Hart, J. C. (1992). The CAVE: audio visual experience automatic virtual environment. Communications of the ACM, 35(6), 64-73.
- 107. Typical CAVE Setup • 4 sides, rear projected stereo images
- 108. Demo Video – Wisconsin CAVE • https://www.youtube.com/watch?v=mBs-OGDoPDY
- 109. CAVE Variations
- 110. Stereo Projection • Active Stereo • Active shutter glasses • Time synced signal • Brighter images • More expensive • Passive Stereo • Polarized images • Two projectors (one/eye) • Cheap glasses (powerless) • Lower resolution/dimmer • Less expensive
- 112. Vehicle Simulators • Combine VR displays with vehicle • Visual displays on windows • Motion base for haptic feedback • Audio feedback • Physical vehicle controls • Steering wheel, flight stick, etc • Full vehicle simulation • Emergencies, normal operation, etc • Weapon operation • Training scenarios
- 113. Demo: Boeing 787 Simulator • https://www.youtube.com/watch?v=3iah-blsw_U