Ryan Damm (Visby): Light Fields & Polygons: Making Sense of New Image Formats
- 1. LIGHT FIELDS & POLYGONS Making Sense of New Image Formats
- 2. WHAT WE’LL COVER: IMAGES • Major image formats of the future • Compare/contrast each • Look at where they fit in Join the conversation @ARealityEvent & #AWE2017
- 3. • No current standards • Display hardware ahead of software / imaging stack • Standards bodies lagging • Market will determine winner • Quality / performance will (probably) win WHY IMAGE FORMATS? Join the conversation @ARealityEvent & #AWE2017
- 4. WHY IMAGE FORMATS? Join the conversation @ARealityEvent & #AWE2017 • No current standards • Display hardware ahead of software / imaging stack • Standards bodies lagging • Market will determine winner • Quality / performance will (probably) win
- 5. WHY IMAGE FORMATS? Join the conversation @ARealityEvent & #AWE2017
- 6. WHY IMAGE FORMATS? Join the conversation @ARealityEvent & #AWE2017 The Software Stack Needs Work
- 7. FORMATS Let’s start with the fundamentals. Join the conversation @ARealityEvent & #AWE2017
- 8. • 360 Video (for VR) • Volumetric (polygons) FORMATS TODAY, TOMORROW Join the conversation @ARealityEvent & #AWE2017 Today Tomorrow
- 9. • 360 Video • Volumetric (polygons) FORMATS TODAY, TOMORROW Join the conversation @ARealityEvent & #AWE2017 • RGB+D Today Tomorrow
- 10. • 360 Video • Volumetric (polygons) FORMATS TODAY, TOMORROW Join the conversation @ARealityEvent & #AWE2017 • RGB+D • Volumetric Today Tomorrow
- 11. • 360 Video • Volumetric (polygons) FORMATS TODAY, TOMORROW Join the conversation @ARealityEvent & #AWE2017 • RGB+D • Volumetric • Light Fields Today Tomorrow
- 12. • As an image format: R, G, B, Depth • Depth carried along with pixel value • Often seen as acquisition / intermediate format RGB + DEPTH Join the conversation @ARealityEvent & #AWE2017 x,y raster of R,G,B,D values: <255,128,16,128>
- 13. • Polygons + pixels • Typically plays back in game engine • Lots of different capture / translation techniques being developed VOLUMETRIC Join the conversation @ARealityEvent & #AWE2017
- 14. • Polygons + pixels • Typically plays back in game engine • Lots of different capture / translation techniques being developed VOLUMETRIC Join the conversation @ARealityEvent & #AWE2017 Mesh (vertex lists, etc) for shape Textures (u,v maps) for color
- 15. • Represent light directly: location and direction • Subsample / shuffle rays for view synthesis • Capture: lots of cameras LIGHT FIELD Join the conversation @ARealityEvent & #AWE2017
- 16. Join the conversation @ARealityEvent & #AWE2017 BUT WHAT ARE THEY GOOD FOR? Good, Bad, and Ugly.
- 17. • It Works! • Extends existing standard • Re-uses robust video tech stack • Compression, delivery, decode • Hardware accelerated, etc RGB + DEPTH: Good Join the conversation @ARealityEvent & #AWE2017
- 18. • Very small parallax • No roundness*, specularity Join the conversation @ARealityEvent & #AWE2017 *high-resolution, stereo 3D can fake this for small ranges RGB + DEPTH: Bad
- 19. • Discrete depths • Infill? RGB + DEPTH: Ugly Join the conversation @ARealityEvent & #AWE2017
- 20. • Discrete depths • Infill? RGB + DEPTH: Ugly Join the conversation @ARealityEvent & #AWE2017
- 21. • It works! • Re-uses robust tech stack • Large movement space • Allows for rich interactivity VOLUMETRIC: The Good Join the conversation @ARealityEvent & #AWE2017
- 22. • Capture is an open question • Compression is tough VOLUMETRIC: The Bad Join the conversation @ARealityEvent & #AWE2017
- 23. VOLUMETRIC: The Ugly Join the conversation @ARealityEvent & #AWE2017
- 24. VOLUMETRIC: The Ugly Join the conversation @ARealityEvent & #AWE2017 (Also true of RGB+D, but masked by limited movement range.)
- 25. • Photorealistic with 6DOF • Potentially most accurate: light in, light out LIGHT FIELD: Good Join the conversation @ARealityEvent & #AWE2017
- 26. • Data! • Movement range defined by cameras • Aliasing, edges LIGHT FIELD: Bad Join the conversation @ARealityEvent & #AWE2017
- 27. • Doesn’t actually exist LIGHT FIELD: Ugly Join the conversation @ARealityEvent & #AWE2017
- 28. • Doesn’t actually exist (yet) LIGHT FIELD: Ugly Join the conversation @ARealityEvent & #AWE2017
- 29. SYNTHESIS: Polygons + Light Join the conversation @ARealityEvent & #AWE2017 • Hybrid approach: Polygons + light • Light field + polygon clipping masks • Solves interpolation problem • ‘BRDF’ / “dynamic pixel map”
- 30. • “Virtual objects / real environment” • Games / hybrid experiences • Interactive experiences Volumetric SYNTHESIS: A Mix of Formats? Join the conversation @ARealityEvent & #AWE2017 • “Dynamic posters” • Near-term format • User-generated content, live- streaming RGB+D • “Windows and crystal balls” • Realistic experiences • Transportation and immersion Light Field
- 31. THANK YOU For keeping it real. Join the conversation @ARealityEvent & #AWE2017