SlideShare a Scribd company logo
Computational Imaging with
Multi-Camera Time-of-Flight Systems
Felix Heide*1,2Shikhar Shrestha*1 Wolfgang Heidrich2,3 Gordon Wetzstein1
2University of British Columbia 3KAUST1Stanford University
www.computationalimaging.org
* joint first authors
Time-of-Flight Imaging
Kinect for Xbox One Google Tango Phone
Direct Sampling Transient Imaging
[Velten et al. 2013]
Google Self-Driving Car
Facial capture in VR
[Li et al. 2015]
Correlation Transient Imaging
[Heide et al. 2013]
Multi-Camera ToF System
• Multiple Camera/Sources
• Phased Array
• Control over modulation
• Extensible
• Reproducible
light source
camera
Time-of-Flight ImagingTime-of-Flight Imaging
Image formation
Electron
Field
Photon
Charge
storage
bin 1
Charge
storage
bin 2
Image formation
Image formation
1,2,3,4
Image formation
1,2,3,4
Static frame
Captured with TI OPT8241
Slow-Motion Video
Captured with TI OPT8241
Time-of-Flight ImagingTime-of-Flight Imaging – Dynamic Scene
Doppler shift:
Orthogonal
Frequencies:
Doppler Shift
Image formation – Multicam
Image formation – Frequency and Phase Synchronization
Image formation – Frequency and Phase Synchronization
Multi-Camera ToF System
• Exposure Synchronization
• Individual Sensor/Source
Modulation Signals
• High Frequency Accuracy
• Compatible with Recent
ToF Cameras
• Extensibility
Multi-Camera ToF System – High-Level System Architecture
Multi-Camera ToF System – Key System Components
Direct Digital Synthesis (DDS)
Ch 0
Ch 1
Ch 2
Ch 3
- Programmable Frequency/Phase
- Resolution : 0.12 Hz/0.02°
- Range : 0-200 MHz/0-360°
- Control over SPI
• Custom Board for End-to-End
Signal Conditioning
• Combines a High Speed
Comparator with Exposure
Gating IC
• Interfaces Directly with TI ToF
Camera
Signal Conditioning Circuitry
ToF Daughter BoardAD9959
Real-Time Controller
STMF4 Discovery Board
• Controller responsible for
triggering frames, controlling
waveform generation and
synchronization
• Runs low-level firmware based
on an interrupt driven state
machine architecture
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
MCU Host
Real-Time Controller
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
PC
ToF Cameras
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 500 MHz)
Configure Modulation Parameters
ToF Cameras
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Configure Camera Registers
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Initiate a Frame
Multi-Camera ToF System – System Workflow
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Start of Micro-Exposure/Quad
Multi-Camera ToF System – System Workflow
Enable Modulation
Multi-Camera ToF System – System Workflow
Modulation Signals to Sensor/Illum
Multi-Camera ToF System – System Workflow
Raw Capture Data
Multi-Camera ToF System – System Workflow
Multi-Camera ToF System – Demo Unit Design
DC-DC Voltage Regulators
Host MCUDDS Board
ToF Daughter Board
ToF Camera
Demo
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Phased Camera Depth Imaging
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Simultaneous Doppler Velocity Imaging
Simultaneous Doppler Velocity Imaging
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Non-Line-of-Sight Motion Detection
Non-Line-of-Sight Motion Detection
Non-Line-of-Sight Motion Detection
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Motion in Scattering Media
Motion in Scattering Media
Multi-Camera ToF System
Open-Source Hardware Platform:
www.computationalimaging.org

More Related Content

PPTX
Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
Matthew O'Toole
 
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 4)
Matthew O'Toole
 
PPTX
Optical Computing for Fast Light Transport Analysis
Matthew O'Toole
 
PPTX
Primal-Dual Coding to Probe Light Transport
Matthew O'Toole
 
PPTX
3D Shape and Indirect Appearance by Structured Light Transport
Matthew O'Toole
 
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 2)
Matthew O'Toole
 
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)
Matthew O'Toole
 
PPT
Raskar Keynote at Stereoscopic Display Jan 2011
Camera Culture Group, MIT Media Lab
 
Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
Matthew O'Toole
 
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 4)
Matthew O'Toole
 
Optical Computing for Fast Light Transport Analysis
Matthew O'Toole
 
Primal-Dual Coding to Probe Light Transport
Matthew O'Toole
 
3D Shape and Indirect Appearance by Structured Light Transport
Matthew O'Toole
 
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 2)
Matthew O'Toole
 
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)
Matthew O'Toole
 
Raskar Keynote at Stereoscopic Display Jan 2011
Camera Culture Group, MIT Media Lab
 

What's hot (20)

PPTX
Light field
Ujjayanta Bhaumik
 
PPTX
Light Field Photography Introduction
Camera Culture Group, MIT Media Lab
 
PPTX
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging
Gordon Wetzstein
 
PPT
Introduction to Camera Challenges - Ramesh Raskar
Camera Culture Group, MIT Media Lab
 
PPTX
Optical volume holograms and their applications
Hossein Babashah
 
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 1)
Matthew O'Toole
 
PPTX
mihara_iccp16_presentation
Hajime Mihara
 
PPTX
Single photon 3D Imaging with Deep Sensor Fusion
David Lindell
 
PPTX
Non-line-of-sight Imaging with Partial Occluders and Surface Normals | TOG 2019
StanfordComputationalImaging
 
PDF
Light Field Technology
Jeffrey Funk
 
PPTX
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
Emanuele Simioni
 
PDF
Nadia2013 research
Nadia Barbara
 
PDF
Lecture 02 yasutaka furukawa - 3 d reconstruction with priors
mustafa sarac
 
PPTX
Wave-Based Non-Line-of-Sight Imaging Using Fast f–k Migration | SIGGRAPH 2019
David Lindell
 
PDF
Keynote at Tracking Workshop during ISMAR 2014
Darius Burschka
 
PPTX
Evaluation of geometrical parameters of buildings from SAR images
Federico Ariu
 
PDF
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
Joo-Haeng Lee
 
PDF
DimEye Corp Presents Revolutionary VLS (Video Laser Scan) at SS IMMR 2013
Patrick Raymond
 
PDF
Goddard-DR-2010
Attila Kovacs
 
PDF
Introduction of slam
Hung-Chih Chang
 
Light field
Ujjayanta Bhaumik
 
Light Field Photography Introduction
Camera Culture Group, MIT Media Lab
 
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging
Gordon Wetzstein
 
Introduction to Camera Challenges - Ramesh Raskar
Camera Culture Group, MIT Media Lab
 
Optical volume holograms and their applications
Hossein Babashah
 
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 1)
Matthew O'Toole
 
mihara_iccp16_presentation
Hajime Mihara
 
Single photon 3D Imaging with Deep Sensor Fusion
David Lindell
 
Non-line-of-sight Imaging with Partial Occluders and Surface Normals | TOG 2019
StanfordComputationalImaging
 
Light Field Technology
Jeffrey Funk
 
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
Emanuele Simioni
 
Nadia2013 research
Nadia Barbara
 
Lecture 02 yasutaka furukawa - 3 d reconstruction with priors
mustafa sarac
 
Wave-Based Non-Line-of-Sight Imaging Using Fast f–k Migration | SIGGRAPH 2019
David Lindell
 
Keynote at Tracking Workshop during ISMAR 2014
Darius Burschka
 
Evaluation of geometrical parameters of buildings from SAR images
Federico Ariu
 
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
Joo-Haeng Lee
 
DimEye Corp Presents Revolutionary VLS (Video Laser Scan) at SS IMMR 2013
Patrick Raymond
 
Goddard-DR-2010
Attila Kovacs
 
Introduction of slam
Hung-Chih Chang
 
Ad

Viewers also liked (6)

PDF
"Time of Flight Sensors: How Do I Choose Them and How Do I Integrate Them?," ...
Edge AI and Vision Alliance
 
PDF
Facebook F8
OptimediaSpain
 
PDF
Melexis Time of Flight Imager for Automotive Applications 2017 teardown rever...
Yole Developpement
 
PPTX
Kinect sensor
bhoomit morkar
 
PDF
"The Evolution of Depth Sensing: From Exotic to Ubiquitous," a Presentation f...
Edge AI and Vision Alliance
 
PDF
Tobias Rothermel (pmd technologies): pmd ToF – the Swiss Army Knife of 3D Sen...
AugmentedWorldExpo
 
"Time of Flight Sensors: How Do I Choose Them and How Do I Integrate Them?," ...
Edge AI and Vision Alliance
 
Facebook F8
OptimediaSpain
 
Melexis Time of Flight Imager for Automotive Applications 2017 teardown rever...
Yole Developpement
 
Kinect sensor
bhoomit morkar
 
"The Evolution of Depth Sensing: From Exotic to Ubiquitous," a Presentation f...
Edge AI and Vision Alliance
 
Tobias Rothermel (pmd technologies): pmd ToF – the Swiss Army Knife of 3D Sen...
AugmentedWorldExpo
 
Ad

Similar to Multi-camera Time-of-Flight Systems | SIGGRAPH 2016 (20)

PDF
Survey of tof and imaging systems
manusuryavansh
 
PDF
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
Edge AI and Vision Alliance
 
PDF
"How to Choose a 3D Vision Sensor," a Presentation from Capable Robot Components
Edge AI and Vision Alliance
 
PPT
Raskar Computational Camera Fall 2009 Lecture 01
Camera Culture Group, MIT Media Lab
 
PPTX
Unit 3 machine vision
rknatarajan
 
PPTX
Unit 3 machine vision
Dr.G.Saravanan
 
PDF
Tfg ros-sis
Coral Alonso Jiménez
 
PDF
TOFD PRO System
Svitlana Priannikova
 
PDF
Wave optics analysis of camera image
IJCSEA Journal
 
PDF
Wave Optics Analysis of Camera Image Formation With Respect to Rectangular Ap...
IJCSEA Journal
 
PDF
03 cameras & their geometry
Sarhat Adam
 
PDF
Time of Flight and Structured Light Depth Cameras Technology and Applications...
radixbedrerr
 
PDF
“Introduction to Depth Sensing,” a Presentation from Meta
Edge AI and Vision Alliance
 
PPTX
Digital Image Processing
Athanasios Anastasiou
 
PPTX
Camera Sensor Technologies – An Overview [2023]
Logic Fruit Technologies
 
PPT
Raskar Ilp Oct08 Web
Camera Culture Group, MIT Media Lab
 
PPTX
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
AugmentedWorldExpo
 
PPTX
Digital radiography-Avinesh Shrestha
Avinesh Shrestha
 
PPTX
Coded Photography - Ramesh Raskar
Camera Culture Group, MIT Media Lab
 
Survey of tof and imaging systems
manusuryavansh
 
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
Edge AI and Vision Alliance
 
"How to Choose a 3D Vision Sensor," a Presentation from Capable Robot Components
Edge AI and Vision Alliance
 
Raskar Computational Camera Fall 2009 Lecture 01
Camera Culture Group, MIT Media Lab
 
Unit 3 machine vision
rknatarajan
 
Unit 3 machine vision
Dr.G.Saravanan
 
TOFD PRO System
Svitlana Priannikova
 
Wave optics analysis of camera image
IJCSEA Journal
 
Wave Optics Analysis of Camera Image Formation With Respect to Rectangular Ap...
IJCSEA Journal
 
03 cameras & their geometry
Sarhat Adam
 
Time of Flight and Structured Light Depth Cameras Technology and Applications...
radixbedrerr
 
“Introduction to Depth Sensing,” a Presentation from Meta
Edge AI and Vision Alliance
 
Digital Image Processing
Athanasios Anastasiou
 
Camera Sensor Technologies – An Overview [2023]
Logic Fruit Technologies
 
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
AugmentedWorldExpo
 
Digital radiography-Avinesh Shrestha
Avinesh Shrestha
 
Coded Photography - Ramesh Raskar
Camera Culture Group, MIT Media Lab
 

More from StanfordComputationalImaging (17)

PDF
Gaze-Contingent Ocular Parallax Rendering for Virtual Reality
StanfordComputationalImaging
 
PPTX
Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes - Siggraph 2019
StanfordComputationalImaging
 
PPTX
End-to-end Optimization of Cameras and Image Processing - SIGGRAPH 2018
StanfordComputationalImaging
 
PPTX
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
StanfordComputationalImaging
 
PPTX
Accommodation-invariant Computational Near-eye Displays - SIGGRAPH 2017
StanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 5
StanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 4
StanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 3
StanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 2
StanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 1
StanfordComputationalImaging
 
PPTX
VR2.0: Making Virtual Reality Better Than Reality?
StanfordComputationalImaging
 
PPTX
ProxImaL | SIGGRAPH 2016
StanfordComputationalImaging
 
PPTX
Light Field, Focus-tunable, and Monovision Near-eye Displays | SID 2016
StanfordComputationalImaging
 
PPTX
Adaptive Spectral Projection
StanfordComputationalImaging
 
PPTX
The Light Field Stereoscope | SIGGRAPH 2015
StanfordComputationalImaging
 
PPTX
Compressive Light Field Projection @ SIGGRAPH 2014
StanfordComputationalImaging
 
PPTX
Vision-correcting Displays @ SIGGRAPH 2014
StanfordComputationalImaging
 
Gaze-Contingent Ocular Parallax Rendering for Virtual Reality
StanfordComputationalImaging
 
Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes - Siggraph 2019
StanfordComputationalImaging
 
End-to-end Optimization of Cameras and Image Processing - SIGGRAPH 2018
StanfordComputationalImaging
 
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
StanfordComputationalImaging
 
Accommodation-invariant Computational Near-eye Displays - SIGGRAPH 2017
StanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 5
StanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 4
StanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 3
StanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 2
StanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 1
StanfordComputationalImaging
 
VR2.0: Making Virtual Reality Better Than Reality?
StanfordComputationalImaging
 
ProxImaL | SIGGRAPH 2016
StanfordComputationalImaging
 
Light Field, Focus-tunable, and Monovision Near-eye Displays | SID 2016
StanfordComputationalImaging
 
Adaptive Spectral Projection
StanfordComputationalImaging
 
The Light Field Stereoscope | SIGGRAPH 2015
StanfordComputationalImaging
 
Compressive Light Field Projection @ SIGGRAPH 2014
StanfordComputationalImaging
 
Vision-correcting Displays @ SIGGRAPH 2014
StanfordComputationalImaging
 

Recently uploaded (20)

PDF
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
PDF
Zero Carbon Building Performance standard
BassemOsman1
 
PDF
Cryptography and Information :Security Fundamentals
Dr. Madhuri Jawale
 
PDF
2025 Laurence Sigler - Advancing Decision Support. Content Management Ecommer...
Francisco Javier Mora Serrano
 
PDF
Unit I Part II.pdf : Security Fundamentals
Dr. Madhuri Jawale
 
PPTX
22PCOAM21 Session 2 Understanding Data Source.pptx
Guru Nanak Technical Institutions
 
PDF
Zero carbon Building Design Guidelines V4
BassemOsman1
 
PPTX
22PCOAM21 Session 1 Data Management.pptx
Guru Nanak Technical Institutions
 
PDF
Chad Ayach - A Versatile Aerospace Professional
Chad Ayach
 
PPTX
Tunnel Ventilation System in Kanpur Metro
220105053
 
PPTX
Module2 Data Base Design- ER and NF.pptx
gomathisankariv2
 
PDF
FLEX-LNG-Company-Presentation-Nov-2017.pdf
jbloggzs
 
PPTX
sunil mishra pptmmmmmmmmmmmmmmmmmmmmmmmmm
singhamit111
 
PDF
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
PDF
The Effect of Artifact Removal from EEG Signals on the Detection of Epileptic...
Partho Prosad
 
PDF
Construction of a Thermal Vacuum Chamber for Environment Test of Triple CubeS...
2208441
 
PDF
LEAP-1B presedntation xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
hatem173148
 
PDF
Machine Learning All topics Covers In This Single Slides
AmritTiwari19
 
PPTX
Civil Engineering Practices_BY Sh.JP Mishra 23.09.pptx
bineetmishra1990
 
PDF
2010_Book_EnvironmentalBioengineering (1).pdf
EmilianoRodriguezTll
 
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
Zero Carbon Building Performance standard
BassemOsman1
 
Cryptography and Information :Security Fundamentals
Dr. Madhuri Jawale
 
2025 Laurence Sigler - Advancing Decision Support. Content Management Ecommer...
Francisco Javier Mora Serrano
 
Unit I Part II.pdf : Security Fundamentals
Dr. Madhuri Jawale
 
22PCOAM21 Session 2 Understanding Data Source.pptx
Guru Nanak Technical Institutions
 
Zero carbon Building Design Guidelines V4
BassemOsman1
 
22PCOAM21 Session 1 Data Management.pptx
Guru Nanak Technical Institutions
 
Chad Ayach - A Versatile Aerospace Professional
Chad Ayach
 
Tunnel Ventilation System in Kanpur Metro
220105053
 
Module2 Data Base Design- ER and NF.pptx
gomathisankariv2
 
FLEX-LNG-Company-Presentation-Nov-2017.pdf
jbloggzs
 
sunil mishra pptmmmmmmmmmmmmmmmmmmmmmmmmm
singhamit111
 
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
The Effect of Artifact Removal from EEG Signals on the Detection of Epileptic...
Partho Prosad
 
Construction of a Thermal Vacuum Chamber for Environment Test of Triple CubeS...
2208441
 
LEAP-1B presedntation xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
hatem173148
 
Machine Learning All topics Covers In This Single Slides
AmritTiwari19
 
Civil Engineering Practices_BY Sh.JP Mishra 23.09.pptx
bineetmishra1990
 
2010_Book_EnvironmentalBioengineering (1).pdf
EmilianoRodriguezTll
 

Multi-camera Time-of-Flight Systems | SIGGRAPH 2016