![References
• [Dhillon] “Inverse Eigenvalue Problem in Wireless
Communications”
• [Gilles] “Image Decomposition: Theory, Numerical Schemes,
and Performance Evaluation”.
• [Strang] “The Search for a Good Basis”.](https://image.slidesharecdn.com/gdc2012framessparsityandglobalillumination-adobe-130302121609-phpapp02/85/Gdc2012-frames-sparsity-and-global-illumination-76-320.jpg)
Gdc2012 frames, sparsity and global illumination
- 1. Frames, Quadratures and Global Illumination: New Math for Games Robin Green – Microsoft Corp Manny Ko – PDI/Dreamworks
- 2. WARNING • This talk is MATH HEAVY • We assume you understand the basics of: – Linear Algebra, Calculus, 3D Mathematics – Spherical Harmonic Lighting, Visibility, BRDF, Cosine Term – Monte Carlo Integration, Unbiased Spherical Sampling – Precomputed Radiance Transfer, Rendering Equation • This is bleeding edge research. • There are still a lot of unanswered questions.
- 3. The Mission • We need to find a spherical basis that is – Is defined natively on the sphere – Retains the norm as a Parseval Tight Frame – Allows us to select the number of coefficients – Is spectrally and spatially concentrated – Is cheap to project – Is cheap to rotate – Exhibits rotational invariance
- 4. Spherical Harmonics • The Real SH functions are a family of orthonormal basis function on the sphere.
- 6. SH the Good • Analysis is simple projection due to orthonormal basis • Reconstruction as weighted-sum • Successive approximation property • Product-integral is dot-product of two coefficient vectors
- 7. SH Deficiencies • SH produces signed values yet all visibility functions, BRDFs and light probes are strictly positive. • SH projections are global and smooth, visibility functions are local and sharp. • SH reproduces a signal at the limit. There is no guarantee the result is close to the original at low orders. Even at high orders it “rings” esp when restricted to the hemisphere.
- 8. SH Deficiencies II • SH support is well localized in frequencies but its spatial support is global. Limiting them to the hemisphere produces ringing – Gibbs. • More important it is not just a problem with SH, all ONB will have the same issue (see Strang ‘The search for a good Basis’ http://www-math.mit.edu/~gs)
- 9. Strang • “Smooth variations are well represented by low frequency terms. But edges that are easy in the standard basis have become extremely expensive – because of the slow 1/k decay of Fourier coefficients, and the ripple from the Gibbs phenomenon when the series is truncated. These shadows near a discontinuity are called ringing.” G. Strang
- 10. Haar Wavelets • Haar wavelets are spatially compact and produce a lot of zero coefficients. • Generating 6 times the coefficients, papers rely on compression and highly conditional code. • Projecting cube faces onto the sphere introduces distortions, and seams for filtering and rotation.
- 11. Spherical Wavelets? • Spherical wavelets typical are tensored-1D wavelets lifted onto the sphere using inverse stereographic projection – Often realized as separable filters. However, directions in images are not limited to the two axes. • All the vanishing moments and other properties don’t automatically carry over from 1D to 2D • The parameterization often is non-manifold since they are based on tangent-plane arguments. • Antonine & McEwen’s directional wavelets still worth a look
- 12. Radial Basis Functions • Radial Basis Functions are also used, usually sums of Gaussian lobes. • Need to solve two variables – direction and spread. Leads to conditional code that is not GPU friendly. • Zonal Harmonics are another form of steerable RBF built out of orthogonal parts.
- 13. Smoothness vs. Localization • Haar and SH are two ends of a continuum – one smooth and global, the other highly local and unsmooth. This is Spatial vs. Spectral compactness. Q: What lives in the middle ground?
- 14. Spatial vs. Spectral • It turns out, the Spatial vs. Spectral problem is exactly Heisenberg’s Uncertainty Principle. • You cannot have both spatial compactness and spectral compactness at the same time – e.g. The Fourier transform of a delta function is infinitely spread out spectrally. • But… thanks to a solution by David Slepian to the Concentration Problem you can get pretty close.
- 15. Fundamental Questions 1. Where do these Orthonormal Basis Functions come from? 2. How can we loosen the rules so we can define better functions for our own use cases? 3. What are the key properties we need to retain for our functions to be useful?
- 16. What You Need To Know • We are going to introduce Frame Theory and Spherical Quadrature, just enough to understand two key concepts: Parseval Tight Frames Spherical t-Designs
- 17. Hilbert Spaces
- 19. Orthonormal Bases 1.57 1 3.14 2 3.14 2 x 1.57 1
- 20. Orthonormal Bases 1 1 1 x 1
- 26. General Bases • We use Orthonormal Bases all the time • Every rotation matrix in 3D is an Orthonormal Basis
- 27. General Bases • What if you chose vectors that are not orthogonal?
- 28. General Base • We can still represent points, but we need a “helper” basis to get us there.
- 29. General Bases
- 31. Matrix Notation
- 32. Matrix Notation
- 33. Breaking the Rules • What happens if we add another vector to the basis? • Now we have an overcomplete system, and coordinates are now linearly dependent
- 35. Breaking the Rules • We can still project a point and reconstruct it
- 37. Frames
- 40. PTF-Mercedes Benz is Self Dual
- 42. Frame Bounds
- 43. Frame Bounds • We can categorize frames based on their construction Unit Frame Tight Frame Parseval Tight Frame • Any tight frame can be factored into a PTF
- 44. Tight Frames • Self-dual – Dual preserves any structure in the frame – e.g. wavelet property, or spatial/spectral locality. – Not true for general frames • Computational tractable • The ratio B/A for a frame is critical as it controls the condition- number of the frame operator. The closer B/A=1 the better.
- 45. Gram Matrix
- 46. Gram Matrix II
- 47. TF vs. ONB • Analysis is no longer simple projection – Frame operator etc. • Reconstruction as weighted-sum • Successive approximation property • Product-integral is dot-product of two coefficient vectors – Need to add the Gram-matrix
- 49. Integrating on the Sphere
- 50. Gaussian Quadrature • If you are integrating a fixed order polynomial over a closed range, Gaussian quadrature can find the integral using a small number of evaluations Simpson’s rule graph • Trapezium Rule is a quadrature for linear curves. • Simpson’s Rule is a quadrature for quadratic curves.
- 53. Minimum Order t-designs order 2 order 3 order 4 verts 4 verts 6 verts 14 order 5 order 6 order 7 verts 20 verts 26 verts 24
- 55. Simplifications
- 59. Spherical Needlet quadrature A single direction needlet Legendre over the quadrature Littlewood-Paley sphere weight weighting polynomial
- 61. What does this integrate to?
- 62. What does this integrate to?
- 63. Needlet B=2.0 and j=1
- 64. Needlet B=2.0 and j=2
- 65. Needlet B=2.0 and j=3
- 66. Needlet B=3.0 and j=1
- 67. Needlet B=2.4 and j=1
- 68. Spherical Basis
- 70. Needlet vs. SH
- 71. Monte Carlo Sampling • Sampling needlets correctly requires non-uniform sampling
- 72. Fast Projection Plot error of lerp LUT versus actual function.
- 73. Key Features of a Spherical Basis • Radially symmetric basis – Allows fast projection – Allows fast and stable rotation • Defined from natively embedded atoms – No parameterization problems – Use lifting to construct a more performant basis – Spherical concentration shows that localization is possible • Using Frames – Allows simpler definition of the problem – Who needs successive approximation anyway?
- 74. Future Work • Littlewood-Paley is just one partition of unity optimized for spectral concentration. Other papers have optimized for spatial and other metrics. • Ridgelets, Curvelets, Bandlets, Shearlets etc. – all utilizes frame construct and theories • Compressive Sensing & Sparsity
- 75. Key References • D. Marinucci et al, “Spherical Needlets for CMB Data Analysis”, arxiv.org/pdf/7070.0844.pdf, 2008 • F. Guilloux et al, “Practical Wavelet Design on the Sphere”, Applied and Computational Harmonic Analysis, 2008 • J. Kovacevic et al, “Life Beyond Bases: The Advent of Frames”, Signal Processing Magazine, IEEE, Vol.24, No.4, July 2007 • T. Hines, “An Introduction to Frame Theory”, Aug 2009, http://mathpost.asu.edu/~hines/docs/090727IntroFrames.pdf
- 76. References • [Dhillon] “Inverse Eigenvalue Problem in Wireless Communications” • [Gilles] “Image Decomposition: Theory, Numerical Schemes, and Performance Evaluation”. • [Strang] “The Search for a Good Basis”.
- 77. quadrature A single direction needlet Legendre over the quadrature Littlewood-Paley sphere weight weighting polynomial