From moments to sparse representations, a geometric, algebraic and algorithmic viewpoint. B. Mourrain. Part 2

From moments to sparse representations,
a geometric, algebraic and algorithmic viewpoint
Bernard Mourrain
Inria M´editerran´ee, Sophia Antipolis
Bernard.Mourrain@inria.fr
Part II

Decomposition algorithms
1 Decomposition algorithms
2 Low rank classiﬁcation
3 The variety of missing moments
B. Mourrain From moments to sparse representations 2 / 39

Univariate series:
Kronecker (1881)
The Hankel operator
Hσ : CN,finite → CN
(pm) → ( m σm+npm)n∈N
is of finite rank r iff ∃ω1, . . . , ωr ∈ C[y] and ξ1, . . . , ξr ∈ C distincts s.t.
σ(y) =
n∈N
σn
yn
n!
=
r
i=1
ωi (y)eξi
(y)
with r
i=1(deg(ωi ) + 1) = r.

Multivariate series:
Theorem (Generalized Kronecker Theorem)
For σ = (σ1, . . . , σm) ∈ (R∗)m, the Hankel operator
Hσ : R → (R∗
)m
p → (p σ1, . . . , p σm)
is of rank r iﬀ
σj =
r
i=1
ωj,i (y) eξi
(y) ∈ PolExp, j = 1, . . . , m
with r = r
i=1 µ(ω1,i , . . . , ωm,i ). In this case, we have
VC(Iσ) = {ξ1, . . . , ξr }.
Iσ = Q1 ∩ · · · ∩ Qr with Q⊥
i = ω1,i , . . . , ωm,i eξi
(y).
If m = 1, Aσ is Gorenstein (A∗
σ = Aσ σ is a free Aσ-module of rank 1)
and (a, b) → σ|ab is non-degenerate in Aσ.

The decomposition from the algebraic structure
Decomposition problem
Given a (truncated) sequence of moments σα, α ∈ A, ﬁnd
ξi = (ξi,1, ξi,1, . . . , ξi,n) ∈ K
n
disctint, ωi ∈ K. s.t. σ = i ωi eξi
Hankel operator: For σ ∈ R∗,
Hσ : R → R∗
p → p σ
Quotient algebra: Aσ = R/Iσ where Iσ = ker Hσ.
0 → Iσ → K[x]
Hσ
−→ A∗
σ → 0
p → p σ
Isomorphism between Aσ and A∗
σ = I⊥
σ .
(Aσ Gorenstein, i.e. ∃τ = σ ∈ A∗
σ s.t. A∗
σ = Aσ τ is a free Aσ-module).

Find the points ξi as the roots of Iσ and the weights ωi from the
idempotents of Aσ.

The structure of Aσ
For σ = r
i=1 ωi eξi
, with ωi ∈ C {0} and ξi ∈ Cn distinct.
rank Hσ = r and the multiplicity of the points ξ1, . . . , ξr in V(Iσ) is 1.
For B, B be of size r, HB ,B
σ invertible iﬀ B and B are bases of
Aσ = K[x]/Iσ.
The matrix Mi of multiplication by xi in the basis B of Aσ is such that
HB ,xiB
σ = HB ,B
xi σ = HB ,B
σ Mi
The common eigenvectors of Mi are (up to a scalar) the Lagrange
interpolation polynomials uξi
at the points ξi, i = 1, . . . , r.
uξi
(ξj)=
1 ifi = j,
0 otherwise
uξi
2 ≡ uξi
, r
i=1 uξi
≡ 1.
The common eigenvectors of Mt
i are (up to a scalar) the vectors
[B(ξi)], i = 1, . . . , r.

Decomposition algorithm
Input: The ﬁrst coeﬃcients (σα)α∈A of the series
σ =
α∈Nn
σα
yα
α!
=
r
i=1
ωi eξi
(y)
1 Compute bases B, B ⊂ xA s.t. that HB ,B invertible and
|B| = |B | = r = dim Aσ;
2 Deduce the tables of multiplications Mi := (HB ,B
σ )−1HB ,xi B
σ
3 Compute the eigenvectors v1, . . . , vr of i li Mi for a generic
l = l1x1 + · · · + lnxn;
4 Deduce the points ξi = (ξi,1, . . . , ξi,n) s.t. Mj vi − ξi,j vi = 0 and the
weights ωi = 1
vi (ξi ) σ|vi .
Output: The decomposition σ = r
i=1
1
vi (ξi ) σ|vi eξi
(y).

Demo

Multivariate Prony method (1)
Let h(t1, t2) = 2 + 3 2t1
2t2
−3t1
, σ = α∈N2 h(α)yα
α! = 2e(1,1)(y) + 3e(2,2)(y)−e(3,1)(y).
Take B = {1, x1, x2} and compute
H0 := HB,B
σ =




h(0, 0) h(1, 0) h(0, 1)
h(1, 0) h(2, 0) h(1, 1)
h(0, 1) h(1, 1) h(0, 2)



 =




4 5 7
5 5 11
7 11 13



,
H1 := HB,x1 B
σ =




5 5 7
5 −1 17
811 178 23



, H2 := HB,x2 B
σ =




7 11 13
11 17 23
13 23 25



 .
Compute the generalized eigenvectors of (aH1 + bH2, H0):
U =




2 −1 0
−1/2 0 1/2
−1/2 1 −1/2



 and H0 U =




2 3 −1
2 × 1 3 × 2 −1 × 3
2 × 1 3 × 2 −1 × 1



 .
This yields the weights 2, 3, −1 and the roots (1, 1), (2, 2), (3, 1).

Multivariate Prony method (2)
h(t1, t2) := r
i=1 ωi ef1t1+f2t2 with
f :=








−0.5 1.0 + 3.141592654 i
0.1 + 21.36283005 i 1.5 + 32.67256360 i
0.1 + 21.36283005 i −0.5 + 79.16813488 i
−2.5 + 145.7698991 i −10.0 + 517.1061508 i








ω :=








1.375328890 + 0.9992349291 i
1.046162168 + 0.3399186938 i
0.9
−9.2








For the sampling [ 1
50, 1
170], B = {1, x1, x2, x1x2}, the SVD of HB,B
σ is
[33.1196344300301391, 14.3767453860219057, 0.244096952193142480, 0.0230734326225932214]
and the computed decomposition is
f
∗
=








−2.49999999703636711 + 145.769899153890435 i −9.9999999913514852 + 517.106150711515852 i
0.0999940670173818935 + 21.3628392917863437 i −0.500045063743692286 + 79.1681566575291527 i
0.100028305341504586 + 21.3628527756206275 i 1.50002358381760881 + 32.6725933609709571 i
−0.499926454593063452 + 0.0000142466247443506387 i 1.00008814016387371 + 3.14161379568963772 i








ω
∗
=








−9.19999999613861696 − 0.00000000772422142913953280 i
0.899999936743709261 − 0.00000156202814849404348 i
1.04615643213670850 + 0.339923495269889020 i
1.37533468654902213 + 0.999231697828891208 i









Sparse interpolation
f (x) = r
i=1 ωi xαi ⇒ σ = γ f (ϕγ) yγ
γ! = r
i=1 ωi eϕαi (y)
Example: f (x1, x2) = x33
1 x12
2 − 5 x1x45
2 + 101.
Compute σα = f (ϕα1
1 , ϕα2
2 ) for α1 + α2 ≤ 3 and ϕ1 = ϕ2 = e
2iπ
50 .
Compute the Hankel matrix H1,2
σ :


97.00000 97.01771 + 3.93695 i 95.50360 − 1.47099 i 98.46280 + 4.88062 i 97.42748 + 1.82098 i 9
97.01771 + 3.93695 i 98.46280 + 4.88062 i 97.42748 + 1.82098 i 102.35770 + 3.77300 i 99.50853 + 5.29465 i 9
95.50360 − 1.47099 i 97.42748 + 1.82098 i 95.73130 − .33862 i 99.50853 + 5.29465 i 95.42134 + 1.47250 i 9
Deduce the decomposition of σ = 3
i=1 ωi eξi
:
Ξ =


0.99211 + 0.12533i 0.80902 − 0.58779i
1.00000 + 4.86234e−11
i 1.00000 − 6.91726e−10
i
−0.53583 − 0.84433i 0.06279 + 0.99803i

 ω =


−5.00000 − 4.43772e−7
i
101.00000 + 4.65640e−7
i
1.00000 − 1.92279e−8
i


and the exponents 50 Ξ
2πi mod 50 of the terms of f :


1.00000 − 0.414119e−7
i −5.00000 + 0.270858e−6
i,
0.386933e−9
+ 0.137963e−8
i −0.550458e−8
− 0.38761e−8
i
−17.00000 − 0.100085e−6
i 12.00000 + 0.700984e−6
i



Symmetric tensor decomposition
τ = (x0 − x1 + 3 x2)
4
+ (x0 + x1 + x2)
4
− 3 (x0 + 2 x1 + 2 x2)
4
= −x0
4
− 24 x0
3
x1 − 8 x0
3
x2 − 60 x0
2
x1
2
− 168 x0
2
x1x2 − 12 x0
2
x2
2
−96 x0x1
3
− 240 x0x1
2
x2 − 384 x0x1x2
2
+ 16 x0x2
3
− 46 x1
4
− 200 x1
3
x2
−228 x1
2
x2
2
− 296 x1x2
3
+ 34 x2
4
τ∗
= e(−1,3)(y) + e(1,1)(y) − 3e(2,2)(y) (by apolarity)
H
2,2
τ∗ :=















−1 −2 −6 −2 −14 −10
−2 −2 −14 4 −32 −20
−6 −14 −10 −32 −20 −24
−2 4 −32 34 −74 −38
−14 −32 −20 −74 −38 −50
−10 −20 −24 −38 −50 −46















For B = {1, x2, x1},
H
B,B
τ∗ =





−1 −2 −6
−2 −2 −14
−6 −14 −10





, H
B,x1B
τ∗ =





−6 −14 −10
−14 −32 −20
−10 −20 −24





, H
B,x2B
τ∗ =





−2 −2 −14
−2 4 −32
−14 −32 −20






The matrix of multiplication by x1 in B = {1, x2, x1} is
M1 = (H
B,B
τ∗ )
−1
H
B,x1B
τ∗ =





0 −2 −2
0 1
2
3
2
1 5
2
3
2





.
Its eigenvalues are [−1, 1, 2] and the eigenvectors:
U :=





0 −2 −1
1
2
3
4
1
2
− 1
2
1
4
1
2





.
that is the polynomials
U(x) = 1
2 x2 − 1
2 x1 −2 + 3
4 x2 + 1
4 x1 −1 + 1
2 x2 + 1
2 x1 .
We deduce the weights and the frequencies:
H
[1,x1,x2],U
τ∗ =





1 1 −3
1 × −1 1 × 1 −3 × 2
1 × 3 1 × 1 −3 × 2





.
Weights: 1, 1, −3; frequencies: (−1, 3), (1, 1), (2, 2).
Decomposition: τ∗
(y) = e(−1,3)(y) + e(1,1)(y) − 3 e(2,2)(y) + (y)4

Phylogenetic trees
Problem: study probability vectors for genes [A, C, G, T]
and the transitions described by Markov matrices Mi .
Example:
Ancestor : A
Transitions : M1 M2 M3
Species : S1 S2 S3
For i1, i2, i3 ∈ {A, C, G, T}, the probability to observe i1, i2, i3 is
pi1,i2,i3 =
4
k=1
πk M1
k,i1
M2
k,i2
M3
k,i3
⇔ p =
4
k=1
πk uk ⊗ vk ⊗ wk
where uk = (M1
k,1, . . . , M1
k,4), vk = (M2
k,1, . . . , M2
k,4), wk = (M3
k,1, . . . , M3
k,4).

p is a tensor ∈ K4 ⊗ K4 ⊗ K4 of rank ≤ 4.

Its decomposition yields the Mi and the ancestor probabibility (πj ).

A general framework
F the functional space, in which the “signal” lives.
S1, . . . , Sn : F → F commuting linear operators: Si ◦ Sj = Sj ◦ Si .
∆ : h ∈ F → ∆[h] ∈ C a linear functional on F.
Generating series associated to h ∈ F:
σh(y) =
α∈Nn
∆[Sα
(h)]
yα
α!
=
α∈Nn
σα
yα
α!
.
Eigenfunctions:
Sj (E) = ξj E, j = 1, . . . , n ⇒ σE = ω eξ(y).
Generalized eigenfunctions:
Sj (Ek ) = ξj Ek +
k <k
mj,k Ek ⇒ σEk
= ωi (y)eξ(y).

If h → σh is injective ⇒ unique decomposition of f as a linear
combination of generalized eigenfunctions.

Sum of polynomial-exponential functions
F = PolExp(x),
Sj : h(x) → h(x1, . . . , xj−1, xj + δj , xj+1, . . . , xn) shift of xj by δj ,
∆ : h(x) → ∆[h] = h(0) the evaluation at 0.
Generating series of h: σh(y) = α∈Nn h(α1δ1, . . . , αnδn) yα
α! .
Eigenfunctions: ef·x; generalized eigenfunctions: ω(x)ef·x;
h(x) = r
i=1 gi (x)efi x + r(x) with gi (x) ∈ C[x], fi ∈ Cn and r(δ α) = 0,
∀α ∈ Nn, iﬀ
σh(y) =
r
i=1
ωi (y)eξi
(y)
with ξi = efi ∈ V(ker Hσh
) ⊂ Cn , ωi (x) = α gi,αωα for gi = α gi,αxα.

Decomposition from the moments σα = h(α1δ1, . . . , αnδn).

Sparse interpolation of PolyLog functions
F = PolyLog(x) = { (β,γ)∈A hβ,γ logβ
(x)xγ, A ﬁnite},
Sj : h(x1, . . . , xn) → h(. . . , xj−1, λj xj , xj+1, . . .) for λj ∈ C − {1},
∆ : h(x1, . . . , xn) → ∆[h] = h(1, . . . , 1).
Generating series of h: σh(y) = α∈Nn h(λα1
1 , . . . , λαn
n ) yα
α! .
Eigenfunctions: xγ; generalized eigenfunctions: logβ
(x)xγ.
h = r
i=1 β∈Bi
ωi,β logβ
(x) xγi iﬀ the generating series σh is of the form
σh(y) =
r
i=1
ωi (y)eξi
(y)
with ξi = (λ
γi,1
1 , . . . , λ
γi,n
n ) ∈ Cn and ωi (y) = β∈Bi
ωi,βyβ ∈ C[y].

Decomposition from the moments σα = h(λα1
1 , . . . , λαn
n ).

Sparse reconstruction from Fourier coefficients
F = L2(Ω);
Si : h(x) ∈ L2(Ω) → e
2π
xi
Ti h(x) ∈ L2(Ω) is the multiplication by e
2π
xi
Ti ;
∆ : h(x) ∈ OC → h(x)dx ∈ C.
The moments of f are
σγ =
1
n
j=1 Tj
f (x)e
−2πi n
j=1
γj xj
Tj dx
Eigenfunctions: δξ; generalized eigenfunctions: δ
(α)
ξ .
For f ∈ L2(Ω) and σ = (σγ)γ∈Zn its Fourier coefficients,
Γσ : (ρβ)β∈Zn ∈ L2
(Zn
) →


β
σα+βρβ


α∈Zn
∈ L2
(Zn
).
Γσ is of finite rank r if and only if f = r
i=1 α∈Ai ⊂Nn ωi,αδ
(α)
ξi
with
ξi = (ξi,1, . . . , ξi,n) ∈ Ω, ωi,α ∈ C and r = r
i=1 µ( α∈Ai
ωi,αyα)

Other applications
Decomposition of measures as sums of spikes from moments (images,
spectroscopy, radar, astronomy, . . . )
Decomposition of convolution operators of ﬁnite rank
Vanishing ideal of points: σ = r
i=1 eξi
(y)
Change of ordering for Grobner bases or change of bases for
zero-dimensional ideals: σα = u, N(xα) ,
. . .

Low rank classiﬁcation
1 Decomposition algorithms
2 Low rank classiﬁcation
3 The variety of missing moments

Low rank decomposition of Hankel matrices
Rank 1 Hankel matrices: Hξ = [ξα+β]α∈A,β∈B for some ξ ∈ Kn or Pn.
Rank r Hankel matrices are not necessarily the sum of r rank one Hankel
matrices.


0 1 0
1 0 0
0 0 0

 = λ1


1 ξ1 ξ2
1
ξ1 ξ2
1 ξ3
1
ξ2
1 ξ3
1 ξ4
1

 + λ2


1 ξ2 ξ2
2
ξ2 ξ2
2 ξ3
2
ξ2
2 ξ3
2 ξ4
2


but


0 1 0
1 0 0
0 0 0

 = lim
→0
1
2


1 2
2 3
2 3 4

 −
1
2


1 − 2
− 2
− 3
2
− 3 4


Symbol: y = lim →0
1
2 (e y − e− y ).

Structured Low rank Decomposition
Decomposition in sum of Hankel operators associated to symbols
ω(y)eξ(y) with ω(y) ∈ K[y], ξ ∈ Cn.

σ = r
i=1 ωi eξi
(y) ⇒
HA,B
σ = VA(ξ1, . . . , ξr ) ∆(ω1, . . . , ωr ) VB(ξ1, . . . , ξr )t
HA,B
g σ = VA(ξ1, . . . , ξr ) ∆(ω1g(ξ1), . . . , ωr g(ξr )) VB(ξ1, . . . , ξr )t
where VA(ξ1, . . . , ξr ) = [ξαi
j ]1≤i≤|A|,1≤j≤r , ∆(· · · ) diagonal matrix.

σ = r
i=1 ωi (y) eξi
(y) ⇒
HA,B
σ = WA;Γ(ξ)∆Γ
ωWB;Γ(ξ)t
HA,B
g σ = WA;Γ(ξ)∆Γ
g ωWB;Γ(ξ)t
where WA;Γ(ξ) Wronskian, ∆Γ
g ω block diagonal.

Symmetric tensors of low rank (joint work with A. Oneto)
For ψ ∈ Sd of degree d, with a decomposition ψ = r
i=1 ξi , x d and for
0 ≤ k ≤ d − k,
Hd−k,k
ψ∗ = Vd−k(Ξ) V t
k (Ξ)
where Ξ = (ξ1, . . . , ξr ) ∈ (Kn+1)r , Vk(Ξ) is the Vandermonde matrix of Ξ
at the monomials of deg. k.
Notation
ψ⊥
k = ker Hd−k,k
ψ∗
h(k) = dim Sk/ψ⊥
k = rankHd−k,k
ψ∗
I(Ξ) deﬁning ideal of the points Ξ
Apolarity lemma
Ξ is apolar to ψ (i.e. appears in a decomposition of ψ) iﬀ I(Ξ)k ⊂ ψ⊥
k for
any k ∈ N.
Proof. ψ∗ ∈ eξ1 , . . . , eξr ⊂ S∗
d .

The regularity of Ξ is ρ(Ξ) = min{k ∈ N | ∃u1, . . . , ur ∈ Sk s.t. ui (ξj ) = δi,j }.
Regularity lemma
Let ψ ∈ Sd and let Ξ be a minimal set of points apolar to ψ. Then,
I(Ξ)k = ψ⊥
k for 0 ≤ k ≤ d − ρ(Ξ).
Proof. ψ⊥
k = ker Hd−k,k
ψ∗ = Vd−k(Ξ) V t
k (Ξ), I(Ξ)k = ker V t
k (Ξ) and
Vd−k(Ξ) injective for d − k ≥ ρ(Ξ).
Theorem
Let ψ ∈ Sd and let Ξ be a minimal set of points apolar to ψ. If
d ≥ 2ρ(Ξ) + 1, then
I(Ξ) = (ψ⊥
≤ρ(Ξ)+1).
Moreover, Ξ is the unique minimal set of points apolar to ψ.

A set of essential variables of ψ is a minimal set of linear
forms 1, . . . , N ∈ S, such that ψ ∈ C[ 1, . . . , N].
Proposition
[Car06] the number of essential variables is hψ(1);
[CC017] any minimal decomposition of ψ involves only linear forms
in the essential variables.
The Waring locus of ψ is the locus of linear forms that can appear in a
minimal decomposition of ψ, i.e.,
Wψ := [ ] ∈ P(S1) | ∃ 2, . . . , r , r = rank(ψ), s.t. ψ ∈ d
, d
2 , . . . , d
r ;
The complement is forbidden locus denoted Fψ := Pn Wψ.

Tensor with 2 essential variables (Sylvester method)
Let ψ(x0, x1) ∈ Sd = K[x0, x1]d .
The Hilbert function of Aψ∗ is of the form:
with (ψ⊥) = (G1, G2) of degree 0 ≤ d1 ≤ d2 ≤ d with d1 + d2 = d + 2.
If G1 has simple roots, then ψ is of rank d1 = deg(G1) and the roots
of G1 are the unique minimal set apolar to ψ.
Otherwise, ψ is of rank d2 = deg(G2) and Wψ is dense in P1. For a
generic choice of A ∈ Sd2−d1 , the roots of AG1 + G2 are a minimal set
apolar to ψ.

Cases of rank 4
(a)
Collinear
(b) Coplanar,
with 3 collinear.
(c) General
coplanar.
(d) General
points.

For ψ ∈ Sd of rank 4.
ψ has two essential variables (hψ(1) = 2):
ψ⊥ = (L1, . . . , Ln−1, G1, G2), where deg(Gi ) = di and
d1 ≤ d2. In particular, it has to be d ≥ 4 and:
(i) if d = 4, 5, 6, then d2 = 4, and minimal apolar sets
of points are deﬁned by ideals
I(Ξ) = (L1, . . . , Ln−1, HG1 + αG2), for a general
choice of H ∈ T6−d and α ∈ C;
(ii) if d ≥ 7, then d1 = 4 and the unique minimal
apolar set of points is given by
I(Ξ) = (L1, . . . , Ln−1, G1).

ψ has three essential variables (hψ(1) = 3) and a
minimal apolar set Ξ of type (b):
(i) if d = 3, then V(ψ⊥
2 ) = P + D, where P is a
reduced point and D is connected scheme of
length 2 whose linear span is a line LD;
Any minimal apolar set is of the type P ∪ Ξ , with
Ξ ⊂ LD;
(ii) if d = 4, then hψ(2) = 4, V(ψ⊥
2 ) = P ∪ L, where P
is a reduced point and L is a line not passing
through P;
Any minimal apolar set is of the type P ∪ Ξ ,
where Ξ ⊂ L.
(iii) if d ≥ 5, then hψ(2) = 4, V(ψ⊥
2 ) = P ∪ L, where P
is a reduced point and L is a line not passing
through P and (ψ⊥
3 ) deﬁnes the unique minimal
apolar set.

ψ has three essential variables (hψ(1) = 3) and a
minimal apolar set Ξ of type (c):
(i) if d = 3, then V(ψ⊥
2 ) = ∅ and Wψ is dense in the
plane of essential variables;
(ii) if d ≥ 4, there is a unique minimal apolar set of
points given by I(Ξ) = (ψ⊥
2 ).
ψ has four essential variables:
there is a unique minimal apolar set of points given by
I(Ξ) = (ψ⊥
2 ).

Classiﬁcation/algorithm for rank ≤ 5

From moments to sparse representations, a geometric, algebraic and algorithmic viewpoint. B. Mourrain. Part 2

More Related Content

What's hot (12)

Similar to From moments to sparse representations, a geometric, algebraic and algorithmic viewpoint. B. Mourrain. Part 2 (20)

Recently uploaded (20)

From moments to sparse representations, a geometric, algebraic and algorithmic viewpoint. B. Mourrain. Part 2