abhinav kumar joint work with satya lokam, vijay patankar...

Matrix rigidity and elimination theory

Abhinav Kumarjoint work with Satya Lokam, Vijay Patankar and Jalayal Sarma M.N.

April 25, 2012

Abhinav Kumar (MIT) Matrix rigidity and elimination theory April 25, 2012 1 / 1

Rigidity of a matrix

Definition

Let A be an n × n matrix with entries in a field F . The rigidity Rig(A, r)of A for target rank r is the smallest number of entries of A that need tobe changed to make the rank at most r .

Most of the time we’ll work with F = C. If F is a subfield of C (e.g. Q)we may want to consider changes in some finite extension field L/F , anddefine Rig(A, r , L).

Definition

Valiant’s theorem

Valiant defined matrix rigidity in his study of lower bounds on complexityfor computing linear forms in n variables, for circuits using gates which cancompute linear combinations of two variables.

To compute one such linear combination, we can use a binary tree to get acircuit of depth log2(n) and size n.

But: say we want to use such a circuit to simultaenously compute n linearforms in n variables, given by the entries of the vector Ax , wherex = (x1, . . . , xn) is the column vector of inputs.

Valiant showed that this cannot be done with circuits which are bothshallow (of depth O(log n)) and small (of size O(n)).

Valiant’s theorem

Valiant’s theorem II

More precisely,

Theorem

Let A1,A2, . . . be an infinite family of matrices, where An is an n × n real

matrix and for some κ, c , ǫ > 0, we have Rig(A, κn) ≥ cn1+ǫ. Then given

any fixed c1, c2 > 0, there does not exist a family of straight line programs

for the corresponding sets of linear forms that achieve size c1n and depth

c2 log n simultaneously for all n.

Remarks:

The proof uses some graph theoretic property of the circuit (namely,that it’s a grate.

We’ll see that most matrices have quadratic rigidity, so asking forsuper-linear rigidity is perhaps not so bad.

More precisely,

Theorem

Remarks:

More precisely,

Theorem

Remarks:

More precisely,

Theorem

Remarks:

Easy upper bound

Proposition

Any n × n matrix X has rigidity at most (n − r)2 for target rank r .

Proof.

If the rank less than r , rigidity is zero. Else assume w.lo.g. that the topleft r × r block is nonsingular. Write X as

Modify D to CA−1B .

Easy upper bound

Proposition

Any n × n matrix X has rigidity at most (n − r)2 for target rank r .

Proof.

If the rank less than r , rigidity is zero. Else assume w.lo.g. that the topleft r × r block is nonsingular. Write X as

Modify D to CA−1B .

Generic rigidity

An extension of this argument shows that generic matrices have maximalrigidity, i.e. the subset of matrices of rigidity less than (n − r)2 iscontained in a proper Zariski-closed subset.

Proof: if rigidity is k < (n − r)2, then the matrix is in the image of a mapfrom a variety of matrices of the form

C CA−1B

(which has dimension ≤ n2 − (n − r)2) times Ak . Sincek + n2 − (n − r)2 < n2 = dimMatn, the image is contained in a properclosed subset.

Generic rigidity

An extension of this argument shows that generic matrices have maximalrigidity, i.e. the subset of matrices of rigidity less than (n − r)2 iscontained in a proper Zariski-closed subset.

Proof: if rigidity is k < (n − r)2, then the matrix is in the image of a mapfrom a variety of matrices of the form

C CA−1B

(which has dimension ≤ n2 − (n − r)2) times Ak . Sincek + n2 − (n − r)2 < n2 = dimMatn, the image is contained in a properclosed subset.

Questions

All this leads to the natural question:

Construct an explicit family of reasonably natural matrices of super-linearrigidity for target rank linear in the dimension n.

Remarks:

Or even better, quadratic rigidity.

Or better still, maximal rigidity (n − r)2.

Note that these extensions will not give a better lower complexity bound,but they are natural questions from the algebraic point of view.

Questions

Remarks:

Questions

Remarks:

Questions

Remarks:

Questions

Remarks:

Examples I

One guess for highly rigid matrices: totally regular matrices, i.e. all minorsare nonsingular.

This guess is not correct! Valiant shows:

Proposition

For each n there is an n × n totally regular matrix A such that

Rig(A, n(log log log n)/(log log n)) ≤ n1+O(1/ log log n).

Remarks:

That is, one can bring rank down to o(n) by changing o(n1+ǫ) entries.

Proof uses superconcentrators, and is also non-explicit.

Examples I

Proposition

Remarks:

Examples I

Proposition

Remarks:

Examples I

Proposition

Remarks:

Examples I

Proposition

Remarks:

Examples II

Nevertheless, here are some natural familes.

Vandermonde matrices: row j is 1, xj , x2j , . . . , x

Discrete Fourier transform matrices: n = p prime say, Vandermondewith xj = e2π

√−1j/p.

Generalized Hadamard matrices H: entries complex numbers hij ofabsolute value 1, and such that HH† = nIn.

Circulant matrices: each row is a shift of the previous one.

Cauchy matrix Cij = 1/(i + j − 1).

Examples II

√−1j/p.

Examples II

√−1j/p.

Examples II

√−1j/p.

Examples II

√−1j/p.

Progress so far

Matrices Ω(.) References

Vandermonde n2

rRazborov ’89, Pudlak ’94Shparlinsky ’97, Lokam ’99

Hadamard n2

rKashin-Razborov ’98

Parity Check n2

rlog(n

r) Friedman ’93

Pudlak-Rodl ’94

Totally regular n2

rlog(n

r) Shokrallahi-Spielmann-Stemann ’97√

pij n(n − 16r) Lokam ’06

ζij = e2π

−1pij (n − r)2 This work.

Statement of our result

We construct matrices with maximal rigidity, but over a number field ofrelatively high degree.

Theorem

Let ∆(n) = 2n2n2and let pij > ∆(n) be distinct primes for 1 ≤ i , j ≤ n.

Let A(n) have (i , j) entry ζij = e2π√−1/pij . Then Rig(A(n), r) = (n − r)2.

Determinantal ideal

Recall that the variety of matrices of rank at most r is defined by thedeterminantal ideal I (n, r) with generators all (r + 1)× (r + 1) minors of

x11 x12 · · · x1nx21 x22 · · · x2n...

......

xn1 xn2 · · · xnn

It’s an irreducible variety of dimension n2 − (n − r)2.

Elimination ideals

Let π be a pattern (Valiant calles it a mask) of positions where we allowchanges, of cardinality k strictly less than (n− r)2 say. The set of matriceswhose rank can be brought down to at most r by changing entries in π liein the image of a variety of dimension n2 − (n − r)2 + k .

Its closure is defined by the elimination ideal I (n, r , π) = I (n, r) ∩ Q[xπ]where π means positions not in π.

Elimination ideals

Let π be a pattern (Valiant calles it a mask) of positions where we allowchanges, of cardinality k strictly less than (n− r)2 say. The set of matriceswhose rank can be brought down to at most r by changing entries in π liein the image of a variety of dimension n2 − (n − r)2 + k .

Its closure is defined by the elimination ideal I (n, r , π) = I (n, r) ∩ Q[xπ]where π means positions not in π.

Sketch of proof

As a generic matrix has maximal rigidity, these elimination ideals I (n, r , π)are all non-zero, as π runs over all patterns of size strictly less than(n − r)2.

So the ideal I (n, r , π) is nonzero. We’ll use effective Nullstellensatz boundsto show that there’s a multivariate polynomial of low enough degree, andthen use Galois theory to show that it cannot vanish on the roots of unityconstructed. Therefore, the matrix with entries Aij = e2π

√−1/pij does not

lie in V (I (n, r , π)) for any π, and therefore has maximal rigidity.

Sketch of proof

As a generic matrix has maximal rigidity, these elimination ideals I (n, r , π)are all non-zero, as π runs over all patterns of size strictly less than(n − r)2.

So the ideal I (n, r , π) is nonzero. We’ll use effective Nullstellensatz boundsto show that there’s a multivariate polynomial of low enough degree, andthen use Galois theory to show that it cannot vanish on the roots of unityconstructed. Therefore, the matrix with entries Aij = e2π

√−1/pij does not

lie in V (I (n, r , π)) for any π, and therefore has maximal rigidity.

Hypersurfaces

In fact, we can show pretty easily that the corresponding union ofsubvarieties V (I (n, r , π)) is a (reducible) hypersurface.

Explicit equations for these, or understanding of their geometry, might beuseful in trying to understand why some families of matrices (e.g.Vandermonde) seem to have maximal rigidity.

Hypersurfaces

In fact, we can show pretty easily that the corresponding union ofsubvarieties V (I (n, r , π)) is a (reducible) hypersurface.

Explicit equations for these, or understanding of their geometry, might beuseful in trying to understand why some families of matrices (e.g.Vandermonde) seem to have maximal rigidity.

Effective Nullstellensatz

We’ll use the following bounds, which rely on the Bezout inequality fordegrees proved by Heintz (1983)

Theorem (Dickenstein, Fitchas, Giusti, Sessa ’91)

Let I = 〈f1, . . . , fs〉 be an ideal in the polynomial ring F [Y ] over an infinite

field F , where Y = y1, . . . , ym. Let dmax be the maximum total degree

of a generator fi . Let Z = yi1 , . . . , yiℓ ⊂ Y be a subset of indeterminates

of Y . If I ∩ F [Z ] 6= (0) then there exists a non-zero polynomial

g ∈ I ∩ F [Z ] such that g =∑s

i=1 gi fi , with gi ∈ F [Y ] anddeg(gi fi ) ≤ dm(dm + 1), where d = max(dmax , 3).

Applying it to our situation, where d = r + 1 and m = n2, we get a boundof ∆(n) = 2n2n

2for the total degree of g .

Effective Nullstellensatz

We’ll use the following bounds, which rely on the Bezout inequality fordegrees proved by Heintz (1983)

Theorem (Dickenstein, Fitchas, Giusti, Sessa ’91)

Let I = 〈f1, . . . , fs〉 be an ideal in the polynomial ring F [Y ] over an infinite

field F , where Y = y1, . . . , ym. Let dmax be the maximum total degree

of a generator fi . Let Z = yi1 , . . . , yiℓ ⊂ Y be a subset of indeterminates

of Y . If I ∩ F [Z ] 6= (0) then there exists a non-zero polynomial

g ∈ I ∩ F [Z ] such that g =∑s

i=1 gi fi , with gi ∈ F [Y ] anddeg(gi fi ) ≤ dm(dm + 1), where d = max(dmax , 3).

Applying it to our situation, where d = r + 1 and m = n2, we get a boundof ∆(n) = 2n2n

2for the total degree of g .

Roots of unity

Finally, we have the following lemma.

Let N be a positive integer, and θ1, . . . , θm be algebraic numbers such that

Q(θi ) is Galois over Q and such that

[Q(θi ) : Q] ≥ N and Q[θi ] ∩Q(θ1, . . . , θi−1, θi+1, . . . , θm) = Q for all i .

Let g(x1, . . . , xm) ∈ Q[x1, . . . , xm] be a nonzero polynomial such that

deg(g) < N. Then g(θ1, . . . , θm) 6= 0.

The proof is easy by induction, using the linear disjointness property.

Conclusion of proof

To finish the proof of the theorem, we note that choosing distinct primespij for 1 ≤ i , j ≤ n, and setting θij = ζpij := e2π

√−1/pij , the linear

disjointness property is satisfied. So we just need to make sure thatpij − 1 > ∆(n) := 2n2n

If we want real matrices, we can take θij = ζpij + ζ−1pij

, these generate thetotally real subfields of the cyclotomic fields, so we just need to ensure,(pij − 1)/2 > ∆(n).

Of course, this falls far short of our desired goal, since the roots of unityhave very high degree. What we would like is to have rational entries, andpreferably some systematic family of these.

Conclusion of proof

Semicontinuity?

One approach to constructing rigid matrices over the rationals is byconstructing rigid matrices over some totally real extension (as we havejust done), and then approximating these real numbers by rationalnumbers, hoping that rigidity does not change.

This will work reasonably well in our situation, since we’ve actuallyconstructed a matrix with a Zariski neighborhood (and therefore Euclideanneighborhood) disjoint from the less-than-maximally-rigid locus.

But one might wonder whether some kind of semicontinuity property holdsfor rigidity in general, as it holds for the rank function of matrices. Wewould like it if Rig(A, r) ≥ ℓ, that the same held true in a neighborhood ofA.

Semicontinuity?

Examples I

Unfortunately, this expectation is false. Let a, b, c , d , e be non-zerorational numbers. Consider

d 0 0e 0 0

We have rank(A) = 2; Rig(A, 1) = 2.

Examples II

Now, for any ǫ > 0 let

A(δ) =

d bdδ cdδe beδ ceδ

Change a to 1δ , rank of the matrix goes down to 1. Rig(A(δ), 1) = 1.

Examples III

One can even produce such counterexamples which are maximally rigid.For instance,

d e 0e 0 i

has Rig(A, 1) = 4 = (3− 1)2.

Examples IV

A(δ) =

d e cdδe bgδ i

has rigidity Rig(A(δ), 1) = 3, since one can change the diagonal entries toget

1/δ b c

d bdδ cdδe bgδ cgδ

Some more thoughts, questions

Perhaps we can try to use similar techniques to show that the eliminationideals do not contain the ideal of the locus of Vandermonde matrices.Rational normal curves?

Can we systematically improve the bounds for effective Nullstellensatz,with some hypothesis on the degrees or type of generators. For example,for elimination ideals of determinantal ideals.

Coming back to the Valiant example of totally regular matrices with lowrigidity, can we find explicit examples of these, using algebraic geometryrather than graph theory?

Reference: “Using Elimination Theory to construct Rigid Matrices”,Abhinav Kumar, Satya V. Lokam, Vijay M. Patankar and Jayalal SarmaM.N. arXiv:0910.5301.

Thank you!

abhinav kumar joint work with satya lokam, vijay patankar...

Documents