Ancient history: localized electrons in a disordered 1d wire

1

0

0

1 + 1

0

s [ , ], 0.5

N

j jj

j j j j

j

H t j j t j j

t t t s s

s

tight binding model with random hopping disorder 

“All states are localized in one dimension…”

jt

jt1 2 N j 1j

11 + 1

N

j jj

H t j j t j j

In fact, there is an extended state at the band center…

3 2( ) ~ 1/ [ ln (1/ )]

singular density of states at   ε = 0

2

0

diverging localization length at = 0lim ( ) ~ ln(1/ ) 1/ ( )loc

inverse localization length = κ(ε)

1/20 0

Variable range hopping in a partially filled band leads to highly non-Ohmic current-voltage relations in 1d at 0 ~ exp[ ( / ) ]

TI I V V

Four localized eigenstates near top of  band

( )x

x

Non-Hermitian Localization in Ecological and Neural Networks

Non-Hermitian matrices, with complex eigenvalue spectra, arise naturally in simple models of complex ecosystems, and neural networks.

Striking departures from the conventional wisdom about localization arise in the one-dimensional non-Hermitian random matrices that describe sparse neural and ecological networks.

An intricate eigenvalue spectrum controls the spontaneous activity and induced response. Directed rings of neurons lead to a hole centered on in the density of states in the complex plane.

All states are extended on the rim of this hole, while the states outside the hole are localized.

Naomichi HatanoUniversity of Tokyo 

Ariel  AmirHarvard/SEAS

Advice from Y. Lue, V.  Murthy  and H. Sompolinsky

Eigenvectors and eigenvalues in biology: rabbits vs. sheep

1 2

1 2

linearize about the fixed point at (3,2)'( ) ( ) 3, '( ) ( ) 2

'( ) / 3 0 '( )'( ) / 0 2 '( )

'( ) '(0) , '( ) '(0)two real eigenvalues:

3, 2, stable fix

t t

x t x t y t y tdx t dt x tdy t dt y t

x t x e y t y e

ed point

decoupled model: two logistic equationsx(t) = number of rabbitsy(t) = number of sheep

Re λ

Im λ

λ1 λ2

(3,2)

mutualism

3 (1 / 3)

2 (1 / 2)

dx x xdtdy y ydt

x(t)

y(t)

2

3rabbits

sheep

Eigenvectors and eigenvalues in biology: rabbits vs. sheep

or yeast vs. bacteria in the same chemostat… vs.

3 (1 / 3 )2 / 3

22 ( / /1 2 )

dx x xdtdy y yd

y

xt

competitive exclusion

vs.

or a genetic toggle switch:   T. S. Gardner, C. R. Cantor, and J. J. Collins. Construction of a genetic toggle switch in Escherichia coli, Nature 403, 339 (2000)

Eigenvectors and eigenvalues in biology: Rabbits vs. Sheep

S. H. Strogatz, Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. Westview press, 2014.

vs.

3 (1 / 3 )2 / 3

22 ( / /1 2 )

dx x xdtdy y yd

y

xt

1 2

linearize about the fixed point at (1,1)'( ) ( ) 1, '( ) ( ) 1

'( ) / 1 2 '( )'( ) / 1 1 '( )

two real eigenvalues control dynamics:

1 2, 1 2due to interactions, t

x t x t y t y tdx t dt x tdy t dt y t

here is now one

stable and one unstable eigendirection

either sheep or rabbits win or “fix” at long times…

Re λ

Im λ

λ1λ2

Rabbits vs. Foxes: complex eigenvalues lead to oscillations…

1

2

( ) number of rabbits( ) number of foxes

const. density of grass

Y tY tX

‐Thiparet Chotibut

Neutrally stable limit cycles

Neutrally stable limit cycles

Re λ

Im λ

λ+

λ-

3 1i c c X

Random matrix theory applied to N-species ecology models (N >> 1)

1

1. Assume each species in isolation would obey a stable logistic equation with stable eigenvalue -1

then

switch on random interactions of either s

(1 ) ; Let '

n

)

ig

(N

ii i ij i j i

j

dx x x B x x x t xdt

1

'( )2. '( ), '( ) is an N-component vector of species

deviations from an logistic fixed point.

3. , where is an N-component interaction matrix with zero mea

( ) 1

n for

Ni

ij jj

i

dx t A x t x td

t

t

A B I B

each element and each with standard deviation

Spectrum of is a uniform distribution of complex eigenvalues in unit circle

in the complex plane of radius "Girko's Law"

B

N

R. M. May. Nature, 238 413 (1972)

Any ecological system becomes unstable for sufficiently large N!

Re λ

Im λ

-1

unstable modes

Visual stimulus s(t) transferred from retinal neurons LGN V1 region of the visual coretexSpike rate r(t) 

depends on orientation of bar moving across the visual field

spike rate r(t)

signal s(t):   orientation in degrees

Pathway from the retina through the lateral geniculate nucleus (LGN) to the primary visual cortex

Neuroscience Motivation

Dayan and Abbott:  Theoretical Neuroscience

Random matrix models of the brain

K. Rajan, 2009 Spontaneous and Stimulus‐driven NetworkDynamics. Doctoral Dissertation, Columbia University.

Random neural connections  can be formed during development, with many stochastic attachments of axons and dendrites to other neurons.  

Over time, pruning and strengthening/weakening of  connections  allow neural circuits to "learn" various functions. 

The spectra and eigenfunctions of completely random neural networks with a mixture of inhibitoryand excitatory connections, can describe neural activity during the early stages of development.

Re λ=0

The simplest models of neural networks assume long range connectivity between individual neurons in the brain, leading to synaptic matrices M(i,j) with statistical properties independent of the separation rij = |ri – rj| in three dimensions.

The eigenvalue spectrum of M(i,j) controls the spontaneous activity and induced response of the network, and much is known when its elements are chosen from simple random matrix ensembles.

H. J. Sommers, et al., Phys. Rev. Lett. 60, 1895 (1988)

Dales Law:   K. Rajanand L. Abbott, et al., Phys. Rev. Lett. 97, 188104 (2006)

What happens when M(i,j) ~ exp[-|ri – rj| /ξ] , where ξ is a length spanning spanning as many as 50 neurons? On scales larger than ξ, the relevant random matrices are banded about the diagonal.

Will localized eigenfunctions dominate the dynamics?

Sensory inputs, possibly after a processing step, are sent via feed forward couplings into a circular ring of N neurons Note that M(1,2) and M(2,1) can not only be unequal, but also of opposite sign, if one direction is excitatory and the other inhibitory.    

Sensory Inpu

ts

Processing

(1, 2)M

2

1

(2,1)M

N

1

firing rate deviation from background of the i neuron in recurrent network

input firing rate of the neuron in the input (feed forward) network

tanh ,

thi

thj

Ni

i ij j i i ij jj

u j

dv v M v h h W udt

1

1 (linear approximation)

N

j

Ni

i ij j ij

dv v M v hdt

Random matrix model of a sparse neural network

Non-Hermitian neural networks with random excitatory (M(i,j) > 0) and inhibitory (M(i,j)< 0) connections

1

2

1

1 + s 1

provides a systematic clockwise (g > 0) or counterclockwise (g < 0) directional bias

Ng g

j jj

M s e j j e j j

g

, 1, indep.

random variables;j js s

Study eigenvalues and eigenvectors of directed, banded non-Hermitian random matrices

1

1 2

2

1

1

0 0 ...0 0

0 0: 0

... 0 0

g gN

g g

g

gN

g gN N

s e s es e s e

M s es e

s e s e

Set g = 0 for now random sign model of J. Feinberg and A. Zee, PRE 59 6433 (1999)

Many eigenvalues have condensed onto the real and imaginary axes

Result of exact diagonalization of 10,000 N x  N matrices with N = 5000 and g = 0

What are the localization properties of this spectrum??

Eigenvalue distribution in the complex plane λ = λ1 + iλ2

More about the density of states

Although the eigenvalues are in general complex, ~20% appear have condensed onto the real axis and ~20% have condensed onto the imaginary axis

The eigenvalues are symmetric under reflections across the real and imaginary axes, as well as           

45 +/‐ 45 degree lines  in the complex plane. 

The remaining eigenvalues (approximately 60%) appear within an intricate, diamond‐shaped structure with a fractal‐like boundary. 

Almost all eigenfunctions are localized.  The localization length is smallest near the boundary of the spectrum in the complex plane.

Numerics suggest that the localization length diverges as the modulus of the eigenvalues tends to zero. 

Fractal density of states

Density of states along real axis; similar results along imaginary axis

What does “localization” mean?

Eigenfunctions within circle on right side are highly localized w/real eigenvalues

1 2 2 21 2 1 2

localization length diverges near the origin1( , ) .

( )const

N = 1000g = 0.0

Eigenfunctions in an annulus closer to the origin are more extended

What is the effect of the bias parameter g?

1, 1 with equal probability

0 (no Dale's law for now)j js s

g

1

1 2

2

1

1

0 0 ...0 0

0 0: 0

... 0 0

g gN

g g

g

gN

g gN N

s e s es e s e

M s es e

s e s e

excitatory connection

inhibitory connection

1

4 3

25

g > 0

As g increases from 0, it tunes down the amount of back propagation in a “feed clockwise” recurrent network…

Similar layered neural nets used for image classification, etc. in machine learning algorithms.Many layers “deep learning”δ3

δ2

δ1

ξ1

ξ2 0 g g J. Hertz et al.

N = 5000,   g = 0.0

�2 �1 1 2

�2

�1

1

2

Effect of a directional bias around the chain (g > 0)

N = 5000,   g = 0.002

�2 �1 1 2

�2

�1

1

2

Effect of a directional bias around the chain (g > 0)

N = 5000,   g = 0.01

�2 �1 1 2

�1.5

�1.0

�0.5

0.5

1.0

1.5

Effect of a directional bias around the chain (g > 0)

N = 5000,   g = 0.05

�2 �1 1 2

�2

�1

1

2

Effect of a directional bias around the chain (g > 0)

A gap or hole appears in the eigenvalue spectrum in the complex plane…

�2 �1 1 2

�2

�1

1

2

N = 5000,   g = 0.1

Effect of a directional bias around the chain (g > 0)

N = 5000,   g = 0.2

�2 �1 1 2

�2

�1

1

2

Effect of a directional bias around the chain (g > 0)

N = 5000,   g = 0.5

�2 �1 1 2

�1.5

�1.0

�0.5

0.5

1.0

1.5

Effect of a directional bias around the chain (g > 0)

Density of states for g > 0

Exact diagonalization of 10,000 N x N matrices with N = 5000 and g = 0.1

Density of states along real axis; similar along imaginary axis

A g‐dependent hole surrounding the origin with angular corners opens up in the complex plane

A very large density of extended states appears along the rim of hole

The eigenvalues of the localized states outside the hole are unchanged “spectral rigidity”

Transfer matrix approach to the inverse localization length κ(E,g) =1/ξloc(E,g)

Defines a computational tractable inverse localization length for any energy E in the complex plane

Closely related via an electrostatic “Thouless relation” to the density of states

1

1 2

2

1

1

0 0 ...0 0

0 0: 0

... 0 0

g gN

g g

g

gN

g gN N

s e s es e s e

H Es es e

s e s e

21 11

1

/ /=

1 0

g gn n nn n n

nn n n

Ee s s e sT

1 1

Iteration of the transfer matrix

( , ) then defines the Lyapunovexponent via

( , ) lim ln /

matrix norm

disorder average

n

N NN

T E g

E g T T T N

M

1 2

21 2 1 2

Contours of constant inverse localization length ( , ) obtained by solving the electrostatic "Thouless equation"

( , ) ( , )

E E

E E E E

�2 �1 1 2

�2

�1

1

2

The shape of the hole

1 2 1 2

Inverse localization length for 0 given by

( , , ) ( , ,0) -g

E E g E E g

Localization lengths and effect of boundary conditions

4 2

Define inverse participation ratio

/

~ inverse localization length

j jj j

IPR

IPR

Eigenvalue spectrum for g = 0(or, for any g with open boundary conditions!)

4 2

0

4 2

extended state: ~ 1/ ,

/ ~ 1/ 1

localized state, ~ exp[- | | / ]

/ (1)

j

j jj j

j j loc

j jj j

N j

IPR N

x x

IPR O

Localization length diverges on the rim of the hole when g > 0  extended states

Eigenvalue spectrum for g = 0.1with periodic boundary conditions

What about Dale’s law? All neurons must be purely excitatory or inhibitory….

1

4 3

25G

N = 5, g = 0.0

11 + s 1

Ng g

j jj

M s e j j e j j

11 + 1

Ng g

kk

G e k k e k k

Replace 2N random variables with only N of them…

The spectra and eigenfunctionsof M and G are essentially identical! The spectral properties are determined in both cases by above/below diagonal products such as

( , 1) ( 1, ) and ( ,j jM j j M j j s s G j j 11) ( 1, ) ,

which have identical statistics!!j jG j j

Large g perturbation theory for extended states

1

1 2

2

1

1

0 0 ...0 0

0 0: 0

... 0 0

g gN

g g

g

gN

g gN N

s e s es e s e

M s es e

s e s e

After a similarity transformation… becomes M= A +B

j j jb s s

A

4N

Eigenvectors of A are plane waves

+ = random walk

Large g limit:  Plane wave states, all eigenfunctions delocalized

Trajectories of eigenvalues for N=100 and values of g decreasing from  1 down to zero.

Eigenvalues "flow" in the complex plane.

Motion stops once eigenvalues localize

The gap rimmed by extended states is robust…

1, s 1, =1000,

0.1, but with diagonal randomnessj js N

g

Single box distributionN = 1000, g = 0.5

-1 1 s

P(s)

extended states

g = 0

g = 0.2

1000 triangularneural clusters, obeying Dale’s law, and coupled together to form a ring

Energy gap and rings of extended states also appear for coupled neural clusters

“Band theory” for neural networks?

Layered neural network with tunable back propagation

Non-Hermitian Localization in Ecological and Neural Networks

Naomichi Hatano

Ariel  Amir

Non-Hermitian matrices, with complex eigenvalue spectra, arise naturally in simple models of complex ecosystems, and neural networks.

Striking departures from the conventional wisdom about localization arise in the one-dimensional non-Hermitian random matrices that describe sparse neural and ecological networks.

An intricate eigenvalue spectrum controls the spontaneous activity and induced response. Directed rings of neurons lead to a hole centered on in the density of states in the complex plane.

All states are extended on the rim of this hole, while the states outside the hole are localized.

Thank you!