An Introduction to Compressive Sensing

Speaker: Ying-Jou ChenAdvisor: Jian-Jiun Ding

Compressive

Compressed

SensingSamplingCS

Outline

• Conventional Sampling & Compression• Compressive Sensing• Why it is useful?• Framework• When and how to use• Recovery• Simple demo

Review…

Sampling and Compression

Nyquist’s Rate

• Perfect recovery

Transform Coding

• Assume: signal is sparse in some domain…• e.g. JPEG, JPEG2000, MPEG…1. Sample with frequency . Get signal of length N2. Transform signal K (<< N) nonzero

coefficients3. Preserve K coefficients and their locations

Compressive Sensing

Compressive Sensing

• Sample with rate lower than !!

• Can be recovered PERFECTLY!

Comparison

Nyquist’s Sampling Compressive Sensing

Sampling Frequency

Recovery Low pass filter Convex Optimization

Some Applications

• ECG• One-pixel Camera• Medical Imaging: MRI

Framework

Φ¿𝑦 𝑓

NM

N

M

N: length for signal sampled with Nyquist’s rateM: length for signal with lower rate Sampling matrix

𝐲=𝚽𝐟

When? How?

Two things you must know…

When….

• Signal is compressible, sparse…

Ψ

𝑥Φ¿

𝑦 𝑓

NM

N

M

Example… ECG

: 心電圖訊號: DCT (discrete cosine transform)

How…

• How to design the sampling matrix?• How to decide the sampling rate

(M)?

Ψ

𝑥Φ¿

𝑦M

N

Sampling Matrix

• Low coherence

Ψ

𝑥Φ¿

𝑦 Low coherence

Coherence

• Describe similarity

– High coherence more similar Low coherence more different

Example: Time and Frequency

0 10 20 30 40 50 60 70 80 90 100-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

• For example, • ,

Fortunately…

• Random Sampling– iid Gaussian N(0,1)–Random

• Low coherence with deterministic basis.

More about low coherence…

Random Sampling

Sampling Rate

• Can be exactly recovered with high probability.

Theorem

C : constant S: sparsityn: signal length

Recovery

Ψ

𝑥Φ¿

𝑦

BUT….

M

N

M

N

𝑓

N

Recovery

• Many related research…– GPSR (Gradient projection for sparse reconstruction)– L1-magic– SparseLab– BOA (Bound optimization approach)…..

Total Procedure

f Find an incoherent matrix e.g. random matrix

Sample signal

𝒂𝒓𝒈 �̂�𝒎𝒊𝒏‖�̂�‖𝟏𝑠 . 𝑡 . 𝐲=𝚯�̂� �̂�=𝐇�̂�

Sampling (Assume f is spare somewhere)

Recovering

已知 :

Demo Time

Reference• Candes, E. J. and M. B. Wakin (2008). "An Introduction To Compressive Sampling."

Signal Processing Magazine, IEEE 25(2): 21-30.• Baraniuk, R. (2008). Compressive sensing. Information Sciences and Systems, 2008.

CISS 2008. 42nd Annual Conference on.• Richard Baraniuk, Mark Davenport, Marco Duarte, Chinmay Hegde. An

Introduction to Compressive Sensing.• https://sites.google.com/site/igorcarron2/cs#sparse• http://videolectures.net/mlss09us_candes_ocsssrl1m/

Thanks a lot!

Key Points

1. Nyquist’s Rate2. CS and Transform coding…3. Sampling in time V.S. Sampling as inner

products4. About compressibility5. About designing sampling matrix6. About L1 norm explanation by geometry!7. Application( MRI, One-pixel camera…)