reconstruction algorithms for compressive sensing i

ACCESS IC LAB

Graduate Institute of Electronics Engineering, NTU

Reconstruction Algorithms for Compressive Sensing I

Presenter: 黃乃珊Advisor: 吳安宇教授

Date: 2014/03/25

ACCESS IC LAB Graduate Institute of Electronics Engineering, NTU

2

Schedule

19:30 @ EEII-225日期內容 Lab & HW Speaker

3/11 Introduction to Compressive Sensing System Nhuang

3/25 Reconstruction Algorithm Nhuang

4/8 Reconstruction Algorithm Lab1 Nhuang

4/15 Break; 決定期末題目方向4/22 Sampling Algorithm: Yumin

4/29 Midterm Presentation (Tutorial, Survey)

5/6 Application: Single Pixel Camera Lab2 Yumin

5/13 ~ 6/10 期末報告討論6/24 Final Presentation


Outline

Review Compressive Sensing Reconstruction Algorithms for Compressive Sensing Basis Pursuit Orthogonal Matching Pursuit Reference

3


4

Compressive Sensing in Mathematics

Sampling matrices should satisfy restricted isometry property (RIP) Ex. Random Gaussian matrices

Reconstruction solves an underdetermined question Linear Programming (ex. Basis Pursuit) Greedy Algorithm (ex. Orthogonal Matching Pursuit) Iterative Thresholding

Sampling ReconstructionChannel

𝒚𝑴=𝚽𝑴×𝑵 𝒙𝑵

𝒙𝑵 �̂�𝑵

𝒚𝑴+𝒏𝒐𝒊𝒔𝒆

(1−𝛿) ∙𝑀𝑁 ∙‖𝑥‖22≤‖Φ 𝑥‖2

2≤ (1+𝛿) ∙𝑀𝑁 ∙‖𝑥‖22


5

Reconstruction

Original underdetermined question

Linear programming question

Two condition Restricted Isometry property (RIP) Sparse signal

min𝑥

‖𝒙‖0 s .t .𝚽 𝒙=𝒚 ,‖𝒙‖0≔¿{𝑘:𝑥𝑘≠0 }

min𝑥

‖𝒙‖1 s . t .𝚽 𝒙=𝒚 ,‖𝒙‖1≔∑𝑖

¿ 𝑥 𝑖∨¿¿

NP-hard!


6

Recovery Algorithms for Compressive Sensing

Linear Programming Basis Pursuit (BP)

Greedy Algorithm Matching Pursuit

Orthogonal Matching Pursuit (OMP) Stagewise Orthogonal Matching Pursuit (StOMP) Compressive Sampling Matching Pursuit (CoSaMP) Subspace Pursuit (SP)

Iterative Thresholding Iterative Hard Thresholding (IHT) Iterative Soft Thresholding (IST)

Bayesian Compressive Sensing (BCS) Approximate Message Passing(AMP)


7

Basis Pursuit (BP) [3][4]

Find signal representation in overcomplete dictionaries by convex optimization

BP-simplex Optimize by swapping element

BP-interior Optimize by modifying coefficient More common

↑BP-simplex

↑BP-interior


8

Compressive Sensing in Linear Algebra

Reconstruction is composed of two parts: Localize nonzero terms Approximate nonzero value

Do correlation to find the location of non-zero terms Solve least square problem to find the value

Projection (pseudo-inverse)coefficient

basis

=Measurement Input


9

Matrix Inverse

Matrix inverse for invertible square matrix A square matrix with nonzero determinant Non-square matrix has enough rank To find inverse matrix

Gauss-Jordan elimination, LU decomposition QR decomposition

Pseudo inverse To find least square solution


10

Orthogonal Matching Pursuit (OMP) [5]

Use greedy algorithm to iteratively recover sparse signal Procedure:

1. Initialize2. Find the column that is most correlated3. Set Union (add one col. every iter.)4. Solve the least squares 5. Update data and residual6. Back to step 2 or output

[14]


11

Stagewise Orthogonal Matching Pursuit (StOMP) [6]

Derive from OMP, but with small fixed number of iteration Procedure:

1. Initialize2. Find the column that is most correlated3. Hard thresholding4. Set Union (add some col. every iter.)5. Find corresponding x by projection6. Update data and residual7. Back to step 2 or output

better global optimizationcorrelation


12

Compressive Sampling Matching Pursuit (CoSaMP)[7]

Inspired by the RIP, the energy in proxy approximates the energy in target signal

Procedure:1. Initialize2. Proxy3. Set Union4. Signal estimation by projection5. Prune approximation6. Update data and residual7. Back to step 2 or output


13

Subspace Pursuit (SP) [8]

Re-evaluate all candidates at each iteration Procedure:

1. Initialize2. Proxy3. Set Union4. Signal estimation by projection5. Prune approximation6. Update data and residual7. Back to step 2 or output


14

Next Lecture Linear Programming

Basis Pursuit (BP) Greedy Algorithm

Matching Pursuit Orthogonal Matching Pursuit (OMP) Stagewise Orthogonal Matching Pursuit (StOMP) Compressive Sampling Matching Pursuit (CoSaMP) Subspace Pursuit (SP)

Iterative Thresholding Iterative Hard Thresholding (IHT) Iterative Soft Thresholding (IST)

Bayesian Compressive Sensing (BCS) Approximate Matching Pursuit (AMP)


15

Reference [1] E. J. Candes, and M. B. Wakin, "An Introduction To Compressive Sampling," Signal Processing

Magazine, IEEE , vol.25, no.2, pp.21-30, March 2008[2] G. Pope, “Compressive Sensing – A Summary of Reconstruction Algorithm”, Swiss Federal Instituute of

Technology Zurich[3] E. J. Candes, and T. Tao, "Decoding by linear programming," IEEE Transactions on Information Theory,

vol.51, no.12, pp. 4203- 4215, Dec. 2005[4] S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM J. Sci

Comp., vol. 20, no. 1, pp. 33–61, 1999.[5] J. A. Tropp, A. C. Gilbert, “Signal Recovery from Random Measurements via Orthogonal Matching

Pursuit,” IEEE Transactions on Information Theory, vol.53, no.12, pp. 4655-4666, Dec. 2007[6] D. L. Donoho, Y. Tsaig, I. Drori, and J.-L. Starck, “Sparse solution of underdetermined linear equations

by stagewise Orthogonal Matching Pursuit (StOMP),” Information Theory, IEEE Transactions on , vol.58, no.2, pp.1094,1121, Feb. 2012

[7] D. Needell, and J. A. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples." Applied and Computational Harmonic Analysis 26.3 (2009): 301-321.

[8] W. Dai, and O. Milenkovic, "Subspace Pursuit for Compressive Sensing Signal Reconstruction," Information Theory, IEEE Transactions on , vol.55, no.5, pp.2230,2249, May 2009

reconstruction algorithms for compressive sensing i

Documents