ECE 513: Vector Space Signal Processing

Spring 2009

Lectures: Tuesdays, Wednesdays, 11-12:25 p.m.; 203 Transportation Bldg.

Instructor: Prof. Yoram Bresler ( ybresler at illinois.edu, 112 Coordinated Science Lab, 244-9660)

Office Hours:  Wed 3-4PM, 112 CSL  (+ appointments by email).

TA:     Nitin Aggarwal  aggarwa1@illinois.edu

TA Office Hours: Tuesdays, 4:00-5:30pm; Everitt Lab 330L.

 


Overview:
Rigorous presentation of key mathematical tools in a vector space framework, and their applications in signal processing, including: finite and infinite dimensional vector spaces, Hilbert spaces, linear operators, inverse problems (e.g. deconvolution, tomography, Fourier imaging), least-squares methods, conditioning and regularization, matrix decompositions, subspace methods, bases and frames for signal representation (e.g. generalized Fourier series, wavelets, splines), Hilbert space of random variables, random processes, signal and spectral estimation.

Topics:

  • Inverse problems and matrix theory (12 hours): linear inverse problems; orthogonal projections; minimum-norm least squares solutions; Moore-Penrose pseudoinverse; singular value decomposition; matrix decomposition and approximation; conditioning and regularization.
  • General linear vector spaces (15 hours): finite and infinite dimensional vector spaces; Hilbert spaces; projection theorem; inverse problems in infinite dimensional vector spaces; approximation and Fourier series; pseudoinverse operators; iterative methods for optimization and inverse problems; bases and frames for signal representation;
  • Hilbert space of random variables (6 hours): random processes; least-squares estimation; Wiener filtering; Wold decomposition; discrete-time Kalman filter.
  • Applications in signal processing (12 hours, during the course): deconvolution, optimal filter design, temporal and spatial spectrum estimation, tomography, harmonic retrieval, subspace methods, sensor array processing, extrapolation of band-limited sequences, generalized sampling, wavelets, splines, subset selection, sparse approximation.


Handouts:


Lecture Notes: (access restricted to only people registered in the course)

 

Chapters 6-10 posted 4/5/09

Extra Notes:



Homework : (access restricted )

 

Problem Set

Date Posted

Date Due

Update/Comments

Solutions

Date Posted

Update/Comments

HW1

1/29/09

2/5/09

 

Sol 1

2/16/09

 

HW2

2/10/09

2/17/09

Typo in Q2(b) corrected (Feb 16, 8:00pm)

Sol 2

3/10/09

 

HW3

2/19/09

2/26/09

Typos corrected

(Feb 25, 4PM)

Sol 3

3/10/09

 

HW4

3/05/09

3/12/09

 

Sol 4

 

 

HW5

4/02/09

4/9/09

 

Sol 5

4/18

 

HW6

4/15/09

4/22/09

Turn in to Nitin Aggarwal (CSL #339)

By 5PM

Sol 6

 

Solutions to be posted on 4/22 5:10 PM, so no late HW can be accepted

 

 


Exams (access restricted)

Midterm 1

Monday, March 16, 2009 7 – 9 PM. Location: 241 Everitt Lab.

Coverage: Ch. 1 – Ch. 4.3

Closed book test. You are allowed one two-sided sheet of paper.

Midterm 1, 2009

·         Grade Improvement Opportunity

You are being offered an opportunity to improve your grade on Midterm 1, by re-working the problems of the midterm and turning the paper for grading by Friday April 9, at 5 PM. Your final score on the midterm will be: in-class Exam score + 0.3* (at home Exam score - in-class Exam score).

You may use the course notes and HW solutions handed out in the course, but no other materials, in whatever form (printed, electronic, etc.). Also, you must solve the problems yourself, and may not use any help in doing so.

On your paper include the following statement and sign:

“I certify that I did not use any materials, in whatever form, other than the course notes and HW solutions handed out in the course, and did not receive any help in preparing the solutions I am turning in for grading.

Signature:
--------------
........”

Previous Exams

Midterm 1, 2007

Midterm 1, 2008

Midterm 2

·         Thursday, April 23, 2009, 7 – 9 PM. Location: 260 EVRT

·         Review Session: Wednesday, April 22, 2009, 7 – 9 PM. Location: 241 EVRT

·         Coverage: Ch. 1 - Ch. 7 of BBC (incl. material on HWs 1- 6).

·         Closed book test. You are allowed two two-sided sheet of paper.

Previous Exams

Midterm 2, 2007

Midterm 2, 2008


Final Projects

  • Project Proposals: Due Monday April 13, 5PM
  • Final Project Report: Due Wednesday May 13, 5 PM
  • Presentation Schedule May 5-6.
  • Note: Everyone must be present at all presentations to receive credit for their own presentation.

Time

Name

Project Title

Tuesday

Rm. 239 CSL*

 

 

6:00 PM

Logan Niehaus

Consistent Sampling and Signal Recovery

6:30

Sai Prasad R

Sparse Representation of data using a union of subspaces

7:00

Jeung Kim

Signal Detection through Projection

7:30

Onur Ertuk

Restricted Orthogonality Property

Wednesday

Rm. 239 CSL

 

 

9:00 AM

Myra Nam
Shape metrics on the space of planar curves

9:30

Bo Zhao
Matrix Completion for Image Reconstruction in MRI Dynamic Imaging

10:00

Shu Xinabiao

Curvelets--A Surprisingly Effective Nonadaptive Representation for Objects with Edges

Rm. 114 CSL

 

 

2:00 PM

Andrew Bean

Signal sampling and reconstruction

2:30

Pilwon Hur

Some mathematics in learning theory and application

3:00

Chao Ma

Simultaneous Sparse Approximation and Its Application in MRI

3:30

Martin McCormick

Brain Computer Interfacing in the Context of Vector Space Signal Processing

4:00

Hoa Pham

Greedy algorithms for Sparse Approximation

4:30

Adam Gustafson

Improvements of Greedy Methods for Inverse Problems and Compressed Sensing

 

* CSL outside doors lock at 6PM.

  • Presentation Contents:
    • Deliver power point or PDF presentation
    • 20 minutes presentation, 5 minutes for questions
    • Must be present for all presentations, and participate in questions

·         General guidelines and criteria for evaluation of the presentations:

1.      Clear statement of the problem being addressed or of the main ideas and purpose of the theory being introduced.

2.      Precise statement of theoretical results (e.g., key theorem(s), key algorithm),
 and explanation of the significance and role of assumptions needed for these results to hold.

1.      Explanation of

a.       the meaning of the results

b.      their significance

c.       their implications (for applications, and/or for the development of additional theory)

d.      their limitations (when they break down, do not apply)

2.      Understanding and ability to explain (in a mathematically precise way, but also providing the intuition) the technical derivation of one of the key results.

a.        You need to be able to teach your audience something new they have not seen before in the course.

b.      Spend at least 5 minutes on this -- but remember to allocate your time to cover the other criteria/components.

3.      Clear illustration of the application of the results/theory by appropriate example(s) -- either your own simulation or analysis, or from the paper(s) you have read.

4.      Suggestions for future work (brief): what are open problem, or what extension/applications might be interesting to pursue.

5.      Ability to answer questions.

 

  • Project Report: Due in electronic form (WORD or PDF) by 5 PM, May 13.
    • Criteria: similar to the above, but greater focus on technical contents.
    • References: Include PDF versions of the key papers used as references.
    • Format:
      1. Length: between 8 to 16 pages
      2. Single or double column layout
      3. Margins: 1” all around
      4. Spacing: 1.5-spaced or double-spaced
      5. Font: 11pt or 12pt Times New Roman or Georgia font
      6. Figures: Avoid large areas of dark color (e.g., black) to avoid waste of toner, and use at least 8 pt. font in figures to ensure readability