Mathematics of Signals and Systems

The files in this repository are developed for ECEn 671 taught in the Electrical and Computer Engineering Department at Brigham Young University. The purpose of the repository is to aid graduate students in learning common vector space and linear algebra techniques used in signal processing and control. The repository consists of two directories: The first containing lecture notes, written in beamer, and the second containing jupyter ipython notebooks written and edited by former students in my class. These contain working code examples and homework problems.

The textbook for the class is Moon & Stirling

Lecture Notes

• ecen671_chap2.tex - metric, vector, normed, and inner product spaces, topology, orthogonality, linear operators, projections, Gram Schmidt Orthogonalization
• ecen671_chap3.tex - approximation theory, dual approximation, underdetermined problems, generalized Fourier series
• ecen671_chap4.tex - linear operators, matrix norms, adjoint operator, fundamental subspaces, matrix inverses, matrix condition number, Schur complement, recursive least squares.
• ecen671_chap5.tex - LU Factorization, Cholesky Factorization, QR Factorization
• ecen671_chap6.tex - Eigenvalues and eigenvectors, Jordan form, Cayley-Hamilton theorem, self adjoint matrices, invariant subspaces, quadratic forms, eigenfilters
• ecen671_chap7.tex - singular value decomposition, pseudo inverse rank reducing approximations
• ecen671_chap14.tex - gradient descent, LMS adaptive filtering, Gauss-Newton, Levenberg-Marquardt
• ecen671_chap18.tex - constrained optimization, Lagrange multipliers, Kuhn-Tucker conditions

Student Designed Jupiter Notebooks

The following links have been developed by BYU graduate students enrolled in ECEn 671 Mathematics of Signals and Systems during Fall Semester 2018, and revised during Fall 2020.

Topic 1. Vector Spaces

Topic 2. Vector norms: 1-norm, 2-norm, p-norm, infinity-norm

Topic 3. Inner product and inner product spaces

Topic 4. Linear independence

Topic 5. Orthonormal bases for vector spaces

Topic 6. Projection operators

Topic 7. Gram-Schmidt orthogonalization

Topic 8. Linear regression (least squares)

Topic 9. Dual approximation (min-norm solutions)

Topic 10. Generalized Fourier series

Topic 11. Matrix norms

Topic 12. Linear operators

Topic 13. Adjoint operators

Topic 14. Matrix Inverses and pseudo-inverses

Topic 15. The matrix inversion lemma

Topic 16. Recursive least squares

Topic 17. LU Factorization

Topic 17. LU Factorization-2

Topic 18. Cholesky Factorization

Topic 19. QR Factorization

Topic 20. Eigenvalues and eigenvectors

Topic 21. The matrix exponential

Topic 22. Differential equations and invariant subspaces

Topic 23. Quadratic Forms

Topic 24. Singular Value Decomposition

Topic 25. The four fundamental spaces of a matrix

Topic 26. Rank reducing approximations of a matrix

Topic 27. Gradient Descent

Topic 28. Lagrange Multipliers

Topic 29. Kuhn-Tucker Conditions

Appendix. Mathematical Preliminaries

Applications

Splines Basis Construction

Spline Basis Construction - Julia

Dynamic Mode Decomposition

Multisource Statistically-Optimized Nearfield Acoustical Holography

How to Install Jupyter Notebooks

For the viewer only option: http://nbviewer.jupyter.org/github/randybeard/ece671_Math_of_Signals_Systems/blob/master/jupyter/table_of_contents.ipynb

In a Linux/MacOS terminal, first set up pip:

sudo apt install python3-pip

Then install jupyter notebooks:

pip install notebook

Change directory to the local git repository and run notebook:

jupyter notebook

Use similar instructions for Windows.