1 Introduction to Vectors 1
1.1 Vectors and Linear Combinations 2
1.2 Lengths and Dot Products 11
1.3 Matrices 22
2 Solving Linear Equations 31
2.1 Vectors and Linear Equations 31
2.2 The Idea of Elimination 46
2.3 Elimination Using Matrices 58
2.4 Rules for Matrix Operations 70
2.5 Inverse Matrices 83
2.6 Elimination = Factorization: A = LU 97
2.7 Transposes and Permutations 108
3 Vector Spaces and Subspaces 122
3.1 Spaces of Vectors 122
3.2 The Nullspace of A: Solving Ax = 0 and Rx = 0 134
3.3 The Complete Solution to Ax = b 149
3.4 Independence, Basis and Dimension 163
3.5 Dimensions of the Four Subspaces 180
4 Orthogonality 193
4.1 Orthogonality of the Four Subspaces 193
4.2 Projections 205
4.3 Least Squares Approximations 218
4.4 Orthonormal Bases and Gram-Schmidt 232
5 Determinants 246
5.1 The Properties of Determinants 246
5.2 Permutations and Cofactors 257
5.3 Cramer’s Rule, Inverses, and Volumes 272
6 Eigenvalues and Eigenvectors 287
6.1 Introduction to Eigenvalues 287
6.2 Diagonalizing a Matrix 303
6.3 Systems of Differential Equations 318
6.4 Symmetric Matrices 337
6.5 Positive Definite Matrices 349
7 The Singular Value Decomposition (SVD) 363
7.1 Image Processing by Linear Algebra 363
7.2 Bases and Matrices in the SVD 370
7.3 Principal Component Analysis (PCA by the SVD) 381
7.4 The Geometry of the SVD 391
8 Linear Transformations 400
8.1 The Idea of a Linear Transformation 400
8.2 The Matrix of a Linear Transformation 410
8.3 The Search for a Good Basis 420
9 Complex Vectors and Matrices 429
9.1 Complex Numbers 430
9.2 Hermitian and Unitary Matrices 437
9.3 The Fast Fourier Transform 444
10 Applications 451
10.1 Graphs and Networks 451
10.2 Matrices in Engineering 461
10.3 Markov Matrices, Population, and Economics 473
10.4 Linear Programming 482
10.5 Fourier Series: Linear Algebra for Functions 489
10.6 Computer Graphics 495
10.7 Linear Algebra for Cryptography 501
11 Numerical Linear Algebra 507
11.1 Gaussian Elimination in Practice 507
11.2 Norms and Condition Numbers 517
11.3 Iterative Methods and Preconditioners 523
12 Linear Algebra in Probability & Statistics 534
12.1 Mean, Variance, and Probability 534
12.2 Covariance Matrices and Joint Probabilities 545
12.3 Multivariate Gaussian and Weighted Least Squares 554
Matrix Factorizations 562
Index 564
Six Great Theorems/ Linear Algebra in a Nutshell 573