# Linear Algebra

• Elements

>Scalars, vectors, matrices

• Operations

>Addition & subtraction (vector, matrix)

>Multiplication (scalar, vector, matrix)

>Division (scalar, matrix)

• Applications

>Signal processing, others...

**Scalars & Vectors
**• Scalar: fancy word for a number:

>e.g., 3.14159, or -42.

>Scalar can be integer, real, or complex.

• Vector: a collection of numbers:

>[age, height, weight] or [x, y, z, roll,

pitch, yaw]

>Why bother?

• Compact notation: p = [x, y, z, roll, pitch, yaw]

> So p_{i} is the ith element, e.g., in above, p_{2}=y

• Ease of manipulation (examples later).

**Visualize a vector
**

**Vectors
**• Row vector:

• Column vector:

• The transpose operator:

>If , then

**Operations: Scalar multiplication
**• Vector multiplied by scalar:

>Multiply each vector element by the

scalar:

• Scalar multiplication increases length

of vector.

• If the scalar is negative, also

reverses direction of the vector.

**Scalar multiplication
**-

**Addition of vectors
**• Element-by-element:

>So,

>Associative, commutative

>Add two like-shape vectors (row & row).

**Linear combinations
**• Linear combinations of vectors:

>Example: u = c

_{1}v

_{1}+ c

_{2}v

_{2}

**Linear independence
**• Set of all linear combinations of a

set of vectors is the linear space

spanned by the set.

• A set of vectors is linearly

independent if none of the vectors

in the set can be written as a linear

combination of the other members

of the set.

**Inner product
**•Also called “dot product:”

> x•y = x

_{1}y

_{1}+ x

_{2}y

_{2}+ … + x

_{n}y

_{n}

• Maps two vectors to a scalar.

• Length (also called norm):

> x•x = x

_{1}x

_{1}+ x

_{2}x

_{2}+ … + x

_{n}x

_{n}

>x•x = ||x||

^{2}

> ||x|| is the length of the vector

> Note ||ax||=|a| ||x||

**Angle between two vectors
**• Defined using inner product:

**Example of compact notation
**• Set of data: x=[x

_{1},…,x

_{n}], y=[y

_{1},…,y

_{n}]

• Correlation coefficient:

**Basis vectors
**• A basis for V is a set of vectors B in V that

span V. Vectors in B are linearly

independent, so any v in V can be written as

a linear combination of vectors in B.

• Orthogonal bases -> simple combinations.

> Orthogonal if u•v = 0, as cos θ = 0.

• Usual basis (of infinitely many) for R3 is:

>B = [1,0,0]

^{T}, [0,1,0]

^{T}, [0,0,1]

^{T}

> coefficients of v using B are coordinates.

**Projections of vectors
**• Projection of v onto w:

>In 2 dimensions, x = ||v|| cos θ, or in higher

dimensions is:

**Projection onto basis
**

**Matrices
**• Matrices are arrays of numbers.

>Convenient, compact notation.

• Are operators, mapping between vector

spaces (e.g., change of basis or

coordinates).

• Some special names:

>Square has same number of rows and

columns.

>Diagonal has non-zero elements only on

diagonal.

• if elements are all = 1, call it I identity matrix,

mapping vector space to itself.

>Symmetric is square with

**Matrix operations
**• If one considers a matrix as a set of

vectors:

>Can multiply matrix by scalar.

>Can add conforming matrices (same #

rows, columns).

• Multiply vector by matrix:

**Matrix multiplication
**• Can concatenate mappings:

> If we have,

**u = Wv**and

**v = Mz,**then

u = W(Mz) = Pz

u = W(Mz) = Pz

•

**W**must have same number of rows as

**M**has columns, so multiplying an rxs

matrix by sxt matrix gives an rxt matrix.

>Associative, distributive, but not

commutative!

**Inner and outer product
**• Inner product: Maps two vectors to a

scalar:

• Outer product: Maps two vectors to a

matrix:

**Matrix inverse
**• The inverse of

**M**is that matrix which,

when multiplied by

**M**, gives the identity

matrix:

**> MM**

• Use the inverse for the solution to n

^{-1}= M^{-1}M = Isimultaneous linear equations:

>If

**y = A x**, then

**x = A**

^{-1}y.•

**M**may not exist, then

^{-1}**M**is singular.

**Eigenvectors & eigenvalues
**•Matrix mapping

**u = Wv**takes domain V to

range U.

• Generally, elements in V are changed in

length and direction.

• Elements that are changed in length only are

called eigenvectors of

**W**.

>If , then v is an eigenvector of

**W**with

eigenvalue

•

**W**can have more than one eigenvector; they

form a basis for the column space of

**W**.