Before we begin our discussion of the statistical models and methods, we review
elements of matrix algebra that will be quite useful in streamlining our presentation and
representing data. Here, we will
note some basic results and operations. Further results and definitions will be
discussed as we need
them throughout the course. Many useful facts here are stated systematically in
this chapter, thus, this
chapter will serve as a reference for later developments using matrix notation.
2.2 Matrix notation MATRIX: A rectangular array of numbers, e.g.
As is standard, we will use boldface capital letters to
denote an entire matrix.
DIMENSION: A matrix with r rows and c columns is said to
be of dimension (r × c).
It is customary to refer generically to the elements of a matrix by using 2
a11 = 3, a12 = 5, etc. In general, for a matrix with r
rows and c columns, A, the element of A in the
ith row and the jth column is denoted as aij , where i = 1, : : : , r and j =
1, : : : , c.
VECTOR: A column vector is a matrix with only one column,
A row vector is matrix with only one row, e.g.
It is worth noting some special cases of matrices.
SQUARE MATRIX : A matrix with r = c, that is, with the same
number of rows and columns is called
a square matrix. If a matrix A is square, the elements aii are said to lie on
the (principal) diagonal
of A. For example,
SYMMETRIC MATRIX: A square matrix A is called symmetric if
aij = aji for all values of i and j.
The term symmetric refers to the fact that such a matrix "reflects" across its
Symmetric matrices turn out to be quite important in
formulating statistical models for all types of
IDENTITY MATRIX: An important special case of a square,
symmetric matrix is the identity matrix
-a square matrix with 1's on diagonal, 0's elsewhere, e.g.
As we will see shortly, the identity matrix functions the
same way as "1" does in the real number system.
TRANSPOSE: The transpose of any (r × c) A matrix is the
(c × r) matrix denoted as A' such that
aij is replaced by aji everywhere. That is, the transpose of A is the matrix
found by "flipping" the
matrix around, e.g.
A fundamental property of a symmetric matrix is that the
matrix and its transpose are the same, i.e.,
if A is symmetric then A = A'. (Try it on the symmetric matrix above.)
2.3 Matrix operations
The world of matrices can be thought of as an extension of the world of real
(scalar) numbers. Just as
we add, subtract, multiply, and divide real numbers, we can do the same in with
matrices. It turns out
that these operations make the expression of complicated calculations easy to
talk about and express,
hiding all the details!
Note that these operations only make sense if the two
matrices have the same dimension - the
operations are not defined otherwise.
MULTIPLICATION BY A CONSTANT : The effect of multiplying a
matrix A of any dimension by a
real number (scalar) b, say, is to multiply each element in A by b. This is easy
to see by considering
that this is just equivalent to adding A to itself b times . E.g.
MATRIX MULTIPLICATION: This operation is a bit tricky, but
as we will see in a moment, it proves
most powerful for expressing a whole series of calculations in a very simple
• Order matters
• Number of columns of first matrix must = Number of rows of second matrix, e.g.
E.g. (1)(2) + (3)(0) + (5)(1) = 7 for the (1, 1) element.
• Two matrices satisfying these requirements are said to
conform to multiplication.
• Formally, if A is (r × c) and B is (c × q), then AB is a (r × q) matrix with (i,
Here, we say that A is postmultiplied by B and, equivalently, that B is
premultiplied by A .
EXAMPLE: Consider a simplelinear regression model:
suppose that we have n pairs (x1, Y1),....., (xn, Yn),
and we believe that, except for a random deviation, the relationship between the
covariate x and the
response Y follows a straight line. That is, for j = 1,....., n, we have
is a random deviation representing the amount by which the actual observed
deviates from the exact straight line relationship. Defining
we may express the model succinctly as
SPECIAL CASE: Multiplying vectors. With a row vector
premultiplying a column vector, the result is
a scalar (remember, a (1 × 1) matrix is just a real number!), e.g.
i.e. (1)(2) + (3)(0) + (-5)(3) + (1)(-2) = -15
With a column vector premultiplying a row vector, the
result is a matrix. e.g.
MULTIPLICATION BY AN IDENTITY MATRIX: Multiplying any
matrix by an identity matrix of
appropriate dimension gives back the same matrix, e.g.
• A(B + C) = AB + AC, (A + B)C = AC + BC
• For any matrix A, A'A will be a square matrix.
• The transpose of a matrix product - if A and B conform to multiplication, then
of their product
These latter results may be proved generically, but you
may convince yourself by working them out for
the matrices A and B given above.
LINEAR DEPENDENCE: This characteristic of a matrix is
extremely important in that it describes
the nature and extent of the information contained in the matrix. Consider the
Refer to the columns as c1, c2, c3. Note that
2c1 + -c2 + -c3 = 0,
where 0 is a column of zeros (in this case, a (3 × 1)
vector). Because the 3 columns of A may be combined in a linear function to yield a vector of nothing but zeros, clearly,
there is some kind of
relationship, or dependence, among the information in the columns. Put another
way, it seems as
though there is some duplication of information in the columns.
In general, we say that k columns c12, c2,...., ck of a
matrix are linearly dependent if there exists a
set of scalar values
Linear dependence implies that each column vector is a
combination of the others, e.g.,
The implication is that all of the "information" in the
matrix is contained in a subset of the columns
- if we know any (k - 1) columns, we know them all. This formalizes our notion
of "duplication" of
If, on the other hand, the only set of
values we can come up with to satisfy (2.2) is a set of all zeros,
then it must be that there is no relationship among the columns, e.g. they are
"independent" in the
sense of containing no overlap of information. The formal term is linearly
Start solving your Algebra Problems
in next 5 minutes!
Download (and optional CD)
Click to Buy Now:
2Checkout.com is an authorized reseller
of goods provided by Sofmath
Attention: We are
currently running a special promotional offer
for Algebra-Answer.com visitors -- if you order
Algebra Helper by midnight of
you will pay only $39.99
instead of our regular price of $74.99 -- this is $35 in
savings ! In order to take advantage of this
offer, you need to order by clicking on one of
the buttons on the left, not through our regular
If you order now you will also receive 30 minute live session from tutor.com for a 1$!
You Will Learn Algebra Better - Guaranteed!
Just take a look how incredibly simple Algebra Helper is:
: Enter your homework problem in an easy WYSIWYG (What you see is what you get) algebra editor:
Step 2 :
Let Algebra Helper solve it:
Step 3 : Ask for an explanation for the steps you don't understand:
Algebra Helper can solve problems in all the following areas:
simplification of algebraic expressions (operations
with polynomials (simplifying, degree, synthetic division...), exponential expressions, fractions and roots
(radicals), absolute values)