User Manual

96 Section 4: Using Matrix Operations
96
Any n × p matrix X can be factored as X = Q
T
U, where Q is an n × n orthogonal matrix
characterized by Q
T
= Q
−l
and U is an n × p upper-triangular matrix. The essential property
of orthogonal matrices is that they preserve length in the sense that
.
)r()(
2
2
F
T
TT
T
F
r
rr
QrQr
QQrrQ




Therefore, if r = y – Xb, it has the same length as
Qr = Qy – QXb = Qy – Ub.
The upper-triangular matrix U and the product Qy can be written as
.
rows)(
rows)(
and
rows)(
rows)(
ˆ
pn
p
pn
p
















f
g
Qy
O
U
U
Then
2
2
2
2
22
ˆ
r
F
F
F
F
FF
f
fbUg
UbQy
Qr




with equality when
0bUg 
ˆ
. In other words, the solution to the ordinary least-squares
problem is any solution to
gbU 
ˆ
and the minimal sum of squares is
2
F
f
. This is the basis
of all numerically sound least-squares programs.
You can solve the unconstrained least-squares problem in two steps:
1. Perform the orthogonal factorization of the augmented n × (p + 1) matrix
 
VQyX
T

where Q
T
= Q
−1
, and retain only the upper-triangular factor V, which you can then
partition as
rows)1(
row)(1
rows)(
ˆ












pn
p
q
00
0
gU
V
(1 column)
(p columns)