Zoom out Search Issue

ManualsBrandsContents Manualsaudio & home theatreZoom in

161

162

163

164

165

166

167

168

169

170

[

lecture

NOTES

]

Dave Zachariah

and Petre Stoica

Cramér–Rao Bound Analog of Bayes’ Rule

he estimation of multiple pa-

rameters is a common task in

signal processing. The Cra-

mér–Rao bound (CRB) sets a

statistical lower limit on the re-

sulting errors when estimating parameters

from a set of random observations. It can be

understood as a fundamental measure of

parameter uncertainty [1], [2]. As a general

example, suppose

i denotes the vector of

sought parameters and that the random ob-

servation model can be written as

,yx w=+

(1)

where x

is a function or signal parame-

terized by

i and w is a zero-mean Gauss-

ian noise vector. Then the CRB for

i has

the following notable properties:

1) For a fixed

i the CRB for i decreas-

es as the dimension of y increases.

2) For a fixed ,y if additional parameters

are estimated, then the CRB for

increases as the dimension of

increases.

3) If adding a set of observations y

requires estimating additional param-

eters

then the CRB for

i decreas-

es as the dimension of y

increases,

provided the dimension of

does not

exceed that of y

[3]. This property

implies both 1) and 2) above.

4) Among all possible distributions of

with a fixed covariance matrix, the CRB

for

i attains its maximum when w is

Gaussian, i.e., the Gaussian scenario is

the “worst case” for estimating

[4]–[6].

In this lecture note, we show a general

property of the CRB that quantifies the

interdependencies between the parameters

i The presented result is valid for more

general models than (1) and also general-

izes the result in [7] to vector parameters.

It will be illustrated via two examples.

RELEVANCE

In probability theory, the chain rule and

Bayes’ rule are useful tools to analyze the

statistical interdependence between multi-

ple random variables and to derive tractable

expressions for their distributions. In this

lecture note, we provide analogs of the

chain rule and Bayes’ rule for the CRB

associated with multiple parameters. The

results are particularly useful when esti-

mating parameters of interest in the pres-

ence of nuisance parameters.

PREREQUISITIES

The reader needs basic knowledge about

linear algebra, elementary probability the-

ory, and statistical signal processing.

PRELIMINARIES

We will consider a general scenario in which

we observe an

n 1# random vector .y Its

probability density function (pdf) ;p y

parameterized by a k 1# deterministic vec-

tor .

i The goal is to estimate ,i or subvec-

tors of ,

i given .y

Let ;lpln y_

denote the log-

likelihood function, and let

be any unbi-

ased estimator. Then the mean square error

(MSE) matrix E[()()]P

_ iiii--

bounded from below by the inverse of

the Fisher information matrix

EJ _ -

2 i

where

denotes the second-

order differential or Laplacian operator with

respect to

i That is, ,PJ

assuming

from hereon that J

is nonsingular. This is

the Cramér–Rao inequality [2], [8], [9].

The determinant of the MSE matrix,

||,P

is a scalar measure of the error

magnitude. For unbiased estimators, ||P

equals the “generalized variance” of errors

[10]. By defining ,CRB J

the

generalized error variance is bounded by

.CRBP $

In the following, we are interested in

subvectors or elements of .

i Letting i =

[],

<<<

we can write the Fisher informa-

tion matrix in block form,

22 2

222

ab ab

ba b

aab

(2)

MAIN RESULT

Let

a and b be two random vectors. Two

useful rules in probability theory are the

chain rule

() (|)()ppp,ab a b b= (3)

and Bayes’ rule

()

(|)

()

().p

p |a

ab=

(4)

Now consider two parameter vectors

a and .b When both are unknown, their

joint CRB bound is given by

,.CRB

ab =

(5)

The bound for

a with known b is simply

CRB J

ab=

(6)

and the bound for a with unknown b is

J .CRB JJ J

a =-

aab

(7)

[Equation (7) follows by evaluating the

inverse in (5) and extracting the upper-

left block corresponding to .

a ] Equations

(6) and (7) are the respective CRB analogs

of conditional and marginal distributions

for random variables.

Digital Object Identifier 10.1109/MSP.2014.2365593

Date of publication: 12 February 2015

THE WORLD’S NEWSSTAND

THE WORLD’S NEWSSTAND