Bayes Home
Jaynes Errata
next up previous

Commentary: Note on exchangeability and de Finetti's Theorem

In several places Jaynes refers to exchangeability and de Finetti's Theorem without defining these; finally, in Chapter 18 section 16 (p. 586 onward) he says a little bit about just what de Finetti's Theorem is. For those readers who are unfamiliar with this topic, Bernardo and Smith's book Bayesian Theory has a nice discussion in sections 4.2 and 4.3. Here is a brief, simplified summary of the definitions and theorems given therein, translated into the vocabulary and notation of Jaynes.

Notation. We write $x_1^n$ for $x_1,...,x_n$, and $x_1^{\infty}$ for the infinite sequence $x_1,x_2,\ldots$. For simplicity, we assume that the set of possible values for each variable $x_i$ is the same finite set $S$.

Definition of finite exchangeability. The variables $x_1^n$ are said to be (finitely) exchangeable for a state of information $I$ if, for any constant values $X_1^n$, we have

\begin{displaymath}P(x_1^n = X_1^n \mid I) = P(y_1^n = X_1^n \mid I) \end{displaymath}

whenever $y_1^n$ is a permutation of the variables $x_1^n$.

Definition of infinite exchangeability. The infinite sequence of variables $x_1^{\infty}$ is said to be infinitely exchangeable for a state of information $I$ if every finite subsequence of $x_1^{\infty}$ is exchangeable for $I$.

Theorem. Let $W$ be the set of probability mass functions over $S$. If $x_1^{\infty}$ is an infinitely exchangeable sequence of variables for $I$, then there exists a probability density $f$ over $W$ such that

P(x_1^n = X_1^n \mid I) = \int_{w\in W} f(w) \prod_i w(X_i)

for any constant $X_1^n$. (Note that $f$ may involve delta functions, to assign positive probability to a single specific value in $W$.)

In other words, we may reason as if there exists some additional variable $u$ such that the variables $x_1^n$ are independently and identically distributed when the value of $u$ is known, and $P(x_i = s \mid u = w, I)$ is $w(s)$.

next up previous