ksvanhorn.com
Home
Bayes Home
Jaynes Errata
Articles
Books
Software
Contact

# Chapter 17: Principles and pathology of orthodox statistics

• p. 512, first full paragraph: ...it seems obvious that, at least for large , this has made things worse instead of better.'' I believe large'' should be small.''

• p. 513, footnote 2: indavertently'' should be inadvertently.''

• p. 518, first line after (17.27), and last line on page: Schwartz'' should be Schwarz.''

• p. 519, eqn. (17.30): (in the denominator) should be .

• p. 519, eqn. (17.32): should be .

• p. 520, third full paragraph, start of third line: Schwartz'' should be Schwarz.''

• p. 530, eqn. (17.65): '' should be ''.

• p. 538, eqn. (17.93): should be .

• p. 541, eqn. (17.107): As far as I can tell, this equation is wrong. The final term of the rightmost expression should be

• p. 543, first line after (17.118): Schwartz'' should be Schwarz.''

• p. 544, eqn. (17.125): subscripts of and are missing from the summation symbol ; the factor for the middle expression should be ; and the rightmost expression should be .

• p. 544, eqn. (17.126): should be .

• p. 545, eqn. (17.128): The denominator needs to be squared--that is,

• p. 545, eqn. (17.129): this should read

• p. 545, eqn. (17.130): should be .

• p. 546, eqn. (17.135): , here and in the rest of the section, should be . Alternatively, in (17.132) should be replaced with .

• p. 546, eqn. (17.136): I think the series should be , as provides only a constant term, which is subsumed by the trend. Likewise, in (17.137), replace '' with ''.

• p. 548, middle of page:  is the matrix of model functions.'' We called this matrix on the previous two pages.

• p. 549, eqn. (17.164): This is redundant; it just repeats eqn. (17.161), which appeared half a page earlier.

• p. 551, third full paragraph, third-to-last line: student'' should be Student.''

## Miscellaneous Commentary

• p.517, eqn. (17.23): the normalization constant is not a typo, even though it is the same as the normalization constant in (17.17). That's right: defining , we have that both and are equal to .

• p. 517, eqn. (17.23) and preceding line: Since is a scale factor, it might seem more reasonable to use an ignorance prior instead of a uniform prior over . The only effect this has is to replace with in the posterior and the corresponding posterior expectation formula (17.23).

• p. 518, footnote 6: This story about Kendall and Jeffreys is probably based on a confusion with G.U. Yule. Kendall was a student at St John's, but he did not become a Fellow. It was Jeffreys and Yule who were both Fellows for many years. [Contributed by Nick Cox.]

• p. 519, eqn. (17.31): Note that the change of parameters'' mentioned in the preceding text is not a reparameterization of a distribution. Recall that to achieve the minimum variance, there must exist some such that

We then define implicitly via ; the purpose is to make (17.32) come out neatly.

• p. 519, eqn. (17.33): Note that is a vector of observables, not a single observable. Writing for the vector , (17.33) may be written more explicitly as

so to make (17.33) correspond to the results of Chapter 11, we must choose . If the estimators are related by

then the distribution that minimizes also minimizes , assuming the are assigned independent and identical distributions:

is identical to (17.33) for arbitrary if we choose , , and .

• p. 530, end of section, and p. 549, end of section: A more detailed discussion of the topic can be found in Larry Bretthorst's Ph.D. dissertation, Bayesian Spectrum Analysis and Parameter Estimation (Bretthorst got his doctorate under Jaynes.) This was published as Lecture Notes in Statistics 48, but is out of print; the best way to get it now is to download it from Bretthorst's web page on Probability Theory as Extended Logic (near the end of the page).

• p. 531 (Section 17.7, The folly of randomization'') Jaynes's example of using Monte Carlo methods to do a simple one-dimensional integral is a bit misleading, in that most uses of MC methods involve high-dimensional integrals and sampling from from distribution to compute an expectation over that distribution; in such cases, using an -dimensional grid to numerically integrate is impractical. However, the emergence of techniques such as Latin squares sampling and the use of quasi-random sequences to improve the convergence of MC integration certainly seems to support Jaynes's contention (on p. 532) that Whenever there is a randomized way of doing something, there is a nonrandomized way that yields better results from the same data, but requires more thinking.'' I would only add the caveat that the better, nonrandomized way often enough seems to require much more thinking, and years of research.

• p. 545, end of section 17.10.5: Chapter 17 is one of those roughed-out chapters that Jaynes never really finished, and it's unclear where he intended to go with this comparison of the Bayesian vs. orthodox estimators using the orthodox criterion of performance. However, let's continue where Jaynes left off. Using the corrected versions of equations (17.129) and (17.130) [see errata above], the expected squared error for the Bayesian estimator is

whereas the expected squared error for the orthodox estimator is

the extra term coming from the bias of the estimator.

To better understand the behavior of , let us assume that for some integer , and let (a complete number of cycles) for ease of analysis. It is easily shown that . Using

and

and

we find that

This goes to zero quite rapidly; even for the minimum values of k=3 and n=1, giving N=3, we have . As a result, the difference in the expected squared error of the two estimators is usually small. Experimenting with different values for , , and the ratio , I found substantial advantage for the Bayesian estimator only for small values of and ; in particular, for I found the expected square error for the orthodox estimator to become much higher than that for the Bayesian estimator as gets smaller than 0.02. For other cases I found only very small differences, often favoring the orthodox estimator.