# Good Thinking: The Foundations of Probability and Its Applications

I. J. Good
Edition: NED - New edition
Pages: 352
https://www.jstor.org/stable/10.5749/j.ctttsn6g

1. Front Matter
(pp. i-iv)
(pp. v-vi)
3. Acknowledgments
(pp. vii-viii)
4. Introduction
(pp. ix-xviii)

This is a book about applicable philosophy, and most of the articles contain all four of the ingredients philosophy, probability, statistics, and mathematics.

Some people believe that clear reasoning about many important practical and philosophical questions is impossible except in terms of probability. This belief has permeated my writings in many areas, such as rational decisions, statistics, randomness, operational research, induction, explanation, information, evidence, corroboration or weight of evidence, surprise, causality, measurement of knowledge, computation, mathematical discovery, artificial intelligence, chess, complexity, and the nature of probability itself. This book contains a selection of my more philosophical and less mathematical articles...

5. Part I. Bayesian Rationality
• CHAPTER 1 Rational Decisions (#26)
(pp. 3-14)

I am going to discuss the following problem. Given various circumstances, to decide what to do. What universal rule or rules can be laid down for making rational decisions? My main contention is that our methods of making rational decisions should not depend on whether we are statisticians. This contention is a consequence of a belief that consistency is important. A few people think there is a danger that overemphasis of consistency may retard the progress of science. Personally I do not think this danger is serious. The resolution of inconsistencies will always be an essential method in science and...

• CHAPTER 2 Twenty-seven Principles of Rationality (#679)
(pp. 15-19)

In the body of my paper for this symposium I originally decided not to argue the case for the use of subjective probability since I have expressed my philosophy of probability, statistics, and (generally) rationality on so many occasions in the past. But after reading the other papers I see thatenlightenmentis still required. So, in this appendix I give a succinct list of 27 priggish principles. I have said and stressed nearly all of them before, many in my 1950 book, but have not brought so many of them together in one short list. As Laplace might have...

• CHAPTER 3 46656 Varieties of Bayesians (#765)
(pp. 20-21)

Some attacks and defenses of the Bayesian position assume that it is unique so it should be helpful to point out that there are at least 46656 different interpretations. This is shown by the following classification based on eleven facets. The count would be larger if I had not artificially made some of the facets discrete and my heading would have been “On the Infinite Variety of Bayesians.”

All Bayesians, as I understand the term, believe that it is usually meaningful to talk about the probability of a hypothesis and they make some attempt to be consistent in their judgments....

• CHAPTER 4 The Bayesian Influence, or How to Sweep Subjectivism under the Carpet (#838)
(pp. 22-56)

… There is one respect in which the title of this paper is deliberately ambiguous: it is not clear whether it refers to thehistoricalor to thelogicalinfluence of “Bayesian” arguments. In fact it refers to both, but with more emphasis on the logical influence. Logical aspects are more fundamental to a science or philosophy than are the historical ones, although they each shed lighten the other. The logical development is a candidate for being the historical development onanotherplanet.

I have taken the expression the “Bayesian influence” from a series of lectures in mimeographed form (#750)....

6. Part II. Probability
• CHAPTER 5 Which Comes First, Probability or Statistics (#85A)
(pp. 59-62)

The title of this note was selected so as to provide an excuse for discussing some rather general matters. Let us first consider the question of which of probability and statistics came first historically. This question is like the one about eggs and chickens. The question whether eggs or chickens came first could in principle be given a meaning by using arbitrarily precise definitions of eggs and chickens, and even then probably nobody would be able to answer the question.…

Let us consider some examples of statistical principles. For each of them we shall run into trouble by regarding them...

• CHAPTER 6 Kinds of Probability (#182)
(pp. 63-72)

The mathematician, the statistician, and the philosopher do different things with a theory of probability. The mathematician develops its formal consequences, the statistician applies the work of the mathematician, and the philosopher describes in general terms what this application consists in. The mathematician develops symbolic tools without worrying overmuch what the tools are for; the statistician uses them; the philosopher talks about them. Each does his job better if he knows something about the work of the other two.

What is it about probability that has interested philosophers? Principally, it is the question whether probability can be defined in terms...

• CHAPTER 7 Subjective Probability as the Measure of a Non-measurable Set (#230)
(pp. 73-82)

I should like to discuss some aspects of axiom systems for subjective and other kinds of probability. Before doing so, I shall summarize some verbal philosophy and terminology. Although the history of the subject is interesting and illuminating, I shall not have time to say much about it.

In order to define the sense in which I am using the expression “subjective probability” it will help to say what it is not, and this can be done by means of a brief classification of kinds of probability (Poisson, 1837; Kemble, 1941; #182).

Each application of a theory of probability is...

• CHAPTER 8 Random Thoughts about Randomness (#815)
(pp. 83-94)

In this paper I shall bring together some philosophical and logical ideas about randomness many of which have been said before in scattered places. For less philosophical aspects see, for example #643.

When philosophers define terms, they try to go beyond the dictionary, but the dictionary is a good place to start and one dictionary definition of “random” is “having no pattern or regularity.” This definition could be analyzed in various contexts but, at least for the time being, I shall restrict my attention to sequences of letters or digits, generically called “digits.” These digits are supposed to belong to...

• CHAPTER 9 Some History of the Hierarchical Bayesian Methodology (#1230)
(pp. 95-105)

In 1947, when few statisticians supported a Bayesian position, I had a nonmonetary bet with M. S. Bartlett that the predominant philosophy of statistics a century ahead would be Bayesian. A third of a century has now elapsed and the trend supports me, but I would now modify my forecast. I think the predominant philosophy wil be a Bayes/non-Bayes synthesis or compromise, and that the Bayesian part will be mostly hierarchical. But before discussing hierarchical methods, let me “prove” that my philosophy of a Bayes/non-Bayes compromise or synthesis is necessary for human reasoning, leaving aside the arguments for the specific...

• CHAPTER 10 Dynamic Probability, Computer Chess, and the Measurement of Knowledge (#938)
(pp. 106-116)

Philosophers and “pseudognosticians” (the artificial intelligentsia¹) are coming more and more to recognize that they share common ground and that each can learn from the other. This has been generally recognized for many years as far as symbolic logic is concerned, but less so in relation to the foundations of probability. In this essay I hope to convince the pseudognostician that the philosophy of probability is relevant to his work. One aspect that I could have discussed would have been probabilistic causality (Good, #223B) in view of Hans Berliner’s forthcoming paper “Inferring causality in tactical analysis,” but my topic here...

7. Part III. Corroboration, Hypothesis Testing, Induction, and Simplicity
• CHAPTER 11 The White Shoe Is a Red Herring (#518)
(pp. 119-120)

Hempel’s paradox of confirmation can be worded thus, “A case of a hypothesis supports the hypothesis. Now the hypothesis that all crows are black is logically equivalent to the contrapositive that all non-black things are non-crows, and this is supported by the observation of a white shoe.”

The literature of the paradox is large and I have myself contributed to it twice (##199, 245). The first contribution contained an error, but I think the second one gave a complete resolution. The main conclusion was that it is simply not true that a “case of a hypothesis” necessarily supports the hypothesis;...

• CHAPTER 12 The White Shoe qua Herring Is Pink (#600)
(pp. 121-121)

Hempel (1967) points out that I (#518) had misunderstood him and that in his context [of no background knowledge] a white shoe is not a red herring. But in my context (#245) it was a red herring; so in its capacity as a herring it seems to be pink. I shall now argue its redness even within Hempel’s context.

Let H be a hypothesis of the form that class A is contained in class B, for example, “all crows are black.” Let E be what I call a “case” of H, that is, a proposition of the form “this object...

• CHAPTER 13 A Subjective Evaluation of Bode’s Law and an “Objective” Test for Approximate Numerical Rationality (#603B)
(pp. 122-128)

This paper is intended in part to be a contribution to the Bayesian evaluation of physical theories, although the main law discussed, Bode’s law or the Bode-Titius law, is not quite a theory in the usual sense of the term.

At the suggestion of the Editor [ofJASA], a brief discussion of the foundations of probability, statistics, and induction, as understood by the author, is given in Part 2 [of the paper], in order to make the paper more self-contained. In Part 3 Bode’s law is given a detailed subjective evaluation. At one point the argument makes use of a...

• CHAPTER 14 Some Logic and History of Hypothesis Testing (#1234)
(pp. 129-148)

The foundations of statistics are controversial, as foundations usually are. The main controversy is between so-called Bayesian methods, or rather neo-Bayesian, on the one hand and the non-Bayesian, or “orthodox,” or sampling-theory methods on the other.¹ The most essential distinction between these two methods is that the use of Bayesian methods is based on the assumption that you should try to make your subjective or personal probabilities more objective, whereas anti-Bayesians act as if they wished to sweep their subjective probabilities under the carpet. (See, for example, #838.) Most anti-Bayesians will agree, if asked, that they use judgment when they...

• CHAPTER 15 Explicativity, Corroboration, and the Relative Odds of Hypotheses (#846)
(pp. 149-170)

In this paper I shall discuss probability, rationality, induction and the relative odds of theories, weight of evidence and corroboration, complexity and simplicity (with a partial recantation), explicativity, predictivity, the sharpened razor, testability and metaphysicality, and gruesomeness.

I agree with some of the things that Popper has said about several of these topics, in his stimulating writings, but I by no means have a Popper fixation.

I used to call myself a Bayesian when it was not misleading to do so. I have not changed my position, but the meanings of words change. As Winston Churchill said, his political opinions...

8. Part IV. Information and Surprise
• CHAPTER 16 The Appropriate Mathematical Tools for Describing and Measuring Uncertainty (#43)
(pp. 173-177)

In this paper I shall be concerned less with decisions that are made than with those that are rational. But the paper will have some relevance to economics since the decisions ofHomo Sapiensare not entirely irrational. Moreover the “theory of rational decisions” or rational behavior includes a completely general theory of probability and is therefore applicable to economics, whether human decisions are rational or not. The economist himself should attempt to be rational.

… The function of the theory is to introduce a certain amount of objectivity into your subjective body of judgments, to act as shackles on...

• CHAPTER 17 On the Principle of Total Evidence (#508)
(pp. 178-180)

Ayer (1957) raised the question of why, in the theory of logical probability (credibility), we should bother to make new observations. His question was not adequately answered in the interesting discussion that followed.… The question raised by Ayer is related by him to a principle called by Carnap (1947), “the principle of total evidence,” which is the recommendation to use all the available evidence when estimating a probability. Ayer’s problem is equally relevant to the theory of subjective probability, although, as he points out, it is hardly relevant to the theory of probability in the frequency sense.

In this note,...

• CHAPTER 18 A Little Learning Can Be Dangerous (#855)
(pp. 181-183)

It has been proved that, under certain assumptions, it pays you “in expectation” to acquire new information, when it is free. A precise formulation of this thesis, together with a proof, was given in #508 with historical references to Carnap, Ayer, Raiffa and Schlaifer, and Lindley. The “expectation” in this result isyour ownexpectation. In the present note it will be pointed out that the result can break down when the expectation is computed by someone else. It is of course familiar that an experiment can be misleading by bad luck, but this is not by itself a justification...

• CHAPTER 19 The Probabilistic Explication of Information, Evidence, Surprise, Causality, Explanation, and Utility (#659)
(pp. 184-192)

My purpose in this paper is to review some of my life’s work in the mathematics of philosophy, meaning the application of mathematics in the philosophy of science. Apart from the clarification that the mathematics of philosophy gives to philosophy, I have high hopes for its application in machine intelligence research, just as Boolean logic, a hundred years after its invention, became important in the design of computers. I think philosophy and technique are both important, but I do not intend to argue the case for the use of subjective probability on this occasion. I would just like to quote...

• CHAPTER 20 Is the Size of Our Galaxy Surprising? (#814)
(pp. 193-194)

Eddington (1933/52, p. 5) after pointing out that the earth and the Sun are of middling size,quaplanet and star respectively, says, “So it seems surprising that we should happen to belong to an altogether exceptional galaxy.” It is not quite as exceptional as Eddington thought; thus van de Kamp (1965, p. 331) said that the photometric studies by Stebbins and Whiteford in 1934 “did much to do away with the notion that our galaxy was a ‘continent’ and that the others were ‘islands.’ ” But still our galaxy is a large one, and Eddington’s remark raises an interesting...

9. Part V. Causality and Explanation
• CHAPTER 21 A Causal Calculus (#223B)
(pp. 197-217)

This paper contains a suggested quantitative explication of probabilistic causality in terms of physical probability. (Cf. Reichenbach, 1959, Chap. 3; Wiener, 1956, pp. 165-190.) The main result is to show that, starting from very reasonable desiderata, there is a unique meaning, up to a continuous increasing transformation, that can be attached to “the tendency of one event to cause another one.” A reasonable explicatum will also be suggested for the degree to which one event caused another one. It may be possible to find other reasonable explicata for tendency to cause, but, if so, the assumptions made here will have...

• CHAPTER 22 A Simplification in the “Causal Calculus” (#1336)
(pp. 218-218)

A quantitative explication was given in #223B forQ(E:F), defined as the degree to which an event F tends to cause a later event E. The argument depended on assigning to a causal network a “resistance”Rand a “strength”S. By considering a parallel network, a functional equation was found forS(p. 205). On the other hand, by considering a special causal Markov chain F → E₁ → E₂, in which$P(F)=x$,$P(E_1|F)=p_1$,$P(E_1|\overline{F})=0$,$P(E_2|E_1)=p_2$,$P(E_2|\overline{E}_1)=0$, and then coalescing E₁ and E₂ into a single event$E=E_1 & E_2$,we can obtain the further functional...

• CHAPTER 23 Explicativity: A Mathematical Theory of Explanation with Statistical Applications (#1000)
(pp. 219-236)

ByexplicativityI mean the extent to which one proposition or event F explains why another one E should be believed, when some of the evidence for believing E might be ignored. Both propositions might describe events, hypotheses, theories, or theorems. For convenience I shall not distinguish between an event and the proposition that states the event. In practice usually only putative explanations can be given and this is one reason for writing “should be believed” instead of “is true,” but explanation in the latter sense can be regarded as the extreme case where belief is knowledge.

The word “explanatoriness”...

10. References
(pp. 239-248)
11. Bibliography: Main Publications by the Author
(pp. 251-266)
12. Subject Index of the Bibliography
(pp. 269-313)
13. Name Index
(pp. 313-317)
14. Subject Index
(pp. 317-332)
15. Back Matter
(pp. 333-333)