It worked for Maid Marion and Robin Hood.

Again, I greatly appreciate your summaries and responses. This took a lot of time and thought, a big help to me. So I owe you a drink if ever we meet.

(1) You are right, I’m actually articulating ways to get around the very problem he’s talking about (which I certainly acknowledge: note, for example, my discussion of it on pp. 110-14). Where we can’t, we can’t claim to know anything (Axiom 3, page 23; also Axiom 5, page 28). His question how we derive “the probability of the New Testament” is unclear (what exactly does he mean?), but I address something quite close to it on pp. 77-79, using GMark rather than the whole NT (and I get even more specific in my discussion of emulation criteria: pp. 192ff.). If he ignores that, then he is not critiquing my book, but some straw man of it.

In any event, the problem is addressed by (a) being less ambitious in what you will attempt to prove (a lesson historians often need to learn anyway) and (b) being more clear and precise in laying out what evidence it is that you think produces a differential in the consequents (most evidence simply will not, and therefore can be ignored). Thus “the whole NT” is irrelevant to historicity; likewise even “the exact content of Mark.” We will need to get much more specific than that (What is it in Mark that makes any significant difference? And why? We can model your answer using BT). And in my next book I do that.

Ironically, he is committing the very fallacy here that I warn against in the book: if we cannot estimate P(E|H), then historical knowledge is simply impossible. Because all historical conclusions implicitly rely on estimates of P(E|H) (and/or P(E|~H)). All historical conclusions ever reached before now, and all that will ever be reached by anyone ever. Thus, if BT can’t solve this problem, no method can. And if he thinks otherwise, it’s his task to produce that method, a method by which (a) a historian can get a conclusion about history without (b) ever relying on any implicit assumption about any P(E|H) or P(E|~H). Good luck with that. Because it can’t be done. If he’d tried it, he’d know.

(2) You have already effectively rebutted this point. He’s just unable to grasp the way BT has to be employed in the humanities, and what it takes to translate how people in the humanities already reason, into terms definable within BT.

(3) You have already effectively rebutted this point, too. He seems to be conflating “not knowing x precisely” with “not knowing x at all.” I explicitly address this fallacy several times in the book (early in chapter six, and in my discussion of arguing a fortiori: pp. 85ff.). In short, historians don’t need the kind of precision Ian seems to want. In fact, as I explain in the book, that they can’t get it even if they wanted it is precisely what demarcates history from science: pp. 45-49.

(4) You have already effectively rebutted this point as well. Like you say, we’ll have to see what he comes up with (do let me know). But I suspect he is ignoring everything I explained in the book about the fact that historians have to live with some measure of ambiguity and uncertainty far beyond what scientists deal with; that’s what makes it different from science (again: pp. 45-49; and again, Axioms 3 and 5). He seems to either want history to be physics, or he seems to think I want history to be physics. The one is foolish; the other betrays a failure to really read my book (e.g. pp. 60-67).

(5) As you note, his criticism here is too vague to even know what he means. At present no response is required.

(6) doesn’t seem mathematically informed, unless he means errors in P(E|H) and P(E|~H) can pull in different directions; otherwise, P(H) and P(~H) always sum to 1, so they actually consist of a single estimate, not two, so they can’t pull against each other. If you estimate P(H) against yourself, you have also estimated P(~H) against yourself, by definition. And if you do the same with both P(E|H) and P(E|~H), they can’t ever pull in opposite directions, either. The compounded error between them will then only make your argument more a fortiori. So there isn’t any discernible criticism here.

So far I am not impressed. It doesn’t look like he’s taking any argument in the book seriously. It’s almost like some Cato Institute effort at refuting a policy that they don’t really have any valid argument against but they have to refute it anyway so they come up with whatever trivia and handwaving they can think of.

Part 3.

Now I’ll turn my attention to Ian’s 2nd post: http://irrco.wordpress.com/2012/09/12/probability-theory-introductio/

Most of his post is taken up with an explanation of conditional probability and Bayes’s theorem, which is actually pretty good; it would’ve been a good idea to devote a few pages in your book to something like this. But so far there’s no criticism of your book, or even really much mention of it. I’ll start the numbering over again from 1 to list his criticisms, which eventual start to appear.

1. For historical questions, there usually is no easy way to calculate, or even estimate in many cases, P(E). He says, “I’ve never seen any credible way of doing so. What would it mean to find the probability of the New Testament, say?…I’m not sure I can imagine a way of calculating either P(H∩E) or P(E|H) for a historical event. How would we credibly calculate the probability of the New Testament, given the Historical Jesus? Or the probably of having both New Testament and Historical Jesus in some universe of possibilities?”

[He writes as if he didn’t actually read your book, although I know that he has, because he is going by his own knowledge of using Bayes’s theorem and not looking at the examples where you apply it. As I pointed out in a previous post, you get around the issue of estimating P(E) and P(H) by conditioning on general statements of the evidence, so that you’re calculating P(H|F) and P(E|(~H)&F). These need to be estimated somehow, but may be easier since the evidence F is more commonly encountered. It’s funny that he observed you doing this in his first post but then never thought through the implications for the rest of his posts. I suggest that if you explained what you were doing mathematically and how it differed from the way scientists usually state and apply Bayes’s theorem there would not be so much confusion.]

2. He then asks whether using the expanded-denominator version of the formula [which he again annoyingly attributes to you and Craig as if you are the only ones who write it this way, and he’s not talking about the inclusion of the background here either] could ameliorate this problem with estimating P(E):
“This is just a further manipulation. The bottom of this equation is still just P(E), we’ve just come up with a different way to calculate it, one involving more terms. We’d be justified in doing so, only if these terms were obviously easier to calculate, or could be calculated with significantly lower error than P(E).”
[He doesn’t seem to think that P(E|~H) is often easier to calculate than P(E), nor does he notice the advantage of using a variable, P(E|~H), that is independent of the other 2, P(H) and P(E|H), unlike P(E), which isn’t. Perhaps if he looked at your examples or even the standard example of medical testing he’d see why the expanded form often works better.]

3. His next criticism is a bit bizarre, as he complains about having to use estimates:
“If these terms are estimates, then we’re just using more estimates that we haven’t justified. We’re still having to calculate P(E|H), and now P(E|~H) too. I cannot conceive of a way to do this that isn’t just unredeemable guesswork. And it is telling nobody I’ve seen advocate Bayes’s Theorem in history has actually worked through such a process with anything but estimates.”
[Of course you use estimates – even in the sciences one does. Unless you’re doing a problem with dice or cards, the numbers one plugs in are always estimates. And you present several examples where you attempt to justify your estimates. It would help if he actually addressed one to show why it was nothing but “unredeemable [sic] guesswork.”]

He sums up in a similar vein:
“So ultimately we end up with this situation. Bayes’s Theorem is used in these kind of historical debates to feed in random guesses and pretend the output is meaningful.”

4. As a teaser for what he intends to write about in a later post, he states why he thinks a fortiori reasoning doesn’t work:

“But, you might say, in Carrier’s book he pretty much admits that numerical values are unreliable, and suggests that we can make broad estimates, erring on the side of caution and do what he calls an a fortiori argument – if a result comes from putting in unrealistically conservative estimates, then that result can only get stronger if we make the estimates more accurate. This isn’t true, unfortunately, but for that, we’ll have to delve into the way these formulas impact errors in the estimates. We can calculate the accuracy of the output, given the accuracy of each input, and it isn’t very helpful for a fortiori reasoning.”

[His characterization of your a fortiori argument – “if a result comes from putting in unrealistically conservative estimates, then that result can only get stronger if we make the estimates more accurate” – is easily demonstrably true: P(H|E) is monotonically increasing in P(H) and P(E|H) and decreasing in P(E|~H), so it follows immediately that if one takes a maximum estimate for P(H) and P(E|H) and a minimum one for P(E|~H) then the estimate for P(H|E) using Bayes’s theorem is a maximum, and similarly one derives a minimum for P(H|E). Furthermore, tightening the possible ranges for these variables yields a tighter range for P(H|E), so the a fortiori argument is in fact valid.

I think what he is intending to say is that while one may in principle get a possible range for P(H|E) from possible ranges for the other probabilities, in practice, the range turns out to be too wide to be useful. Whether that turns out to be the case will be seen when you actually try to apply it.]

His final comment supports my reading of him: “It doesn’t take much uncertainty on the input before you loose any plausibility for your output.”

[If this were true it would true for all uses of BT, so how does he account for its use in science? It’s not like those estimates for false positives in DNA testing (in his example) lack errors.]

5. He hints at another problem here but says he’ll explain some other time:
“[I]n subjective historical work, sets that seem not to overlap can be imagined to overlap in some situations. This is another problem for historical use of probability theory, but to do it justice we’ll need to talk about philosophical vagueness and how we deal with that in mathematics.”

6. There are 4 footnotes to the post, only the last of which could be taken as a criticism of using BT with uncertainty (specifically the form of BT with the denominator expanded). Here it is in full:
“You’ll notice, however, that P(E|H)P(H) is on both the top and the bottom of the fraction now. So it may seem that we’re using the same estimate twice, cutting down the number of things to find. This is only partially helpful, though. If I write a follow up post on errors and accuracy, I’ll show that errors on top and bottom pull in different directions, and so while you have fewer numbers to estimate, any errors in those estimates are compounded.”

[Since P(E|H)P(H) appears in both the numerator and denominator, an increase in it would increase both the numerator and denominator, so the effects from each would offset, not compound! But P(H) appears in the second term in the denominator, so it is not quite that simple. Changes in P(E|H) would partially cancel in the numerator and denominator, making P(H|E) less sensitive to changes in it, but for P(H) depending on the ratio of P(E|H) and P(E|~H), the effect of changes in the prior on the top and bottom of the fraction can indeed compound, but not for the reason he stated.]

That is his whole criticism. Basically he doubts that BT can be practically used in BT because he feels the inputs are too subjective and have uncertainties that are too wide. We’ll have to wait for your next book to see if you can pull it off.

(4) remains unintelligible to me. It seems like a total non sequitur. If he doesn’t deal with the formal deductive argument at all (pp. 106-14), it’s useless as a critique. Indeed, that is the point of formalizing the argument: so critics would be able to identify any errors that invalidate the conclusion. If he’s not even going to do that…?

(5) is just ad hominem. I am sure I don’t know all the arguments published on this point (it must be in the thousands, counting books and articles), and if anyone has articulated the same conclusion before, I’d love to accumulate those references (so if he provides them, please copy them here). That is neither here nor there. Whether he dislikes my tone or thinks it’s arrogant or condescending is not a valid rebuttal to whether it is correct. I also don’t think it’s an objective assessment, since I am responding to the debate as framed in recent literature by leading professors of mathematics (some of which I actually cite in the book), so he is actually critiquing them for not knowing the solution I propose. After all, if they don’t know about this unoriginal argument of mine (and if they did, they’d have resolved the debate with it in the literature I cite), then why is it condescending to suggest it to them?

(6) I’ll certainly concede that my colloquial discourse will lead to ambiguities that chafe mathematicians; but this is precisely the kind of shit they need to get over, because they are simply not going to be able to communicate with the humanities if they don’t learn how to strategically use ambiguity to increase the intelligibility of the concepts they want to relate. It’s like Heisenberg’s Uncertainty Principle: you can have precision with unintelligibility to almost everyone but extremely erudite specialists, or you can have ambiguity with intelligibility to everyone else. The more ambiguity, the greater the clarity, but the lower the precision. This is really a fundamental principle of all nonfiction literature, and a basic principle of science popularization. I have a particular audience. I am writing for them.

(7) the difference between me and Craig is revealed by all of Ian’s qualifying remarks (like “coincidentally did not seem to result in incorrect conclusions,” a backhanded way to admit I’m actually using it correctly, unlike Craig, thus negating his analogy). As you know, the whole point of my book is to prevent Craig-style abuses by making it clear how to use it correctly and how to spot its being used incorrectly. And indeed I repeatedly emphasize that anyone who wants to use it needs to be clear in how they are using it so it can be vetted and critiqued, thus avoiding the “dazzling with numbers” tactic by arming the reader with the ability to see through it.

So, overall, all I see here is an out-of-touch mathematician who doesn’t understand how to communicate math concepts to humanities majors, and thus ironically is grossly condescending to humanities majors (whom I’m actually writing for) in the very effort to accuse me of being condescending to mathematicians (whom I’m not writing for). Not a very useful critique, IMO.

Part 2.

So far all of his criticisms have been stylistic (either about how equations were expressed or how there were explained), rather than truly mathematical. The rest of his post is no different.

4. He takes you to task for using Bayes’s formula as a synonym for Bayesian reasoning. In particular, he ridicules this quote from your book: “any historical reasoning that cannot be validly described by Bayes’s Theorem is itself invalid.” His objection seems to be that, while BT is of course true, there are other equations that one could derive from the definition of conditional probability that couldn’t be derived from BT.

[Actually, one could derive the formula for conditional probability from BT if one had some sort of definition of conditional probability that implied P(AB|B) = P(A|B) and P(A|AB) = 1: If one assumes BT (i.e., P(H|E) = P(E|H)P(H)/P(E)) then P(H|E) = P(HE|E) = P(E|EH)P(HE)/P(E) = P(HE)/P(E). But this is besides the point, since you only limited your claim to “historical reasoning,” a term which you unfortunately didn’t define.]

He further states your attempt to prove this “laughable” assertion is not credible, but gives no other reason than what I just stated above.

5. He then goes on to discuss your “cheeky” proposal to unify frequentist and Bayesian interpretations of probability. His criticisms here are that your proposal is “unnuanced” and presented as if were original, when it is not. (Not that he is accusing you of taking credit for others’ ideas but rather of being possibly unaware of previous work in the field.) He also states that this “hubris” is typical of “a tone of arrogance and condescension that I consistently perceived throughout the book.”

6. As a final comment on the mathematics he raises 2 issues but doesn’t elaborate on either: “I felt there were severe errors with his arguments a fortiori…and his set-theoretic treatment of reference classes was likewise muddled (though in the latter case it coincidentally did not seem to result in incorrect conclusions).” This is the entire extent of his discussion of these, from his perspective, problems.

[I’ll more to say about a fortiori reasoning in my next posted comment.]

7. In his conclusion he has some positive things to say about the book:
“Outside the chapters on the mathematics, I enjoyed the book, and found it entertaining to consider some of the historical content in mathematical terms….History and biblical criticism would be better if historians had a better understanding of probability….

I am also rather sympathetic to many of Carrier’s opinions, and therefore predisposed towards his conclusions. So while I consistently despaired of his claims to have shown his results mathematically, I agree with some of the conclusions, and I think that gestalts in favour of those conclusions can be supported by probability theory.”

But here is his final critique:
“But ultimately I think the book is disingenuous. It doesn’t read as a mathematical treatment of the subject, and I can’t help but think that Carrier is using Bayes’s Theorem in much the same way that apologists such as William Lane Craig use it: to give their arguments a veneer of scientific rigour that they hope cannot be challenged by their generally more math-phobic peers.”

As you can see, he hasn’t presented any concrete objection to the mathematics in the book – just the way the mathematics was presented and explained and the overall tone of the book.

Thanks. This is helpful.

(1) and (2) are pedantic and belie an ignorance of pedagogical principles and therefore are not really a worthwhile criticism. For example, a humanities major is not going to understand what P(E) is or how it derives from the sum of all probabilities; they are also going to have a much easier time estimating P(E|H) and P(E|~H) because estimating those probabilities is something they have already been doing their whole lives–they just didn’t know that that’s what they are doing. Much of my book is about pointing this out. This is also why I keep B in all terms (as I even explain in an endnote), so that laymen won’t lose sight of its role at every stage. And why I don’t bore or waste the reader’s time by explaining how BT was derived or proved; instead, I refer interested readers to the literature that does that. That’s how progress works: you don’t repeat, but build on existing work. I don’t have to prove BT or explain how it was derived; that’s already been done. I just have to show how BT can be applied to my field (history).

(3) likewise ignores the fact that historians need to do different things than scientists. Thus the way I demarcate B from E is what is most useful to historians (again, I even explain this explicitly in an endnote). Historians are testing two competing hypotheses: that a claim is true vs. the claim is fabricated (or in error etc.), but to a historian that means the actual hypotheses being tested are “the event happened vs. a mistake/fabrication happened,” which gives us the causal model “the claim exists because event happened vs. the claim exists because a mistake/fabrication happened”; and B contains the background evidence relating to context (who is making this claim, where, to what end, what kind of claim is it, etc.), which gives us a reference class that gives us a ratio of how often such claims turn out to be true usually, vs. fabricated (etc.). You then introduce additional indicators that distinguish this claim from those others, to update your priors. And you can do that anywhere in the chain of indicators (so you can start with a really general reference class, or a really narrow one–and which you should prefer depends on the best data you have for building a prior, which historians rarely have any control over, so they need more flexibility in deciding that).

You could, if you wanted, build out the whole bayesian chain (e.g. see endnote 11, page 301), all the way from raw data, but why should historians trouble themselves with that? They already have context-determined estimates of the global reliability of statements based on their experience. If they get into an argument over conflicting estimates there, then they can dig into the underlying assumptions and build out the whole bayesian case from raw data (or at least from further down the chain of underlying assumptions). But it’s a massively inefficient waste of their time to ask them to do that all the time, or even a lot of the time.

Ultimately, all bayesian arguments start in the middle somewhere. If they didn’t, they’d all have priors of 0.5 (or whatever ramifies equally the spread of all possible hypotheses). Ian might prefer to start somewhere past the assembly of raw sensory data and toward the frequency-sets of basic event-occurrences (so he would try to answer the question “did Winston Churchill exist?” by starting with questions like “what is the physical probability that a man named Winston Churchill would be born in time to be Prime Minister of England during WWII?,” which would be a ridiculous way to proceed), but even that is doing what I am doing (he is skipping a step: in this case, how we know the frequency data about names is correct, given a certain body of sensations Ian experiences). Historians usually skip all the science steps. Because they’re doing history. Not science (in the narrow sense). But one can always go back in and check those steps. If you had to for some reason.

In short, historians need to be more flexible in how they model questions in Bayes’ Theorem. Ian’s pedantry wouldn’t help them at all. Because it really doesn’t matter how you build the model. As long as you can articulate what you are doing, and it’s correct (as I explain in chapter six, especially). Because then it can be vetted and critiqued. Which is all we want. And that is all my book arms the historian to do. And that’s all she needs to get started.

Indeed, having these conversations (about what models to use and what frequencies fall out of them, and thus how to define H and demarcate E from B) is precisely what historians need to be doing. My book gives them the starting point for doing that. Because otherwise it won’t be the same for every question, because the data-availability differs for each case and thus historians have to demarcate differently in different cases.

He has written 2 posts on your book so far, with a promised 3rd to come. His 1st post starts out by stating:
“I will be unable to deal with every mathematical problem in the book in a short review, so will limit myself to issues arising from the first mathematical chapter [Chapter 3].”

1. Your form of Bayes’s theorem is “confusing and unnecessarily complex.” His preferred form of BT is P(H|E) = P(E|H)P(H)/P(E).

He has 2 objections to your form of the formula: (1) The denominator has been expanded, which he feels is unnecessary. [I pointed out to him that most textbooks actually state BT with the expanded denominator and that most applications of the theorem that one encounters use it in that form as well, but he replied that in his own field (AI) he only needs P(E), so maybe this is just his personal preference from his own experience. Moreover, you give both forms of BT in the appendix.]

(2) Adding the background explicitly is “highly idiosyncratic,” “condescending,” and “irksome,” reminiscent of William Lane Craig. [I agree that it is unusual and unnecessary, but it is not wrong, as even he acknowledges. Moreover, it is easier to transition to the form of the equation that you actually are using, whereby some of the evidence is incorporated into the background, but you never make this explicit.]

2. He also criticizes you for failing to explain the derivation of BT or discussing the definition of conditional probability generally: “While Carrier devotes a lot of ink to describing the terms of his long-form BT, he nowhere attempts to describe what Bayes’s Theorem is doing. Why are we dividing probabilities? What does his long denominator represent?” Consequently BT becomes “a kind a magic black box.”

He then states cryptically: “In this Carrier allows himself to sidestep the question whether these necessarily true conclusions are meaningful in a particular domain. A discussion both awkward for his approach, and one surely that would have been more conspicuously missing if he’d have described why BT is the way it is.”

[I’m not sure what point Ian is making here, but I think he is alluding to the difficulty of calculating P(E), which he discusses in his 2nd post. As I’ll point out later, his criticism is based on a misunderstanding of how you are applying BT.]

3. His next criticism is one that I partially share: “Carrier correctly states that he is allowed to divide content between evidence and background knowledge any way he chooses, provided he is consistent. But then fails to do so throughout the book.” He cites as an example, p. 51, where the prior is defined to explicitly include evidence in it. [The prior should be the probability that the hypothesis is true before any consideration of the evidence.] He continues with this quote from your book [which I also find objectionable]: “For example, if someone claims they were struck by lightening five times … the prior probabilty they are telling the truth is not the probability of being struck by lightening five times, but the probability that someone in general who claims such a thing would be telling the truth.”

This is his response: “This is not wrong, per se, but highly bizarre. One can certainly bundle the claim and the event like that, but if you do so Bayes’s Theorem cannot be used to calculate the probability that the claim is true based on that evidence. The quote is valid, but highly misleading in a book which is seeking to examine the historicity of documentary claims.”

Ian’s point is that BT is defined as P(H|E) = P(E|H)P(H)/P(E), with the prior by definition being P(H), i.e., without any conditioning on the evidence. In the example of the claim of being struck by lightning 5 times, the hypothesis H would normally be “someone was struck by lightning 5 times” and the evidence E would be “he claims to have been struck by lightning 5 times.” Then the prior would indeed be the probability of being struck by lightning 5 times. You instead have as your prior the conditional probability that someone is telling the truth when he claims “in general such a thing,” which would be (part of) the evidence for the claim.

[Effectively what you are doing is treating the evidence E (“someone claims to have been struck by lightning 5 times”) as if it were an intersection of E with a larger set, F (“someone claims such a thing”), that is more general, and then absorbing F into B. The form of BT that you are using is then:
P(H|EB) = P(E|HFB)P(H|FB)/P(E|FB).

Now, this trick may be useful for actually calculating P(H|BE), since then you avoid having to calculate P(H|B) or P(E|B), but you haven’t been entirely upfront with the reader about what you are doing.] Moreover, as he points out in the quote above, you still are left with P(H|FB), which is very similar to P(H|EB), so you haven’t really used BT to solve the problem.

[In the actual applications in your book, what you generally do is use BT in this way to reduce the problem to conditional probabilities with more general evidence, and then use an empirical frequency to estimate them. This avoids the problem that he raises about having to estimate, say, the probability of the NT existing.]

To be continued….

“What would it mean to find the probability of the New Testament, say?”

The way that Dr. Carrier uses Bayes’s theorem avoids these sorts of difficulties. This is because he is usually conditioning all the terms in the numerator and denominator on a more general event, such as “religious books like NT,” and then ends up with priors like “how often in general are protagonists in religious texts like the NT mythical?” rather than “what is the a priori probability that Jesus didn’t exist?” Of course, this then makes the prior itself a conditional probability, which could either itself be computed by another application of Bayes’s formula or else by a direct frequency calculation.

Noen,

You’re right! Zeus and Hera must’ve been real after all!

I answer those questions in Proving History. I have quite a lot to say about it there (in fact arguably half the book is about answering that), so if he is not responding to any of it, his critique is useless.