It’s not common that major, respected academic journals exhibit a catastrophic failure in their peer review process. But it happens. Everything from creationist dribble to just about anything in any field has “slipped past” peer review protocols in legitimate journals—usually leading to retractions when caught (since the epistemic reliability of a journal literally hinges on what they will or won’t retract). Which means you need to know how to detect when this happens, so you can continue to rely on the otherwise reliable source-vetting that peer review standards are intended to create (see my Vital Primer on Media Literacy and On Evaluating Arguments from Consensus). Most often you’ll benefit from the expert community already catching this and thus exposing it. But that can take time. And in humanities fields it is very common for no one to notice, and thus for a bogus paper to survive undetected.

The example I use today is not such. Since it pushed an extremely alarming position in a controversial subject many people had constant eyes on, it was caught and exposed as fraudulent almost immediately upon the annunciation of its acceptance for publication. But the question of how such a paper survived peer review at a serious journal (rather than a fake journal whose peer review is a joke, a problem plaguing all sciences and humanities) remains. And explaining that gives us lessons in what peer review is supposed to do in the field of history, and thus what constitutes an actual failure of it; how we as mere citizens can sleuth such failures on our own when we need to; and why (this may come as a surprise here) feminism is necessary to the epistemic reliability of every field of knowledge. Yes. I will elucidate all three propositions, proving even the third.

The Set-Up

Recently a research paper by J. Mark Ramseyer, a Harvard professor of Japanese law, passed peer review and was accepted for publication by the International Review of Law and Economics (and released online before going to print; likely to be retracted, it probably will not appear in the print issue). The paper is titled “Contracting for Sex in the Pacific War” and argues that the widely-confirmed Japanese mass-rape atrocity, in which literally tens (and possibly hundreds) of thousands of women (especially Koreans; Korea then being an occupied Japanese colony) were forced into brutal sex slavery by the Japanese military in WWII, is a “fiction” that never happened and that all those women consented to and signed lucrative prostitution contracts they could depart from at any time, and that this can be proved with “Game Theory” (he also repeated these assertions in an editorial published in the Japanese nationalist far-right web forum Japan Forward, which may indicate something of his own political leanings).

Several experts, including Nobel laureates, have compared his argument to Holocaust denial, and that comparison is spot on, morally and epistemically. Indeed Ramseyer’s paper was so grossly in violation of even basic epistemic standards as to be fairly described as fraudulent (it fakes evidence and deliberately misquotes sources, as well as consciously omits contrary evidence that is in fact overwhelming). In my opinion, Ramseyer should lose his position at Harvard and be blacklisted from any future position at any respectable university; and not because he “dared” to challenge the factual reality of a mass atrocity against women, but because no scholar who engages in this kind of epistemic dishonesty should ever again be trusted as a scholar, or allowed to teach. This paper demonstrates he is literally a fraud. That should end your career.

To catch up on all the ins and outs of this controversy there is a good article by The Associated Press (reproduced at NBCNews.com) and another by Anne Branigin at The Lilly (a publication of The Washington Post) and a solid Op Ed at Harvard’s campus periodical The Crimson, as well as some crucial gotcha-style sleuthing reported on by Cho Ki-weon & Kim So-youn for the South-Korean newspaper The Hankyoreh. Most importantly, there is a letter of demonstration of Ramseyer’s fraud signed by over a thousand experts in relevant fields worldwide: “Letter by Concerned Economists Regarding ‘Contracting for Sex in the Pacific War’ in the International Review of Law and Economics,” which I’ll refer to as the Economists Letter. Likewise a Statement by Andrew Gordon and Carter Eckert, Harvard professors who did their own fact-check of Ramseyer’s paper and found alarming evidence of malfeasance. Everything to follow comes from these sources or directly from Ramseyer’s writings or my own personal and professional knowledge.

The Standards

The Economists Letter correctly concludes “this article” by Ramseyer constitutes academic malpractice, and indeed “goes well beyond mere academic failure or malpractice in its breach of academic standards, integrity, and ethics.” And it makes clear “our chief concern is” that it “attempts to use the language of economics to make historical claims that have no basis in evidence,” and also does so in aid of immoral and unconscionable ends, “using economics—more specifically game theory and law and economics—as a cover to legitimize horrific atrocities.” They make clear the difference between opposing such immoral fraud and opposing academic freedom. “Academics should not shy away from analyzing controversial and disturbing topics” nor be intimidated or punished for doing so. But that is not the issue here. Here the issue is epistemic malpractice. That it was deployed in furtherance of misogynistic nationalist propaganda only makes it worse.

Actual evidence (and it is vast, extending from recovered state and military documents from Japan to a huge database of eyewitness testimony) demonstrates quite conclusively that “most” of the tens or hundreds of thousands of women pressed into “comfort service” were “between eleven and twenty” years old (yes, you read that right), they were transported into hundreds of brothels by “Japanese military vessels, under the supervision of military police,” “recruitment methods included abduction, deception, threats, and violence,” and the victims of this process endured widespread “rape, forced abortions, physical torture, and sexually transmitted diseases” and most (informed estimates come out around 75%) “died from this experience” (many were mowed down en masse by the Japanese during military retreats, to conceal evidence of their war crimes). Though long denied by the Japanese government, overwhelming evidence uncovered in subsequent lawsuits decades ago forced the Japanese state to admit to it all. Several independent legal investigations of Japanese war crimes have reached the same conclusion. The overwhelming consensus of experts in Japanese and Korean history concurs.

If you are going to challenge a historical consensus built on such a vast and multiply-corroborated network of evidence, you had better damn well have the goods. And peer reviewers are responsible for ensuring that. I’ve discussed how peer review works in the field of history before. Their principal role is to “pre-vet” a paper or book, saving the rest of us the laborious time of doing so. Passing peer review thus signals a work is worth our time reading, considering, and engaging with, because it meets the minimal standards for that. Passing peer review thus does not indicate the conclusion a paper or book argues is “true”; peer reviewers are not tasked with nixing anything they disagree with. They are only tasked with answering basic questions like “are there any obvious errors of reasoning or method,” “is enough evidence presented to establish the conclusions reached,” “are there any obvious omissions or misrepresentations of fact,” “is this author familiar with and engaging with established knowledge and positions in the subject,” and “does this thesis comport with well-known background facts.” For examples of general peer review standards see the PLOS Guide and the Wiley Guide. Everything they say applies to history manuscripts as much as any in science.

Peer review is often blind (single or double), meaning the author is usually not told who the peer reviewers are, and often the peer reviewers are not told who the author is (that information will be stripped from the headline, for example). Blinding peer review is not necessary, but it does allow reviewers to be more honest without community or professional backlash, and it can forestall some biased outcomes (resulting from professional grudges for example). A serious journal, one whose peer review process is considered of high quality, will only use reviewers who have relevant PhDs and publication histories (meaning, they have gotten their own work in the same or closely related topics published through peer review). And it is standard to have at least two, operating independently of each other, as such duplication significantly reduces the risk and impact of bias and error.

And finally, it is very common for an academic work to be rejected (often for trivial reasons, like word length or a belief the work is redundant or not subject-suited to the journal or publisher), then revised and resubmitted to the same or other journals or publishers, going through multiple venues until accepted. However, this will not operate merely as forum shopping. With every peer review (resulting in a rejection or acceptance), there will be a peer reviewer’s report (in fact, more than one), and if it contains any critique of the work (reasons for rejection, or conditions for acceptance, or even just recommendations that fall short of requirements), this will be provided to the author. Who can then use that information to improve the work (or even have to). Which is one of the principal benefits of peer review as a process: anything that passes peer review (at any respectable journal or publisher), you know has gone through this process, and therefore will be of higher quality—it has already faced and been revised in light of expert criticism. This makes peer reviewed work substantially more reliable than “just anything published on the internet,” or through a non-academic publisher, or one of dubious credentials (like the many fraudulent and shoddy academic journals there are out there: on which, see “How Do You Know a Paper Is Legit?”, “How Do You Know a Journal Is Legitimate?”, and “Scholarly Articles: How Can I Tell?”).

This does not mean work that isn’t peer reviewed is unreliable; just that it requires more critical vetting from the reader themselves, which means it is safe to dismiss it if you don’t have or want to burn time on doing that before trusting it, or lack the needed skills and professional knowledge to do it (and no one who does has already done this and made available their critical report or assessment). Peer review is like a check mark that says “that preliminary vetting by real experts has been done and it passed.” Which helps you allocate your time more efficiently, zeroing in on stuff highly likely to be worth taking seriously. Peer review is thus one example of the core advantage of civilization generally: utilization of specialized expertise and the division of labor.

Which is all great, until it fails. How did the Ramseyer article even pass peer review? Literally hundreds of serious experts are actually asking this question now. It is a shocking failure of process. The International Review of Law and Economics professes it is now conducting an investigation to find out how this happened. Indeed, their reputation has been seriously tarnished by this, and it might not recover—they will need to earn back academic trust by showing what mistakes occurred and how they plan to ensure they won’t be repeated. But until such a report comes out, we can explore what at minimum had to have happened, and thus what reforms the Review will certainly have to implement, regardless of what it finds. It cannot likely be that the peer reviewers only pretended to read the paper, giving it a rubber stamp not even really knowing what’s in it, and by coincidence this paper just happened to be assigned simultaneously to two reviewers engaging in such malpractice, because professional peer review reports make this impossible: the editor must receive reports from each reviewer describing the contents of the work and answering some assigned questions about it (see, again, the PLOS and Wiley guides for example). So even the editor would have to be engaging in this malpractice, and that’s remarkably unlikely. So how could two independent peer reviewers have written reports giving Ramseyer a pass?

The Failures

Here’s a short-list of the frauds in the Ramseyer piece:

• Ramseyer’s article repeatedly talks about “contracts” between private procurers and the women and girls involved and states as fact their existence and even content, and the entire thesis of his paper is built on these “facts” as essential premises. But as the Economists Letter puts it, “despite the centrality of this assumption,” that such contracts existed and had that content, “to [his explanatory] model, the article presents no evidence to justify this.” There are no contracts. Ramseyer made that up. When confronted about this after the paper was accepted for publication, he admitted “I don’t have any Korean contracts…I haven’t been able to find” any.
• Tessa Morris-Suzuki, an emeritus professor of Japanese history, puts it succinctly: in his paper “Ramseyer provides no reference to a single contract actually signed between a ‘comfort woman’ and her employers, and cites no oral testimony from any former ‘comfort woman’ who recalls signing a contract of the type he describes, nor from any third party who witnessed the signing of such a contract.” In other words, the paper did not even include a fake citation of evidence. It literally cited no evidence.
• Ramseyer wrote that a 10-year-old Japanese girl named Osaki had voluntarily agreed (sic) to become a prostitute. In his words: “When Osaki turned ten, a recruiter stopped by and offered her 300 yen upfront if she would agree to go abroad. The recruiter did not try to trick her; even at age ten, she knew what the job entailed.” But when experts checked the source he cited, “they found that Osaki and other young girls complained to the recruiter that he hadn’t told them they would be doing that kind of work.” Confronted with this, Ramseyer claimed “I don’t know how this happened, but I did in fact make a mistake here.” Such a “mistake” is essentially impossible though; I think only deliberate fraud can explain this.
• Ramseyer claims as a representative example a one Mun Okchu was so financially successful as one of these “voluntary” prostitutes that she lived “most flamboyantly” in result. His own sources don’t say anything of the kind, but in fact report that Mun was actually “prevented from retrieving her money during the war and afterward, even in 1993” when she attempted to claim the money she was never paid in a lawsuit. Other victims who led successful lawsuits were awarded the equivalent of a mere few thousand dollars, so little were they even falsely promised to be paid by their enslavers (and usually, in fact, for other work—countless witnesses attest they were lied to about what jobs they were being shipped off to do). As one victim reports, “We only received clothes two times per year and not enough food, only rice cakes and water. I was never paid for my ‘services.’” No source cited in Ramseyer’s article evinces any other outcome.
• Ramseyer claims that “some Korean comfort women in Burma worked on contracts as short as six months to a year,” but the source for this he cites contains no such information; rather, it only reproduces “a sample contract,” not a signed contract, “written in Japanese,” not Korean, and “in 1937,” which was “years before the Japanese military was fighting in Burma.” Worse, this “is a sample for contracting with Japanese prostitutes,” not Korean subjects, and “specifies a two-year term,” not “six months to a year.” It would appear Ramseyer wholly fabricated this claim and cited a bogus source for it, hoping no one would check.
• Ramseyer claims “regularly, comfort women from [one Burma] brothel completed their terms and returned to their homes,” but the source material he cites only documents one, and maybe two, women (of numerous hundreds) were let go (their fate is not recorded; nor why they were released). This is not “regularly.” Nor does it constitute completing a contract. That an occasional prisoner of war is released cannot evince even they were imprisoned voluntarily (!), much less that all (!) war prisoners were. That these few victims’ families bought them back, by bribing Japanese officials, is a far more plausible explanation of the data.
• Ramseyer’s paper never discusses the well known fact that victims of similar sex-trafficking are often tricked or forced to sign “contracts” which often they cannot even read and are lied to regarding the contents of, and are forced by threats of ominous violence or harm to agree to anyway. Therefore, even if he had such contracts, that would not alone suffice to evince a consensual practice. To not acknowledge or answer this, but instead pretend no such possibility has to be ruled out with evidence, is a bizarrely egregious failure of historical reasoning.
• As pointed out in the Economists Letter FAQ, Ramseyer’s paper “does not meet current disciplinary standards for being a contribution to economic theory or game theory … because it does not make its arguments in a way which is now considered standard and complete,” e.g. it presents no mathematical or logical model—it never proves anything by any method accepted in the field—but argues solely by assertion and hunchwork—which is the approach of an amateur, not a professional. It also presents no thesis that is “novel, interesting, or rises above common sense,” apart from its claim that there were voluntary contracts, which claim is established by no accepted method in any professional field.
• As the Economists Letter points out, Ramseyer’s “article implies that economics, in particular, game theory, justifies his conclusion” that the contracts were consensual, but “invoking game theory does not establish the absence of violent exploitation or predation,” and game theory “does not allow one to conclude that such interactions were consensual.” In other words, Ramseyer’s inference model rises to the level of professional incompetence: a completely irrational (and frankly, sub-amateur) deployment of the theories and terminology of the field.

Any one of these findings would nix a paper in the peer review process of any even half-respectable history journal. I struggle to think of any explanation for how multiple peer reviewers did not notice that no evidence was ever cited for these alleged contracts or their contents. One of the fundamental questions peer reviewers are asked to make of a work is precisely whether evidence is cited for its essential claims. They are not expected to fact-check that cited evidence (there is some trust that scholars will not perpetuate a fraud; a trust that has gotten even leading science journals in trouble), but they are expected to fact-check a representative sample of citations for the most surprising or controversial or unexpected claims.

Peer reviewers, being experts, can often already tell at-sight when claims are true, because they already know the evidence-base well enough, so they might not check a source citation for an unsurprising claim. But they definitely will check something that shocks or surprises them, to confirm an author is citing a real source and representing its contents fairly, and most especially when a claim is both surprising and essential to the thesis being argued (and of course, once you catch one incident of fraudulent citation, you will then start checking far more citations than otherwise you might, looking for more). Which means Ramseyer’s peer reviewers were for some inexplicable reason not shocked or surprised by any of his claims, and thus never moved to check them and thus discover what other experts subsequently did (such as for the Osaki or Mun or Burma claims).

Another of the fundamental questions peer reviewers are asked to make of a work is whether the conclusions follow from the premises by any recognizable logic. Which means Ramseyer’s peer reviewers did not think to ask why his inference model never addresses in any evidence-based way the obvious counter-hypothesis that the contracts he’s talking about were coerced. This is a glaring hole in the paper’s logic. They also did not “notice” that Ramseyer’s misapplication of game theory as an explanatory device was bogus; it did not meet even minimal standards in the field regarding how that would actually work. This is quite strange, as any real expert reading a paper that misused a basic term or theorem would notice that right away—it’s one of the most common red flags for an amateur poser, so it garners the most ready attention and suspicion. It would be like a paper in Jesus studies assuming, without explanation, that Q was usually employed to explain the content of the Gospel of John. Any expert would flag that as indicating the author does not know what they are talking about, which would trigger an even more thorough audit of the paper’s sources, claims, and arguments. It would also evoke a correction demand; the claim would never make it into print.

Why We Need Feminism

That’s all true, and profoundly mysterious. The failure of the International Review of Law and Economics peer review process is disturbingly extensive here, which is why the expert community is so alarmed by it. But there is one other point I sort of swept right past, which is not about the academic and epistemic standards peer reviewers are supposed to police, but which falls squarely in the province of social morality: why did none (!) of Ramseyer’s peer reviewers catch that his paper actually rests on the argument that a ten year old girl can consent even to sex—much less to be sex-trafficked? This isn’t just epistemically inexplicable; it’s morally terrifying. This means not only Ramseyer thinks children can be consensual sex partners, and even consent to being shipped off to undertake that as a regular profession (the moral core of even ordinary child labor laws be damned), but also multiple expert peer reviewers thought so, and the editor of the journal thought so (as the paper had to be edited and formatted for publication; so the editor definitely had to have read it). Think about that.

And let’s be clear. We mean actual children here; not teenagers. Osaki was 10, and no aberration; the evidence establishes preteen victims of this wartime industry were quite numerous. And yet even Japanese law at the time held that women below the age of twenty could not consent to a sexual relationship or even sign a contract (and even with parental approval, girls still had to be at least sixteen). Which means even if these contracts were “consensual” (in whatever sense Ramseyer, his peer reviewers, and the journal’s editor were imagining), they were illegal—even by the perpetrators’ own laws! It is also of course academic malfeasance to neither know nor mention this in the article itself; but what I am calling attention to here is that no one flagged this argument as constituting an immorally delusional worldview, and to such an extent as in fact to undermine the paper’s entire thesis. At the very least, after being really creeped out and hoping they never meet this author, a reviewer should have insisted Ramseyer qualify this part of his paper with an acknowledgement that the scenario he is describing cannot evince consent—morally or legally, even under the applicable law—and thus actually is evidence against his paper’s thesis, not “for” his thesis, as he presents it: any institution that is contracting ten-year-olds for this service should be suspected of not running a consensual operation of mutual benefit for anyone.

Feminism gets a bad rap, largely because of delusional, misogyny-cult Protocols-of-Zion-style fantasies about Cultural Marxism (which doesn’t exist, and has nothing to do with actual feminism as an idea or movement), and partly because human brains are shitty at grasping reality (on which general point see my Vital Primer on Media Literacy). Human beings tend to be fools, loons, or idiots prone to Dunning-Kruger their way through life; and feminists are human beings, and therefore suffer the same proportion of such clowns as non-feminists do, and—ironically—one of the things seeing the world through a sexist lens causes you to do is only “see” the bad examples confirming your biases, while you ignore all the far more numerous good examples as “irrelevant data,” as they don’t confirm your thesis. That same lens can also distort what you see to conform to what you expect to see (your caricatures and mythologies), rather than what’s really there; most anti-feminists cannot correctly describe a feminist argument made to them five minutes prior, much less the evidence that was presented them. Thus, when someone hears “feminism” what they might “see” in their head is a gallery of fools, loons, or idiots Dunning-Krugering their way through any conversation about feminism (whether real or imagined), and not an actual collection of reasonable, professional, or well-informed feminist activists and intellectuals. So, I do not here mean that delusional, selection-biased version of feminism—the straw man—but the actual, expert version—the steel man (or woman or other; scarecrows and ferrous entities have no real gender). On that see my articles A Primer on Fourth Wave Feminism and Intersectionality: A Guide for the Perplexed.

Baseline feminism—mainstream feminism—is simply the conjunction of five core beliefs:

• That women are people, in the sense of autonomous actors with agency, intelligence, competencies, individualities, opinions, feelings, and rights;
• That women are not so different from nor inferior to men, and deserve to be understood and treated on a basis of factual realities about them rather than myths and tropes and group biases;
• That despite what any laws say or can do, and regardless whether by conscious or unconscious motives, women aren’t consistently treated that way in society (in some places the disparity is greater than others, but it’s hard to find any population-group lacking any noticeable disparity);
• That nevertheless women ought to be consistently treated that way everywhere in society;
• And that we ought to do something about that.

That much all feminists agree on; and as such, that is the definition of feminism qua feminism. After that can ensue all manner of disagreements, leading to various different kinds of feminists and different positions within feminism and among feminists. Disagreements about “what ought to be done,” “what disparities, and where and when,” “what realities,” and even who counts as a “woman” in this equation, are all to be found, and as such no single position on any of these subjects can be definitive of feminism. See my article Why Atheism Needs Feminism; which also goes more into the detail of my next point:

Feminism is an essential component of any successful epistemic enterprise. Not only because it’s an epistemic canary. If your epistemology does not lead you to even baseline feminism (as just now defined), it has failed a basic epistemic test and is therefore exposed as broken; which is a serious social and academic problem: see What’s the Harm for the epistemic point; though there applied to religion, it applies as well to rejecting feminism. The canary role here operates the same way as depicted in my old flow chart on systemic racism (on which topic see Actually, Fryer Proved Systemic Racism in American Policing), which I have reproduced here:

But a more direct way feminism is essential to any epistemic enterprise is that once you grasp the facts and ideas inherent in feminism, you cannot fail to understand concepts of autonomy and consent, and how coercion and bias operate in a social system in overt and subtle ways. And this will make you more epistemically competent to assess presentations like Ramseyer’s. A feminist would immediately flag the ten-year-old-girls-can-consent thing; a feminist would immediately notice Ramseyer’s argument omits any effective response to the ubiquitous reality of coerced contracts in sex-trafficking, and identify that omission as a fatal methodological flaw; and a feminist would be pervasively skeptical of Ramseyer’s thesis and claims and thus strongly motivated to check the validity (or indeed even presence!) of his sources, instead of “not noticing” he didn’t have any, or employed them fraudulently. For example, a feminist would be “shocked and surprised” by the claim that he had the evidence he claimed (such as regarding Mun and Burma), and thus been motivated to fact-check his sources.

And not simply because of some sort of feminist bias. A competent feminist peer reviewer (as most actual feminist peer reviewers will be), who did all that and still found all Ramseyer’s sources and evidence-claims checked out, would still approve the paper on a proviso of conditional revisions, such as that he address the logical problems of ten-year-old-consent and how to ascertain the absence of coerced contracts. In other words, any competent feminist would not block publication of such a shocking and controversial article if it met the epistemic, logical, and professional requirements of the field and claim. It just didn’t. And feminism is what would have ensured that was detected—probably at every point of the peer review process, but certainly at least at one point in that process, thereby ensuring this fraud was not endorsed by publication; or that its author fixed all the epistemic, logical, and professional problems before attempting to publish it again.

I doubt the International Review of Law and Economics will realize or admit this in particular. But I await with bated breath the results of their “investigation” into how all of this incompetence, fraud, and pedophilic ideology made it through several layers of supposedly professional vetting. I also await Harvard’s excuses for why it does not fire this disturbing, mendacious hack, but continues to allow him to represent their school and teach its students. But as they say, reactions against feminism typically justify feminism.

-:-

Update: J. Mark Ramseyer has since attempted to reply to these facts against him, but so speciously and disingenuously as to spark even greater outrage.