The rolling stone of confirmation bias

Rolling Stone published a report concerning their article ‘A Rape on Campus’. The Columbia School of Journalism conducted the report. It reveals a host of problems with Rolling Stone’s article, its writer Sabrina Rubin Erdely, and its single source named “Jackie.”

The report follows the complete discrediting of Jackie’s rape allegations against the fraternity Phi Kappa Psi. Police conducted a thorough investigation after the Washington Post reported the numerous inconsistencies in Jackie’s story and Rolling Stone’s failure to engage in basic fact-checking. How the situation got to this point is a long story. The report is 13,000 words. Yet it shows what happens when one allows confirmation bias to dictate how one reports.

The issue seems to fall back on one basic element: the decision to believe Jackie’s story. While it is understandable that someone hearing Jackie’s tale would empathize and seek to believe her, Erdely and the rest of the magazine’s staff appear to have allowed that desire to trump all the warning signs that something was off about Jackie’s story. For instance, when Jackie finally named the lifeguard she claimed was the ringleader of the gang rape, she could not remember how to spell his last name. Erdely claims:

“An alarm bell went off in my head,” Erdely said. How could Jackie not know the exact name of someone she said had carried out such a terrible crime against her – a man she professed to fear deeply?

That alarm bell did not stop her from continuing with the article.

Other issues, such as contacting Jackie’s friends, proved to be a problem. According to the report, Jackie never asked Erdely not to contact her friends. Instead, Erdely chose not to contact Jackie’s friend out of concern that doing so would drive Jackie away. Jackie later told Erdely that she bumped into one of her former friends:

On Sept. 11, Erdely traveled to Charlottesville and met Jackie in person for the first time, at a restaurant near the UVA campus. With her digital recorder running, the reporter again asked about speaking to Ryan. “I did talk to Ryan,” Jackie disclosed. She said she had bumped into him and had asked if he would be interested in talking to Rolling Stone. Jackie went on to quote Ryan’s incredulous reaction: “No! … I’m in a fraternity here, Jackie, I don’t want the Greek system to go down, and it seems like that’s what you want to happen. … I don’t want to be a part of whatever little shit show you’re running.”


“Ryan is obviously out,” Erdely told Jackie a little later.

The failure to double check Jackie’s claims led to Erdely missing important information. As the report notes:

If Erdely had reached Ryan Duffin – his true name – he would have said that he had never told Jackie that he would not participate in Rolling Stone‘s “shit show,” Duffin said in an interview for this report. The entire conversation with Ryan that Jackie described to Erdely “never happened,” he said. Jackie had never tried to contact him about cooperating with Rolling Stone. He hadn’t seen Jackie or communicated with her since the previous April, he said.

In short, had Erdely bothered to check she would have caught Jackie in a massive lie. The same is true about the other two friend, Kathryn Hendley and Alex Stock:

If Erdely had called Kathryn Hendley and Alex Stock – their true names – to check their sides of Jackie’s account of Sept. 28 and 29, they would have denied saying any of the words Jackie attributed to them (as Ryan would have as well). They would have described for Erdely a history of communications with Jackie that would have left the reporter with many new questions. For example, the friends said that Jackie told them that her date on Sept. 28 was not a lifeguard but a student in her chemistry class named Haven Monahan. (The Charlottesville police said in March they could not identify a UVA student or any other person named Haven Monahan.) All three friends would have spoken to Erdely, they said, if they had been contacted.

The report lists more examples like this where a simple investigation of Jackie’s claims would have revealed gross inconsistencies with her story. These inconsistencies would have been enough to question whether her claims were even true. Yet no one involved in the process, from Erdely to the fact-checkers to the editors, bothered to check any of this out, even after they realized there were problems with the story.

It gets worse. Take this example:

Last October, as she was finishing her story, Erdely emailed Stephen Scipione, Phi Kappa Psi’s local chapter president. “I’ve become aware of allegations of gang rape that have been made against the UVA chapter of Phi Kappa Psi,” Erdely wrote. “Can you comment on those allegations?”

It was a decidedly truncated version of the facts that Erdely believed she had in hand. She did not reveal Jackie’s account of the date of the attack. She did not reveal that Jackie said Phi Kappa Psi had hosted a “date function” that night, that prospective pledges were present or that the man who allegedly orchestrated the attack was a Phi Kappa Psi member who was also a lifeguard at the university aquatic center. Jackie had made no request that she refrain from providing such details to the fraternity.

Erdely decided on her own to withhold information from the people she questioned, information that could have led to new revelations about Jackie’s story. The report states that had Erdely given Scipione and Shawn Collinsworth, Phi Kappa Psi’s former national executive director, the full details they might have investigated the claims. That is what they did following the article’s publication. They found:

[…]  that a review of the fraternity’s social media archives and bank records showed that the fraternity had held no date function or other party on the night Jackie said she was raped. A comparison of fraternity membership rolls with aquatic center employment records showed that it had no members who worked as lifeguards, Scipione added.

It would appear that Erdely was so intent on believing that the University of Virgina was corrupt and unwilling to help female rape victims that she never bothered to check to see if that were true. Worse, she took the school’s inability to speak candidly as evidence of their negative intent. She did not factor in that the school is barred by federal law from divulging students’ private information. They could not discuss certain things with her, and one would think Erdely would know that the law prevented them from doing so. To the contrary, she jumped to the conclusion that they were giving her the runaround, and continued on believing her single source.

This is the same source that disappeared for two weeks when Erdely pressed her for the name of the lifeguard. Erdely agreed not to publish the name. She only wanted to verify his existence. Jackie did not request that Erdely not contact the alleged ringleader. She told Erdely to check the fraternity roster to find him, which Erdely failed to do.

As if that were not bad enough:

In December, Jackie told The Washington Post in an interview that after several interviews with Erdely, she had asked to be removed from the story, but that Erdely had refused. Jackie told the Post she later agreed to participate on condition that she be allowed to fact-check parts of her story. Erdely said in an interview for this report that she was completely surprised by Jackie’s statements to the Post and that Jackie never told her she wanted to withdraw from the story. There is no evidence of such an exchange between Jackie and Erdely in the materials Erdely submitted to Rolling Stone.

At this point it would be fair to call Jackie a compulsive liar. She appears to lie every chance she gets. Again, this was Erdely’s sole source of proof of the gang rape.

This was also a source who told conflicting accounts. She initially told her friends and the assistant dean of students Nicole Eramo that several men forced her to perform oral sex at a frat party. She did not name the fraternity or attackers. During a second meeting with Eramo, Jackie introduced new accounts of assaults:

She told the dean that she was now coming under pressure for her visible activism on campus with assault prevention groups such as Take Back the Night, according to the UVA sources. Three weeks earlier, she said, she had been hit in the face by a bottle thrown by hecklers outside a Charlottesville bar. She also added a new piece of information to her earlier account of the gang rape she had endured. She named Phi Kappa Psi as the fraternity where the assault had taken place, the police said later. Moreover, she mentioned to Eramo two other students who she said had been raped at that fraternity. But she did not reveal the names of these women or any details about their assaults.

When police attempted to investigate the claims, Jackie refused to give any details about the gang rape. They also found inconsistencies in her story about being attacked with a bottle.

Apparently Jackie told the story about the other two women to Emily Renda, a woman working with staff to address sexual assaults on campus. Renda believed the story, but when asked about it by Erdely she:

[…] told the reporter that it might be unwise for Rolling Stone to name Phi Kappa Psi in its story because “there are two other women who have not come forward fully yet, and we are trying to persuade them to get punitive action against the fraternity.” Renda wrote later in an email for this report that she had tried to dissuade the writer “because of due process concerns and the way in which publicly accusing a fraternity might both prevent any future justice, but also infringe on their rights.” Renda’s warning to Erdely – a notice from a UVA employee that Phi Kappa Psi was under university scrutiny over allegations made by Jackie and two others – added to the impression that UVA regarded Jackie’s narrative as reliable.Yet all the information that anyone had about these two women came solely from Jackie. There is one anonymous report filed through UVA’s online system, but nothing to verify who sent it. No one has heard anything from the two women since that report.

Anita Sarkeesian wishes she was this good at conning people. No one involved doubted Jackie’s story even as they learned about the discrepancies. Even the police felt compelled to state “something” could have happened to Jackie after releasing the results of their investigation. The evidence, both individually and as a whole, suggests otherwise. It appears Jackie lied, and did such a good job of it that people are unwilling to call her on it.

However, one cannot put this all on Jackie. A liar is only as good as the people willing to believe them. Rolling Stone’s staff proved to be incredible patsies. At every stage where they should have stepped back and questioned publishing the article, they pressed on.

For example, instead of questioning Jackie’s friends, they set it aside. Eventually they decided to use pseudonyms, yet failed to report in the article that all the statements attributed to “Randall,” “Cindy,” and “Andrew” came from Jackie. Erdely states in the report:

“In retrospect, I wish somebody had pushed me harder” about reaching out to the three for their versions, Erdely said. “I guess maybe I was surprised that nobody said, ‘Why haven’t you called them?’ But nobody did, and I wasn’t going to press that issue.”

That was not going to happen. Her principal editor Sean Woods dropped the issue after pressing for the interviews. Why?

He accepted this because “I felt we had enough.” […] Woods said he ultimately approved pseudonyms because he didn’t want to embarrass the three students by having Jackie’s account of their self-involved patter out there for all their friends and classmates to see. “I wanted to protect them,” he said.

Will Dana, the managing editor for Rolling Stone, and Woods capitulated when Jackie refused to give them a name for the lifeguard. They told Erdely to tell Jackie that they would not try to find the lifeguard, and they used a pseudonym to cover up their lack of investigation.

They put everything into one source, basing their assumption that Jackie’s story was true solely on other people saying Jackie told them her story. As the report notes:

The problem of confirmation bias – the tendency of people to be trapped by pre-existing assumptions and to select facts that support their own views while overlooking contradictory ones – is a well-established finding of social science. It seems to have been a factor here. Erdely believed the university was obstructing justice. She felt she had been blocked. Like many other universities, UVA had a flawed record of managing sexual assault cases. Jackie’s experience seemed to confirm this larger pattern. Her story seemed well established on campus, repeated and accepted.

“If I had been informed ahead of time of one problem or discrepancy with her overall story, we would have acted upon that very aggressively,” Dana said. “There were plenty of other stories we could have told in this piece.” If anyone had raised doubts about how verifiable Jackie’s narrative was, her case could have been summarized “in a paragraph deep in the story.”

No such doubts came to his attention, he said. As to the apparent gaps in reporting, attribution and verification that had accumulated in the story’s drafts, Dana said, “I had a faith that as it went through the fact-checking that all this was going to be straightened out.”

Does he mean this fact-checker?

In this case, the fact-checker assigned to “A Rape on Campus” had been checking stories as a freelancer for about three years, and had been on staff for one and a half years. She relied heavily on Jackie, as Erdely had done. She said she was “also aware of the fact that UVA believed this story to be true.” That was a misunderstanding. What Rolling Stone knew at the time of publication was that Jackie had given a version of her account to UVA and other student activists. A university employee, Renda, had made reference to that account in congressional testimony. UVA had placed Phi Kappa Psi under scrutiny. None of this meant that the university had reached a conclusion about Jackie’s narrative. The checker did not provide the school with the details of Jackie’s account to Erdely of her assault at Phi Kappa Psi.

Even the fact-checker bought the story without bothering to check its veracity.

One would think that at this point those running Rolling Stone would want to change their practices. One would think they would see the problems their current system allowed to occurred. One would be wrong:

Yet Rolling Stone‘s senior editors are unanimous in the belief that the story’s failure does not require them to change their editorial systems. “It’s not like I think we need to overhaul our process, and I don’t think we need to necessarily institute a lot of new ways of doing things,” Dana said. “We just have to do what we’ve always done and just make sure we don’t make this mistake again.” Coco McPherson, the fact-checking chief, said, “I one hundred percent do not think that the policies that we have in place failed. I think decisions were made around those because of the subject matter.”

I will post that again, just so it can sink in:

Yet Rolling Stone‘s senior editors are unanimous in the belief that the story’s failure does not require them to change their editorial systems. “It’s not like I think we need to overhaul our process, and I don’t think we need to necessarily institute a lot of new ways of doing things,” Dana said. “We just have to do what we’ve always done and just make sure we don’t make this mistake again.” Coco McPherson, the fact-checking chief, said, “I one hundred percent do not think that the policies that we have in place failed. I think decisions were made around those because of the subject matter.”

Every procedure they had in place failed because they relied solely on one source and assumed their source was honest. That they would rely on one source and never bother to fact-check her beyond asking people who believed her story is the problem. They followed their procedures, and yet those procedures failed to catch this before print. By the time anyone realized something was off, the article was out and being used to create new policies across college campuses.

Every aspect of this monumental screw-up comes down to people playing off their confirmation bias. They wanted this story to be true because it fit the narrative they wanted to tell. It did not matter whether the story or the narrative itself were true. All that mattered is that it served their intended goal. Likewise, because of their desire to believe female victims, they allowed themselves to be conned by Jackie.

People may remember that shortly after Rolling Stone published the article, HBO ran an episode of The Newsroom covering a similar topic. The episode caught heat from feminists and activists because one of the reporters pushed erring on the side of caution when dealing with reporting rape cases rather than simply taking the accuser’s side. Feminists and activists were outraged, yet that scene is a perfect example of how to avoid this kind of mistake.

Yes, you can believe the accuser, but you ought to check their claims. Check the locations, check the names of the accused, check any information you are given. This is not because you are trying to prove the accuser wrong. This because you want to make sure you have the story right. It is also to protect you, your publication, and anyone accused if it turns out that the story is false. You will be able to catch it before you run the story. Then you will not have to have college students investigate your mistakes.

Of course, this story could not end without the obligatory non-apology. Erdely’s reputation is damaged, so she needed to say something to redeem herself:

The past few months, since my Rolling Stone article “A Rape on Campus” was first called into question, have been among the most painful of my life. Reading the Columbia account of the mistakes and misjudgments in my reporting was a brutal and humbling experience. I want to offer my deepest apologies: to Rolling Stone’s readers, to my Rolling Stone editors and colleagues, to the U.V.A. community, and to any victims of sexual assault who may feel fearful as a result of my article.

Over my 20 years of working as an investigative journalist — including at Rolling Stone, a magazine I grew up loving and am honored to work for — I have often dealt with sensitive topics and sources. In writing each of these stories I must weigh my compassion against my journalistic duty to find the truth. However, in the case of Jackie and her account of her traumatic rape, I did not go far enough to verify her story. I allowed my concern for Jackie’s well-being, my fear of re-traumatizing her, and my confidence in her credibility to take the place of more questioning and more facts. These are mistakes I will not make again.

Reporting on rape has unique challenges, but the journalist still has the responsibility to get it right. I hope that my mistakes in reporting this story do not silence the voices of victims that need to be heard.

Noticeably missing is any apology to the men and the fraternity falsely accused of rape.

9 thoughts on “The rolling stone of confirmation bias

  1. Jackie doesn’t lie to *everyone*… she’s apparently got enough sense to tell her lawyer enough truth that the lawyer recognizes her client’s position and is protecting her from future lawsuits.

    Amazing how there is absolutely *zero* discussion of apology to the men who were falsely accused. Take heart men, we’re all just livestock to these female liars.

  2. Jackie was a feminist. Erdely was a feminist hunting for a story. The result is very, very predictable. Is there actually a case, where feminists don’t lie?

  3. Her non-apology to those she and Jackie falsely accused speaks volumes as to both their political intentions. I continually have to remind “always believe the woman” feminists, that the Salem Witch Trials were based on the sole testimony of a few teenage girls who said they were clairvoyant… honest! And the Pilgrims believed them.

    CNN says that it looks like no one’s going to get fired over this. As someone the comments argued, they may be trying to avoid legally admitting fault, and the firings will start after the lawsuit. Which may also be why Erdely’s not apologizing to the men accused.

    apollohaan on April 10, 2015 at 7:38 am said:
    I like how even when she’s not a victim, she isnt identified.

    Well, they wouldn’t want to cause harassment, would they?

  5. Pingback: UVA fraternity files $25 million lawsuit against Rolling Stone | Toy Soldiers

  6. Pingback: A Dose of Stupid v120 | Toy Soldiers

  7. Pingback: Judge decides UVA false accuser must testify | Toy Soldiers

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s