Again, this is sort of a trial run. The idea is to work out the best approach for doing it. I agree with the commenters who say that someone other than me should take on this project. If someone is interested in taking it on, perhaps along with a similar project for the NYT, they should get in touch with me, and we can brainstorm how to fund it.
My current thinking is that there are three types of bias. First, the headlines and lead paragraphs sometimes do a lot of editorializing, as in the first story on the phone call between Mr. Trump and Taiwan’s President.
Second, there are double standards in choice of emphasis, as when the Post tells the “Steve Bannon is controversial” story in a way that makes Bannon seem totally beyond the pale but tells the “Keith Ellison is controversial” story in a way that endorses neither Ellison nor his opponents. I do not have a problem with the Ellison story, but I do have a problem with the disparate treatment given to Bannon.
Third, there is the “world view confirmation” bias of the Metro section, the Style section, and the Sunday Outlook section. That is, each contains essays or stories that make progressives feel good about their world view, with much fewer pieces that might give progressives reason to doubt or reconsider in any way.
Also, I am finding that a binary classification system of “bias or no bias” is not the best scheme. I would feel much better assigning bias points to pieces, which could range, say, from 0 to 5, with 0 for no bias to 5 for extremely high bias. Using this system, the initial story on the China phone call would get the full 5 points for editorializing. For each week, you would have a scorecard giving the number of progressive bias points and the number of anti-progressive bias points.
For now, and probably in general, the actual scoring is less important than explaining my thinking on the various pieces.
Monday, December 5
The piece on Trump that is most aggressively hostile can be found in the Style section. It is by Margaret Sullivan. I give it a bias point in the “world view confirmation” category.
The lead story is again on Mr. Trump’s phone call with Taiwan’s President. I code it as neutral, particularly in comparison with last week’s editorializing. There is also a story on page 4 that I code as neutral on Mr. Trump’s threat to impose tariffs on companies that move jobs overseas.
Tuesday, December 6
The lead story is on the appointment of Dr. Ben Carson as HUD secretary. It leads with three paragraphs of negative spin on Dr. Carson. It does not mention that HUD is a swamp in need of draining, in that audit reports on HUD have been terrible for years. I am inclined to give it 2 bias points for editorializing.
On page 2, Dana Milbank blames Mr. Trump for stoking violence, using as evidence a deranged individual’s attack in a local pizza place that appears to have been stimulated in part by fake news. It is hard to rate that story as neutral. When police officers have been murdered in recent months, neither Milbank or others at WaPo have blamed the Black Lives Matter movement. In general, I think that the best approach in all of these cases is to err on the side of not blaming people who were not personally involved in the shooting. I give it a bias point in the double-standards category.
Also on page 2, there is a story that Mr. Trump’s son-in-law donated to Israeli settlements in the West Bank. If you actually read the story, it seems to have very little significance. If it belongs anywhere in the newspaper, it is not on page 2. I cannot imagine that a figure associated with a Democratic leader would have charitable contributions scrutinized so carefully in order to give negative spin. I give it a bias point in the double standards category.
On page 5, there is a story about Mr. Trump meeting with Al Gore to discuss climate change. The story seems neutral.
On page 16, there is a story claiming that Trump’s national security adviser Michael Flynn and his son “use social media accounts to promote baseless claims.” The examples cited seemed to me to be pretty damning. I think that the Post should do an entire story on Michael Flynn’s twitter comments, because I imagine that there are others that should frighten people. I code the story as neutral, because its negative tone strikes me as justified.
The Style section has an essay talking about Mr. Trump’s cabinet selection process as a reality TV show. It is rather gentle in tone, and I would code it as neutral.
Wednesday, December 7
The lead story is on the DC family leave act, which uses phrases like “most generous” and “at the forefront.” I give it 3 bias points out of 5 for editorializing.
There is yet another front-page story on the gunman who went to a pizza parlor (but did not kill anyone. Did he even shoot anyone?). I give the story 2 points for worldview confirmation and double standards, in that it tells progressives that right-wing conspiracy theorists are a really big deal, thereby ironically feeding one of the left’s favorite conspiracy theories.
On p. 4, there is a story headlined “GOP on hill seeks harmony, but Trump clash is possible.” It concerns Republican opposition to Trump’s tariff threats. Many of my progressive friends have expressed the fear that Republicans would fall into line and turn Trump into a dictator. I think that the facts of this story go against that view. However, the story sticks to the facts without making any editorial analysis, which is fine. I code it as neutral.
Also on p. 4, there is a story headlined “Electors for Trump urged to have second thoughts.” It is a mild story that gives a mostly sympathetic treatment to the people trying to get electors in states that Mr. Trump won not to vote for him. My guess is that if it were a Democrat who had duly been elected, the people doing the “urging” would be described as extremists and threats to democracy. I give the piece 3 bias points for double standards.
The op-ed page has a piece by Kathleen Parker urging electors to vote against Trump. Because it is an opinion piece, I give it no bias points. But sheesh! If Trump is not chosen, what does she expect to happen? (a) The millions of people who voted for Mr. Trump to say, “Right. We made a mistake. Glad you overturned it;” or (b) Civil War. I am pretty sure that we would get (b). And I am pretty sure that Ms. Parker has not even considered the possibility, because she lives in a bubble in which nobody supports Trump. I know that she is a regular columnist, but I don’t think that the editors are obliged to run a crazy column without asking her to put a little more thought into it.
Thursday, December 8
On page 8, there is a headline “Trump picks border-security hawk to lead DHS.” This is a news story. On page 1, however, there is a much more prominent story headlined “Trump’s DHS pick is another general.” It basically is an editorial saying that Mr. Trump’s team includes too many former generals. As if only a Cabinet full of lawyers and political hacks can be regarded as sound. I give this story two bias points for editorializing.
There are many other stories on the Trump Administration, all of which struck me as neutral. One of the most interesting stories is on page 17 of the front section, headlined “2 in GOP want careful eye on Trump.” It reports that Senator Tom Cotton and Representative Trey Gowdy plan to work to rein in executive power and try to re-assert the role of Congress. I think it would have been more educational for the Post‘s readers to have this as the page 1 story rather than the “pick is another general” editorial.
Friday, December 9
Another day, another Trump Cabinet nomination (labor secretary), and another chance for the Post to put the actual story on a non-front page (page 4) while using the lead story to editorialize. In this instance, the editorializing is about Mr. Trump as a threat to labor. The page 4 story shows how the Post can write when it wants to be objective. It begins,
Andrew Puzder, the fast-food executive President-elect Donald Trump intends to nominate as head of the Labor Department, has been a harsh critic of raising the federal minimum wage to $15 an hour. . .
In contrast, the page 1 story begins,
President-elect Donald Trump’s Twitter attack this week on a union official, followed by his choice of a labor secretary who has criticized worker protections…
We only learn deep in the jump page that Trump’s Twitter attack was on the leader of a union involved in the Carrier deal who had attacked that deal. We never learn that the “worker protections” are considered by some respectable economists to be contrary to the interests of many workers.
If they had labeled this story “analysis” and taken it off the front page, while using the objective story on the front page, then I would not have charged them with any bias points, forgiving the failure to explain the context for the Twitter attack closer to the lead paragraph. As it is, I give this story 4 bias points for editorializing.
Saturday, December 10
For me, the most interesting piece in today’s paper is by Paul Farhi in the Style section.
the nation’s newspapers and major digital news sites — the dreaded mainstream media — are facing a shortage of people able, or more likely willing, to write opinion columns supportive of the president-elect.
Farhi points out, correctly, that many conservative intellectuals dislike Mr. Trump. The “usual” conservative columnists for the NYT and the WaPo were opposed to Mr. Trump.
However, Farhi makes it sound as if there was no columnist to be found to support Mr. Trump. I think that is not quite correct. Dilbert creator Scott Adams, although coy on his personal politics, wrote blog posts in praise of Mr. Trump’s strategy and tactics in the campaign. Roger Simon was a Trump cheerleader. I am sure that Peter Thiel would have been happy to write frequent columns.
But I do not think that finding pro-Trump columnists is an important issue. For the editorial sections, my main concern is what I call “world-view confirmation,” meaning essays that serve no purpose other than to make progressives feel good about themselves. There are days when I feel bombarded with these.
However, my concerns about the editorial sections are relatively mild. The big issue is how the mainstream media frames what it calls news.
The lead story is about Russian interference in the election. It suggests that the Russian government preferred Mr. Trump and that it was behind the theft and publication of emails from the Democratic National Committee. It appears to me to be a fair story.
Another front-page story reports on the Trump transition team’s efforts to get information about the Department of Energy. It construes the effort as improperly threatening toward scientists associated with climate issues. If it turns out that the Trump Administration punishes civil servants for their viewpoints, then there may be a story here. But if all the Trump folks are doing is trying to get a handle on how they can change the policy directions, then raising conspiracy theories strikes me as unwarranted. I do not think that there is enough substance in this story to warrant front-page coverage, so I give it one bias point for choice of emphasis.
Meanwhile, what the Post is not covering is stories like this one, about actions that President Obama is taking that could hamper Mr. Trump’s ability to change policy. A really complete story about the transition process would discuss all of the dubious tactical moves being undertaken by both sides, not just those made by the Trump team. The Post is in danger of earning double-standards points on this.
There is another front-page story on the views of ex-generals in the Trump Administration. The story itself is fair, balanced, and informative. But the headline, “Trump’s generals see world as good vs. evil” is biased and misleading. I give this story one bias point for editorializing, based solely on the headline.
Sunday, December 11
The front-page headline is “Election Allegations Roil Washington.” The subject has already been on page 1, and this story adds no new important information.
Since the election, Democrats have been doing all that they can to make the result seem illegitimate. One of their approaches has been to blame Russian hacking. On page 2, Dan Balz argues that “Democrats ought not to believe that their problems have been caused primarily by the actions of a foreign government.” I think that is right. The Russian hacking story is not exactly fake news, but making it a lead story gives a lot of credence to a “stabbed in the back” narrative for the left.
This is a very interesting “what if the shoe were on the other foot?” situation. Suppose Mr. Trump had lost by a small margin and the CIA was making noise that Russian influence was a factor? In that hypothetical case, aren’t we pretty sure that Mr. Trump and his supporters would be going ballistic? Would not Mrs. Clinton and the Democrats be scorning them as bitter losers and nutjob conspiracy theorists? Indeed, would they not be the ones questioning the CIA’s competence?
My opinion on the substance is that the charge of Russian interference is sufficiently plausible and sufficiently important that it deserves to be investigated aggressively and fairly. However, that is not what this blog post is about.
The issue here is whether or not the Post is showing bias. The Post is giving a lot of play to the narrative that the Democrats are pushing. My guess is that, in the “shoe on the other foot” scenario, it would instead be giving a lot of play to the “bitter losers and nutjob” narrative. Based on my assessment of this hypothetical, I am inclined to assign 2 bias points for double standards. Obviously, this is a difficult call, and I am not confident about it. In fact, it is almost surely wrong. The Post deserves either 5 bias points or 0 bias points, but we will never know, because the “shoe on the other foot” experiment will not actually take place.
In the op-ed section, Maggie Orth writes,
Those who believe in liberal democracy must fight Little Brother’s lies and aggressions with everything we have. We cannot allow legitimate resistance to authoritarianism to be squashed by an armed mob of angry Little Brothers. We must fight back.
By Little Brother, she means Trump supporters in the public and in social media.
I put this op-ed in the category of “Civil War priming.” It uses the phrase “must fight” twice in three sentences. Conservative intellectuals favored using political and Constitutional means to oppose President Obama’s policies. And they waited for Obama to actually propose policies before they expressed their opposition. I get the sense that some left-wing intellectuals do not believe in such niceties, and they would be happy to have their views expressed in street violence. Because it is an op-ed, I assign no bias points to this piece, However, as with Kathleen Parker’s essay yesterday, I find myself alarmed by the casual eagerness for political conflict.
If you say that Mr. Trump and his supporters have some dangerous belligerent tendencies, I agree. And going forward, when they stray outside of formal Constitutional restraints or traditional behavioral norms, I absolutely do not want to see anyone cave in. But the argument that there is an immediate need for aggressive confrontation is not one that I find persuasive. If your real goal is to restrain Mr. Trump, extreme and implacable opposition is not what will bring it about–it is likely to have the opposite effect.
Doesn’t this exercise by default feed the 2 party paradigm?
Mire constructively, what about taking the full list of cognitive biases and measuring the stories as against them. Then the left media just happens to be wittingly or unwittingly using them against their audience, but the fact that they are biased for their side becomes more incidental.
Scott Adams just did something on this. Maybe he would help fund it.
http://blog.dilbert.com/post/154336783261/fake-news-versus-misleading-news
“I’m watching the mainstream media have fits over so-called “fake news.” The theme they are pushing is that fake news stories are more damaging to society than normal news that includes the following:
1. True stories told out of context to intentionally mislead.
2. Biased reporting that the media doesn’t realize is biased.
3. Giving a spotlight to people who are lying.
4. Misleading by putting emphasis on some things and not others.
5. True stories too complicated for the public to understand.
6. True reports of sources that happen to be lying but we don’t know it. (That gives you the Iraq war, for example.)
7. Having boths sides represented when one side is clearly lying or wrong.
8. Simplification to the point of misleading.
9. Showing clear disdain for the opinions on one side but not the other.”
I’m pretty uncomfortable with his “reality is impossible so stop trying it” vibe.
Great public service. If the Washington post were really interested in presenting news objectively they should run it on their op-ed page. I con’t think this is going to happen.
Since the election, Democrats have been doing all that they can to make the result seem illegitimate.
I call BULLSHIT on this….I think most Democrats are sick of the 2016 shitshow and I think the lesson here is the election norms have changed in the future. In reality this is not news and Conservatives ‘welcomed’ or turned a blind eye to it as the hacking to help Trump beat HRC. And all Democrats should fight back against NSA spying and accept that every e-mail we write can be used against us.
And by 2020 maybe liberals should work with Chinese hackers.
I have serious doubts about Russian hacking. And besides that, it is irrelevant. You or I could have “hacked” Hillary.
The real issue is information availability undermining establishment patterns. Dems chose to cheat their primary to ensure “safe” establishment candidate won. They zigged when they should have zagged.
It turns out a disavowed anti-establishment candidate beats a hated establishment candidate in an anti-establishment era.
My take is the hacking happened by the Russians outside the Trump campaign so he is not guilty here. And frankly the Wikileaks e-mails were had little information although the press could continue the “E-mail” headlines against HRC. (It was dangerous emails stories that hurt more than the hack.) And what does the Primary have to do with anything?
My problem with conservatives is there are a lot of issues of hacking could create and they seem accept the reality only because they agree with the results. Now it seems OK to hack Large Companies or private citizens. Being we can find out what Koch Brother companies secrets! Or now what would be wrong if the Chinese started hacking Trump administration? And at this point, they may already be doing so or working on hacking ExxonMobile to find material on Tillerman. (And finally it does give Ds their ‘Russianghazi’ so they act like Republicans did during Obama.)
I predicted before he said it that Trump would not prosecute Hillary. He is still touting the wall and I’m still predicting no wall (depending on your definition of wall).
So, campaign season hyperbole is not new to me. Yes, Benghazi was politically motivated. And I still suspect that the reality is something Republicans wouldn’t want to come out (The CIA funneling weapons to proto-ISIS). But all that is not relevant.
What is relevant is you basically can’t hide this stuff anymore. And anything that is built on secrecy is going to have to change.
Believe me, I’m not “okay” with it. I’m not either “side” either. I’m on my side.
So, it just occurred to me a bifurcation between our views.
Your view is based on the “someone did this to someone” view. Mine is, “this is happening.” Trump doesn’t use e-mail. His tweets are open and he gets plenty of heat for them. I suspect if Republicans tried to cheat their primaries (the dirty laundry of which also was also aired very publicly) we’d probably have those leaks as well. Instead we got NeverTrumpers.
It just so happens that, probably quite luckily for Republicans, Hillary was secretive and outted whereas Republicans looked like clowns in the open. I also dispute that it had that big an effect on the election anyway. Everyone knew Hillary was crooked, so at best it was confirmation. It was mostly baked in the cake. Just like Trump’s negatives were HEAVILY advertised by Democrats. It is weird to believe that Hillary’s negatives released by Russians are so much more impactful than Trump’s negatives.
And by the way, bad cyber-security is not a point for Hillary and Democrats. In fact, evidence of foreign interests hacking in is one of the things Comey said would have raised the level of her infraction (along with the much more discussed intent/motive).
I also don’t see where Republicans are suddenly okay with everyone being hacked. There are indeed several who support the inquiry into the alleged Russian involvement. They happen to also be Trump critics.
So, campaign season hyperbole is not new to me.
All campaigns are hyperbole but Trumps was in another dimension hyperbole. In general, there was a lot we can dismiss (HRC in jail) but most Presidents act on what they campaign on the most. (Not everything during 18 months but the core issues.) So I do believe Trump has to lower illegal immigration, bring back manufacturing jobs, rolling back abortion rights and not cut Medicare/Social Security. They were said numerous times during the campaign.
However, Trump is no longer campaigning but as President and his statements have consequences. This is one why Obama was effective that he tended to be careful of what he said (and in fact Syria statement looked like a failure.) And something will go wrong and he can not be hyperbole in his reaction.
TRUMP: Well, it could be. I think President Obama has been terrific. He’s been, you know, very respectful of the process and everything else. So, I saw that. But – and–and I want it, too. I think it’s great. I think — well, I don’t want anyone hacking us, and I’m not only talking about countries. I’m talking about anyone, period. But if you’re gonna to do that, I think you should not just say “Russia.” You should say other countries also, and maybe other individuals. It’s not necessarily just –
Additionally if NYT and WaPo framing as the Anti-Trump is increasing it’s subscriptions to audience why should they not continue? There is no true MSM press here. CNN rating increased when they focused on Trump in 2015 and 2016 so this just business at this point.
You should understand that is basic free market of the press and that is happening naturally in the market.!
I’d suggest you add the truly awful Valerie Strauss’ content to your list.
https://www.washingtonpost.com/news/answer-sheet
While purportedly on education, it mostly the reflexive union/lefist take on everything, and about 75% anti-Trump content. I don’t get the physical paper anymore, so don’t know if any of that makes it into print, but it’s pretty awful.
I’d also take a look at wonkblog, but it’s really scary there. Smart people that believe ludicrous things about markets, high levels of racism from Jeff Guo…
It looks like you want to do a content analysis. If you are really going to spend time doing this work then I suggest you get some qualitative software (e.g. Atlas.ti or NVivo). You can start by doing some open-coding (which it seems like you are doing) and then begin to narrow down your codes. Using software will allow you to do a few things:
1) Create and keep track of the themes, codes and sub-codes
2) Look at themes over time or within subjects (e.g. all codes related to stories about climate change)
3) Code parts of an article (e.g. is the headline biased?)
4) Multiple people can access the same articles so you can do some inter-coder reliability
5) Turn your qualitative analysis into a quantitative analysis: Compare codes and number of times codes/themes appear in different types of articles or sections of the newspaper
I’m sure there are grad students you can pay to do this work. Your biggest expense will be the software.
Good luck!
If Arnold keeps this up, WaPo is going to put him on the list of Fake News Russian sites.
Arnold – this exercise does a great job of articulating the nature of the bias.
But it is mostly explaining the details and structure of something almost anybody who is interested already knows – in other words, it is telling people who do not trust the WaPO why, in detail, they don’t trust the WaPO.
But that is mostly people who don’t bother with the WaPO to begin with. The ones who are devoted are unlikely to pay any attention to such observations.
Mind you, I think this is a fine thing to do, but I’m skeptical of it having much effect.
Bryan said it — you’re doing a great job … but it’s unlikely to have much effect.
Because the important bias is 0 or 5… “The Post deserves either 5 bias points or 0 bias points, but we will never know, because the “shoe on the other foot” experiment will not actually take place.”
Actually, we DO have the “other foot” — before the election, Hillary and the Dems were complaining about how treasonous Trump was for suggesting that the election is rigged.
Like anybody who opposes PC, the Dems claim such anti-PC folk are evil, and therefore already guilty — so no more need for any presumption of innocence. The purpose of the demonization and Civil War prep is promote Dem radicals against Trump.
There’s an abundance of other feet. It’s like shooting feet in a barrel.
I can remember when the FBI was, for about six days, a hotbed of treason and coup d’etat. This was the objective, certified, real news according to CNN, NYT, SNL. I can also remember one month prior to that, when the same people held up for our admiration as a bastion of non-partisan, rule-of-law, just-the-facts proceduralism the same FBI that was now recklessly, criminally, trying to swing an election.
So the state-of-play at this exact moment and always subject to change is that the FBI may or not be a nest of vipers conspiring against democracy and America, but at least the CIA is a bastion of non-partisan, rule-of-law, pure unbiased advice. Until the media freaks out again like it did a week before the election, trying to destroy the FBI for the sake of Hillary Clinton’s campaign.
How many people and institutions and cake shops would need to be destroyed over the next four years, just obliterated, under the full force of the journalism-entertainment-celebrity complex, had the election gone the other way? It would have been four years of circling the wagons to defend her administration from all opponents, no matter how small. One week it would have been “death to nuns” every hour on CNN. A week later they’d train their guns on some freshman wearing a pantsuit at a college costume party. “Lionel Shriver is out of line.” “HGTV is the American Taliban.” The breaking news scrolling across every airport TV screen: “Britney Spears hates America and little puppies too.”
1. I say keep going qualitative. Quantitative bias measures depend on “inter-coder agreement,” which means that the biases of the coders are baked in rather than being explained. And they are unconstructive. Only if you explain in detail why a judgment of bias is being made is the judgment capable of persuading anyone.
2. However, there’s a genuine substantive problem with qualitative coding when it comes to withholding bias points (or judgments) on the grounds that a story seems plausible (to the coder). My suggestion is to not conflate bias with untruth or unbiased news with true news. A biased story may be making a good point and an unbiased story can be entirely wrong. We can’t expect journalists, who are only human, to be right all the time. What we can expect is that they try to be unbiased in their reporting.
3. So I think the goal should be to set and apply standards that are fairly objective, such as (1) balance: at least two sides on every issue should be heard from, and equally; the move to abolish “false equivalency” is the surest way for the MSM to destroy its credibility forever; (2) assumption identification: journalists should be taught (by you, if necessary) to identify their own assumptions and then try to excise them; (3) no double standards: reporters should get into the habit of asking themselves if they would cover a like story if the ideological shoe were on the other foot. That’s off the top of my head, partly inspired by the standards you’re already applying.
4. What is the point? If this were done regularly and got enough views, it might actually affect coverage (although I don’t think the WaPo is important enough to most intelligent Americans that a WaPo Bias Watch will ever get enough views, as compared to an NYT Bias Watch). And if it were done in the right spirit, it might actually be welcomed by some editors and reporters.
On the other hand, if it has no effect, this very fact might get people to examine the assumption that balanced, fair journalism is a realistic possibility. If it isn’t, that has large implications for our views of politics and government.
5. I don’t think attributing to journalists the desire to make readers feel good about themselves is in the right spirit. I doubt that journalists think that way, and they’d therefore have every right to be resentful of such accusations. More likely is that they simply believe in the importance of what they report–just like their readers do. I.e., they have a worldview, and this determines what they think is newsworthy, even in the Style section.
6. Bravo for keeping the focus on the non-editorial sections. In fact, other than noting imbalances in the viewpoints expressed in the op-ed pages, I suggest letting editorialists and op-ed writers say whatever silly things they want to say without comment. I don’t see what quasi-objective standards could be applied here, so complaints will just amount to political grievances. And readers know that opinion pieces are not supposed to be “objective.” The far more important worldview shaping goes on in the parts of the paper (and the wider culture) that are supposed to be delivering “just the facts.”