Gender Bias in Peer Review

Here are the eLife Podcast we try to create audio abstracts of some of the most important papers carried by the journal in recent weeks. But in each episode we also try to shine the spotlight on some significant issues faced by practising scientists. One of those is the rigour of peer review, where new science is scrutinised by authorities in the field before it’s accepted for publication. But is it done fairly? In the spirit of transparency, eLife made available to one of its editors, Jennifer Raymond, details of every manuscript submitted to the journal since its launch. She told Chris Smith what she and her colleagues discovered when they analysed the data…

Jennifer - Well the fundamental question is whether journals when they review scientific papers are finding the best science or whether there are other factors biases that could be influencing the review process.

Chris - The journals used to decide which research to publish now would you say finding the best science of course journals don't go looking for publications is the other way round isn't it correct.

Jennifer - The processes authors submit a manuscript that they would like to have published in the journal and then the editors first make a preliminary decision. Is this work interesting enough or potentially interesting enough to send out to members of the scientific community for peer review and typically three reviewers will write comments about the paper send it back to the editors and a decision is rendered.

Chris - That sounds fair enough and in fact peer review is held up as the gold standard in how we should do things. So what's your beef with that.

Jennifer - The problem is that scientists are people and we know that people are influenced by all kinds of cognitive biases.

Chris - So can I paraphrase by saying Then I get your paper and I decide for whatever reason best known to me. I don't like you. So I'm actually going to scan your paper. I'm going to say it's really bad. It's lots of methodological flaws or whatever. And I'm going to say let's kill it. Let's not publish it.

Jennifer - Sure that can happen. What we found in this study is more that people tend to give kind of an advantage to people who share certain characteristics with themselves and those characteristics include gender and nationality.

Chris - That's quite a claim. So how did you actually do the study. Let's look at the method first and then we'll actually look at the results so talk us through how you approach this.

Jennifer - First of all we were very excited to get access to these data. Every manuscript ever submitted to the Journal is life since its inception. First of all we did but it has been done a number of times. Look at the differences in success rates of authors with different demographic characteristics of women versus men. People from the U.S. U.K. Germany China Japan.

Jennifer - So we asked what are the differences in success rates. But there could be differences in success rates that are not reflecting bias because it may be that papers coming from certain groups are just of better quality either because they've had certain advantages like access to research resources or time or encouragement or other kinds of things. So seeing differences in success rates doesn't mean there's bias and conversely equal success rates doesn't mean there's no bias because it could be that one group was actually better but had that equal success rate. So to try to get beyond that we looked at how successful men were when reviewed by all male reviewers versus reviewers of mixed gender and how successful they were when reviewed by reviewers from the U.S. versus other countries.

Chris - And just to be clear when you say success that's a paper get sent to the journal it passes muster and ends up in print. That's right. And what does that show when you feed these data through your sieve. What trends come out.

詹妮弗-我们发现决定the reviewers made about whether a paper should be published was not just influenced by the contents of the paper but it was influenced by the demographics by the gender and by the nationality of the authors and by the gender nationality of the reviewers male reviewers were more likely to accept a paper from a male author people from a given country were more likely to accept a paper from somebody from their own country.

Chris - So we call this homo Philly was it just that one way street was it just men favoring men or did women tend to give more benefit of the doubt to other women authors.

Jennifer - There was a suggestion that it may be the case that women also give the advantage to women but there were not enough cases of papers from women being reviewed by all female reviewer panels because women are highly underrepresented at every stage of science and in fact we found that they were underrepresented as reviewers too. And when I say underrepresented I don't just mean that not 50 percent. I mean compared to their compared to the number of women in the scientific community they were invited to review papers less often than their male colleagues.

Chris - It's about 30 percent isn't it if you look at the workforce about one in three scientists is female. Two in three a male. That's right. And that's going to inevitably be reflected. If you think about well that's the pool of expertise we've got to rely on. It's probably going to be reflected in editorial boards and review panels and referee ship as well.

Jennifer - It's worse than that. We see that women are less represented as reviewers than they are as authors so that the authors pool should be the same as the reviewer pool. And so we find that there's even less representation of women than we'd expect based on how many of them are submitting papers.

Chris - So you find that there's this a monthly effect. Men look after men women look after women. Does that apply regardless of who else is in the author list. So if you've got say a female first author but a male senior author Does that skew things or does it all ride on whether there's a woman's name there.

Jennifer - We did not find any effect of the gender of the first author. And it's really the senior author kind of the head of the research laboratory whose gender matters. And that makes sense because I think often the reviewers know the head of the lab but might not know who the graduate student or postdoctoral fellow is who's doing the research

克里斯-是一种解释,如果我的卧室rk for a female senior author and I send my paper off - I'm male - I nonetheless might face a publication bias against me owing to the fact that I've got a boss who's a female?

Jennifer - That's true; it looks like papers coming from a given laboratory are influenced by the demographics of the senior author.

Chris - And these were statistically significant differences?

Jennifer - These were statistically significant differences, yes.

Chris - But, obviously, as a statistician knows, that could mean if you do a big enough study, which you did do a huge study it's thousands of papers, you can detect a very very tiny difference; so how big was the difference?

Jennifer - In practical terms what we saw was overall scripts submitted by female scientists were accepted at a rate of thirteen point six percent whereas fifteen point four percent of the manuscripts from male researchers were accepted. So you know some people have said well that's tiny. That's 2 percent. Why are we worrying about that. But if you think of it another way it means that the papers from the women are being accepted at only 88 percent. The rate of the men so there is kind of a 12 percent success gap there. And the effects are even bigger for some nationality effects. So for example papers from scientists submitting from China had success rate of only four point nine percent compared to twenty two point three percent for researchers from the US...