FX's blog: musings on chemistry, among other things…

To content | To menu | To search

peer review

Entries feed - Comments feed

Friday 6 March 2015

Tired of “issues”, “concerns” and “worrying signs” in published papers

I don't know if bad academic conduct in scholarly publishing is increasing, or if I just notice it more often now that I have an active Twitter presence… but it's seriously annoying to see the accumulation of issues in published papers. And because I don't like it, and I think it's everyone's responsibility to act to prevent it (including me), I end up spending time to write to editors, etc. Time that could be used for other stuff, like, I don't know… my own research!

Recent examples that annoy me significantly:

  • TEM images with worrying signs of manipulation (and that's using very cautious language for something that looks clear-cut) in 4 papers by the same authors… including 3 papers in a journal where the author is also an editor.

  • Plagiarism accidents where, after 10 months of careful consideration, the editor decides it was “not intentional”. Like, you were cleaning your keyboard, and repeatedly pressed copy-change window-paste by accident? (Full story over there)
  • A paper with (in my view) unwarranted citations to my work, on an unrelated topic. I wrote to the editor to report it, and posted my concerns on Twitter, and the editor told me they didn't like that. Also, they decided that it was OK for the authors to cite me since “ they have put some thought into their selection”, and it was up to cite to cite whomever they wanted. Turns out, commenters on PubPeer later found other issues with that paper, including figures/tables duplication with other papers (which are, though, on different topics). I wrote again to the editors, but I am not holding my breath: too many seem to like the old-school, closed-door, nothing-to-see-here conduct.

Gaming intermission: spot the differences between the tables from this 2012 paper

and from this 2015 paper

I'm tired of this. Just how common really is academic misconduct in chemistry publications?

Monday 8 December 2014

Why correcting the scientific record is hard

Here I explain, from a very simple example at a modest scale, why correcting the scientific record is difficult in today's academic world. I think it highlights, in a very simple way, some flaws of our systems that we can hopefully address. (Also worth noting: although the example comes from the condensed matter field, you don't have to know anything about physics to read the text below!)

I have recently worked, with a MSc student in my group, on the understanding of elastic behavior of porous materials. During the course of his internship, we spent a lot of time in understanding exactly how elastic stability criteria work in the most generic case. Stability conditions, know as Born stability criteria, have very simple and well-known mathematical expressions for cubic crystals, but things get more complex for systems with lower symmetry. While the fundamentals have been set forth very clearly a long time ago by Max Born (in particular in his 1954 book, Dynamics Theory of Crystal Lattices), his and subsequent books usual give explicit expressions only for common high-symmetry crystals (cubic, hexagonal, etc.)

Now, during the course of the MSc project, we stumbled onto some papers that quote incorrect mathematical formulations of the Born conditions, while usually citing the original Born book (in which these expressions are not found). Most of the errors arise from people incorrectly generalizing the "cubic" conditions. We looked a bit more, and found more examples of such errors, in papers between 2007 and 2014. Now, if you find several mistakes in series of related equations, in a dozen papers published in a given field throughout a decade, what do you do?

Over the course of a few days, we wrote a short paper, explaining the general form of the conditions, and explicitly listing their mathematical expression for each possible crystal class. We uploaded it to the arXiv, and sent it for publication to a well-read journal in the field (Phys. Rev. B). I highlighted, in the cover letter to the editor (see below), that it was meant to be a concise and pedagogical reference that would benefit the research community, and it seemed to us worthy of publication.

First, it took the editor some time to send the paper out for review. He first sought input from a member of the editorial board, we were told. My cover letter was apparently convincing, as the manuscript was sent for review and we got three referee reports on this. I won't quote them in full (I don't have their permission), but they can be summarized as:

  1. After a considerable amount of thinking, this is appropriate for publication in PRB. The multiple erroneous references cited show it is valuable, though it shouldn't be needed in an ideal world. [Then some useful comments for improving the work. Thanks!]
  2. The basic principles are well-established, the derivations are mathematically simple. Reject.
  3. I’m in favor of publishing the paper; whether in PRB is an editorial decision best left to the PRB editors. [Also known as the “no risk” approach to refereeing ;-)]

Editor decided paper couldn't be published, but we could resubmit if we felt we had a strong response. It underwent further review, during which one of the referees said something very interesting:

I have checked "Enough significant new physics? No" both times, but still tend to agree with the third referee -- this should be published somewhere

I think this is interesting, because it shows why correcting the scientific record is hard: it is considered not new science. The current standards for publication highlight (with some good reason) the original and the sexy. But while doing so, we need to keep room for corrections, comments and allow the discussion to go on, through peer-reviewed articles.

Another interesting point, made by the reviewer, is the unstated comment that maybe PRB is too good a journal for this sort of work. I think it also shows that corrections and pedagogical papers are deemed less important than your “regular” research article.

Finally, in our case, the paper was accepted and is now published. We managed to convince the reviewers and editor of the value of our paper to the community, which should be in my eye an important yardstick for publication.

Let me know your thoughts, in comments or on twitter. Also: what do you think should come next? Contacting the authors? Post-publication peer review?