Via Teh Grauniad, science correspondent Ian Sample reported today on a phenomenon that is at once hilarious and extremely concerning for the academic science research community.

Allow me to summarize a part of Ian’s report.

  • Three enterprising MIT grad students, Jeremy Stribling, Dan Aguayo and Maxwell Krohn, wanted to run a sting operation to reveal the practice, by some scientific conferences of dubious provenance, of indiscriminately accepting papers of dubious significance with a view to bringing in registration fees.
  • They wrote a computer program — Naturally! — called SCIgen, that generated “random Computer Science research papers, including graphs, figures, and citations”, using a “hand-written context-free grammar“. Needless to say, the outputs are rather… interesting! (Check the SCIgen page link for glorious examples of papers and randomly-generated talks.)
  • SciGen ably churned out (“churn”, used by Ian in his report, is le mot juste here) science-y-sounding nonsensical prose, which they compiled in the format of a Computer Science paper, sent it out, and lo! And behold! It was accepted by a conference; another product of the same program was, however, rejected. Upon asking for reasons for the rejection, they received a strange reply – complete with references – stating why the reasons could not be provided. (Do read this gem!) This was in 2005.
  • M/s Stribling et al., by way of public service, provided SciGen free under a GP License.

Harmless “gotcha” fun aimed at bogus conferences, you think? Well, think again. As Ian reports:

This week, Nature reported, French researcher Cyril Labbé revealed that 16 gobbledegook papers created by SCIgen had been used by German academic publisher Springer. More than 100 more fake SCIgen papers were published by the US Institute of Electrical and Electronic Engineers (IEEE). Both organisations have now taken steps to remove the papers.

YIKES!!

The above-mentioned Nature report by Richard Van Noorden, however, does mention an important detail which didn’t quite come across in Ian’s Guardian report, and it IS an important distinction, to me at least. These spurious computer-generated papers were mostly not actually peer-reviewed research articles published in peer-reviewed journals, but were in the format of work presented at conferences, which finds its way into conference proceedings that are usually published by the journals as separate issues dedicated to the specific conferences. I don’t know if I am making an error in this interpretation (please feel free to let me know in the comments), but in bioscience-related disciplines (including mine), conference presentations are always considered preliminary work, and “work-in-progress”, and the only acceptable ratification of such work comes from it passing through a peer review process and being published as a research article.

The good news in both Nature and Guardian reports is that M. Labbé, who is a computer science researcher at the Joseph Fourier University in Grenoble, France, has devised a web-based way to detect SCIgen-generated gobbledygook (Oh, how I love that word!). The not-so-good news comes in form of an ominous possibility from one of the original authors of SCIgen, Maxwell Krohn. Ian reports:

Krohn sees an arms race brewing, in which computers churn out ever more convincing papers, while other programs are designed to sniff them out. Does he regret the beast he helped unleash, or is he proud that it is still exposing weaknesses in the world of science? “I’m psyched, it’s so great. These papers are so funny, you read them and can’t help but laugh. They are total bullshit. And I don’t see this going away.”

Begun, the computer-generated fake paper war has?

An uncharitable part of me (which I am suitably ashamed to admit) did feel a wave of schadenfreude breeze over momentarily, bolstered by the thought that this would have never happened in a bioscience-related discipline, which emphasizes laboratory-based, hypothesis-driven, empirical “wet-work”. And for that moment, I was glad for that much-maligned process of Peer Review, as well as peer reviewers (yes, including the notorious — and much loved on the internet — Reviewer #3!), which – when done properly – acts as a sieve to separate the grain from the chaff.

But my bubble burst when I realized that “Peer Review” as a process is only as good as the peer reviewer. Biological Sciences have been a witness to a worrisome trend, the steady infiltration of “quackademia” into mainstream sciences and the burgeoning of less-than-stellar publications (that’s putting it mildly) with their uncritical, often-credulous parodies of peer review. In addition, there is the deplorable issue of Science-by-Press-Release and introduction of spin in the communication of scientific research.

Therefore, while Ian’s report – which inspired me to write this blog post today – may not pertain to my discipline directly, it acutely highlights the importance of scientific integrity and scholarly ethics on part of both researchers and journal editors, and not sacrificing these tenets of good science at the altar of “publish-or-perish”.