So… it has come to this.

Nature Medicine (December 2012 Volume 18, Issue 12) Table of Contents, delivered to my inbox yesterday, immediately drew my attention to an interesting news item written by Elie Dolgin, a science-writer and news editor at Nature. Kudos to Mr. Dolgin and/or whoever came up with the eerily engaging headline; I was riveted by it: Biomedical grant awarded by ‘American Idol’-style public vote.¹

The Biomedical Research Institute (BRI) of the Brigham and Women’s Hospital (BWH), a hospital affiliated to the Harvard University in Boston, was trying to find a solution to an age-old problem: from a multitude of applications, which biomedical research project to fund? The dilemma before them was, as Elie Dolgin explained in the report:

With so many top-notch research proposals seeking funding but only limited grant money to go around, deciding which among the best of the best projects to support is no easy task. What if you have a number of equally commendable applications and you don’t know how to break the tie?

So how did BRI/BWH resolve this issue? To use a mot juste neologism: crowdsourcing.

… But in an unconventional twist, the Brigham and Women’s Hospital (BWH) has opted to let the general public act as scientific judge and jury. After six weeks of online voting and nearly 6,500 votes cast, the decision was in… a project designed to explore how best to integrate genomic sequencing into routine medical care for healthy newborns had won the inaugural BRIght Futures Prize. The project’s leader, clinical geneticist Robert Green, and his team received a $100,000 research grant from the BWH’s Biomedical Research Institute (BRI).

Those familiar with the public judging format in American Idol (confession: guilty! Yes, I watch the episodes and vote via the Facebook App) will immediately see the parallel. In the American Idol competition, the panel of judges rate the hopeful contestants in several rounds, holding auditions in several cities, eventually winnowing the list to a much smaller and more manageable number. From that stage onwards, fans and followers of the competition start sending in their votes by phone, text message and/or web-based applications every week. The votes counts become crucial in determining which contestants stay on and who are eliminated during these rounds. In the final round, the judgement of the winner of the competition is completely by popular votes.

So, it has come to this?

The BRI/BWH approach to science-funding’s eternal dilemma: Leaving the fate of critical scientific projects cavalierly to the mercies of a swath of humanity who is potentially unable to comprehend the necessity, nor the significance, of such projects? Or, truly an unconventional, unusual, remarkable remedy – with a touch of whimsy?

Funding agencies (such as the National Institutes of Health, NIH) usually convene a panel of experts to perform a scientific review (a.k.a. Study Sections) of submitted project proposals; they evaluate the scientific merits, as well as weaknesses, of the hypotheses presented, grading/scoring the proposals on diverse criteria, such as significance, approach, innovation, abilities and track-records of the investigators, and the physical environment in which the study would be done. The deliberations are done in camera, following which consensus scores are considered to determine which proposals would be funded.

Although the process may seem like background machinations and arcane rituals of a secret, elite cabal, in reality it works out rather well. Each application is assigned primary, secondary, and tertiary reviewers. Reviewers, each respected academics in their own fields, bring diversity of opinions and ideas to the discussion, which oftentimes end up helping an applicant enhance the scientific merit of the application. Aside from funding an individual proposal, a goal of these scientific reviews is also to identify and assess a candidate scientist’s potential to become an important, productive and independent contributor towards the advancement of science.

However, as with any other collaborative process, there are several downsides to the reality of these scientific review groups. Usually, as a means to dealing with the sheer workload, reviewers submit preliminary grades and critiques a week prior to the study section, during which grant proposals are reviewed and discussed in order of the preliminary scores; proposals with lower preliminary scores, generally the bottom 40%, are not even reviewed. Even with the proposals that end up being discussed, the assigned primary reviewer’s comments are most important and may sway the group in a positive or negative direction. In addition, the review committee has no say in the final funding decision. (Information courtesy: a presentation by Dr. Lynn Schnapp, Pulmonary and Critical Care Medicine, University of University of Washington School of Medicine.)

Therefore, obviously, in the current funding model, the process is not optimal and there is no dearth of challenges. In fact, in yesterday’s issue of Nature, Joshua Nicholson (Virginia Tech), and John Ioannidis (Stanford University) discuss² the growing concern that “funding systems based on peer review, such as those currently used by the NIH, encourage conformity if not mediocrity, and that such systems may ignore truly innovative thinkers“, noting that “many US authors of the most innovative and influential papers in the life sciences do not receive NIH funding“, a fact which doesn’t speak well of the current system. In their comment, Nicholson and Ioannidis indicate “More alternative funding modes should be tested in pilot schemes and in experimental controlled studies of optimizing funding processes.“

Et voilà! Enter the crowdsourcing of funding decisions. From the news report by Elie Dolgin:

Jacqueline Slavik, executive director of the BRI, maintains that the decision process was scientifically “solid,” as every research proposal “went through all of the traditional scientific review steps and a rigorous peer review” before reaching the final voting stage.

First, review committees with expertise in personalized medicine and systems immunology—the two subject areas for which the BRI solicited proposals for the prize—winnowed the list of applicants down to a series of semifinalists. Each selected applicant then made an in-person pitch to the BRI’s Research Oversight Committee, which ultimately chose the three proposals that were presented to the public. Finally, the three research groups behind those projects worked together with the hospital’s public affairs team to create a series of videos and brief nontechnical descriptions about the proposals that were hosted on the voting site.

“It’s a new way of trying to decide who gets the money when you have equally meritorious projects,” says Slavik. “We could flip a coin,” she quips. Instead, by engaging the public, “we achieve several goals at once,” without sacrificing scientific rigor.

“Our goal was really to engage the Brigham community at large,” notes Lesley Solomon, director of strategy and innovation at the BRI. “We want the world to know about the breadth and depth of the research that goes on here.”

That, right there, is a very laudable goal, and one that I, as someone who understands the importance of science communication and dissemination of scientific information, can readily fall behind. In fact, the experience of Robert Plenge, a rheumatologist and geneticist at BWH and one of the finalists prior to the popular vote stage, corroborates this, and is exciting and immensely significant in many ways:

For the prize, (Plenge) proposed to study the molecular signature generated by people with rheumatoid arthritis in response to anti–interleukin-6 therapy. After being selected as a finalist, several small news outlets in his local town of Wellesley, Massachusetts, ran articles and broadcasts featuring his work. “Suddenly, I could talk to friends and neighbors who aren’t in science about what I do for a living,” Plenge says. “At the end of all this, even though I didn’t win the prize, I have actually benefited from the process.”

This is not a traditional form of engagement of a working scientist with other scientists and the general populace. But these are the days of heightened awareness and web-presence, and the ubiquity of online social networks has transformed the modes in which information can be interchanged. I do think that the engagement of the general populace in the funding decision-making opens up the opportunities of dialogs between hithertofore dissociated parties, namely, the scientists and general public. This can dispel the oft-portrayed image of scientists in ivory towers and can bring greater enlightenment and scientific discourse to society, kind of a win-win.

At the same time, there are two major caveats that need to be adequately considered as well. One, the types of scientific proposals, focusing on Basic versus Applied Science. Scientific questions in biomedical disciplines are plentiful; the need for knowledge and understanding in several critical areas of biomedical sciences is yet unmet – such as studies of various human, primate and other mammalian disease systems, pathogenesis, progression, diagnostics and therapeutics – areas of knowledge that, if gained, will assist in the alleviation of human sufferings. This kind of studies (especially, what is known as translational research), is understandably easier to communicate to the layperson and seek their approval for, in terms of funding. However, it is very, very important to remember that basic science of today provides the ideas and tools for applied science of tomorrow, but may not excite the imagination and understanding of the layperson to the same extent as would the applied sciences with the ready promise of visible results.

For example, the study of Candida albicans gene mutations may not seem immediately important at the first glance, especially because most such cases of Candida infections, seen clinically, may easily be treated with available antifungal drugs using a “nail, meet hammer” approach. However, because someone had once done just that in the past – namely, studied the effects of certain genes in that microbe – we can now in the position to speculate that a particular protein of Candida may be helping the bug to gain traction in the mouths of HIV-infected individuals and cause the condition known as oropharyngeal thrush. Long term benefits of basic science research may not be immediately visible to those who are not engaged in it, and I fear that depending upon the enthusiasm of the lay public as an indicator of the scientific merit of a grant proposal may (a) put a spotlight on applied science research to the detriment of basic science, and/or (b) force basic scientists to take to unsavory spin techniques in order to stay relevant in the public eye. One way this eventuality may be warded off is if the universities and other institutions for higher learning were to evolve mechanisms for alternative funding, and support the basic science research efforts on a different track.

The second caveat has to do with the kind of people that are engaged during these public discourses. To make the BRI/BWH model work, the scientific bodies and funding agencies need to put in place mechanisms for adequate chaff-grain separation. I may sound bitter and cynical when I say I often find it difficult to accept prima facie opinions from laypersons about scientific matters, but I assure you, gentle reader, that such cynicism is strictly the result of encountering – and sometimes wading through – the comment streams after newspaper reports and/or blog posts, as well as in social media. Suffice it to say, oftentimes, the quality (or more precisely, the resounding lack thereof) of the comments makes me weep for the future of humanity.

Ideally, scientists would expect serious, informed opinions from laypersons who would evaluate scientific proposals in real-life terms in order to determine their fundability. This is conferring a huge power to the vox populi, which, of course, comes with great responsibility – and needs a mechanism to ensure the said responsibility. When ill-informed, ignorant, uninitiated, ideology- or faith-driven public opinions run amok and gain political traction, we often end up with well-funded temples to pseudoscience; NCCAM is a testament to that. Would that kind of an opinion pool be beneficial and useful to the progress of real science and knowledge? If a study proposes to check for, say, efficacy and penetration of the already-successful-elsewhere oral polio vaccination in some remote corner of the world, and seeks funding to do it, would it really help to have a deluge of comments and hate-mails from dedicated anti-vaccination groups and/or homeopathy proponents? If a proposed study involves absolutely necessary animal experiments to investigate infectious disease pathogenesis, would the opinions of radical anti-science anti-animal experiment folks, whose minds are closed to even any scientific necessity, be of any benefit?

Crowdsourcing of any kind works best with an informed crowd; if people are given the facts, they will make up their own minds – of that I am hopeful. Differing opinions and debates are not hindrances in such a situation; they may well assist in the refinement of grant proposals. Therefore, effective science communication is a must; a considerable amount of thought and efforts, on part of scientists and funding agencies, needs to be expended towards that end, prior to the august endeavors to harness the crowdsourcing power.

Further reading:

1. Nature Medicine, 18:1721 (06 December 2012); doi:10.1038/nm1212-1721
2. Nature, 492:34–36 (06 December 2012); doi:10.1038/492034a