Heh! Right now I have this stupid grin on my face, because I caught this glaring error in a published paper. Okay, it is a paper on homeopathy referenced in a godawful homeopathy website (that I mentioned in my yesterday’s write-up), but nevertheless.
Imagine, I start the introduction to a scientific paper like this:
Disease A is very important and prevalent in certain regions like blah-blah-blah. The causal organism of disease A is known to be virus X with wide seroprevalence in the said regions.
—Paragraph—
Doing cartwheel requires strength of the arms as well as of the muscles of the lower back. People who are in poor shape are afraid of doing cartwheels.
—Paragraph—
Currently, there are no treatments for virus X, nor is any vaccine available.
—Paragraph—
Therefore, this study was aimed at discovering whether doing cartwheels can eradicate the virus X.
I kid you not, my dear readers. As strange as it may seem, this is exactly how a 2010 paper that purportedly examined the effect of homeopathic belladonna on Japanese Encephalitis (JE) Virus infection starts (Ref: American Journal of Infectious Diseases 6 (2): 24-28, 2010). If you don’t believe me, read the PDF here.
The authors start with a description of the problem of JE and its geographical distribution and prevalence in Southeast Asia (which is fine), abruptly seguing into two paragraphs on the provenance of belladonna, alkaloids derived from that plant (which are not in doubt), before veering again into features of JE and vaccine prophylaxis for JE, which has been effective in several countries in reducing the incidence, commenting on the relative lack of specific antiviral therapy against JE, and finally touching upon a couple of new investigational therapies. That’s one hell of a run-on sentence, but I hope you are with me so far.
And then they end the intro with this aim statement:
There is thus a real need for antivirals that can reduce the toll of death and neurological sequelae resulting from infection with JE virus… Therefore, this study was aimed to see whether ultradiluted belladonna has a role in this infection.
Boom! … Wait, what?
Yeah. There is nothing in the introduction that remotely prepares the reader for this massive stretch from theory to an implausible study aim, this veritable leap from fact to fiction. Nothing in the introduction mentions anything about ‘ultradiluted belladonna’, nothing provides any inkling as to its purported efficacy, or indeed, as to why this particular nostrum was chosen for study.
Let me be very clear: Atropa belladonna and its derived alkaloids ≠ ultradiluted homeopathic belladonna used in this study. Ain’t quite the same.
I should also clarify that the doses used (3, 6, 30 and 200 – the authors don’t mention the designation) possibly refer to 3, 6, 30 and 200X (the ‘X’-designation is prevalent in India, the source of the preparations), which mean, respectively, the dilutions of a thousandth (10-3), a millionth (10-6) – both being fairly plausible dilutions, with reasonable amounts of the original alkaloid left – to be followed by 10-30 and 10-200 – both of which are ridiculous dilutions in which no remnant or trace of the original alkaloid can possibly remain. At these dilutions, only the solvent and dissolved impurities are around.
The experiment itself used a fairly straightforward technique of embryonic chick egg inoculation and enumeration of JE virus-induced pock marks on the chorioallantoic membrane. The virus, deposited on this membrane inside the egg, grows in situ and produces the pock marks; higher number of these marks indicates higher viral proliferation, and this is the variable authors estimated.
The test group basically checks the number of pock marks in presence of their therapy. Naturally, the appropriate controls are (a) virus control, i.e. virus without therapy, (b) therapy control, i.e. the treatment without the virus, and (c) controls for buffer (BSA in PBS) and vehicle (sterile water), treated in the same way (‘potentized’) as the therapy. The authors seem to have these covered in the methods section. So far, good.
Let’s delve into the observations. The authors contend that:
Encouraging results were obtained with all ultradiluted Belladonna 3, 6, 30, 200 preparations used in this study showing inhibition of viral growth on CAM (‘chorioallantoic membrane’). [N.B. Caveat lector: Although they mention in the discussion beneficial effects on JE in mice also, there is no mouse data presented in this paper]
Where does this extraordinary claim come from? The authors lay out their results in the following table.
As you can see, they claim significance of the homeopathic dilutions, because the number of pock marks is significantly less with their administration, than with the virus alone. It is indeed significant (but not in the way the authors intended) that they chose not to show any data from the controls, just restricting the observation to ‘no significant finding’ for the control.
The authors also chose to ignore a very vital comparison: Even in absence of the homeopathic therapy (i.e. in the virus control), the virus-induced pock marks vary significantly (p<0.001, I plugged the numbers to check) from experiment to experiment – say, between the controls for the belladonna 3 and 30 test groups, which should have been nearly identical, because conditions are the same. This inherent/experimental variability in their control groups immediately invalidates their conclusion of difference between control and test groups, because one cannot satisfactorily estimate if the observed effect is due to the treatment or simply random chance events.
The authors also fail to mention why a higher dilution (therefore, more potent according to homeopathic principles) of 6X ends up with more pock marks than a lower dilution (3X). They plot the average counts in the paper, but the graph contains no error bars, leaving no visual means to ascertain the dispersal of their test data vis-à-vis the control.
So, in conclusion, for a study that is hailed by the homeopathy website(s) as the evidence that a homeopathic nostrum works against the JE virus, the study is inadequate (only ex vivo), as well as plain darn wrong because it draws wrong conclusions stemming from a major logical flaw: confirmation bias. The authors saw in the observations only what they wanted to see, and didn’t analyze the data critically. This study is sloppy, and I honestly wonder how it even passed peer review to be published in the American Journal of Infectious Disease (admittedly not a journal with whose academic standards I am familiar).
But what irritates me the most – perhaps a tad irrationally – is that the primary authors are from my own city and are employed in prestigious and historic institutions for clinical and basic research. To me, this only reflects the sad reality of the inexplicable iron grip on the general populace, including – shamefully – scientists, that various pseudoscientific, nonsensical, faith- and superstition-based modalities, such as homeopathy, continue to exercise. Geek Nation, anyone?
Hi Kausik,
Thanks for an interesting read. It seems almost absurd that homeopathic remedies would be considered as a therapy for Japanese encephalitis when research focus, time and money could instead be geared to looking for other drug therapies.
I was going to ask why they picked Atropa belladonna , however after reading the below, a quote from the paper, I think it speaks volumes in itself:
As you discussed in your previous posts, I think that this hope of curing horrible illnesses with these ancient remedies stems from religious beliefs, that almost boarder on magical. These dogmas are often instilled since childhood and are difficult to detach from, even for scientists working at prestigious Indian institutions. (Must read Geek Nation!)
Thank you, Laura. Let me make sure that I have conveyed my concerns correctly. Their description of Atropa belladonna has nothing to do with the ‘belladonna’ 30X or 200X that they have used in the study to look for efficacy. That’s what makes this description so irritating and veritably misleading. It’s almost as if they were trying to convey – it would be fraudulent if they did – that their study had anything to do with the effects of Atropa alkaloids or metabolic dervivatives thereof. It didn’t. They used a homeopathic preparation called ‘Belladonna’, prepared elsewhere, and nothwithstanding whatever it is called, the 30X and 200X dilutions (found ‘most effective’ according to their figure 2) cannot have any meaningful trace of the original substance in it.
You’re right about the connections between belief in homeopathy and alternative medicines in general, and religious faith. It is quite possible that one presupposes the other, because both belief systems thrive on irrationality.
I had to plot the data to see what’s happening:
It looks like there’s a clear effect, but the dose-response in the control is odd. It’s also not clear from the methods section that the controls and treatments were treated in the same way: it seems as if in the control the dummy treatment (i.e. water) was applied at the same time as the virus, whereas it was applied several minutes earlier in the treatment.
It’s also odd the way the control and treatment are presented in the methods, in different sections. Why aren’t they in the same section, in which it’s stated that they were both applied in the same way? It implies that they weren’t together. Or is this a standard way of presenting results in medical journals?
Nice graph, Bob! Actually, they have plotted the exact same results in Figure 2, which is identical to what you’d get if you joined your control and treatment points with lines.
The ‘effect’ that you mention is clear, yes, as long as you consider only the relative differences between treatment and control. This is what they capitalized on also. But consider this: according to their experimental protocol (described in methods), ALL their controls are identical, JE virus diluted at 10-3, delivered to the chorioallantoic membrane of 200 or 500 eggs. Only the treatment groups differ. Therefore, all their controls (your black circles) should have shown approximately same mean pock counts. They don’t. The numbers vary significantly – within what constitutes the same group. It is almost as if it were a single group, with significantly high within-group variance. And that casts a shadow on the validity of their treatment-control comparisons.
And no, that is not the standard way of presenting results in medical journals. It is simply sloppy writing and sloppier reviewing/editing that have resulted in this mess of a paper.
Looking at the graph, I’m pretty sure there would still be a significant effect of treatment, even accounting for the variation between groups. But the similarity in the dose response is odd.
Would you mind explaining how? I mean, the situation is this: enumerating the pock counts
virus alone (91), virus alone (56) – difference significant
virus alone (91), virus + treatment (21) – difference significant
Just how does one draw a conclusion of treatment efficacy from this?
A part of it may just be ex vivo phenomenology. When one increases the N, even small differences may become statistically significant, no matter if they are not biologically relevant. And N of 200 or 500 is pretty large, no?
I wish I had access to their raw data and seen the numbers for myself. In addition, they should have described more of their treatment in Materials and Methods. The 3X and 6X dilutions are still reasonable – but of course I have no clue what they started with, and apart from their buffers, what else there was in their diluted treatments.
Hey know-all-enthu-cutlet – When no designation is mentioned, the potency is “C”.
Geeez!
>>I should also clarify that the doses used (3, 6, 30 and 200 – the authors don’t mention the >>designation) possibly refer to 3, 6, 30 and 200X (the ‘X’-designation is prevalent in India, >>the source of the preparations),
Correction:
Hey know-all-enthu-cutlet – When no designation is mentioned about the potency, the dilution is “C”.
Geeez!
>>I should also clarify that the doses used (3, 6, 30 and 200 – the authors don’t mention the >>designation) possibly refer to 3, 6, 30 and 200X (the ‘X’-designation is prevalent in India, >>the source of the preparations),
thanks for info,
i am a homoeopahic graduate dr.those who have doubt and denying real fact, can i know they are doctors or scientist!!!!!!!!!!!!!!!!!,so what are they??????????????????just search the net.it was researched in school of tropical medicine kolkata with renowned virologist.pathologists,and professor of materia medica..there are total gr.the material was published in NATURE journal..so apart from medical personal all are layman..they have no right to say..MIND THAT..
only medical postgraduate,undergraduate dr, have right to question..we will clarify their doubts..but remind that illogical questions are thrown out.
i am a homoeopahic graduate dr.those who have doubt and denying real fact, can i know they are doctors or scientist!!!!!!!!!!!!!!!!!,so what are they??????????????????just search the net.it was researched in school of tropical medicine kolkata with renowned virologist.pathologists,and professor of materia medica..there are total gr.the material was published in NATURE journal..so apart from medical personal all are layman..they have no right to say..MIND THAT..there words have no values…
My commiserations. Your time, effort and money would have been better spent in learning another trade. As such, your argument from authority wouldn’t wash much here. I urge you to try answering the legitimate questions about the methodology of this paper that I have raised.
Bob O´Hara shown (independently) similar results as an paper of Bhaswati Bandyopadhya et.al. The Kauski Data “arguments” is poor, ad-hominem fallacy and straw man fallacy. I.e.:
“The authors also fail to mention why a higher dilution (therefore, more potent according to homeopathic principles) of 6X ends up with more pock marks than a lower dilution (3X).”
No, homeopathy does nos says this. The word “potent” does not mean more chemical reactivity. Your misinterpretation explain the awful and poor “debunks” in this blog. The paper is clear cut of evidence the biological activity of high dilution.
Another evidence of homoeopathy:
http://www.sciencedirect.com/science/article/pii/S0167732215303585
Erm… No. Bob simply took the data as presented in that paper and replotted it for better understanding. As I mentioned in reply, this figure corresponds to the figure 2 in the paper.
I am glad you are aware of the names of these common logical fallacies. However, I suggest you take the trouble to actually read up about them – to understand what they are.
Incorrect. I suggest you read Hahnemann’s Organon, 269, where he has specifically defined potentization and what it is expected to achieve. The paper is ‘clear cut evidence’ of nothing.
And the paper you referred to, I cannot comment on until I have read it.