PLOS ONE Meta-analysis on Acupuncture in Pain Management Spins Out Undue Recommendations

Science communicators are no strangers to spin in the reporting of scientific studies, especially in Press Releases. This is a favorite tactic of aficionados and researchers alike in the so-called ‘complementary and alternative medicine’ (CAM), which includes acupuncture — a pre-scientific therapeutic modality originating in ancient China with roots in medical astrology and ignorance of human anatomy and physiology. I have earlier written several times on an issue that I continue to find rather perplexing: when it comes to publishing studies on CAM research, the usually-high publication standards of the premier open access journal PLOS ONE appear to be ignored, in the context of both primary research and systemic, quantitative and analytical reviews.

So, yes. I know this happens (even if I don’t know why). Nonetheless, coming across the same phenomenon – again – in a recent PLOS ONE article was more than a little disconcerting, especially since the abstract bore evidence of a severe disconnect between the results reported and conclusions drawn, and the first section in any scientific research report which readers happen upon is the abstract. I refer to the meta-analytical report published in PLOS ONE last month, authored by a group largely from the Taipei Medical University in Taiwan, with Wu Ming-Shun as the lead, entitled: The Efficacy of Acupuncture in Post-Operative Pain Management: A Systematic Review and Meta-Analysis.

This study is a systematic review and meta-analysis undertaken by the authors to evaluate if, in randomized controlled trials (RCTs) of adult subjects, acupuncture and related modalities are effective in treating postoperative acute pain (resulting from trauma of or during surgery), for which the current standard of medical care is the use of opioid analgesics despite its undesirable side-effects. For the analytical evaluation, the researchers pulled publication data from established sources and repositories, namely, MEDLINE, Cochrane Library, and EMBASE databases, ultimately incorporating 682 adult, middle-aged patients, of whom a little over half received acupuncture or related treatments, whereas the remainder received regular medical therapy (control group).

The abstract being neatly divided into section summaries, let me look at it first, especially since I find it so remarkable. (I have retained the exact text of the abstract, but rearranged the sentences in points and adjusted some fonts for emphasis.)

Conclusion

1. Our findings indicate that certain modes of acupuncture improved postoperative pain on the first day after surgery and reduced opioid use.
2. Our findings support the use of acupuncture as adjuvant therapy in treating postoperative pain.

Peachy, right? Except, not quite. Let’s just look a few lines up at the actual results, as presented in the abstract.

Observations reported by the authors:

The authors found that:

1. “patients treated with acupuncture or related techniques had less pain and used less opioid analgesics on Day 1 after surgery compared with those treated with control (P < 0.001).”
   
   — WOW. That’s something, right? From their analysis of multiple published studies, the authors determined that patients receiving acupuncture had less overall pain and were in less acute pain (which would necessitate use of opioid analgesics on the first day right after surgery). But do their overall observations support this claim?
2. “Sensitivity analysis using the leave-one-out approach indicated the findings are reliable and are not dependent on any one study. In addition, no publication bias was detected.”
   
   — How wonderful! Publication bias in handling a large number of research papers occurs when a smaller number of papers are chosen or given preference based on their conclusion that differs significantly from the overall conclusion drawn from the whole body of research in that field. Being able to rule out publication bias during quantitative reviews (such as a meta-analysis) is, therefore, of cardinal importance.
   — However, the authors acknowledge that the data quality was moderate, with considerable heterogeneity and a high risk of detection bias due to inadequate blinding – which is, of course, a problem common to many, many clinical studies involving CAM modalities.
3. Subgroup analysis [which means, independent analysis of specific endpoints in a subset of the whole group in order to tease out specific patterns and understand their significance] revealed that:
   • When compared to Control (non-acupuncture) treatment, conventional acupuncture and transcutaneous electric acupoint stimulation (TEAS) were associated with less pain on postoperative day 1, while electroacupuncture was similar to control (P = 0.116).
     — Whoa. So, one of the acupuncture modalities had no difference in its outcomes, both pain score and opioid use, compared to the control treatments. Hmmm. (I have earlier noted my concerns about the modality of electroacupuncture elsewhere.)
     — Interestingly, conventional acupuncture and control groups had used similar quantities of opioid analgesics.
   • TEAS was associated with significantly greater reduction in opioid analgesic use on Day 1 post surgery than control (P < 0.001).
     — However, two of the five TEAS studies considered shows no significant difference in pain score (Figure 4) and opioid use (Figure 5) compared to the control.
     — The issue of inadequate control often plagues the electrical acu-stimulation studies, including electroacupuncture. For some reason, investigators prefer not to use the most pertinent control, that is non-specific stimulation at non-acupuncture or irrelevant acupuncture points; without this control, it is impossible to determine that the observed effect (whatever it might be) is due to the acupuncture-procedure, and all the putative mechanisms of acupuncture analgesia (elaborated in the Discussion section) are rather moot. (In this connection, “Tooth Fairy Science” verily jumps to mind.)
   • However conventional acupuncture and electroacupuncture showed no benefit in reducing opioid analgesic use compared with control (P ≥ 0.142).
     — Let that sink in a little bit: when compared with non-acupuncture treatments, conventional acupuncture and electroacupuncture showed NO BENEFIT in the primary endpoint, i.e. reduction in opioid analgesic use following surgery (Figure 5).

So, what exactly is the generalized recommendation for acupuncture use written in the Conclusion based on?

Supportive Evidence in the Body Text?

In the Introduction, in the paragraph describing Acupuncture, the authors include two papers prominently thusly: Two prior meta-analyses evaluated the use of acupuncture in treating postoperative pain [1,8]. One focused on the use of acupuncture following back surgery [8]. The other, which was performed in 2008, evaluated the use of acupuncture more broadly following surgery [1]. Use of the word ‘evaluated’ is interesting, because it doesn’t indicate which way the evaluation went. So I decided to check at the source.

• Reference 1 from 2008 searched clinical databases (Medline, CINAHL, The Cochrane Central Register of Controlled Trials, and Scopus for RCTs) to seek reports on acupuncture in postoperative pain management, gathering 1166 patients more than half of whom received acupuncture.
  — Their own results showed that (a) most of the postoperative pain score studies demonstrated no significant difference between acupuncture and placebo control groups (Figure 2); (b) by subgroup analysis, pain score reduction at 24h (day 1) in acupuncture was not statistically significant compared with placebo, and (c) opioid pain medication consumption was significantly lower in the acupuncture group ONLY when it was administered before the operation (thereby, most likely priming the patient to favor acupuncture in later scores).
• Reference 8 from 2013 was NOT a meta-analysis, nor was it focused on “acupuncture following back surgery”. Rather, it was a multicenter, parallel, randomized, sham-controlled clinical trial done in 3 Korean hospitals (with a total of 116 patients analyzed), comparing acupuncture treatments with sham acupuncture (with semi-blunt, non-penetrating sham needles used at non-acupuncture points) as control in adult patients with non-specific chronic lower back pain. The outcome measures were defined as ‘bothersomeness’, a surrogate for Quality of Life measure, as well as pain intensity, both measured via a subjective visual analog scale, at cessation of treatment, week 8 (primary endpoint), and 3- and 6-month follow up.
  — Their own results showed that (a) regardless of real or sham acupuncture, both outcome measures recorded significant drops in the scores (Table 2), which is probably not unexpected because all patients had positive expectations from the treatment, and the fact that both groups were advised to undertake physical exercise during the study period; (b) there was no difference between real and sham acupuncture in terms of a quantitative disability measurement called Oswestry Disability Index, even though its kinetics mirrored the other outcomes (Figure 3). (Needless to say, the authors’ conclusions have more positive spins on the observed results and putative explanations thereof.)

It appears, then, that the authoritative definitiveness of these two references was, to say the least, not well-established. But their inclusion was interesting to me in a different way, too – in the non-inclusion of some other significant and clinically relevant papers, amongst which were:

• Yuan et al., Spine 2008: A systematic review of the effectiveness of acupuncture for low back pain, incorporating 23 RCTs and 6359 patients, which showed that the effect of real acupuncture treatments was no different than sham acupuncture treatments (thereby corroborating my impression of the 2013 study mentioned above; others have had the same impression).
• Haake et al., Archives of Internal Medicine 2007: A large German multicenter, blinded, parallel-group, acupuncture RCT for chronic low back pain, incorporating 1162 adult patients who were randomly assigned to real or sham acupuncture, or a conventional regimen with medication and exercise. At 6 months, there was no difference in the response rate of true vs. sham acupuncture, though both achieved better response than conventional treatment. Again, this was not surprising given that patients in the acupuncture arms (real and sham) received personal physician attention which likely promoted expectation, as noted by Professor Edzard Ernst and others.

17th Century Acupuncture chart, showing putative link between lungs and acu-points on hand. ©Wellcome Library, London; used under CC-BY-4.0

Why do I consider these papers significant? Aside from their analytic value, these papers (neither of which demonstrate clinical efficacy of acupuncture) were mentioned by a 2010 NEJM article on the use of acupuncture for –you guessed it!– chronic low back pain, authored by Brian Berman, Helene Langevin and Claudia Witt, all considered academic experts in acupuncture and other CAM modalities. [Why yes, I am making an Argument from Authority, just not the exact argument the CAM proponents may be looking for.]

That again brings to me the question I asked above: why, based on what observations and empirical evidence, are the authors of the PLOS ONE paper under discussion making a generalized recommendation for adoption of acupuncture as a therapeutic modality? Given the implausibility of acupuncture and its mythical system of meridians, as well as the observed absence of specificity in its manipulations (no difference in effects between true vs sham acupuncture), this question becomes even more important.

Speaking of Witt, that NEJM article also references Witt et al., American Journal of Epidemiology 2006 — a very large German RCT evaluating the clinical and economic effectiveness of acupuncture for chronic low back pain, incorporating 11,630 adult patients, of whom 3093 were randomly assigned to acupuncture either to be received immediately (treatment group) or 3 months later (control group), and the remaining 8000+ people deliberately chose acupuncture. (Talk about patient expectation!) All patients were allowed to conventional treatment as needed. The back function was measured for these patients at baseline, and then at 3- and 6 months. At 3 months (cessation of treatment for those who were treated), the back function scores increased by a statistically significant 9.4 points for acupuncture treatments over control; once the control started receiving acupuncture after the 3rd month (until the 6th), their scores increased, too, almost to the levels of those who had been receiving acupuncture for the first three months. Clearly, acupuncture showed better response compared to not treating. Interesting, however, were the observations that:

1. 3-months of acupuncture treatment seemed to maintain back function scores steady up until 6 months (Figure 2).
2. About 2 in every 10 patients receiving acupuncture or not were prescribed pain-killer medications; which also means that for 8 out of 10 patients, the pain was likely not so severe as to require rescue pain medication for immediate use.
3. those who chose to receive acupuncture (non-randomized) always showed better scores (higher function, lower pain) compared to the patients who were randomized to the acupuncture group (Table 2); again, patient expectation comes to mind, especially since none of the participants were blinded to the treatment options. This can explain in part why younger and more well-educated patients, and therefore likely to be subject to confirmation bias towards acupuncture (in a situation where acupuncture is accepted, popular, well-used, without taking too deep a peek into its dubious origins) appeared to show better responses.
4. And of course, by the design of this trial, it is impossible to conclude if the acupuncture manipulations were translating to specific, targeted physiological changes — a situation that is compounded by the fact that a large number of patients in this study were treated with acupuncture by German physicians who also offered conventional medicine and other treatments.

Is it any wonder, then, that acupuncture has been aptly termed Theatrical Placebo by neuroscientists and science communicators David Colquhoun and Steve Novella? (I highly recommend reading this freely available 2013 editorial essay in the journal Anesthesia & Analgesia.)

Primary Paper discussed (Others are hyperlinked in the post):

Wu, M., Chen, K., Chen, I., Huang, S., Tzeng, P., Yeh, M., Lee, F., Lin, J., & Chen, C. (2016). The Efficacy of Acupuncture in Post-Operative Pain Management: A Systematic Review and Meta-Analysis PLOS ONE, 11 (3) DOI: 10.1371/journal.pone.0150367