This study evaluated the reporting quality of sham needles in acupuncture trials, in three time periods between the year 2009 and 2018, using systematic methodology adopted and modified from previous studies [18, 20, 21]. For this systematic evaluation, a checklist was developed based on CONSORT, STRICTA, TIDieR, and previous literature investigating placebo effects in acupuncture treatments. While significant differences of reporting scores among categories were observed, there were no significant differences among time periods; similarly, no significant improvement of reporting scores was observed over time. To our knowledge, this is the first paper to systematically analyze the reporting quality of sham needles used in acupuncture trials.
The results showed that overall, a total of 7 items out of 25 were reported in more than 50% of the studies throughout all time periods. These items include: “number of sham needling”, “depths of insertion (including non-penetration)”, “sham needle retention”, “location of sham needling”, “number of treatment sessions”, “total trial period”, and “blinding”. Only the “treatment regimen” category was reported in more than 50% of the studies throughout all time periods. Items which showed a constantly increasing trend over time and were above 50% reporting rate were: “number of sham needling”, “location of sham needling”, “number of treatment sessions”, “frequency and duration of treatment”, “total trial period”, and “blinding”, implying a continuously increasing trend of reporting the above information in published literature. On the other hand, items which showed a constantly decreasing trend over time and were below 50% reporting rate were: “response sought for sham needle (e.g., de qi, muscle twitch)”, “whether the practitioner and the patient have any interaction during the trial”, “whether there was any discussion regarding the symptoms”, and “information include the term ‘placebo’ or ‘sham’.”
Modified items from STRICTA such as “type of sham needle” were not reported in more than 50% of the studies throughout all time periods, while a previous study found significantly increased reporting of the “specific style of acupuncture,” the corresponding item in STRICTA, over time since the publication of the checklist . Moreover, modified items from TIDieR such as “modification of the needling procedure” was one of the least reported items. Previous studies on the impact of STRICTA and CONSORT on reporting qualities of acupuncture trials showed an increased quality of reporting since the publication of STRICTA in 2010 [18, 20, 21]; in contrast, the results from this study illustrated that reporting quality of sham needles did not improve over time.
Furthermore, items that were not listed in either STRICTA, CONSORT or TIDieR but nonetheless discussed in recent studies to have substantial impact on the placebo effect by sham needles, were reported in very few studies. These items include discussions between practitioner and patients before and during the treatments, information given to patients, and confirmation of whether the information given to patients included the term “sham” or “placebo.” Previous studies discussed the importance of the doctor-patient relationship in placebo effects [26,27,28,29]; recent studies also point out that acupuncture treatment, with its long duration of time for the application of the needles and the communication involved, consequently builds a stronger doctor-patient relationship which may lead to stronger placebo effects [2, 39]. While it would be nonsensical to report all types of communications during the trial, the scope of communications between the practitioner and the patient, i.e., whether the conversation was carried out beyond the scope of the experiment, and whether any kind of interaction to build a trusting relationship between the practitioner and the patient was allowed, would enable the readers to understand the potential non-specific effects involved in the process of the treatment.
On the other hand, literature discuss that the extent of disclosure of information may influence the level of placebo effect on patients [30, 32,33,34]. One study showed that the placebo responses caused by information disclosure modifies the drug response . Lastly, blinding is an important aspect of controlled trials [36, 38]. It is also found to be extremely difficult to achieve in acupuncture trials [35, 37], which requires caution in both the experiment design and the reporting of the experiment.
The findings in this study showing low rates of reporting information about sham needle and its treatment protocol would most likely lead to a gap of reporting qualities between that of acupuncture in general and that of sham needles. Previous studies presented that reporting qualities of acupuncture RCTs improved overall since the publication of STRICTA and CONSORT [18, 20]. One study showed that the best-reported item was adhered to in over 90% while poorest reporting value was 51.1% [18, 20, 21]. In contrast, this study showed that only seven items out of 25 were reported in more than 50% of the studies. Similarly, the overall trend of reporting qualities based on STRICTA was shown to constantly improve in the previous studies [20, 21], while the reporting qualities of sham needles reviewed in this study did not show significant improvement over time.
Low reporting qualities of the controls used in acupuncture trials compared to reporting qualities of the intervention in general puts at risk the accurate analysis of specific effect of acupuncture; and this may require a reappraisal of the current understanding of the effectiveness of acupuncture. As with other types of medical interventions investigated in RCTs, the specific effects of acupuncture have often been analyzed through contrast of the reported effectiveness of the treatment to that of control. Therefore, the data presented in this study might potentially imply that the specific effects of acupuncture might have been misinterpreted due to missing information regarding controls. Furthermore, research such as meta-analyses and systematic reviews based on literature, might have even higher chance of misunderstanding of the effectiveness of acupuncture, simply due to the accumulation of fallacies over time. It may not be pure coincidence that placebos in acupuncture trials have been subject to strong effectiveness, sometimes as effective as verum acupuncture, by a number of researchers for decades [6, 39,40,41,42,43], and might be due to the lack of elaboration on controls used in acupuncture trials.
The limitations of this study include that the scoring system based on the checklist to evaluate reporting qualities. STRICTA, CONSORT and TIDieR were never intended to be employed as rating scales [21, 44, 45], and the items in the checklist used in this study were mostly adopted from these guidelines. It is important to emphasize that the score of each paper does not reflect the quality of the research itself. Furthermore, the items in this study were equally weighted based on previous researches [20, 21, 45], which may be subject to further discussion. However, despite these limitations, this study allowed a systematic evaluation of the reporting qualities of the previous researches using sham needles, and allowed a comparison of the results with the reporting qualities of acupuncture trials investigated in previous studies by adopting the time span as well as the items. The systematic methodology used in this study allowed for an objective evaluation of the studies by multiple reviews and discussions.
In conclusion, we found that the reporting qualities on sham needles did not improve over time. Many of the items were reported in less than 50% of the evaluated studies. In contrast to the reporting quality of acupuncture trials in general as reported in previous studies, low reporting qualities regarding controls may influence how researchers understand the effectiveness of acupuncture. Further studies are required to validate the items used in this study to endorse better reporting of sham needles used as controls in acupuncture trials.