This study provides a literature review of the reporting characteristics and quality of massage RCTs in the past 30 years, which have identified a remarkable increase in the number of publications since 2017. Although RCT is regarded as the gold standard among different types of clinical studies, the quality of massage RCTs included in this review appears to be far from optimal. Incomplete or absent reporting of key trial information leads to some difficulties for assessing the quality of the whole study, easily inviting skepticism as to its results. Similar problems were also found in the previous study examining systematic reviews of massage treatments [10].
In this study, we found several problems in the characteristics of articles reporting massage RCTs. First, few studies were published in relatively high-quality journals. For the 96.8% of trials published in Chinese journals, only 702 articles were published in the “Chinese Medical Core Journals” [22]. For the remaining 3.2% published in international (English) journals, only 4 articles were rated as having a relatively high-impact factor (e.g., IF > 5). Higher-quality journals usually have broader reviewer networks and more rigorous reporting requirements. Previous studies have indicated that the reporting quality of clinical trials is positively correlated with journal quality [19]. Secondly, CM-related indicators were rarely mentioned in the design of massage RCTs. Although CM-based massage intervention (e.g., Tuina/Chinese massage) was the most commonly used, less than 15% of trials adopted CM pattern as diagnostic criteria and outcome(s) for evaluation. Previous studies have pointed out that if a CM trial does not consider CM pattern identification (‘zheng’) or use CM ‘zheng’-related outcome(s), participants may not be properly treated and/or the efficacy of CM intervention(s) may not be assessed properly [23]. Thirdly, the common study design was a two-arm-in-parallel group, with 51-100 sample size, open label and a single centre trial. Thus, high-quality massage RCTs with a large sample size and multicentre design are urgently needed. Blinding is not easily implemented in massage interventional RCTs, especially for double-blind. In this study, we identified a total of 16 trials that reported a double-blind method. Except for 10 trials (e.g., 4 Chinese journal papers and 6 international journal papers) which defined the blinded people as subjects and outcome assessors/statisticians, the remaining 6 trials (e.g., 1 Chinese journal paper and 5 international journal papers) actually blinded both subjects and treatment providers. Among these 6 trials, 2 used oil massage (e.g., oil placebo), 1 adopted sham massage device (e.g., the same mechanical massage technique but with different settings of actual massage and sham massage), and 3 achieved sham control through coded different acupoints and locations. The treatment providers in these trials did not know which massage protocol is the placebo control. Finally, nearly half of the included trials did not report any information about ethical approval. Readers thereby cannot fully evaluate such researches as ethical declarations form a vital integral part of clinical trial [24].
The reporting quality of massage RCTs was assessed by three instruments in this review, including the CONSORT checklist, the CONSORT Extension for NPT checklist, and a self-designed massage-specific checklist. Our checklist of message-specific information was designed to identify the critical elements in the rationale for choosing massage as a therapy, the details of massage techniques, and the design of a control for massage interventions. The overall quality of reporting has been unsatisfactory, although a slight improvement has been seen after the CONSORT was updated and issued in 2010.
With regard to the completeness of the CONSORT items, the average reporting rate was 50% (ranging from 0.2 to 99.8%). 18 items were reported poorly (< 50%), of which most (56%, 10/18) related to key methodological domains, such as trial design (item 3a), sample size (items 7a and 7b), sequence generation (items 8a and 8b), allocation concealment mechanism (item 9), implementation (item 10), blinding (item 11b), and additional analyses (item 12b). Previous studies have indicated that the methodology-related items are crucial for the assessment of bias risk and of the reliability of reported effects [25,26,27]. Other items, including baseline data (item 15), ancillary analyses (item 18), harms (item 19), limitations (item 20), registration (item 23), protocol (item 24), and funding (item 25) in the “Results” and “Other information” sections, were also reported poorly. Inadequate reporting of trial registration, protocol, and ethical approval significantly compromises the value of massage RCTs, inviting skepticism and criticism [28].
At the same time, we identified a considerable number of NPT Extension items that were missing or incompletely reported; the average reporting rate was only 10%, with a range of 0.04% to 57%. Nonpharmacological treatments frequently involve multicomponent interventions delivered by multiple care providers, and each component or provider may influence the success of the overall intervention. Even though the CONSORT NPT Extension was developed in 2008 and updated in 2017 [12], the completeness of reporting remains insufficient, especially in terms of (1) adherence of participants to interventions; (2) changes to the intervention delivered from what was planned; (3) information about care providers; and (4) clustering by care providers or centers. Similar to previous studies, space constraints in journals, such as Chinese journals without an online appendix, is one reason for inadequate reporting of interventions in the RCTs of massage [29]. In this study, we also identified that the reporting score of the CONSORT and NPT items was lower in Chinese articles than that in the international (English) articles. Together, authors, editors (especially for Chinese journals), and reviewers should be more rigorous in their demands that authors adhere to the reporting standards of CONSORT and NPT items in trials of nonpharmacological treatments.
For massage- specific items, the average reporting rate was 45%, with a range of 1.8% to 94.4%. The least well reported information about massage interventions can be summarised in the following aspects: (1) the rationale of why the massage was being used and why the comparison was selected; (2) some details of procedure, particular in individualised massage treatments, media used (if any), responses sought of patients, and management for adverse events; and (3) the background of treatment provider(s). In practice, massage is very much practitioner and experience-dependent, thus, precise and complete details of the manipulation and related factors are essential for the transparency and replication of trials. Some previous analyses suggested that “post-2010 publication years” can be used as an independent predictor of the high reporting quality of RCTs [30], but no great improvement was identified in the included massage RCTs. In this study, we found the reporting score of massage-specific items in Chinese journals was better than that in international (English) journals. We surmise that this is because CM practitioners have a richer experience in conducting complex massage therapies and greater motivation to describe the technique details [31]. Much of traditional CM, since the Shang Dynasty, has been and is based on doctors recording their treatments. Modern doctors have read these earlier accounts, develop different Tuina/Chinese massage schools, know the value of techniques inheritance, and hence are more likely to record those details from their own practice [32].
Given the deficiencies of reporting identified in this study, specific improvements are needed. Previous findings confirm that guidelines do help improve the quality of reporting [17, 33]. Therefore, there are two paths forward: either strengthen reporting of the CONSORT and NPT Extension guidelines, or develop a series of standard reporting items specifically relevant to RCTs with massage interventions as another independent Extension to the general CONSORT 2010 statement. Taking the second route, our working group has initiated and registered the “STandards for Reporting Interventions in Clinical Trials Of Tuina/Massage (STRICTOTM): Extending the CONSORT Statement” on the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network [34]. We intend to complete this reporting guideline at the end of 2021.
This study has some limitations. First, this review identified massage RCTs published up to 22 June 2020 in the targeted eight databases. Any records which had not been included in these databases by that cut-off period have not been included. In addition, we included only articles in English and Chinese because of language limitations. As such, we may not have captured otherwise eligible trials published in other languages. Second, we assessed each item with a ‘‘1’’ or ‘‘0’’ score according to whether the author had reported the detailed information listed in the proposed items. All incomplete reporting (e.g., partial and absent) was given as “0”; and some “not applicable” reporting (e.g., open-label without blinding) was categorized as “1”. This over-simplifies the actual situation. Third, we did not assess the methodology quality (e.g., using a Cochrane tool) of each included trial as the primary objective of this study focused on the reporting characteristics. Although the results of this review may not necessarily be comprehensive, we do believe that the general trends indicated by this study are valid.