Acceptance of health measurement scales based on one's perception has increased in recent years [1–3]. Several well-established measures on health status or health-related Quality of Life (QOL) have been widely used [4–8]; however, few were developed on the basis of Chinese medicine which has been important for the well-being of the Chinese people for many years [9] and is now increasingly recognized worldwide [10–12].

Measuring health in accordance with Chinese medicine theory is useful [13, 14]. Chinese medicine practitioners pay close attention to patients' subjective feelings for diagnosis and treatment. Different perspectives of the patients may lead to different prescriptions from the practitioner even if the patients suffer from the same condition [15]. Over the years, Chinese medicine practitioners have accumulated valuable experiences on obtaining information from the patient's perspective that may benefit their treatment strategy and judgment of clinical efficacy. We believe that these experiences can be more easily understood by clinicians and researchers if they were involved in some applicable instruments. Moreover, Chinese medicine and Western medicine work differently towards the end result. In Chinese medicine, information such as adaptation to climates and seasons, aversion to cold, positive feelings concerning the ability of maintaining stable mood is carefully captured as evidence for patients' adaptability with both natural and social environments. Although modern Chinese medicine practitioners may take biomarkers as references, there is a general impression among them that indicators reflect individual's adaptability may possess a more important position when testing the efficacy of a Chinese medicine treatment.

However, these indicators are usually not involved in common patient reported outcomes (PRO) or health related quality of life measures. A few other instruments were developed on the basis of Chinese medicine theory or for integrative medical research but none of them were based on a community sample [16, 17]. In addition, the commonly recognized and accepted set of standardized procedures [18, 19], from concept to measurement scale, makes it possible to develop an instrument based on Chinese medicine theory.

The Health Scale of Traditional Chinese Medicine (HSTCM), which is a generic scale designed for evaluating general health according to subjective feelings [20], is based on health concept of Chinese medicine and the operationalization of this concept's measurement [20, 21]. The HSTCM consists of three domains, namely physical function under natural environment (PFNE), spirit (SP) and social environment (SE), and 2 additional items (see Additional file 1). PFNE domain has four facets: physical functioning (PF), voice (VOC), stool and urine (SU), and adaptability of natural environment (ANE). SP domain also has four facets: confidence and content (CC), self-confidence (SC), energy (EG) and basic ability of thinking (BAT). SE domain has three facets: ability of communicating with people (ACP), adaptability of noisy condition (ANC) and ability of dealing with bad stimulations (ADBS). In 2003, the development of initial Health Scale of Traditional Chinese Medicine (iHSTCM) [20] and the refinement from the iHSTCM (88 items) to HSTCM (47 items) [21] were accomplished. It was shown that the HSTCM was conceptually sound and reliable [22]. However, the test of psychometric properties using various independent samples may still be needed, particularly construct validation, according to the principles of instrument development [23, 24]. Meanwhile, the previous test was based on data collected through iHSTCM (containing 88 items [21, 22]); it is reasonable to test the psychometric properties again with the revised version of HSTCM (containing 47 items).

The purpose of developing the HSTCM is similar to other generic health related QOL measures, e.g. the World Health Organization Quality of Life Measure-Abbreviated Version (WHOQOL-BREF) [25] and MOS 36-Item Short Form (SF-36) [6]. We used the WHOQOL-BREF (Chinese version) [26] as a comparison instrument when examining the validation of the HSTCM because of the excellent validity and reliability of the WHOQOL-BREF. It is also because the Chinese version of the WHOQOL-BREF has become one of the standard measurements of QOL in China.

The aim of this study is to validate the measurement properties of the HSTCM in a convenience sample.

The properties of the HSTCM were tested in a group of participants in the Liwan community in Guangzhou, China. Measurement properties, internal reliability and inter-interviewer reliability of the HSTCM and its domains were satisfactory. The HSTCM had mild convergence with the WHOQOL-BREF (with a moderate correlation). The overall HSTCM scores and most of the domain scores discriminated groups of participants with known differences in health status. Confirmatory factor analysis showed a good structure fitness of the domains and overall structure.

In the reliability test, Cronbach's α was low in subscale ANC (0.32). As the items in the ANC facet met the selection criteria and Cronbach's α was considered to be acceptable on the whole scale and three domains' level; we may need further evidence before removing ANC from the scale.

In this study, we used the ROC analysis to evaluate the discriminate ability of the HSTCM. The AUC indicates that the HSTCM discriminate better than chance between groups with and without good self-reported health status. Therefore, the results of ROC analysis support the clinical validity of the HSTCM. However, this discriminate ability should not be confused with that of diagnostic instruments. Health status measures are not intended to be used at an individual patient level [34].

Construct validation is an ongoing process. The more frequently an instrument is used, the greater our confidence is in its validation [35]. Comparing the previous study [21] with the present one, we found that the construct validities of two independent samples were similar. Similar constructs in two independent studies strengthen the construct validation for the HSTCM. We may find some similar constructs in other health related QOL instruments based on Chinese medicine theory [16].

Items from SC and CC had similar meanings in terms of the HSTCM developmental theory, as did VOC and SU, which may be explained by the exterior-interior relations [36] between the large intestine meridian and lung meridian. The exterior-interior relations of the two meridians not only strengthen the connection between this specific pair of meridians, but also promote the large intestine/lung pair to coordinate with each other physiologically and influence each other pathologically. Items of the VOC facet are key indicators of the status of lung, while the SU facet contains an important predictor of the large intestine.

No test-retest reliability was conducted in the present study; however, results from the previous study suggest good test-retest reliability [22].

The attribute of 'responsiveness' is also considered as a component of a scale validation [37]. Based on the discriminative validation and measures of ROC, we think that the responsiveness of the scale is acceptable [38]. However, we did not estimate the parameter to compare the before-after changes caused by a given intervention, which means the inference may be limited.

In this study, participants were selected from a community by convenience sampling which is widely used in psychometric property testing, especially those of health scales [39–43]. However, a convenience sample is less representative than a random sample and may limit the generalizability of this study.

We did not use item response theory (IRT) in developing and evaluating the HSTCM because the advantage of the IRT methods over traditional methods was not clear to us [44]. However, we think that IRT methods can be used to refine the scale. Generalizability theory (GT) is another powerful tool in analyzing scale reliability with complicated sources of measurement biases. Further studies with IRT to analyze item characteristic of the instrument and with GT to evaluate the validation of the HSTCM are warranted.

The HSTCM meets the basic requirements for a generic health scale and complements the existing health measurement scales for Chinese medicine health care.