Literature Reviews

Literature Reviews

Clinical and experimental studies are not the only types of published research we might encounter as we look for evidence to inform our practices. One of the most useful types is the literature review, which summarizes a group of studies. The Massage Therapy Foundation recently partnered with the Samueli Institute to conduct a review of the literature on massage therapy and pain.1

Literature reviews can be an invaluable resource. They provide an overview of a particular research question and identify key studies in that area. Although review articles will show up in a general search, they can also be searched for specifically by using "review" as a key word. Literature reviews fall into three categories:

  • The narrative review or survey of the literature.
  • The meta-analysis.
  • The systematic review.

Bibliographies of review articles are also a good source for identifying related studies. For someone looking for a synopsis of current thinking on a research question, a recent literature review is an excellent place to start. It can tell you, among other things, which articles are worth your time reading in their entirety.

Narrative Reviews

The narrative review examines a group of studies in a particular subject area and, as all literature reviews do, attempts to come to some conclusion based on a synthesis of the group as a whole. The author determines which studies are selected to form the group, and this feature can be both a strength and a weakness. The primary strength of the narrative review is that the author can discuss the methods and conclusions of important studies in detail, and give an informed opinion about their relative merits. A major weakness of narrative reviews is the risk associated with author selection; in some cases, the author may be unaware of or unable to access all the relevant studies. Poorer quality studies may be included and treated as equal to better quality studies. The author may also choose to ignore studies with outcomes that are not consistent with the hypothesis presented, although this practice is not considered acceptable and would be unlikely to pass a peer review.

There are several examples of narrative reviews that are pertinent to the field of massage therapy. A well-known and excellent example is the review of research on touch compiled by anthropologist Ashley Montague in his book, Touching.2 Montague integrates research from several different fields including biology, psychology, anthropology, and sociology, and makes a compelling case for his argument that touch is a biological necessity for human survival and optimal well-being.

Meta-analyses

The meta-analysis, developed as a way to improve upon the weaknesses of the narrative review, has increasingly gained respect as a form of literature review. Meta-analysis is possible as a result of the development of computerized databases, which make locating studies much easier. This review format is more necessary than ever before in health care due to the expanding volume of research studies.

In a meta-analysis, studies on a particular question are grouped according to pre-established criteria and a literature search using one or more major online databases is conducted to locate the articles that meet those criteria. This practice is analogous to specifying eligibility criteria for clinical study participants prior to their recruitment into the study. Defining the criteria for inclusion in advance reduces the potential for selection bias on the part of the investigator.

Results of the studies that meet the stated criteria are then pooled, that is, combined so that the statistical power of the group of studies is much greater than each individual study alone. Meta-analysis is often used to estimate the size of a treatment effect or to settle a research question when there are several contradictory or inconclusive studies with small numbers of participants. An important consideration in evaluating a meta-analysis is whether the studies that have been pooled are similar enough in terms of their hypotheses tested and outcomes measured. One criticism of this method is that while it can be an improvement over the more subjective narrative review, meta-analysis can sometimes pool such disparate studies that the authors are in effect combining apples and oranges. If the research question is about fruit salad, so to speak, there is no problem. If not, the results can be misleading. This is a particular risk in fields like massage therapy where the specific combinations of techniques and modalities used in an individual treatment session can vary widely.

Another concern with meta-analysis is that publication bias has the potential to skew the results. Publication bias is the tendency for journals to reject studies with negative or inconclusive results and more likely to publish those with positive results. When studies with negative findings are not included in any subsequent analysis, their absence can prejudice the results, for example by making an intervention appear more effective than it really is.

Systematic Reviews

The systematic review is a more refined version of the meta-analysis and has become the new "gold standard" for research evidence. It attempts to compensate for publication bias by making additional efforts to identify every relevant study on a given research question, such as locating dissertations, hand searching journals that are not indexed in online databases, and personally communicating with authors of unpublished studies. These are then usually weighted in terms of the strength and quality of the evidence presented. For example, a study employing random assignment of participants to an intervention or a control group would be weighted more heavily than one without these design features. Standardized methods are used for weighting studies based on internal validity and appropriateness of their statistical analysis methods. Randomized controlled trials are usually given more weight in a systematic review because of their greater capacity to link cause and effect, which places them higher on the hierarchical model of levels of evidence.

Cochrane Collaboration

Systematic reviews have been popularized by the Cochrane Collaboration, a voluntary group of health care providers, consumers, and scientists whose name commemorates the late British physician and epidemiologist Archie Cochrane. Cochrane was a pioneer of the evidence-based medicine movement who strongly believed in the importance of systematic reviews in shifting medical care towards practice based on research evidence. For more information on the Cochrane Collaboration, visit: www.cochrane.org. Abstracts of all published reviews, including a number of reviews on massage therapy and other integrative therapies can be viewed at the Cochrane website.

Another Cochrane review has been conducted on the use of massage for low back pain.3 This study uses a slightly different method of systematic review called best evidence synthesis.4 Best evidence synthesis developed as a way to combine the quantitative features of meta-analysis with the advantages of the qualitative narrative review. It is used when the studies that are located turn out to be too different from one another or lack sufficient data to be pooled in the manner done in a traditional meta-analysis. Instead, reviewers evaluate the selected studies by carefully considering the evidence presented in each one, and then draw an overall conclusion about where the weight of the evidence lies. Systematic reviews can be expected to continue to gain popularity as a way to synthesize a large quantity of research on a disputed question and render a verdict.

Guidelines

A major consideration when evaluating any study is to what extent the research design and methods used are a good fit with the study question. The research question may be clearly articulated and relevant, but if the design and/or methods are poorly suited to answer it, the findings can be inconclusive, misleading, or not interpretable. Particularly in research on massage therapy or other integrative health therapies, the appropriateness of the treatment protocol to the question should be thoughtfully examined. This issue of model fit validity is important for several reasons. Many of the integrative health disciplines do not have large bodies of research, so that even one or two poorly conceived studies can have a negative impact on understanding a therapy's efficacy or practices. A protocol that is inappropriate to the study question also diminishes the usefulness of that research to practitioners, and makes translating research into practice more difficult.

In-depth evaluation of literature reviews is a detailed and technically challenging process whose methods continue to be refined. However, several general guidelines can be applied to assess the quality of a review:

  1. The research question addressed by the review should be specific and clearly defined. If possible, check out the background and expertise of the reviewers, who should represent a balanced and comprehensive team suitable to conducting a review on the subject in question. A statistician familiar with systematic review procedures is a necessity.
  2. The methods used to locate studies should be described, along with the criteria for inclusion or exclusion of studies from the review.
  3. If studies are ranked or weighted, the internal validity of the studies included should be assessed according to objective and reproducible methods, and the weighting methods should be described.
  4. If a meta-analysis was performed, the pooled data should be combined appropriately (apples with apples). A table that lists the individual studies with brief summaries describing the participants, methods, and results of each study helps the reader to judge similarities.
  5. Just as in any research study, the reviewers' conclusions should be supported by the data presented.

References:

  1. Crawford C, Boyd C, Paat CF, Price A, Xenakis L, Yang E, Zhang W; Evidence for Massage Therapy (EMT) Working Group. The Impact of Massage Therapy on Function in Pain Populations-A Systematic Review and Meta-Analysis of Randomized Controlled Trials: Part I, Patients Experiencing Pain in the General Population. Pain Med. 2016 May 10. Accessed June 1, 2016: http://painmedicine.oxfordjournals.org/content/early/2016/05/06/pm.pnw099.
  2. Montague A. Touching: The Significance of the Human Skin. 2nd edition. New York: Harper Books; 1978.
  3. Furlan AD, Brosseau L, Imamura M, Irvin E. Massage for low-back pain. Cochrane Database of Systematic Reviews 2002, Issue 1. Art. No.: CD001929. DOI: 10.1002/14651858.
  4. Slavin RE. Best Evidence Synthesis: An Intelligent Alternative to Meta-analysis. Journal of Clinical Epidemiology. 1995; 48(1): 9-18.