Telehealth: A META-ANALYTIC EVALUATION OF
WORKBOOKS EFFECTIVENESS IN PHYSICAL AND MENTAL HEALTH
by: Joshua Smyth, Ph.D. and Luciano L'Abate, Ph.D.
January, 2000
This is a summary of a chapter to be published in "Distance Writing and
Computer-Assisted Interventions in Psychiatry and Mental Health," edited
by Luciano L'Abate. The chapter will be published in Fall of 2000 by Ablex
Corporation, Stamford, CT.
To supplement or as alternatives to current f2f psychotherapeutic
practices, computer-assisted interventions like workbooks may be used on
their own, without any additional intervention, in preventive or
para-preventive activities. Proponents of these supplements argue that
f2f verbal treatment is inherently inefficient. It dramatically limits
the number of respondents (individuals, couples, and families) that can
receive treatment from mental health professionals. Inclusion of these supplements
and alternatives, hopefully will lead to more cost-effective treatments.
Through Distance Writing (DW) and Computer-Assisted Interventions (CAI),
it should be possible to treat greater numbers of respondents.
A critical assumption of this argument, however, is whether DW/CAI, and
workbooks are effective. Will DW/ CAI or workbooks as supplements to, or
replacements for, more traditional therapies actually lead to improvement?
The goal of this chapter was to examine this assumption within the limited
domain of workbooks. It attempted to determine to what degree workbooks
lead to improvements in mental and/or physical health.
Writing about emotionally traumatic experiences has been shown to be effective
in promoting health and well-being in both healthy and chronically ill
individuals (Esterling, L'Abate, Murray, & Pennebaker, Clinical Psychology
Review, 1999; Smyth, JCCP, 1998). An annotated bibliography of mental
health workbooks (http://www.mentalhealthhelp.com)
reported over 60. Despite this prevalence, very few workbooks have established
an empirical basis for their preventive or clinical use. Such support
would need to include demonstration of both efficacy and efficiency.
Current changes and trends in the health-care system (for both mental and
physical health) suggest that treatments which fail to meet an
increasingly stringent cost-effectiveness criterion will not be made available, or
will not receive reimbursements from insurance and managed care companies.
Conversely, those treatments that are demonstrated to be cost-effective
will become commonplace in the arsenal of managed-care.
Cost-effectiveness, therefore, becomes a critical concept as the
combination of both effectiveness and efficiency. Although the nature of
workbooks make them inherently low-cost and mass-produced, we remain
confronted with the bugaboo of effectiveness. To justify the continued
use of workbooks in clinical settings, as well as to promote more stringent
empirical examination of this issue, this chapter examine if workbooks
show any effectiveness.
Meta-analytical methods for the cumulation and examination of research
studies can thus provide an alternative, and in many ways preferable,
approach to evaluate a research literature. Meta-analysis is a
statistically based approach for generating effect size estimates by
aggregating effect size information from all available sources. There are,
however, some unique difficulties posed by the literature on workbooks.
Vast differences exist between studies in overall design and quality of
methods and statistics used. There are, however, a relatively small
number of studies that contrast workbooks to a control condition (often a
delayed treatment condition).
All studies had to include the use of a workbook in conjunction with, or
solely as, therapy (with the explicit aim of producing positive change on
some outcome measure). Studies were also required to contain some outcome
measure of health, although, for the purposes of this chapter, we defined
health in the broadest possible terms. Such health outcomes could be in
the domain of either mental or physical health, or more general measures
of performance (e.g., student grades, cognitive performance). Studies also
had to contain statistical information necessary to calculate an effect size.
Following these criteria, 18 studies were included in this analysis.
Magnitude and significance of the overall mean weighted effect size was
computed for all outcomes and all studies. One effect size was computed
for each study by generating effect sizes for each outcome in the study, and
then averaging across all outcomes. These study effect sizes were then
cumulated across all studies (corrected for bias) for an overall effect
size of workbooks.
The overall effect size across all outcomes and all studies was d = .30 (r
= .15; 95% CI +0.21 / +0.39, p<.0001). There was considerable variability
in effect sizes between studies, ranging from -.22 to +1.16. Effect
size for workbooks alone was d = .36 (r = .15; 95% CI +0.22 / +0.50),
whereas the effect size for workbooks used in conjunction with other
treatment was d = .26 (r = .13; 95% CI +0.15 / +0.37). The difference
between these two effect sizes was not significant (Qb(1) = 1.27, p =
.26).
Overall effect size for mental health outcomes was d = .44 (r = .21, 95%
CI +0.29 / +0.59). Overal effect size of workbooks on physical health
outcomes was d = .25 (r = .12, 95 %CI +0.14 / +0.35). Unlike mental
health outcomes, the test for homogeneity of effect sizes was significant
(Qw(5) = 10.79, p = .05).
Results of this analysis suggest that workbooks may produce a medium
effect size in mental health and a somewhat lower effect size for physical
health (although there was great variability for such outcomes). This
analysis supports the use of workbooks as additions or as alternatives in
preventive and psychotherapeutic practices, making them tools for
secondary prevention. Furthermore, as long as they are external to traditional
psychotherapeutic practices, they are not yet part of the mental or
physical health mainstream. It behooves practitioners using them to
include informed consent and pre-post objective evaluation as standard
operating procedures in their preventive and clinical practices. Finally,
we explicitly encourage future research on the use of workbooks.
For more information on this topic, subscribe to one or more
of
Dr. Maheu's free email discussion
services.
|
