Methods and Procedures

I have encountered a dilemma. A rather well-done RCT with blinding and a placebo control group (a level 1b study per the Oxford Centre for EBP) finds that post-op TENS was not effective in decreasing the amount of analgesic in post-op pain following TKR (Breit R & Van der Wall, H; 2004). However, upon reading the procedures, the authors did not report the setting used on the TENS unit. Instead, they set it at the maximum output level and told the patients they could change it as necessary.

In a meta-analysis of TENS and post-op pain (Bjordal, Johnson, Ljunggreen; 2002), the authors pooled subjects from only RCTs evaluating TENS effectiveness in reducing the amount of analgesic and found TENS reduced the amount by ~25% (MWD) more than placebo. Further, when a subgroup of "optimal setting TENS" was analyzed, the authors found a 35% (MWD) reduction in analgesic more than placebo. Only 4% difference favoring the analgesic was found when pooling 9 trials without explicit information about sufficient current intensity and adequate frequency.

So, my question is - where do procedures figure into the level of evidence? This question also comes up when evaluating the effects of exercise on function in older adults. Many older studies only used low intensity exercise as the intervention and concluded that exercise was not effective in improving function in older adults. However, when one uses moderate to high intensity exercise (depending on the initial threshold of the subject), results are strongly in favor of functional improvement. And yet - both types of studies are well designed.

Anyone care to comment on how a clinician is to weigh a well designed study (and thus a high level of evidence) that uses ineffectual procedures and still keep the EBP process to Sackett's goal of "20 minutes"? Where do procedures fit into the validity of a study?

Comments

[John Childs]

Dale:

You ask an important question for which I don't think there are any easy answers. My sense is that we depend on investigators to incorporate compelling interventions with the maximal potential for effectiveness, balanced by other considerations such as costs, risks, etc. If this doesn't happen, there is the potential for the scenairo you propose.

If you are not already a member, I would encurage you to subscribe to the UK's EBP listserv. Many of the well known EBP experts from around the world participate in this forum and entertain difficult questions such as these. I would encourage you to join, then post a brief summary of your question. I'd be willing to bet you get an informed response in prompt fashion.

John