Q: I am considering using an intervention with insufficient evidence. Any suggestions?

Great question! I know that as much as we want to practice based on evidence, sometimes the evidence can be lacking on particular interventions. So below are our 4 Apply EBP Tips when you are thinking of trying an intervention that does not have a strong support from research.

*Note that our suggestions below are for interventions or strategies with insufficient evidence. This may include those with mixed evidence or those that do not yet have enough research supporting their effectiveness but have sound theoretical support (more on this below). It does not include interventions that have already been shown by multiple research studies to be ineffective, or that are based on theories that have been debunked – these interventions I would totally avoid!  

You may use the tips below in the order listed, or you may go straight to the pertinent one.

Tip #1:  First, re-check the evidence for the intervention you want to use.

Lots of research are being published every month. So you may have just missed newer ones that may be for or against the use of that intervention.

• • • Go on Google Scholar and search for the “<name of the intervention>” with the word “effectiveness” or “efficacy” (suggested search terms here are in quotes). You can also use other terms such as “<the diagnosis of your client>” or the “<outcome you are looking for (e.g.,motor, behavior, endurance, writing)>.”   
    • Read our previous #AskApplyEBP article “9 Resources for Finding Research Articles that Can Elevate Your Practice” for search tips.

Tip #2: Pause and Ask: Are there any other interventions with sufficient evidence that you have not tried yet?

You can start by reviewing your “toolbox” of interventions to see what effective strategy you have not tried yet.

• • • Nothing there? You can try to search for other interventions by typing in “efficacy” or “effectiveness” with the “<diagnosis>” and/or “<outcome>”.
    • You can add the search term “systematic review” so you can see a comparison of the efficacy of different strategies. If you find one, read the article for its recommendations. For example, if you look for a “systematic review” of “interventions” for children with “cerebral palsy”, you would find an article by Novak et al (2020). 
      1. In this article, they use a traffic light system of: 
         • Green (do it)
         • Yellow+ (probably do it)
         • Yellow- (probably don’t do it,), and 
         • Red light (don’t do it). 
      2. Find your outcome (for example, Motor Outcome), and see if there are any green and yellow+ interventions that you have not yet tried.

Tip #3: Ask Yourself: Are you the right service? Or do you need to refer to another professional?

Discuss the case with your team. Is there somebody else in your team who has an intervention within their expertise that is supported by evidence to address the child’s needs and get them to their outcomes. If so, you should refer with a big smile, as a potential solution is at hand. 

Referring to other experts is a big part of our professional responsibility (see APTA Guide to PT Practice, and AOTA’s OT Practice Framework). It does not mean that you are  giving up on the child. It actually means that you and your team are focused on the child’s progress, no matter who can deliver the best intervention for them. Sometimes it is us, sometimes it is others.

Tip #4: Is your team on its rope’s end and have tried tips 1-3 above, but have not yet found a solution for the student’s needs? 

First: Make sure that the strategy that you want to try is supported by sound theoretical framework (and not by a theoretical framework that has already been disproven). What is the science saying about the theory supporting the intervention you want to try? Try the article search strategies discussed above by typing in “<the theoretical framework>”.  

• • • For example, if you want to try heavy work for self-regulation, what is the theory supporting this? 
      • Is it “that it is believed that proprioception improves attention to task” and “heavy work can provide proprioceptive input”? 
        • Then search for impact of “proprioception” on “attention.” 
        • If you find support for that reasoning, search and reflect on what type of heavy work provides a sufficient amount of proprioceptive input.
      • Or, is it “that it is believed that moderately vigorous activity improves attention to task” and “heavy work can be a moderately vigorous activity?” 
        • Then search for impact of “moderately vigorous activity” on “attention.” 
        • If you find support for that reasoning, search and reflect on what type of heavy work provides the moderately vigorous activity needed to impact attention. 
      • Spoiler alert! Currently, there are more evidence that supports the latter for self-regulation.

Now, if you find that the theoretical support is sound, implement the strategy and closely monitor the impact of the intervention. Here are 3 steps for closely monitoring whether the strategy is working or not:

1. 1. 1. 1. Design a data collection strategy that is observable, reproducible, and  appropriate for the outcome desired.
            • For example, if you think that the heavy work intervention (sample from above) can improve attention during the 20 minutes of journal writing activity, you can follow this strategy that is similar to what researchers have used (Fedewa and Erwin, 2011).
              • Create a chart with 3 columns and 20 Rows.
                • The Column Labels should be: “Minute”, “YES”, and “NO”.
                • The Rows should be labeled from 1 to 20, from top to bottom. 
              • Sit in the back of the room. Put your earphones on that is connected to a timer that beeps once every minute for 20 minutes (I’m sure there’s an app for that too). Every time you here the “minute beep,” observe and record whether the child is attending to his journal writing or not. A “NO” may include any behavior that the teacher has described as time off the writing task, such as bothering his seatmates, standing up to grab a different workbook, etc. At the end of this observation you will have 20 data points. Now, you can calculate the on-task percentage by dividing the numbers of “YES” answers by 20.
              • Or, instead of the above strategy, you may select a writing sample from the journal class and calculate the length of the writing.
            • Another example: If you hypothesize that the reason for a student’s instability while walking is due to the lack of heel strike, and you think that the use of intervention X can increase heel strike when walking, you can follow this strategy that is similar to what researchers have used:
              • If in a school, stay in a hallway that the student’s class passes by, say, before lunch. Videotape the child walking from one end to the other. Watch the video to record how many total steps it took him, and how many of those steps he demonstrated heel strike. You can now calculate the percentage of heel strike, by dividing “number of steps with heel strikes” by the “number of total steps.”
              • If you are a home-based therapist, you can find the longest walkway inside or outside at home, and follow the same strategy above.
              • Needless to say, if your functional outcome is safe ambulation (i.e., decreased falls), you should also gather baseline data for that, and monitor whether falls decreased as increased heel strike is achieved to further test your hypothesis.   
         2. Go and gather the baseline data.
            • In addition to gathering baseline data using the procedures above, write down the location, the day of the week, the time of day and any other relevant context and observations.
         3. Implement the intervention.
            • Go for it! Try your intetvention.
            • Make sure that the intervention is being implemented with fidelity (i.e., the way it is designed) and regularly. Use coaching as a therapy tool to ensure carryover practice.  Check in every so often,  and be a cheerleader if there is poor carryover. 
         4. Gather the outcome data.
            • Set a date to gather data again. 
              • Select a reasonable time to gather data. Don’t wait too long. This study, for example, on context-focused interventions, implemented a “fail fast” strategy. The therapists would evaluate the impact of a strategy after trying it for 2 weeks. This makes sense for strategies that are provided at a high frequency. For example, the use of movement breaks before writing tasks, and writing happens many times a day in class. Or a walking strategy when walking occurs a lot. 
            • Choose the same location, day, time and context as the baseline measure to ensure you are collecting comparable data.
              • Using the “heavy work baseline data” example above, if the “on-task” baseline was gathered on a Monday from 10-10:20 am, in the classroom during journal writing class, gather data during the same day and time of the week with similar circumstances.
              • Or if you used a writing sample for baseline, make sure it is a fairly similar topic in terms of context and difficulty. For example, the topic for both observations was “What I Did this Weekend.” Same topic, even though the writing content would be different.
         5. Analyze the data.
            • Compare the baseline and outcome data. Did the child improve?
              • If yes, is the improvement considered meaningful by the child, teacher and/or family? 
                • If yes, continue the strategy!
                • If not, you may still need to continue the strategy if you think the child will continue to improve to a meaningful level. Or you may consider modifying the strategy.
              • If the child did not improve, then consider the following:
                • Do you or the team have another strategy to try?
                • Or do you need to go back to the team to discuss changing the goal to something that is more achievable?

Now, go for it! Try the tips above, and if you have other suggestions, add them to the comment below. Check out these courses that you may find helpful in providing evidence-informed interventions:

References:

Darrah, J., Law, M. C., Pollock, N., Wilson, B., Russell, D. J., Walter, S. D., … & Galuppi, B. (2011). Context therapy: a new intervention approach for children with cerebral palsy. Developmental Medicine & Child Neurology, 53(7), 615-620.

Fedewa, A. L., & Erwin, H. E. (2011). Stability balls and students with attention and hyperactivity concerns: Implications for on-task and in-seat behavior. The American Journal of Occupational Therapy, 65(4), 393-399.

Novak, I., Morgan, C., Fahey, M., Finch-Edmondson, M., Galea, C., Hines, A., … & Badawi, N. (2020). State of the evidence traffic lights 2019: systematic review of interventions for preventing and treating children with cerebral palsy. Current neurology and neuroscience reports, 20(2), 1-21.