![[ Visit Client Website ]](images/banner.gif)
CATEGORY/THEME: Dissemination and Implementation Science
Background: Two essential components of improving the speed with which an implementation strategy takes place are careful monitoring of progress in implementation stages and feedback of this information for action by those responsible for implementation. Many monitoring and feedback systems are not designed to recognize when appropriate action needs to take place. Often, the design requires human capital that the service delivery system cannot sustain over time, and this is where automated monitoring, using text mining and machine learning, can step in. We propose a three-stage information theory perspective to guide the construction of implementation monitoring and feedback.
Methods: We first describe three sub-component processes involved in information exchange, based on Vermeer (under review). These are Radiation, Transmission, and Reception (RTR). We discuss what information is needed to monitor implementation stages and how it ordinarily gets transmitted and received, often with extensive time lags. We illustrate how text information, recorded by an implementation broker, can be mined to generate a low burden and timely signal (radiation) that can be processed and classified into implementation stages using machine learning (transmission) which can consequently be fed back into the implementation process (reception).
Findings: In Wang et al. (accepted, Implementation Science) we demonstrated that, within a large randomized implementation trial, a supervised support vector machine learning algorithm is able to detect the start and end dates of each implementation stage, with a success rate of 77%. And that these finding are in strong agreement with human coding of these stages (Kappa = 0.72).
Discussion: We propose that these methods can be appropriate in settings where free-text is available by an implementation broker or reports can be mined. Especially when the numbers of communities or organizations that are delivering an evidence-based intervention is large, automating the radiation and transmission parts of the information exchange process could improve the speed at which feedback on implementation can be provided. This in turn allows for immediate action, by focusing support on communities or organizations where implementation has stagnated.