Methods: CTL developed a survey with consultation from the research team, to assess the needs of people in crisis who texted CTL to measure their satisfaction, mental health outcomes, and reasons for texting. Texters from 251,260 conversations were invited to respond to the survey from July 1 2017 to November 1 2017. Survey response rate was strong: 15-19% depending on survey item. As a first step toward understanding the subsample of individuals who responded to the survey (responders) versus those who did not (non-responders), we extracted linguistic features from text messages. Our eventual goal is to assess the quality of the overall delivery for non-responders from an inference process of available data.
Results: We extracted length of message, delay in response in between messages, and linguistic variables such as positive and negative emotions, function words (e.g. articles, prepositions), verb tenses, among others. Differences in these linguistic features are highlighted for the survey responder and non-responder groups. For example, survey responders seem to use more positive emotion words in their text messages than non-responders (mean=93 versus 129, sd=77 versus 83 respectively).
Conclusions: Mobile technology is increasing the reach of crisis support, yet it also requires a scalable method to assess quality of delivery. Internet-based evaluations help assess this quality, however, many users do not respond to questionnaires. Linguistic-based analysis and machine-based methods can complement the tool set of evaluation methods and can improve crisis support delivery.