Recently, I was asked to present some thoughts on outcome measurement for specialist and universal services, for an interview. Since it took some time to prepare, I thought I’d post a version here on my blog. So, here goes!
We have several well established methods of measuring outcomes for specialist services, but measuring the outcomes of our universal work is much more difficult. More difficult, but just as important.
Definition and purpose
Let’s start with a definition.
According to Alison J L Fawcett, outcome measurement “…establish[es] the effects of an intervention on an individual or the effectiveness of a service on a defined aspect of the health or well-being of a specified population…. [it involves] administering an outcome measure on at least two occasions to document change over time…” (Principles of Assessment and Outcome Measurement for Occupational Therapists and Physiotherapists, 2007).
Outcome measurement is important for providing quality assurance (as part of clinical governance), demonstrating value, and contributing to the evidence base.
In clinical practice there are a number of well known methods to measure outcomes, such as Enderby’s Therapy Outcome Measures and the East Kent Outcome System. Services I’ve worked in use their own local systems, which involve writing SMART targets and stating whether they’ve been achieved at the end of a block of therapy.
When we choose an outcome measure, it’s worth considering the time required to use the system, as well as what we’ll use the data for. For example, I’m happy to spend time recording outcomes when I know the data will be used to develop care pathways, or contribute to the local evidence base. But the exercise becomes frustrating if I think the data will just sit in a file somewhere, gathering dust!
Universal services aim to raise awareness of speech and language development for all children; ensuring that environments children spend time in are language rich, to stop difficulties arising. Universal services also support early identification and timely intervention.
Outcome measurement for universal services is messy. There’s no clear target child, SLTs are working indirectly (through other people), and we’re perhaps not so disciplined with our goal setting. We also don’t have any directly applicable tools.
Measuring outcomes for universal services is currently a hot discussion topic; it was addressed at the “SLTs in Children’s Centres” Special Interest Group. James Law suggested that outcomes need to be easy to communicate to non-specialists, while Michael Thompson talked about how we could use focus groups, observations, and workshops to measure outcomes.
An example: “facilitating language” training session
Thinking about how I can apply this to my own practice, here’s an example.
There are a number of ways we could collect outcome measurement data for a training session which aims to teach play workers how to facilitate language. Before the training session we could ask participants to complete a self rating scale, or observe each play worker interacting with a child; or if this isn’t practical, we could ask them to make a video recording. We would then need to repeat these measures after the training session.
I think it’s important to consider when to collect the post-intervention data. If the training session is aiming to teach practical skills, it might be important to give the participants time to go back into their settings to practise these skills, and consolidate their learning. Carrying out post-intervention measures 4 weeks after the training session might provide us with the most useful data.
Given all the challenges and difficulties I’ve mentioned here, why should we even bother to measure universal outcomes?
SLTs working with pre-school children are doing more universal work to try and reduce dependency in the population, and work more efficiently. Our commissioners require us to demonstrate that this universal work is effective. So it’s vital that we continue to develop outcome measures for this area of our work.
Have you had any luck measuring the outcomes of your universal work? Please share them here.