Category Archives: Thoughts about Therapy

Outcome Measurement for Specialist vs. Universal Services

Recently, I was asked to present some thoughts on outcome measurement for specialist and universal services, for an interview. Since it took some time to prepare, I thought I’d post a version here on my blog. So, here goes!

We have several well established methods of measuring outcomes for specialist services, but measuring the outcomes of our universal work is much more difficult. More difficult, but just as important.

Definition and purpose

Let’s start with a definition.

According to Alison J L Fawcett, outcome measurement “…establish[es] the effects of an intervention on an individual or the effectiveness of a service on a defined aspect of the health or well-being of a specified population…. [it involves] administering an outcome measure on at least two occasions to document change over time…” (Principles of Assessment and Outcome Measurement for Occupational Therapists and Physiotherapists, 2007).

Outcome measurement is important for providing quality assurance (as part of clinical governance), demonstrating value, and contributing to the evidence base.

Specialist services

In clinical practice there are a number of well known methods to measure outcomes, such as Enderby’s Therapy Outcome Measures and the East Kent Outcome System. Services I’ve worked in use their own local systems, which involve writing SMART targets and stating whether they’ve been achieved at the end of a block of therapy.

When we choose an outcome measure, it’s worth considering the time required to use the system, as well as what we’ll use the data for. For example, I’m happy to spend time recording outcomes when I know the data will be used to develop care pathways, or contribute to the local evidence base. But the exercise becomes frustrating if I think the data will just sit in a file somewhere, gathering dust!

Universal services

Universal services aim to raise awareness of speech and language development for all children; ensuring that environments children spend time in are language rich, to stop difficulties arising. Universal services also support early identification and timely intervention.

Outcome measurement for universal services is messy. There’s no clear target child, SLTs are working indirectly (through other people), and we’re perhaps not so disciplined with our goal setting. We also don’t have any directly applicable tools.

Measuring outcomes for universal services is currently a hot discussion topic; it was addressed at the “SLTs in Children’s Centres” Special Interest Group. James Law suggested that outcomes need to be easy to communicate to non-specialists, while Michael Thompson talked about how we could use focus groups, observations, and workshops to measure outcomes.

An example: “facilitating language” training session

Thinking about how I can apply this to my own practice, here’s an example.

There are a number of ways we could collect outcome measurement data for a training session which aims to teach play workers how to facilitate language. Before the training session we could ask participants to complete a self rating scale, or observe each play worker interacting with a child; or if this isn’t practical, we could ask them to make a video recording. We would then need to repeat these measures after the training session.

I think it’s important to consider when to collect the post-intervention data. If the training session is aiming to teach practical skills, it might be important to give the participants time to go back into their settings to practise these skills, and consolidate their learning. Carrying out post-intervention measures 4 weeks after the training session might provide us with the most useful data.

Why bother?

Given all the challenges and difficulties I’ve mentioned here, why should we even bother to measure universal outcomes?

SLTs working with pre-school children are doing more universal work to try and reduce dependency in the population, and work more efficiently. Our commissioners require us to demonstrate that this universal work is effective. So it’s vital that we continue to develop outcome measures for this area of our work.

Have you had any luck measuring the outcomes of your universal work? Please share them here.

Dynamic Assessment: Mediation, Metacognition and Maximum performance

I recently attended a Psychology ‘Special Interest Group’ event about Dynamic Assessment. Ruth Deutsch and Natalie Hasson led an engaging session covering the what, why, and how of Dynamic Assessment, as well as its relevance and application to Speech and Language Therapy. Here is an overview of the session and some of my thoughts.

What is Dynamic Assessment?

Vygotsky (1978) believed that the difference between a child’s performance when he works alone and his performance when he collaborates with an adult reflects his ‘zone of proximal development’ or developmental potential. Dynamic Assessment is based on this theme, aiming to assess potential for learning rather than a ‘snapshot’ of a child’s performance in a particular situation on a particular day, as measured by conventional tests.

Why use Dynamic Assessment?

I sometimes complete my set of standardised or formal assessments and end up thinking, ‘now what?’ Dynamic Assessment can highlight which cognitive processes need to be targeted, describe useful mediation techniques, and indicate next steps of learning.

Natalie suggested Dynamic Assessment may be useful for:

differentiating between children with Specific Language Impairment and those with low language levels due to English as an Additional Language.
finding out the modifiability of clients to plan effective intervention.
standardising ways of measuring how responsive an individual is to language learning.
identifying which types of intervention will benefit particular individuals.

Dynamic Assessment in action

Ruth and Natalie showed video clips which demonstrated Dynamic Assessment in action. The clips were a great learning tool; making the idea of mediation much more concrete for me. Ruth explained some techniques of mediational teaching, such as focussing on processes rather than responses, and the development of principles of learning, e.g. ‘when I take my time I can complete the task.’

Mediation can take place before, within, or after a task. There is an emphasis on metacognition (“awareness and understanding of one’s own thought processes” — Oxford American Dictionary); in the video clips the adults asked questions such as, “how did you know?” and, “how did you work it out?” Ruth advised that during mediation the adult shouldn’t over-assist, because this takes away a learning opportunity. The adult should begin with the lowest level of assistance, such as encouraging the child to identify the problem, and then help the child work out their own way of approaching the task.

For more information about Dynamic Assessment try: Hasson, N. and Joffe, V. 2007 ‘The Case for Dynamic Assessment in speech and language therapy‘, Child Language Teaching and Therapy, 23(1), 9-25.

What next?

So, if I want to see what my clients’ maximum performance is, how can I use these ideas in my assessments? Well, I can’t use mediation techniques on standardised assessments, because the mediation will make any re-testing with the same assessment unreliable. I talked to Natalie briefly about using the Renfrew Action Picture Test (RAPT). I think that if I could develop five more RAPTs (sets of ten pictures and questions) I would have enough material for the intervention (or mediation phase) and the re-assessment phase. Developing five RAPTs seems both feasible and a mammoth task, depending on which day I think about it!

In the meantime, it occurred to me that perhaps I could use the picture and question scenarios from the Language for Thinking program. There are 50 scenarios with questions at three levels; plenty to use for assessment, mediation and re-assessment.

Fancy collaborating on some new RAPT style assessments? Used the principles of Dynamic Assessment with under 5s? Please leave a comment, I’d love to hear from you!

An Englishwoman in New York

We present a guest article from Kerry Bray, a Speech and Language Therapist (SLT) colleague and friend from Hackney, East London. Last year Kerry moved to New York City, to marry a lucky American. When her visa allowed it, she started working as a Speech and Language Pathologist (SLP). Here are some of Kerry’s observations on the similarities and differences between working as a SLT in the UK versus a SLP in the USA. Thanks Kerry!

Kerry Bray

I have been working here in New York for the last three months, and finally feel like I am getting to grips with the major similarities and differences in the professional structures and roles. It is pretty different in practice here, because the organisation of health and education is structured and funded so differently. I wouldn’t say that children receive a better or worse service in either country—there are major pros and cons to both systems.

Twice-weekly therapy sessions

Here most SLPs have a full day of pure therapy, carrying out about 10 sessions a day, mainly seeing individuals and small groups in schools or pre-schools. On average, children receive therapy two or three times per week, for half an hour. Speech and language difficulties are almost exclusively addressed at the impairment level, and there is no expectation of carry-over into the classroom or the home. Although there is lots of literature encouraging work at the activity and participation level, many therapists are paid per session—since paperwork, liaison, and training cannot be billed, it seems like there is no incentive or time to work at this level.

Seeing children so frequently, on an ongoing basis, means they make lots of progress, and you have a chance to really get to grips with each child’s strengths and needs. There’s also a surprisingly small amount of paperwork here, compared to the UK. We write just one line of progress notes per session per child, plus statistics. Each child is then evaluated once per year, and a short report and targets are written.

I arrive at 8:30 am, leave at 3 pm, and I’m paid a liveable wage (I haven’t compared salaries, as I am paid per session here.) A lot of people also do extra work in the evenings, and even at weekends.

Buying your own resources

A major downside here is that in some settings you are expected to buy your own resources. I work in a special school one day a week, and a pre-school for the other four days. At the special school, the SLP had to buy her own colour printer and laminator. At the pre-school, these things are supplied, but there are few shared toys, or other therapy materials. As you can imagine, it takes a lot of time and money to build up resources for a huge range of abilities—I need resources for children with Profound and Multiple Learning Difficulties (PMLD), Autistic Spectrum Disorders (ASD), Oro-motor needs, or five-year-olds with language delay.

Trying not to generalise

I need to be careful not to make too many generalisations—I’ve only done volunteer work for a private practice, and paid work in three different settings, through an agency. However, I have talked to a lot of therapists, and I am always trying to find out whether my experience is typical, or not.

Are any of my British friends interested in working here? I’d be happy to answer any questions about working in the USA, versus the UK.

We’d love to hear views from SLTs and SLPs in the USA, the UK, and around the world—what are the similarities and differences between your experiences and Kerry’s? Please add a comment!

Clinical Risk vs. Clinical Need: managing workload and throughput

At the beginning of the year I attended a two day training course: “Advanced Clinical Reasoning and Effective Clinical Decision Making”, facilitated by Kate Malcomess. It was an intense two days, at the end of which my brain hurt!

Kate talked a lot about risk, which she defines as, “the degree to which harm is foreseeable.” This led us to think about who can best manage a child’s risk, which is linked to the three levels of care: universal, targeted, and specialist. At the universal level—that is, for all children—we should be supporting parents to enable them to manage their child’s risk.

We discussed clinical risk, “the degree to which foreseeable harm can be managed by your intervention,” which you can think of as effectiveness. Then there’s clinical need, “the input needed to reduce risk and achieve predicted outcomes, ” which approximates to the amount of clinical input needed. Kate suggests using a clinical risk vs. clinical need grid, to prioritise workload and increase throughput.

Let’s consider a child who has both high clinical risk and high clinical need. An SLT can effectively reduce risk for this child, but a large amount of input is required. In contrast, a child who has high clinical risk but low clinical need, requires only a small amount of input for risk to be effectively reduced.

If we prioritised these high clinical risk, low clinical need children we would increase throughput: the number of children moving through the system, i.e. the children whose referrals are accepted, are assessed, offered intervention and then discharged. Currently it seems like most children are stuck at the intervention stage—we don’t discharge many, so throughput is small. If we could increase throughput, we would reduce waiting times, which may lead to more cheerful parents (and therapists!).

This way of thinking turns the traditional model, that I’m used to, on its head—there is no mention of using severity to make these types of decisions.

So how do we start? Kate talked about caseload profiling as a first step: looking at where on the risk vs. need grid we would place the children currently on the caseload. Then we can work on throughput, while keeping a record of unmet needs, to show to our commisioners. It’s going to be a lot of work, but I’m looking forward to the challenge, and want to start making some changes… I’ll keep you updated!

Meeting diverse needs: the Early Years Foundation Stage

On three recent school visits, I left feeling that the needs of the children on my caseload weren’t being met. So I looked at the Early Years Foundation Stage (EYFS) for some persuasive pointers to use with teachers.

In the practice guidance booklet, on page 6, it states: “Meeting the individual needs of all children lies at the heart of the EYFS. Practitioners should deliver personalised learning… to help children get the best possible start in life.” It goes on to say, “You must plan for each child’s individual care and learning requirements.”

On these occasions, this kind of individual planning seemed to be missing. Perhaps I could refer to the EYFS to make discussions with teachers more constructive?

Something along the lines of: We need to be planning how to meet x’s individual learning requirements, in line with the EYFS. Shall we have a look at the plans you have in place already, and think about how the activities x and I have been doing can be incorporated?

Too confrontational? What do you think?

Does It Take Two To Talk in inner London?

This week I spent 3 days in Nottingham, learning how to run the Hanen “It Takes Two To Talk” (ITTTT) course for parents. ITTTT is an early intervention program designed to teach parents how to facilitate their children’s communication skills. A key component of the program is the use of video feedback; parents are filmed playing with their children, so they can see how they are using the strategies they have been taught.

The course was interesting, the group leader was dynamic and inspiring, and I picked up some really useful tips for running adult training sessions. The information about different adult learning styles and how to accommodate them made sense; I’ll be applying this when I give some training tomorrow! I found practising how to coach parents during the video sessions, and using the 9 steps for feedback a worthwhile exercise.

Although I learnt a huge amount that I can use in my day to day work, I have doubts about how successful ITTTT would be for most of the families I work with in a bilingual patch of inner city London.

I have to work against the medical model: ‘You’re the professional, fix my child.’ Parents often don’t understand the importance of play, or are struggling with financial, housing or health issues; playing with their child simply isn’t a priority. Some parents have limited literacy skills, or negative past experiences of education — and the use of video is an alien concept.

Therapists have tried running ITTTT in other local boroughs with limited success and struggled with a lack of attendance. Have you tried running ITTTT in urban areas with high levels of deprivation? How effective was the course as intervention? Have you successfully adapted the program for use with this type of population? At the moment, I’m planning on working with families on an individual basis, using the ITTTT principles for parent training and video feedback sessions. I’ll let you know how I get on!

Did Not Attend

It’s been a frustrating couple of days at work—six out of the seven children I booked in ‘did not attend’ (DNA). I call families the day before their appointments, and try to see them at a Children’s Centre rather than the clinic, if it’s closer to home, but this obviously isn’t enough—I think it’s time for something radical!

My partner suggested I double book appointments—invite two children to every appointment, on the assumption that one won’t come. I think that’s a bit too radical! But what about ‘over booking’ appointments? At the moment I book appointments in one hour slots. I think this is the norm where I work. It gives me time to write the notes, sort out the therapy room, get a cup of tea etc., between clients.

When I talk about ‘over booking’, I envisage booking in ten children, at 30 minute intervals. Here are my calculations.

Appointments generally last 40 minutes, but I’d book the appointments in half-hour slots. There’d be two one hour slots, which would allow me ‘catch up’ time. So, if all ten children came (which with current form, seems highly unlikely!) four children would be seen on time, four would be seen ten minutes late and two would have to wait 20 minutes for their appointment. More realistically, 50% come, I see five children, do a good day’s work and stop getting so grumpy.

(For clarity: I’d use the following appointment times: 9.30 am, 10.00 am, 10.30 am, 11.30 am, 12 noon, 1.30 pm, 2.00 pm, 2.30 pm, 3.30 pm and 4.00 pm.)

Perhaps I’d be able to ‘reward’ good attenders, by giving them particular slots so they’d be seen on time. Do you think a system like this would be feasible? Have you tried anything similar? What are the pitfalls?

I know I need to investigate why people are not bringing their children to speech and language therapy appointments. Maybe I’ll do a telephone survey, but I find it hard to ask people why they DNA’d without sounding cross! Something to work on then!

Montessori education and language enrichment strategies

At work recently, the issue of whether our standard advice for language enrichment is compatible with Montessori education came up, during peer supervision. When a Montessori school asked for help extending their younger pupils’ language and conversation skills, my colleague began by mentioning that the “home corner” was a great setting for encouraging language. She was told that home corners are contrary to the Montessori ethos.

According to Wikipedia, the Montessori method “is characterized by an emphasis on self-directed activity on the part of the child.” My colleague was told that her suggestions to comment on a child’s play, play games involving following instructions and ask questions about a book were adult-directed and therefore not appropriate. The colleague left the meeting feeling frustrated and unsure how to move forward with the situation.

In supervision it was suggested that perhaps the Early Years Foundation Stage would provide a common ground. When I searched google I found that the Montessori Schools Association have put together a document (PDF) which demonstrates how their approach shares the underlying principles of the Early Years Foundation Stage. I found a number of examples of Montessori practice in the document which SLTs could use to model ways of extending and developing language. These included farm, hospital or any other “small world” play, using puppets and props to re-tell stories, and role-play such as “shop”.

I think it’s a helpful document—hopefully the ideas in it will make the next meeting with the school more productive! The same colleague also told the supervision group about many positive experiences working with Montessori schools. What is your experience of working with staff in Montessori settings? Any tips you could share? SLT games / activities that are compatible with their ethos? It would be great to hear other people’s ideas!

Rhiannan

Tube posting and favourite clients

This week I tried out an idea that Deborah contributed; tube posting. I merrily spent an evening decorating a tube with sparkly paper and got some strange looks carrying it to work the next morning on the bus.

I took the tube along to my first session; I was working with a child who has severe attention and listening difficulties. He focussed on the tube for approximately 3 seconds, watched me post one ball and then got up to find another toy to play with. A disappointing first outing for my sparkly tube!

However, half an hour later, a very different child walked into my therapy room. He couldn’t get enough of the tube game! The tube (and I) held his attention for more than 15 minutes; it was a great informal assessment tool, allowing me to check the child’s key word understanding and how he was able to make choices. Thanks Deborah!

Since I started writing this blog, I have been having a nosey around the web for other SLT-related blogs. So far I haven’t found all that much out there… add a comment if you’ve found (or write) something!

I did find one interesting blog: Tower to Trenches, by Judy Stone-Goldman. I enjoyed Judy’s most recent post. I hadn’t really considered how having a favourite client puts that client under pressure. Judy suggests that we should try to be honest with ourselves about how we feel, that “we like some clients better than others.” She takes a quiet moment before sessions to think both about the child she is going to see and herself. Do you do something similar? I’m going to try this over the next couple of weeks, I’ll let you know how I get on!