I know seeing my graph [of HoNOS outcomes] it’s like a little competition – you want to improve it. I know that I don’t want to go back to where I was so it’s important to improve it."
- Leanne, psychiatric assistant, West Coast DHB and member of the PRIMHD Consumer Group
The profile
In 2005 the West Coast District Health Board (DHB) started an initiative to make HoNOS data more accurate and to improve outcomes. It involves two things:
- a group of service users that monitors and comments on developments in outcome data collection; and
- forums for service users once or twice a year to explain why HoNOS ratings are important to them, how to understand their trends over time and how to request data on their individual HoNOS trends.
Part of HoNOS was realising you were well enough and not that dumb and not too nutty to work. I’d always wanted to work in [the psychiatric] field but because of my history felt it was…out of my reach."
- Leanne, psychiatric assistant,West Coast DHB and member of the PRIMHD Consumer Group
While there has been no formal evaluation of this initiative, anecdotal evidence suggests that it:
- has led to improved accuracy of HoNOS ratings by improving communication - service users are more likely to tell clinicians what is happening in their lives because they understand that it is important for accurate HoNOS ratings. They are also more likely to speak up if they feel their ratings do not reflect their experiences.
- improves outcomes for service users - once people see their trends over time they seem to be more likely to take actions that will improve these outcomes. Seeing positive trends appears to encourage more positive actions, in a positive feedback loop.
- creates a peer culture that supports HoNOS outcome collection - after attending forums and finding out their personal HoNOS trends, service users appear more likely to encourage others to speak up about the accuracy of their ratings and support them to move their HoNOS trends in a positive direction.
When you start working with consumers you get a very different view of the project."
- Sue Brown, PRIMHD project manager
The beginnings
In 2005 Sue Brown, PRIMHD project coordinator, decided to bring together a group of service users to “monitor all things consumer”. Locals who used the mental health services had made it clear to Sue that they wanted to learn about HoNOS data and that they could see that HoNOS outcomes information could be helpful to them. For example, if something had a severe effect on their mental wellbeing at the same time each year, they could plan for it.
In Sue’s experience if people perceive outcome collection as being done ‘to them’ rather than ‘with them’ they start to put up barriers that make it harder to collect accurate data. Sue was concerned that if the clinician didn’t have a good relationship with the person they were collecting data on, accuracy could be reduced because they would learn less about what had been happening in that person’s life.
It’s about the dialogue…between consumer and clinician."
- Sue Brown, PRIMHD project manager
So Sue set a group up in partnership with the local branch of Supporting Families (SF).The group really got active in 2007 when the Ministry of Health drew MH-SMART under the umbrella of PRIMHD (the Project for Integration of Mental Health Data). Sue beefed up the service user group and it was named the PRIMHD Consumer Group. When SF was not able to stay involved, local NGO, PACT West Coast stepped into the breach. To date, the group has cost little to run.
The PRIMHD Consumer Group came up with the idea of organising forums to tell service users about HoNOS, which has had some unexpected positive results. “At the very first forum there were only five people there but they went away and talked to their friends and neighbours,” says Sue. It built up from there to a top attendance of 63. While forums discuss any issues of interest to service users, understanding HoNOS is always on the agenda.
The HoNOS workshop helped me learn that the collecting of information is a partnership – for it to be correct it needs to be a partnership."
- Joe Hall, service user advisor, West Coast DHB and member of the PRIMHD Consumer Group
The process
The steps to putting this model in place are fairly straightforward, both for the PRIMHD Consumer Group and the forums.
1. The PRIMHD Consumer Group
Membership
Almost everyone in the group has personal experience of using mental health services and those who don’t have had people close to them use those services. Sue Brown is employed by the DHB while service user advisor Joe Hall is employed by NGO PACT. Her position is funded by the DHB, largely to ensure consumer participation in mental health services. Sue and Joe liaise with the DHB, PACT and the local consumer network.
Meetings
They meet either when there is a development Sue thinks the group needs to know about or when other members want to, around every two to three months. While Joe usually coordinates the meetings, decisions are made by the whole group.
Overall group role
The main role of the group is to simply keep up with developments in HoNOS data collection, looking at how it affects service users and sometimes suggesting improvements. The group also organises forums to inform and educate service users about HoNOS.
Communication
When the group has a suggestion or feedback Sue or Joe voice their views to the DHB or other relevant organisation. At other times the group may speak as one, via a letter or some other means. Some members of the group are also members of the local consumer network, which aids communication with local service users.
HoNOS is not a tick box. It’s about a relationship, about being able to say to a clinician ‘it’s not right’."
- Joe Hall, Service User Advisor and member of the PRIMHD Consumer Group
Support and liaison role
In the group’s experience, accurate data is collected when clinicians and the people they provide services to form a partnership – one where service users feel comfortable enough to honestly share their experiences. To help build these relationships, group members support anyone who has a question about their HoNOS results to talk to their clinician. Sue estimates that around 12 out of approximately 60 people who have attended forums have done this. If someone finds it too challenging to talk to their clinician about their HoNOS results, either Sue or Joe are happy to do so on their behalf.
Helping service users access and understand HoNOS trend data
When someone attends a forum and becomes motivated to find out more about their HoNOS results they can request them from Sue or Joe. Sometimes attendees go away and mull over what they’ve learned for months until they feel ready to engage with outcomes data. The PRIMHD Consumer Group has set up a network of people who are happy to spend time talking people through their HoNOS information. They let relevant clinicians know when outcome data is provided, and encourage people to talk to their case managers about it.
Organising forums
The group is responsible for organising forums once or twice a year on a range of issues important to service users, including HoNOS. They also work between forums to encourage people to attend.
I’ve had consumers who went to the forum two years ago and have been thinking about it, and have just come to me to do something about it."
- Sue Brown, PRIMHD Project Manager
2. Forums on HoNOS
Forum aim
The forums aim to help West Coast service users understand their HoNOS ratings over time and to feel confident about talking with clinicians about them.
Frequency and duration
The forums proved so successful that they are now held once or twice a year. Each lasts one day, from 10am to 3pm.
Location
Forums are held at a community location where people feel comfortable, such as Marlborough House, the centre for the consumer network. “It’s where consumers decide to have it,” Sue says.
Costs
The main costs are food, venue and transport. Local NGO PACT provides the venue. Support with transport is funded by local consumer fundraising. Staff and attendees donate food, and have a working bee on the morning of the forum to prepare it. Sue says “the little bit of budget we had at the beginning wouldn’t even have been $500 for food and running the event”.
Speakers
Members of the PRIMHD Consumer Group present most of the programme. Sometimes a guest speaker is invited. For example, Carolyn Swanson, service user lead at Te Pou, spoke about the importance of outcome data collection from a service user perspective at one forum.
I also understand some of the [HoNOS] questions – before [the consumer group and forums] I thought they were asking them for nosy reasons. Now I realise that it was all to benefit me, not to use against me."
- Leanne, psychiatric assistant and member of PRIMHD Consumer Group
Forum content
Most of the day at a forum can be taken up talking about HoNOS. The day usually includes the following elements:
- introduction to HoNOS - organisers introduce HoNOS and talk about what it is, why it is used and how it can be helpful to service users. The concept of outcomes is explained, sometimes using the example of fishing, where an outcome equals catching a fish. Speakers ask attendees questions such as ‘What outcome do you want to see? What’s a good outcome for you?’ At one forum an attendee replied that a good outcome for them was seeing ‘just one tiny movement in the trend chart that shows HoNOS scores over time. Other thought-provoking questions include: “How does your clinician get the information to fill out HoNOS?” and “How to do you want them to get the information?”
- familiarisation with how HoNOS works - after the introduction, attendees break into small groups to look at a made-up case history that includes HoNOS ratings and the rater’s comments. They talk about how the HoNOS rater would have arrived at those scores given the imaginary person’s symptoms and circumstances. This takes about an hour and a half, and not only helps people understand HoNOS better but also to see it from a clinician’s perspective.
- explaining how to interpret the summary data over time - next, speakers explain how to interpret HoNOS scores over time, using an overhead of the imaginary person’s results. When Leanne first saw her trend data she felt distressed because her illness appeared to be getting worse. She was delighted when she realised that she’d been reading the graph the wrong way round and she was actually getting better! When her results later did go downhill, Sue asked her what had been going on in her life. Leanne was going through a nasty divorce at the time. Making that connection helped her make sense of the results.
- giving attendees an opportunity to ask for their own HoNOS results - towards the end of the forum Sue invites attendees who want to see their own HoNOS results over time to make an appointment with her to do so. After the forum Sue meets with these people and explains what their information means. After the previous discussion and exercises people are often very interested to find out what their own HoNOS scores show. They are able to ask for updates of the graph of their scores over time as often as every 91 days.
- encouraging forum attendees to keep track of their outcomes - attendees are also encouraged to keep track of their outcome information data once they leave the forum, and notice the impact any changes in their life or treatment might have on them. A presenter might say something like “when you next see your clinician or your meds are changed keep an eye on your trend data.” Sue explains that this “empowers the consumer to take control of their own outcomes.” One woman noticed that she tended to get distressed around the same time every year, before the anniversary of a painful event. Once she made that connection, she was able to plan for that time of year, and make sure she ate well and set up support in case she needed it.
One attendee’s experience
Joe Hall, service user adviser at the DHB, says that one woman who attended the forum found understanding her HoNOS trend data “beyond her” at first. She told Joe “I don’t understand it, but I want to.” When she grasped that the graph reflected the course of her illness, she started to cry. After she learned how to read her trends graph she got “great delight” out of making it work for her. Instead of spending nine months of the year being isolated, she now goes to the peer group at Marlborough House, where she talks about her outcome score and how it’s changed.
Depression problems over time: Graph
A family advisor’s experience
Michelle Curtis, who works for Supporting Families as the West Coast DHB family advisor, says “I see a level of family engagement that probably wouldn’t be there or be obvious in the community had consumers not got their outcome data”. Information on outcomes helps service users and families identify times of year and events that could trigger illness, and mean they can plan for them. Families also become more confident about discussing their relative’s health, and more hopeful as they see that change in mental health is possible.
The unique approach
What is innovative about PRIMHD West Coast’s approach is the way they consult and inform service users about HoNOS outcome data collection. The West Coast was one of the first DHBs (if not the first) in New Zealand, and possibly worldwide, to take this approach.
This is innovative in the international context as well. Literature on HoNOS often discusses:
- how easy clinicians find HoNOS to administer
- how the results can be used to plan and monitor treatment
- how useful the data is to health service administrators and researchers for assessing what impact mental health services are having.
There is little mention of service users, or the contribution that they can make to accurate data collection. The West Coast initiative puts the service user firmly in the picture, apparently making HoNOS an even more powerful tool.
Without our consumers there is no PRIMHD project. If you don’t have consumers, you don’t have anything."
- Sue Brown, PRIMHD project manager
The results
So what impact are the consumer group and forums having? While rigorous evaluation is yet to be carried out, there are intriguing anecdotal hints of three payoffs.
- More accurate information through improved communication - as service users see the value to them of HoNOS data being accurate they are motivated to give clinicians more detailed information about their lives and mental health. For example, after learning that her HoNOS ratings reflected whether her mental health was getting better or worse Leanne queried them twice. The first time she felt that her ratings had not taken into account the crises and stress in her life, and her mental health was actually worse than it appeared. “On paper it looked like everything was going along quite nicely,” she says. “But in fact I had just had an ugly and violent separation from my husband.” The second time she felt that her mental wellbeing was much higher than HoNOS results indicated. Leanne mentioned it to Sue, who took the matter to her clinician.
- Better mental health and life outcomes - there are hints that some people are working to improve their mental wellbeing so they can see positive movement in the graph of their HoNOS ratings over time. Again, good research is needed to confirm this. Leanne said that she consciously works to make her HoNOS graph move in a positive direction. Sue recalls one service user’s family who got involved in trying to improve that individual’s outcome data once they understood the significance of the HoNOS results. Seeing outcomes information change can also give an individual the confidence to take steps that will further improve their mental health, setting up a virtuous cycle. Leanne’s confidence was so boosted by seeing positive trends in her outcome data that after years on a benefit she got a job as a psychiatric assistant. Support from the consumer group also helped. Leanne says that before seeing her HoNOS results improve she felt that she wouldn’t be able to cope with the job. Being employed helped push her HoNOS graph further in a positive direction.
- Creating a peer culture that supports HoNOS - one indirect effect of the forums and consumer group appears to be that some service users talk informally with each other about their HoNOS results – what they mean, and what to do if they don’t agree with them. Joe says such conversations can give individuals courage to speak up where they may not have previously. Evaluation is needed to confirm this trend. It suggests that training service users about HoNOS may be as important for getting good data as training clinicians.
[Forums and being part of the consumer advisory group] made a huge difference and gave me the confidence to go back into the workforce."
- Leanne, psychiatric assistant, West Coast DHB and member of the PRIMHD Consumer Group
Positive evaluations of forums
Feedback from people attending HoNOS forums has been very positive. Attendees commented on evaluation forms that they got following the forum.
- “Understanding of the PRIMHD system and what it can do.”
- “Better understanding of the mental health process and what HoNOS is all about.”
- “More information now than before.”
- “A better understanding that I should be more involved in many aspects of my treatment.” “I learnt how to work the HoNOS 1-12 score out better.”
It’s about empowering consumers to manage their own HoNOS process directly with clinicians."
- Joe Hall, service user adviser and member of the PRIMHD Consumer Group
The lessons learnt
“I think it’s do-able anywhere,” says Sue. “It didn’t cost much,” mainly “time and commitment.” She adds that organising the forums is easy but getting people to come can be a challenge. A lot of work goes on between forums making sure that people are able – and motivated – to attend.
Sue believes the biggest benefit of involving service users in outcome collection is better communication, between “all sorts of people”:
- service users and service user organisations
- service users and the DHB
- service users and clinicians
- service users and NGOs.
The consumer involvement is very different from the clinical involvement – it’s two different languages."
- Joe Hall, service user advisor and member of the PRIMHD Consumer Group
Sue offers some words of wisdom to services thinking of adopting the West Coast model. She says “go for it – but be mindful that it has to have a consumer focus, not a clinical focus”. In her experience, “clinicians view outcomes collection very differently from the way consumers do. Sometimes it’s very scary for consumers to go to the clinicians and ask for outcome information”. This is why having members of the PRIMHD Consumer Group supporting people who want to ask for their outcome data, or discuss it, is so important.
Joe also thinks it is important to involve someone like Sue, who works for the DHB and has a strong consumer focus, in the service user group. This person provides a communication channel between the group and the DHB that Joe thinks is particularly valuable.
So is this model sustainable? Joe Hall worries that if Sue leaves it might not carry on without her. However, Sue is confident that if her role is built into her successor’s job description it will continue. The crucial issue is finding ways to make it part of ‘business as usual’.
The biggest challenge has been getting a clear message across to service users. “I want to run the forums and be clear…without being patronising,” Sue says. “It’s all about them, it’s their journey. At the end of the day, HoNOS collection and outcomes don’t exist without consumers.” What Sue wants people to go away thinking at the end of a forum is “What sort of outcome do I want?” and “How can I get the best outcome for me?”
If only one person got a job it would have been reward enough. What we’ve managed is so many good outcome stories."
- Sue Brown, PRIMHD project manager
There are a couple of things Sue wouldn’t do again. Firstly, she wouldn’t assume that clinicians would talk to service users about outcomes. Not all case managers share graphs of HoNOS ratings over time. This is why it is so important that the PRIMHD Consumer Group lets people know that they can request their data from Sue or Joe.
I want the consumers to walk away from the forums saying ‘wow, I want to know more about it [HoNOS trend data]and I want to tell others about it’."
- Sue Brown, PRIMD project manager
Secondly, Sue would look for sustainable support from the community right from the start. Community support has been a huge part of why the West Coast model has worked. When Supporting Families was no longer able to take part in the Consumer Group the West Coast branch of NGO PACT stepped into the breach. Their support has been crucial to the success of both the group and the forums. PACT provides practical support with the forum venue. Also, support from the local consumer network “is so vital,” explains Sue. “We can’t do it without them.”
Download this story in PDF format
Nothing about us without us - the benefits of informing service users about HoNOS (PDF, 2.2MB)
