Community Alcohol and Drug Services (CADS) provide alcohol and drug services across the Auckland region. They serve a population of 1.4 million people in three district health board areas - Auckland, Waitemata and Counties-Manukau.
ADOM, a two-part questionnaire, was developed because of limitations in the standard outcome measures and screening tools being used by alcohol and drug services.
The benefit is a more accurate picture of substance misuse by the service users and any psychosocial issues they identify before, during and after their treatment programme. It also allows services to plan and improve their treatment programmes, based on the information collected about reductions in substance misuse and improvements to a service user's mental health.
Pitman House, Pt Chevalier, Auckland
Prior to 2005, there were no standard reporting measures applied nationally for alcohol and other drug (AOD) services. The Programme for the Integration of Mental Health Data (PRIMHD), a Ministry of Health project, has been collecting Health of the Nation Outcome Scales (HoNOS) data since July 2008 and people in mental health services with an alcohol and drug problem should have had HoNOS data collected.
However, HoNOS is designed for individuals with mental illness and although it has a section on problem drinking and drug taking, these questions do not adequately measure outcomes for clients attending AOD services. Therefore - the need for a new measure was identified.
The Alcohol and Drug Outcomes Project (ADOPT) team was established as part of the Ministry of Health's Mental Health Standard Measures of Assessment and Recovery (MH SMART) initiative, to develop one standardised outcome measure nationally for alcohol and drug services.
The project team included the Clinical Research and Resource Centre (CRRC) and Auckland Community Alcohol and Drug Services (CADS) from Waitemata District Health Board in collaboration with the National Addiction Centre (NAC), University of Otago. Te Pou funded the development of ADOM on behalf of the Ministry of Health.
The aim was to produce a brief measure to be completed by service users rather than clinicians that is useful for both, clinicians and service users in assessing individual service users' progress.
As the project moved forward its name evolved into Alcohol and Drug Outcome Measure (ADOM). The ADOM is a two-part questionnaire for use with service users attending AOD services. It is an 18 item questionnaire, collecting information about changes in substance use and aspects of health and functioning. Part A has questions about the type and frequency of alcohol and other drug use. Part B has questions about how the use of alcohol and other drugs affects the person's life in different ways, for example, relationships, work, health and accommodation.
"We thought it would help services plan,
because we would be identifying needs
from the information provided."
Polly Websdell, CADS, alcohol and drug clinician
Currently clinicians complete other measures at CADS at various stages of a service user's treatment. They include:
- Alcohol Use Disorders Identification Test (AUDIT) which is used at first assessment of people at CADS
- Leeds Dependent Questionnaire (LDQ) which is used if indicated at first assessment
- Severity of Dependence Scale (SDS) that measures a variety of substances other than alcohol and measures drug dependence. It is used when indicated at first assessment
- Clinical Institute Withdrawal Assessment (CIWA) which rates a service user's withdrawal status and is used by the ‘medical detoxification' services
- Substances And Choices Scale (SACS), this is only used in youth-based alcohol and drug addiction services.
These measures are useful but not designed to provide information on outcomes.
CADS have deliberately chosen a strategy of implementing ADOM internally in a step-by-step, or project-by-project, manner throughout the different units in the service.
Their reasoning is that this approach allows them to test the utility of ADOM in various settings and population groups before introducing it to the entire organisation, and/or the sector.
It enables them to assess the infrastructure implications, which is crucial for a service user base of 15,500 people. They also recognise that in a research environment you can control the training and implementation, but as the measure is taken wider there will be more variability in how the measure is implemented.
Robert Steenhuisen, CADS manager, says this potential for variability and additional infrastructure is one of the reasons why they looked at developing a measure that required a brief, clinical application. He believes applying the measure in this part of alcohol and drug services will make it much easier to embed in clinical practice. Once it has been embedded, less relevant screening measures can be phased out, which in theory will decrease the amount of overall staff time a service is using to report on the measures.
Step one: Measuring outcomes with the ADOM in Community Alcohol & Drug Services Offender project
In the initial phase of the project, researchers reviewed all the outcome measures being used nationally and internationally within mental health and alcohol and drug services. Once this had been completed and a project plan drafted, the go ahead was given in 2007. This project provided an opportunity for CADS to clinically field test the ADOM as part of the brief AOD assessment.
A defined sub-population of 278 offenders with different ethnic backgrounds agreed to participate in the project. This group was referred to CADS by the criminal justice system and formed the pilot project group.
The ADOM measure was applied at treatment entry and at three and six months post-treatment.
The data collected was analysed by Clinical Research and Resource Centre (CRRC), an independent business unit within Waitemata District Health Board. Results were reported at local, national and international forums and/or conferences.
"The pilot was about looking at how individuals change over time. It was a chance to check how ADOM worked in practice. These projects allow us to see how it works in a general population rather than with offenders."
Polly Websdell, CADS, alcohol and drug clinician
Step two: Project to deliver the ADOM tool across contracted Counties-Manukau District Health Board (CMDHB) AOD providers
This independent project by CMDHB is underway and accesses service users in the wider South Auckland area.
The focus of the project is to:
- investigate the suitability of the ADOM as part of the MH SMART and as a standardised outcome tool for all contracted AOD providers within Counties Manukau;
- gain better understanding of the changes that people have made to their alcohol or drug use and their lifestyle through engaging in treatment; and
- obtain information regarding improving local access to treatment.
The project has been contracted out to Odyssey House Trust by CMDHB. The dissemination of the results is planned for June 2011.
Step three: CADS Auckland Visual ADOM project
This project commenced in March 2011 and aims to compare the impact of the new Visual ADOM with the original ADOM within service users attending different treatment modalities for their drug and/or alcohol use. The same methodology is used for both groups.
The new, colourful and visually informative chart evolved from the original standard ADOM text. Clinicians and services users can clearly and easily track and observe any major trends and changes based on the service user's answers over time.
New Visual ADOM graph, © CADS
"The Visual ADOM is intended to provide better clinical utility of the outcome measure for staff and clients alike. It provides the opportunity to match outcomes with client goals and opportunity for the organisation to monitor efficacy of interventions provided."
Susanna Galea, CADS, clinical director
Service users attending the CADS counselling service (at CADS West) and the CADS Auckland Methadone Service will be randomly selected to receive either the ADOM or the Visual ADOM. The measures will be applied at treatment entry and at exit or three months.
Working with these two separate groups of service users will allow CADS to see how the measure functions in varying day to day clinical settings.
The project is an opportunity to get feedback from staff about the new Visual ADOM and will help answer the following questions: Will the visual changes to the measure engage clinicians and their service users more than the previous questionnaire did? Is the Visual ADOM clinically useful? And will the visual element of the chart make a difference to the treatment and outcome?
The unique approach
There are a number of unique aspects to the ADOM initiative that have been deliberately thought out and built into the outcome measure.
- Unlike other measures used, ADOM has been developed in New Zealand for New Zealand alcohol and drug service users. It has also has been tested for cultural appropriateness with Maori and Pacific Island service users as well as having broader service user input in its development.
- Another strength of ADOM is the dual partnership of clinician and service user. This is a service user rated measure.
- The outcome measure is brief. It can be completed in five minutes or less. On occasion, it may take longer if the service user chooses to elaborate on some of these domains. However, any information gained during this elaboration is clinically useful.
- It includes a question about tobacco use - a question that is not always included in commonly used screening instruments.
- Another important element is the questions included in the second part of the measure. ADOM not only asks questions about service users' consumption of substances (Part A), but also addresses how those substances impact and affect services users' lives and wellbeing (Part B). The latter questions provide prompts for a more holistic approach to treatment.
Other unique features have emerged from the pilot projects at CADS.
- The champion/coordinator role of Polly Websdell, alcohol and drug clinician is unique. It plays an integral part in the success of these projects.
- The newly developed colourful and user friendly Visual ADOM chart.
(From left to right) Polly Websdell, alcohol and drug clinician,
Robert Steenhuisen, regional manager and Susanna Galea, clinical director
Data from the pilot was fully analysed alongside qualitative evidence from clinicians and service users. The response from clinicians and service users involved in the pilot has been positive. They describe it as a ‘friendly' measure and can see the value in using it. Clinicians are getting a measure they can use quickly, service users are getting information they find immediately useful and services are getting the information they need.
The pilot's post-programme three-month analysis showed a reduced frequency and amount of substance use, with a significant reduction in alcohol and cannabis use. At six months the frequency of substance abuse continued to reduce, primarily use of cannabis. Amounts of alcohol units consumed on a typical drinking day also continued to decrease at six months, despite an increase in frequency.
At both post-programme assessments most service users reported the CADS programme had helped them reduce their offending and were motivated to continue addressing their substance use. They also reported improvements in both physical and psychological health.
"People have said they can see the value in getting more insight into themselves and their behaviour.
One person said they hadn't realised how much they were drinking and what that means to them -
they'd only focussed on opiate use during the treatment programme."
Polly Websdell, CADS, alcohol and drug clinician
In February 2010 the Ministry of Health endorsed the use of ADOM for use by local Auckland alcohol and drug services.
Additional information about the application and implementation of ADOM continues to be gathered through the additional projects mentioned above.
"The results highlight that we are able to record and track individual's information over time.
The process emphasised that ADOM allows us to get information before and after treatment -
something we've not been able to do previously."
Polly Websdell, CADS, alcohol and drug clinician
The lessons learnt
Below are some of the lessons learned so far by CADS as they have implemented and used ADOM.
Services considering using ADOM need to think about why they are collecting the information and what they want to do with the information once they have it. Some services may already have something similar in place.
CADS believe implementing the ADOM would be fairly straightforward as it can be built into current infrastructures.
However, implementing an ongoing follow up project nationwide in a large service (like CADS), that is an adjunct to core business, would require a significant and substantive resource in people, administration, planning and evaluation.
Be prepared for the time it might take to introduce a new measure and have it up and running efficiently. People are busy and introducing one new thing slows people down. New processes take time to embed in an organisation.
To limit the variability in how the measure is used, and to generate more consistency, it will be important to provide training for staff. Each discipline, from clinicians and nurses to social workers, medical officers and psychologists, comes with a different understanding of the measure and will have their own biases. Once all the projects are completed, CADS intends to develop new guidelines and training for staff. The manual will explain what the measure is for and why the questions are asked. It could potentially be supported with a video and workshops where staff can role-play.
"We will need to sell the measure as an interactive, quick and simple way of gathering
valuable data from service uses - which it is."
Robert Steenhuisen, CADS, regional manager.
There are several ways to improve data collection.
- CADS emphasises that it is essential to have at least one person internally who understands ADOM and can champion it. Polly Websdell, who fulfils this role, reminds clinicians to take the initial measure. She also collects the follow-up information and then lets clinicians know about the ‘good stuff', such as how service users are tracking over time.
- Clinicians and service users want to be engaged by the measure. A tool that is attractive and easy to use is essential to getting them involved. Some clinicians are resistant as ADOM is seen as just one more form for them to fill in amongst many others. Developing a new visual tool that makes it easier for clinicians and service users to see changes over time will make the assessment more valuable.
- Making contact with service users at the three-month and six-month stages can be difficult when they may have moved on from their place of residence and not left any forwarding contact details. CADS now collect more than one contact phone number and an email address at the start. Those details are checked again when service users exit the programme - at which time they are also reminded about the follow-up measures that will be taken three and six months later. Another solution to help make it easier to reach service users is to do the follow-ups in evenings and weekends when people are home.
- The team also recognised that with a large service user group, inputting the data into an electronic database could take a lot of time. So they've decided to have just one person inputting the data from the measure so they can gain an accurate idea of how much administrative time is required and how they might streamline the process. They are using a standard electronic database already in use internally.
Use of data
- It is important to step back and see how the process can be streamlined for staff. Getting feedback from clinicians about how they prefer to receive information is important.
- At the moment, the data collected is just used by the clinician and service user and focuses on the individual treatment plan. It is not extrapolated for other uses.
- In the future, there is the potential for population-based reports to be generated as information collected includes individual substance misuse ‘scores', ethnicity, gender, age, regional location, etc. This use of the data will contribute to more long-term service planning. Teams can see how the service is doing over time and, if applicable, identify and address areas of improvements.
- Be prepared for the results you collect. They might not be what you anticipate, as they could highlight parts of your service that require improvements, so you will need to think about what will you do with results that are different from what you are expecting.
Robert anticipates future challenges as more providers in the sector use ADOM. Implementing the tool nationally could be time and resource intensive depending on the infrastructure required, so there are also implications for management in a wider national rollout.
CADS strategy to implement the measure stage by stage is one that allows challenges and changes to be managed as they come up.
- ADOM web page with background details, current sector activities information and download options for the measure
- More information about other outcome measures
Download this story in PDF format
The information on this page is also available for download in PDF format.
ADOM at Community Alcohol and Drug Services (CADS) (PDF, 204KB).