About the HoNOS family data dashboards
The HoNOS family dashboards Aotearoa summarise Programme for the Integration of Mental Health Data (PRIMHD) data submitted by Te Whatu Ora districts and some NGOs. The dashboards present the Health of the Nation Outcome Scale (HoNOS) family of measures data from services in which HoNOS is the primary measure used.
Two data dashboards are available.
- A combined dashboard for HoNOS (for adults aged 18 to 64 years) and HoNOS65+ (for adults 65 years and older).
- A HoNOSCA dashboard (used with children up to the age of 17 years).
PRIMHD information is only a starting point and should provide more questions than answers. It gives important information about how your services work. Of greater value is the use of the information to guide your curiosity and ask more in-depth questions about our services and how we might improve them.
Data quality
To ensure the data collected is of good quality, clinicians need to ensure that they:
1. use the appropriate glossary to complete the ratings
2. have been trained in the measure and
3. have had some practice in rating.
Using the reports
In many cases, the data is presented graphically for the whole Te Whatu Ora district compared with national data and then presented as a table for the individual teams.
-
Navigation
Use the tabs at the top of the data dashboard to move between report pages.
-
Accessing further information
On each report page within the data dashboard, click the Page User Guide link to access information to help with interpretation.
-
Selecting the report period
The time period displayed can be altered by changing the period selected in the Year quarter slicer at the top right of each report page in the dashboard. The report is intended to be used to view four consecutive quarters, but other time periods can be viewed.
-
Selecting multiple organisations
Hold down the CTRL or Command key to select multiple organisations.
About the dashboards
The HoNOS family of measures data dashboards provide information relating to three areas.
Methodology
Ethnicity
The prioritised ethnicity data protocol has been used for reports within the data dashboard. The ability to filter by ethnic group can help to identify ethnic groups for whom improved approaches or additional resources would be particularly helpful to assist with achieving equitable outcomes and to identify services that achieve good outcomes with particular ethnic groups so their approach can be studied and transferred.
Minimum sample size for inclusion
Any data point that is made from less than 5 cases will not be presented either on graphs or in the tables. In some cases, this has been increased to 10. This is because when the number of cases making up a data point becomes small, the data becomes unreliable and is likely to be misleading.
Cross-sectional outcomes
The data presented in the HoNOS family of measures reports is different from most outcome evaluations. This is because rather than comparing the same people at the beginning and end of their contact with the service, it compares the cohort admitted and the cohort leaving the service at the same time. This is done so that the maximum amount of data collected can be used. In most cases, the nature of referrals over the average length of stay will change little, so this provides a reasonable indication of the outcomes achieved. Where client mix changes significantly this approach may not be valid.
Matched pairs (comparison of admission and discharge data for the same person) are used in one report page but the data set is limited to collections where a valid admission and discharge data is available for the same person.
Confidence intervals
Where appropriate, the statistical confidence interval is presented. This is shown by error bars (small lines above and below the average) on the graphs, and a score range in some tables.
There is a degree of uncertainty about all data which means we don’t know how well the average of the sample we have collected approximates the ‘true’ average value. However, we can calculate the range of values in which the ‘true’ value is most likely to be. The error bars (small lines above and below the average) on the graphs mark the confidence interval which indicates the range in which the true value is likely to be (95% probability). The range of scores covered by the confidence interval is also listed in some tables.
To avoid over-interpreting data (in particular, thinking two things are different when they really aren’t) the convention is to only regard them as actually different if their confidence intervals don’t overlap. If their confidence intervals do overlap, we normally assume there is no real difference between them, even if the difference looks interesting. If confidence intervals don’t overlap, we can assume that the points are statistically significantly different.
This is quite a conservative test, and may not always be correct, but it is a fairly safe way of preventing over-interpreting the data.
Example:
In this case, the confidence intervals for alcohol and other drugs for assessment and admission collection overlap, so we infer that there is no significant difference between categories. However, the confidence intervals for self-harm for assessment and admission collection do not overlap, so we infer that the categories are significantly different.
