.. raw:: html

   <style type="text/css">
     span.bolditalic {
       font-weight: bold;
       font-style: italic;
     }
   </style>

.. role:: bolditalic
  :class: bolditalic

Reports
========

.. contents::
   :local:
   :depth: 2

Accessing reports
------------------

Users can access reports via the Reports page, accessible from the File Details page.

   .. figure:: screenshots/reports_button.png
      :alt: Screenshot of Reports button
      :width: 80%
      
      Reports available from File Details page

.. _structural-reports:

Record integrity report
---------------------------

The **record integrity report** provides a summary of all records, and presents as a tabluar breakdown by record type of detected issues within the file being validated. This report can be used report to identify records that have structural issues, preventing the file from being proposed for review.

Note that it reports in real time so it can be viewed while the validation run is in progress. Doing so can save you from validating an entire file for which it is already evident a rebuild is required.


   .. figure:: screenshots/reports-record-integrity.png
      :alt: Screenshot of Record Integrity Report
      :width: 80%
      
      Record Integrity Summary Table

A count and percentage is displayed for all items flagged as :bolditalic:`OK`, :bolditalic:`Malformed`, :bolditalic:`Duplicate`, :bolditalic:`Orphan`, :bolditalic:`Barren`, :bolditalic:`Miscoded` and :bolditalic:`Other`, along with a total count for each record type and for each flag category.

Record integrity validations ensure:

   * that records have the correct number of fields, as well as valid data within those fields.
   * that records specify valid hierarchical entities. For example, an organisation with a region specifies a valid corresponding region record.
   * that organisations have valid identifiers.

These rules are internal (that is, there is no SQL associated with them) as they are applied while processing the raw uploaded file before the data is extracted into the database.

Types of record integrity issues
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. _table-structural-issues:

.. csv-table:: Types of record integrity issues
   :file: reports_structural_issues.csv
   :header-rows: 0
   :widths: 20,80


.. _miscoded-lines:

.. Miscoded lines
.. ^^^^^^^^^^^^^^^^
..
.. `Draft documentation for Miscoded Lines <https://docs.google.com/document/d/1yXmkLMSZDiy04TtXoGb26FlI0GlKuYQ113_gtE7c4LU>`__
..
.. Rows are marked as miscoded if the most significant error in the row is one or more the following rule violations.
..
.. Note that these rules are internal (there is no SQL associated with them) as they are applied while processing the raw uploaded file before the data is extracted into the database.
..
.. .. _table-miscoded-lines:
..
.. .. csv-table:: Miscoded Lines
..    :file: reports_miscoded_lines.csv
..    :header-rows: 0
..    :widths: 10,10,40,40




.. _data-integrity-reports:

Data integrity reports
----------------------

Consistency validations ensure that data appears to be reasonable and consistent, both within the current reporting periods and when compared to data submitted in previous reporting periods. Many of these validations have emerged over time in response to errors in past-year data.

The Data integrity reports report on data that is invalid or appears to contain anomalies, for example:

   * a male client should not have a female-only diagnosis
   * services should serve a reasonable number of clients
   * contact durations should be of reasonable length

There are a number of common terms used in the reports, they include;

   * :bolditalic:`Record type`: Fields under this heading will change depending on the file type being reported. Information on a file’s expected record types is available in the `metadata <https://validator.net.au/webval/metadata>`_.
   * :bolditalic:`Rule`: Rules are validations that have been created with the aim of improving data accuracy. 
   * :bolditalic:`Issue`: Highlights uploaded data that triggers a ‘Rule’ that requires further attention.

Each submitted file is evaluated against a set of :ref:`rules <rules_vfields>`. These rules are generated from both the :ref:`dataset specifications <dataset_specs>`, as well as rules identified by the Commonwealth and jurisdictions to draw focus to common unusual trends that have been found over the history of the project.

Some rules use :ref:`Virtual Elements (VFields) <rules_vfields>`. These are fields that have not been directly supplied in the data; instead they are calculated from a variety of fields in the submitted data file. VFields and their SQL can be found via the `metadata <https://validator.net.au/webval/metadata>`_ site.


Dataset reports
^^^^^^^^^^^^^^^^^

Data integrity reports available for all datasets:

   * Summary of issues
   * Issues by field
   * Issues by priority
   * Issues by record type
   * Rule by record type
   * Rule by class
   * Organisation by rule (not available to NOCC)
   * Region by class
   * Organisation level issues list 

Data integrity reports available only to **NOCC**

   * Hospital cluster
   * Organisation by age group and setting
   * Sequence errors: Organisation by rule and setting
   * Measure frequency distribution
   * Person sequence frequency

Invalid records download
^^^^^^^^^^^^^^^^^^^^^^^^

The ‘Download Invalid Records’ link generates a CSV download of all of the invalid records in the datafile. Each row represents a single error; records with multiple errors are repeated with one row per error. This enables users to filter the download as required.

   .. figure:: screenshots/reports-invalid-rec-export.png
      :alt: Screenshot of export of HR level issues 
      :width: 80%
      
      Example export of Invalid HR Level Records report



Cross-file comparison reports
-----------------------------

Ad-hoc data explorer
^^^^^^^^^^^^^^^^^^^^

This report allows users to select data elements, such as the number of clients and the number of contacts, for comparison between submissions. 

For example, when comparing the number of clients and contacts between MHE and CMHC, the output provides the difference in clients and contacts between each group both as a number and a percentage. Totals are also provided.

..   .. figure:: screenshots/reports-crossfile-adhoc.png
      :alt: Screenshot of Adhoc Cross file Reports
      :width: 80%
      
      Adhoc Cross file report options

.. _non-matching-entities:

Non-matching entities
^^^^^^^^^^^^^^^^^^^^^

When uploading a new file (MHE/CMHC/RMHC), users have the option to validate the file against that year’s accepted SKL submission. If any entities in the newly uploaded file do not match what is in the SKL file, the :ref:`Non-matching entities report <non-matching-entities>` will helps to identify the unmatched entities.

The Non-matching entities report displays the entities that cannot be matched and lists them in either the :bolditalic:`Not in SKL` column or the :bolditalic:`In SKL Only` column, along with the corresponding rule. Clicking on an entity’s name takes the user directly to the related issue.

..   .. figure:: screenshots/reports-nonmatching-entities.png
      :alt: Screenshot of Non-matching Entity Report
      :width: 80%
      
      Non-matching Entities Report page

MHE-CMHC Clients & Contacts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This report summarises the difference (absolute and percentage) in **number of clients** and **number of contacts** across MHE and CMHC files.

..   .. figure:: screenshots/reports-client-contact.png
      :alt: Screenshot of MHE-CMHC Clients & Contacts report
      :width: 80%
      
      MHE-CMHC Clients & Contacts

MHE-RMHC Episode & Accrued MHC Days
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This report summarises the difference (absolute and percentage) in **number of clients** and **number of contacts** across MHE and RMHC files.

..   .. figure:: screenshots/reports-episode-accrued.png
      :alt: Screenshot of MHE-RMHC Episode & Accrued MHC Days report
      :width: 80%
      
      MHE-RMHC Episode & Accrued MHC Days Report

NOCC-CMHC/RMHC PersId Comparison Reports
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This report shows the amount of overlap between the patient identifiers in the NOCC and the CMHC, and NOCC and the RMHC. It has been designed to assist jurisdictions in understanding the integrity of these identifiers for the purposes of subsequent linkage when reporting coverage estimates as is the case with Mental Health Services Performance Indicator (MHS PI) 14, Outcomes readiness.

The NOCC and CMHC/RMHC Total columns indicate the total number of unique person identifiers found in the respective file for the given entity (i.e., at a jurisdiction, region and organisational level).

The Shared Total column indicates the number of unique identifiers found in both files. The NOCC and CMHC Shared columns indicate the percentage of their dataset's identifiers that were shared with the other dataset.

There are several important considerations when interpreting this report:
   1. The meaning of this report depends on the consistency of region and organisational code sets. This consistency should be assessed initially via validation of the NOCC & CMHC submissions with the MHE Skeleton. If different code sets are used between these collections, there can be no matches with the patient identifiers.
   2. Linkage between the NOCC and CMHC is reported only with those NOCC patient identifiers used for NOCC Collection Occasions in NOCC Ambulatory Mental Health Services Settings and those CMHC patient identifiers where the CMHC person identifier flag indicates a “genuine” unique individual.
   3. The report is generated at the time of initial file validation and compares the current submission of the NOCC/CMHC with the available CMHC/NOCC submission for the given reporting period. With the NOCC data it should be noted that there are additional validation processes applied after acceptance that filter only those data that meet the requirements of the NOCC “business rules” (i.e., one episode at a time, change of setting triggers a new NOCC episode of mental health care).

By way of guidance, we can reasonably expect that “all” consumers recorded on the NOCC in ambulatory settings will have service contacts recording on the CMHC. While there are some differences between jurisdictions, previously we have found that approximately 90-95% of NOCC patient identifiers exist in the corresponding CMHC for that reporting period.

On the other hand, the proportion of CMHC patient identifiers that have NOCC clinical ratings is likely to be in the range 35-40%. This is not surprising given that we have consistently found that approximately 30% of all unique CMHC patient identifiers, in a given reporting period, have service contacts recorded on only one or two service contact days.

It may be useful to generate these reports with previous submissions of the NOCC and CMHC/RMHC to check whether there are in fact new issues being identified in the current submission process (e.g., there may be “known” issues regarding “low” completion rates of NOCC measures by some organisations reporting CMHC service activity).



Trend reports
-------------

CMHC/RMHC Activity and Client Level Statistics
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This CMHC- and RMHC-specific report provides breakdowns of client and activity data, comparing the current and previous year's CMHC/RMHC data.

Client reports present contact and client counts broken down by client demographic information (age, sex, etc.). 

Activity reports present contact, client, contact hours and treatment days broken down by registration status (is the client registered on the system or is the person generated by the presence of a contact).


..   .. figure:: screenshots/reports-dataset.png
      :alt: Screenshot of Data Set Reports
      :width: 80%
      
      CMHC Data Set Reports

MHE: Historical trend report
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This report allows users to select and compare information in recently submitted files with that in files that had been submitted previously. Output provided includes current and previous figures and their difference, along with a percentage figure to represent growth for directly apportioned and indirectly apportioned expenditure. Totals are also provided.


..   .. figure:: screenshots/reports_mhe_historical_trend.png
      :alt: Screenshot of MHE Historical Trend Report page
      :width: 80%
      
      MHE historical trend report

NOCC: Protocol adherence
^^^^^^^^^^^^^^^^^^^^^^^^^^

..   .. figure:: screenshots/reports_nocc_protocol_adherence.png
      :alt: Screenshot of NOCC Protocol Adherence Report page
      :width: 80%
      
      NOCC protocol adherence report

The aim of the **Protocol Adherence Report** is to improve data quality by flagging potential issues of adherence to the NOCC reporting requirements as summarised in Fig 9.1. of the specification:
https://docs.validator.com.au/nocc/04.00/collection-protocol.html#figure-data-ep-setting-age-group

The colour coding of the **Protocol Adherence Report** is as follows:

   * Green - close to 100% conformance on a mandatory measure.

   * Yellow - some conformance, but many items missing from a mandatory measure.

   * Red - low or no conformance on a mandatory measure.

   * Blue - measures received when no reporting requirements apply. These measures will not be included in the standard NOCC reporting, but may be used by the Jurisdiction for its own reporting purposes.

NOCC: Measure trend
^^^^^^^^^^^^^^^^^^^^^

..   .. figure:: screenshots/reports_nocc_measure_trend.png
      :alt: Screenshot of NOCC Measure Trend Report page
      :width: 80%
      
      NOCC measure trend report

Adherence to NOCC reporting requirements in current submission is compared with historical trend data to determine whether the current submission is credible and valid. 

The colour coding of the **Measure Trend Report** is as follows:

   * Green - close to 100% conformance on a mandatory measure.

   * Yellow - some conformance, but many items missing from a mandatory measure.

   * Red - low or no conformance on a mandatory measure.

   * Blue - measures received when no reporting requirements apply. These measures will not be included in the standard NOCC reporting, but may be used by the Jurisdiction for its own reporting purposes.
  

Reporting principles
--------------------

**Person at organisation, cluster and service unit levels**

The person identifiers are defined at ORG level but are reported below SERV on PER records. This leads to various complications. Inconsistent values at a PER level is caught by the *Differs* rules, but it's still possible to have one valid value and some number of Missing values (or all Missing). Counted differently at different SERV and ORG entity levels:

   * Same person (orgid+persid) in two units of the same org
   * Same person (orgid+persid) in two units of the same org on same day

Both could be counted as 2 at reports below ORG and 1 above.

**Variable person attributes**

The general principle is to use the most recent valid value. In situations where multiple values can be taken by a person, either different values on PER records in different SERVs or over time on CON attributes, then the most recent (ContDt) valid value should be consistently used in all reports. "Valid" depends on the particular variable, but generally just means non-missing and mappable.

**Variable person DxPrinc same-day**

Find most-recent contact diagnosis prefix for each person. In the case of a multiple, last-day CONs, the RecordId is taken as a stable, albeit arbitrary, tie-breaker.

**Country of birth priority**

CoB codes in priority order:
   1. Valid: 0001,1000,1100-9999(not 1603)
   2. Not Stated: 0003
   3. Inadequately Described: 0000
   4. Other: ' ' or out of valid 1603,0002,0004-0999,1001-1099

These priority classes are used by the most recent, valid value rule. Some are more valid than others.

**Date of birth priority**

More accurate values of DoBFlag are preferred (if present).

**Person age**

This is calculated at the RepEnd reporting period end date and used in AgeGroup reports.

**Treatment day calculation**

"Treatment Days" are calculated separately at SERV, CLUS and ORG levels with the result that one ORG person seeing two units on the same day will count as 2 at the lower level and 1 at the higher level.