NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
NHS DigitalIndicator Supporting Documentation
IAP00137 People who have a follow-up assessment between 4 and 8 months after initial admission for stroke
FIELD CONTENTSIAP Code IAP00137Title People who have a follow-up assessment between 4 to 8 months after initial admission for strokePublished by Department of Health and Social CareReporting period
Annual
Geographical Coverage
England
Reporting level(s)
National
Based on data from
Royal College of Physicians’ Sentinel Stroke National Audit Programme (RCP SSNAP), Office for National Statistics (ONS) mortality data
Contact Author Name Alison Roe, Senior Service Delivery Manager, HSCIC
Review date 14/12/2016Indicator set CCG Outcomes Indicator Set (OIS) 3.8Brief Description [This appears as a blurb in search results]
This indicator is the percentage of patients in the Sentinel Stroke National Audit Programme who had a follow-up assessment between 4 and 8 months after initial admission for stroke.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Purpose This indicator is an important measure of the effectiveness of rehabilitation within the wider local healthcare system outside of the inpatient setting. It is expected that this indicator will be used to track progress over time and highlight areas for improvement.
Definition Of the number of eligible stroke patients entered into the Sentinel Stroke National Audit Programme (SSNAP), the percentage who had a follow-up assessment between 4 and 8 months after initial admission for stroke.
Stroke is defined within this indicator as intracerebral haemorrhage, cerebral infarction and stroke (not specified as haemorrhage or infarction).
Data SourceRCP SSNAP, ONS mortality data
Numerator The number of patients who had a follow-up assessment between 4 and 8 months after initial admission for strokeDenominator The number of patients entered into SSNAPCalculation The indicator is calculated as a percentageInterpretation Guidelines
This indicator requires careful interpretation and should not be viewed in isolation. It should be considered alongside information from other indicators and alternative sources, such as the other Clinical Commissioning Group Outcome Indicator Set (CCG OIS) stroke measures and the CCG level Stroke Sentinel National Audit Programme (SSNAP) stroke unit key indicators. When evaluated together, these will help to provide a comprehensive view of CCG outcomes and provide a more complete overview of the impact of the CCGs’ processes on outcomes.
A high percentage of stroke patients who have a follow-up assessment between 4 and 8 months after initial admission for stroke is desirable.
There is currently very wide variation across CCGs. Recent analysis of the completeness of the data for this indicator shows that almost a third of eligible patient records do not hold the relevant assessment information and are, therefore, excluded from the indicator.
Caveats Care may vary between organisations in terms of hospital inpatient admission practices and policies.
There may be variation in the prevalence of stroke due to differing levels of deprivation, for other regional or demographic reasons or between patients of different ethnic heritages.
Application FormIndicator and Methodology Assurance Service
Title: Set or domain: CCG OIS 3.8
IAS Reference Code: IAP00137
Version HistoryVersion Date Changed By ChangeV0.1 23/06/201
7Andy Besch Initial uplift to most recent application form
Application FormSection 1 Introduction / Overview
1.1 Title
1.2 Set or domain CCG Outcomes Indicator Set (OIS) 3.8
1.3 Topic area Cardiovascular
1.4 Definition The percentage of people who have a follow-up assessment between 4 and 8 months after initial admission for stroke.
Technical description: Of the number of eligible stroke patients entered into the SSNAP, the percentage who had a follow-up assessment between 4 and 8 months after initial admission for stroke.
Eligible patients exclude;
Patients who died within 8 months of initial admission for stroke and who did not have a follow-up assessment
Patients who died whilst on the stroke care pathway (reported by either an inpatient team or a community team)
Patients who decline an appointment offered
Patients for whom an attempt is made to offer an appointment but are untraceable as they are not registered with a GP
Stroke is defined within this indicator as intracerebral haemorrhage (ICD-10 code: I61), cerebral infarction (I63) and stroke, not specified as haemorrhage or infarction (I64).
It was published for the first time in December 2014 (2013/14 data) and has been published annually in December since then.
1.5 Indicator owner & contact details
1.6 Publication status
Currently in publication
Section 2 Rationale
2.1 Purpose The National Stroke Strategy recognises that people who have had a stroke should be offered a follow-up assessment of their health and social care status and secondary prevention needs. This indicator is to ensure follow-up assessments take place, making it possible for the patient to access further specialist advice, information and rehabilitation where needed. For practical reasons, given the nature of care provided, a follow-up assessment period of
Application FormIndicator and Methodology Assurance Service
Title: Set or domain: CCG OIS 3.8
IAS Reference Code: IAP00137
Version HistoryVersion Date Changed By ChangeV0.1 23/06/201
7Andy Besch Initial uplift to most recent application form
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
4-8 months after discharge is considered appropriate to satisfy the conditions of the indicator.
Reviews should be a multifaceted assessment of need and should encompass:
Medicines/general health needs Ongoing therapy and rehabilitation needs Mood, memory cognitive and psychological status Social care needs, carer wellbeing, finances and benefits,
driving, travel and transport.
2.2 Sponsor
2.3 Endorsement The indicator was constructed following consultation with the following clinical and stroke data experts:
Professor Anthony Rudd, Chair of the Intercollegiate Stroke Working Party, Associate Director for Stroke, Consultant Stroke Physician
James Campbell, Sentinel Stroke National Audit Programme (SSNAP) Intelligence Programme Manager, Royal College of Physicians (RCP)
Including related national incentives, critical business question, NICE quality standard and set or domain rationale, if appropriate
This indicator supports the Quality Standard for stroke (QS2)1 which covers care provided to adult stroke patients by healthcare staff during diagnosis and initial management, acute-phase care, rehabilitation and long-term management.
Guidance suggests that regular reviews after transfer home provide a key opportunity to ensure people get the support they need2. Recovery from stroke can continue over a long time and rehabilitation should continue until it is clear that maximum recovery has been achieved. Some patients will need ongoing support, possibly for many years. These people and their carers should have access to a stroke care co-ordinator who can provide advice, arrange reassessment when needs or circumstances change, co-ordinate long-term support or arrange for specialist care3.
The National Stroke Strategy advised that people who have had strokes (and their carers), either living at home or in care homes, are offered a review from primary care services of their health and social care status and secondary prevention needs, typically within six weeks of discharge home or to care home and again before six
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
months after leaving hospital. The offer of regular review is one way to ensure people continue to feel supported. Of key importance is the enabling and supporting of people in navigating through the system. Self-referral systems that do not take this into account are likely to increase inequalities of access and outcome4.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Section 3 Data
3.1 Data source
RCP SSNAP and Office for National Statistics (ONS) mortality data, via record linkage.
The SSNAP is guided by the Intercollegiate Stroke Working Party (ICSWP) and delivered by the Stroke Programme within the Clinical Effectiveness and Evaluation Unit in the RCP. It is centrally funded by the Healthcare Quality Improvement Partnership (HQIP) on behalf of NHS England as part of the National Clinical Audit and Patient Outcomes Programme (NCAPOP).
The SSNAP is the single source of data on stroke services, processes of care and outcomes. It provides the data for other statutory reporting mechanisms in England, including the NICE Quality Standard and the five other CCG OIS stroke measures; it is also due to be used in the NHS Outcomes Framework. SSNAP metrics are aligned with those in the Cardiovascular Disease Outcomes Strategy.
Hospital Episode Statistics (HES) was considered as a data source for this indicator; however it does not contain the necessary detail required to measure follow-up assessment.
3.2 Data availability
CCG OIS indicators are published annually. SSNAP data for the full financial year is available to produce the indicator approximately 8 months after the financial year end; therefore the indicator is published each year in December.
CCG OIS indicators are official statistics and the publication date is pre-announced. There is no gap between the planned and actual publication date.
The RCP make this indicator, along with a number of others, accessible to the public via RCP reporting, including an Easy Access Version aimed at stroke survivors and carers. It is available via Excel spreadsheets and other formats including graphical representation.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
3.4 Data quality
i) What data quality checks are relevant to this indicator?
Coverage ☐
Completeness ☐ Validity ☐
Default ☐
Integrity ☐ Timeliness ☐ Other ☐If you included ‘Other’ as a data quality check, please describe the check, how it will be measured, and its reason for use below: ii) What are the current values for the data quality checks selected? The period of data the current values are calculated from should be stated. Current values should be recorded as a percentage and calculated as described below. Period of data: Coverage: Calculation: Completeness: Calculation: Validity: Calculation: Default: Calculation: Integrity: Calculation: Timeliness: Calculation: Other: Calculation:iii) What are the thresholds for the data quality checks selected? Coverage: Completeness: Validity: Default: Integrity: Timeliness: Other: iv) What is the rationale for the selection of the data quality checks and thresholds selected above? v) Describe how you would plan to improve data quality should it not meet, or subsequently fall below, the thresholds required for this indicator. vi) Who will own the data quality risks and issues for this indicator? Name: Job Title: Role: Email: Telephone:
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
3.5 Quality assurance
As SSNAP data is subject to strong built-in validation via the secure web tool, it means that it is not possible for providers to enter illogical timings; however, this is double checked during analysis and therefore the accuracy of the indicator is very high. No assumptions are made regarding the arrival and discharge times, apart from when a patient died in hospital.
When submitting SSNAP data, security and confidentiality are maintained through the use of passwords and a person specific registration process. A dedicated helpdesk is in place to answer queries from SSNAP participants, helping to ensure questions are interpreted consistently (which informs updates to FAQs and data set help notes). Users can register for their team on the SSNAP web tool and input data for their team. Once records are complete and correct they can be ‘locked’ at different levels. Records can be ‘locked’ to 72 hours once this information is completed, they can then be locked to discharge once this is applicable. Locking confirms that all data have been clinically signed off and are ready for central analysis. The ‘Lead clinical contact’ role is responsible for ensuring that the overall system of data collection and entry onto the web tool is accurate, robust and functioning. The SSNAP encourage the lead to routinely check data. Only complete and locked to 72 hours records go into data analysis for the 72 hour section and complete and locked to discharge records go into data analysis for the post-72h section.
Eligibility criteria are applied to determine which records can be included in the audit. The criteria are: ICD-10 codes I61, I63, I64, but hospitals have means of checking for eligible patients other than their coding system and participants are encouraged to enter cases prospectively meaning the stroke team have more control over selecting records to be included and can also refer to their stroke register, should they have one.
If this question is not answered, it is interpreted in the RCP SSNAP output as an assessment did not take place; this stance is providing an incentive for stroke teams to record this information.
3.6 Data linkage
SSNAP records are linked with mortality information from ONS. The SSNAP data are securely sent for linkage following each quarterly deadline, and the information on any death notifications is provided back monthly. This enables SSNAP to track mortality other than as reported on SSNAP (i.e. after patients have left care). As well as providing casemix adjusted mortality rates, this is also used for other purposes, such as to determine eligibility for receiving a six month assessment.
3.7 Quality of data linkage
The match rate between SSNAP and ONS data stood at over 98% for 2013/14 data; therefore the indicator is considered an accurate representation of mortality for stroke patients. The RCP are still awaiting 2014/15 mortality data but there are not expected to be concerns with the linkage.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
3.8 Data fields The data fields supplied by the RCP are as follows:
1. Number of patients considered applicable to be assessed at 4-8 months
2. Number of patients alive when due a 4-8 month assessment
3. Percentage of patients alive who are considered applicable to be assessed at 4-8 months
4. Number of applicable patients assessed
5. Number of patients applicable to be assessed
6. Percentage of applicable patients who are assessed at 4-8 months
3.9 Data filters SSNAP-derived records meeting all of the following requirements are valid for the denominator in this indicator:
Audit Question 7.1 – The response to ‘The patient…’ is not recorded as ‘Died’.
Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘Yes’ or ‘No’ or is not completed.
Please note that in order to improve the recording of the six month follow-up assessment, the ‘or is not completed’ element has been introduced into the filter for future data releases. This part of the filter was not present in 2013/14 data.
SSNAP-derived records meeting all of the following requirements are valid for the numerator in this indicator:
Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘Yes’
A patient is included in both the denominator and numerator if they had a six month follow-up assessment, regardless of if they died after receiving it. A patient is excluded from the denominator if they did not have an assessment and died within 8 months of admission; identified using ONS data (Date of Death within 8 months of Admission Date).
3.10 Justifications of inclusions and exclusions
and how these adhere to standard definitions
Audit Question 7.1 is not recorded as ‘Died’ – Identifies patients who did not die whilst on the stroke care pathway (reported by either an inpatient team or a community team). This is implicit within the indicator; if a patient is recorded as having died at this stage, it is not possible for the stroke team to complete Audit Question 8.1.
Audit Question 8.1 is equal to ‘Yes’ or ‘No’ – Identifies patients who had or could have had a follow-up assessment at 6 months (plus or minus 2 months).
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Audit Question 8.1 is equal to ‘Yes’ – Identifies patients who had a follow-up assessment at 6 months (plus or minus 2 months).
The rationale for including a patient in both the numerator and denominator if they had a six month follow-up assessment, regardless of if they died after receiving it, is that credit should be given where six month assessments are completed. Patients should be excluded if they could have received a six month assessment within the recommended time frame (4-8 months) but died before being offered or having that assessment.
The SSNAP uses the following ICD-10 diagnosis codes to identify stroke patients:
I61 - Intracerebral haemorrhage
I63 - Cerebral infarction
I64 - Stroke, not specified as haemorrhage or infarction
The coding advice from the Clinical Classifications Service also includes I60 (Subarachnoid haemorrhage) and I62 (Other nontraumatic intracranial haemorrhage), however this advice would not be endorsed by the RCP as subarachnoid haemorrhage and other non-traumatic intracranial haemorrhage have a different care pathway and outcome.
Subarachnoid haemorrhages and other non-traumatic intracranial haemorrhages are routinely and nearly always managed entirely outside of the stroke unit by neurosurgeons or by interventional neuroradiologists, which is what is recommended in national guidelines for these cases. The indicators need to reflect the care given on appropriate clinical pathways, not arbitrary groupings.
3.11 Data processing
The calculated CCG level indicator is provided by the RCP and includes the percentage, numerator, denominator and contextual information. It is provided with any necessary data suppression.
A 95% confidence interval is calculated by Clinical Indicators for each CCG prior to publication
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Section 4 Construction
4.1 Numerator
Of the denominator, the number of patients who had a follow-up assessment between 4 and 8 months after initial admission for stroke.
4.2 Denominator
The number of stroke patients entered into the SSNAP excluding;
Patients who died within 8 months of initial admission for stroke and who did not have a follow-up assessment
Patients who died whilst on the stroke care pathway (reported by either an inpatient team or a community team)
Patients who decline an appointment offered
Patients for whom an attempt is made to offer an appointment but are untraceable as they are not registered with a GP
A patient is included in both the denominator and numerator if they had a six month follow-up assessment, regardless of if they died after receiving it. A patient is excluded from the denominator if they did not have an assessment and died within 8 months of admission (identified using ONS data - Date of Death within 8 months of admission date).
Patients who decline an appointment offered are patients who are offered a follow-up assessment and decline or do not attend the assessment. These are identified in the SSNAP where Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘No but’ and therefore excluded from the denominator.
4.3 Computation
The percentage p is given by:
p=On×100
where:
O is the numerator and n is the denominator.
4.4 Risk adjustment or standardisation type and methodology
None
Variables and methodology:
4.5 Justification of risk adjustment type and
All eligible patients should receive a follow-up assessment between 4 and 8 months after initial admission for stroke.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
variables
or why risk adjustment is not used
4.6 Confidence interval / control limit use and methodology
Confidence Intervals
Methodology:
Confidence intervals are calculated using the Wilson Score method, as specified in “Commonly used public health statistics and their confidence intervals” (Public Health England (PHE), March 2008 http://webarchive.nationalarchives.gov.uk/20170106081009/http://www.apho.org.uk/resource/view.aspx?RID=48617 ).
The formulae for the 100(1 – α)% confidence interval limits for the proportion p are:
Plower=2O+z2−z √z2+4 oq
2 (n+z2 )
Pupper=2O+z2+z √z2+4 oq
2 (n+z2)
where:
O is the observed number of individuals in the sample/population having the specified characteristic (i.e., the numerator);
n is the total number of individuals in the sample/population (i.e., the denominator);
q = (1 – p) is the proportion without the specified characteristic;
z is the 100(1 – α/2)th percentile value from the Standard Normal distribution. For example for a 95% confidence interval, α = 0.05, and z = 1.96 (i.e. the 97.5th percentile value from the Standard Normal distribution).
4.7 Justification of confidence intervals / control limits used
The preferred PHE confidence interval method for proportions is the Wilson Score method5 which has been evaluated and recommended by Newcombe and Altman6;7. It can be used with any data values and, unlike some methods, it does not fail to give an interval when the numerator count, and therefore the proportion, is zero8.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Section 5 Presentation and InterpretationPresentation
5.1 Presentation of indicator
The indicator is presented on the NHS Digital Indicator Portal in a consistent format to other CCG OIS indicators. It is accompanied by indicator specification and quality statement documents, which provide details of indicator construction, data quality, statistical methods and interpretation considerations http://indicators.ic.nhs.uk/webview.
The data is presented with a detailed header including information on the statistic presented, the reporting period, level of coverage, publication date, data source, and any further notes to be aware of. The customer is also able to make use of drop-down filtering.
Column name Output
Reporting period Financial year
Breakdown England, CCG
Level CCG Code
Level description CCG Name
Percentage The indicator percentage calculation
CI lower Lower 95% confidence interval
CI upper Upper 95% confidence interval
Denominator The number of patients entered into SSNAP
Numerator The number of patients who had a follow-up assessment between 4 and 8 months after initial admission for stroke
Number of records in SSNAP (care delivered within the first 72hrs)
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Estimated expected number of patients (from HES)
The number of cases in HES
Case ascertainment band Case ascertainment between SSNAP and HES
5.2 Contextual information provided alongside indicator
with justification
Alongside the numerator, denominator and percentage, the number of records in SSNAP (care delivered within first 72hrs) is provided for each CCG as contextual information.
The indicator is published in the context of case ascertainment between SSNAP and HES. The ‘Estimated expected number of patients from HES’ figure is the number of patients who have been coded as a primary diagnosis of stroke during their admission in a year’s worth of HES, split by the patient’s CCG recorded in the HES record. Case ascertainment is reported alongside the indicator for all CCGs to highlight audit coverage against HES. MRG requested this further analysis in the original assurance process.
Case ascertainment is reported within the context of the ‘care delivered within the first 72hrs’ cohort of stroke patients for this indicator. The ‘Case ascertainment band’ column in the published output uses the following bandings:
90%+
80-89%
70-79%
50-69%
Less than 50%
The indicator is not reported for any CCGs with lower than 50% case ascertainment or for those with fewer than 20 patients.
5.3 Calculation and data source of contextual information
The contextual information is sourced from the SSNAP and provided by the RCP.
5.4 Use of bandings, benchmarks or targets
with
None. If a CCG believes their figure to be disproportionately low, the factors contributing to this can be investigated and appropriate action can be taken, however data completeness is likely to be an issue for many CCGs.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
justification
5.5 Banding, benchmark or target methodology
if appropriate
N/A
Interpretation
5.6 Interpretation guidelines
A high percentage of stroke patients who have a follow-up assessment between 4 and 8 months after initial admission for stroke is desirable.
As stated in section 3.4, there is currently very wide variation across CCGs. Recent analysis of the completeness of the data for this indicator shows that almost a third of eligible patient records do not hold the relevant assessment information and are therefore excluded from the indicator. As stated previously, it is proposed that in future data releases, if this question is not answered; it will be interpreted as an assessment did not take place (see section 3.10 for the amendment to the data filter).
This indicator requires careful interpretation and should not be viewed in isolation but instead be considered alongside information from other indicators and alternative sources, such as the other CCG OIS stroke measures and the CCG level SSNAP stroke unit key indicators. When evaluated together, these will help to provide a holistic view of CCG outcomes and provide a more complete overview of the impact of the CCGs’ processes on outcomes.
5.7 Limitations and potential bias
The patterns of providing care may vary between organisations in terms of hospital inpatient admission practices and policies.
There may be variation in the prevalence of stroke due to differing levels of deprivation, for other geo-demographic reasons or between patients of different ethnic heritages.
5.8 Improvement actions
It is expected that CCGs will use this indicator to identify improvements in care and how they can be delivered.
Improvements could be made by enhancing aspects of the services CCGs commission for patients. This could come in the form of improving follow-up processes to make it easier to assess discharged patients in a timely manner.
While the vast majority of patients alive at this time after stroke are applicable to receive a six month follow-up, this is recorded to have happened in less than 20% of cases. Clinical teams and commissioners need to work closely together to see recording improve to get the most value from the audit for service improvement.
5.9 Evidence of variability
The data within this section is taken from the December 2014 CCG OIS publication.
During the financial year 2013/14 there were 24,873 patients eligible to receive a follow-up assessment between 4 and 8 months. Of these, 4,059 had a follow-up
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
assessment.
.
The data below shows the ten CCGs with the lowest and the ten CCGs with the highest percentages in 2013/14. Five CCGs have been suppressed due to insufficient case ascertainment between SSNAP and HES and are not included within the data below.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
There is very wide variation in the 2013/14 data with 64 CCGs (30.3%) recording that none of their eligible stroke patients received a follow-up assessment between 4 and 8 months after initial admission. The RCP acknowledged the variation in the SSNAP 2013/14 annual report (https://www.strokeaudit.org/Documents/Results/National/Apr2013Mar2014/Apr2013Mar2014-AnnualReport.aspx) stating that, ‘At present, an increasing number of (stroke) teams are starting to enter information routinely to SSNAP, but based on the patients who could have had a six month (plus or minus two months) follow-up assessment within the financial year, the rate is less than one in five.’
As stated in section 3.4, the subsequent RCP SSNAP Clinical audit October - December 2014 public report (https://www.strokeaudit.org/Documents/Results/National/OctDec2014/OctDec2014-PublicReport.aspx) provides an indication of how widely this section of the audit is being answered. Between July and December 2014, 34,476 patient records should have had an answer for the 6 month assessment. Of these, 10,709 (31.1%) did have an answer
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Section 6 Risks
6. 1Similar existing indicators
This indicator is published in different formats at CCG, trust and stroke team level on the SSNAP results portal http://www.strokeaudit.org/results/Clinical-audit/National-Results.aspx
The Accelerating Stroke Improvement National Plan was a national initiative designed to ensure that maximum implementation of the Quality Markers in the National Stroke Strategy were achieved before the end of the 2010/11 financial year. One of the measures (measure 8) looked at the proportion of stroke patients that were reviewed six months after leaving hospital, with the aim of 95% by April 2011 http://www.stroke-in-stoke.info/otherfiles/Accelerating%20Stroke%20Improvement%20National%20Plan.pdf
6.2 Coherence and comparability
The methodology and results for this indicator are consistent with the same indicator published on the SSNAP results portal.
6.3 Undesired behaviours and/or gaming
Gaming would involve stroke teams purposefully not completing the non-mandatory 6 month follow-up assessment field when the assessment did not take place. However, this issue will be addressed in future releases of data by being interpreted in the SSNAP output as an assessment did not take place where the question is not answered.
6.4 Approach to indicator review
The Indicator Governance Board (IGB) set a review period of one year when the indicator was originally assured, due to the relative immaturity of the SSNAP data set at that time. The time period for the next review will again be set by IGB.
User feedback and comments on this indicator are welcomed via NHS Digital Enquires [email protected] or the CCG OIS mailbox [email protected]
6.5 Disclosure control
Case ascertainment used is the proportion of patients per CCG with primary ICD-10 codes I61, I63 and I64 in HES data who are included in SSNAP for the same time period.
Case ascertainment is reported alongside the indicator for all CCGs. The indicator is not reported for any CCGs with lower than 50% case ascertainment or for those with fewer than 20 patients, instead replacing the SMR with ‘*’.
Ratios are rounded to two decimal places before publication.
6.6 Copyright
There are no restrictions on the use of these data. Any subsequent use or publishing of these data should reference the RCP SSNAP.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
through including null responses in the denominator, and the quality of the indicator otherwise, MRG are endorsing it for inclusion in the Library.
Summary Recommendation to IGB:
MRG endorse the indicator for inclusion in the Library, however suggest that data quality is reassessed upon review. There are small improvements which could be made to the metadata, specifically around justifying the data source, how and why HES is used to measure case ascertainment, and the interpretation guidelines. In addition, there is currently no named sponsor for the indicator.
Please find a detailed description of recommendations and actions in the appraisal log at the end of the document.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
What do the Assurance Ratings mean?
Rating Description
Fit for use This indicator can be used with confidence that it is constructed in a sound manner that is fit for purpose.
Fit for use with caveats The indicator is fit for use, however users should be aware of caveats and/or recommendations for improvement that have been identified during the assurance process.
Use with caution The indicator is based on a sound methodology for which the assurance process endorse the use, however issues have been identified with the national data source which have implications for its use as an indicator.
Not fit for use Issues have been identified with the indicator which have resulted in the assurance process currently not endorsing its use as a quality indicator.
Not enough information provided
There has not been enough information supplied to the assurance process to be able to accurately give the indicator a level of assurance.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Appraisal Log ClarityRec. no
Issue or recommendation Raised by / Date
Response or Action taken by applicant Response date
Resolved
Sign off by / Date
☐RationaleRec. no
Issue or recommendation Raised by / Date
Response or Action taken by applicant Response date
Resolved
Sign off by / Date
2a A sponsor for the indicator needs to be identified.
MRG
10/09/15
The sponsor of the CCG OIS is Richard Owen, Outcomes Strategy Lead, NHS Medical Directorate, NHS England.
2b The definition should be clear as to the types of stroke included in the indicator.
MRG
10/09/15
A sentence is included in the definition section of the IAS application form and Indicator Quality Statement, stating: Stroke is defined within this indicator as intracerebral haemorrhage (ICD-10 code: I61), cerebral infarction (I63) and stroke, not specified as haemorrhage or infarction (I64).
2c The paperwork should be clearer regarding what constitutes a follow-up.
MRG
10/09/15
The National Stroke Strategy recognises that people who have had a stroke, either living at home or in care homes, should be offered a review of their health and social care status and secondary prevention needs. Reviews should be a multifaceted assessment of need and should encompass:
Medicines/general health needs Ongoing therapy and rehabilitation
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Social care needs, carer wellbeing, finances and benefits, driving, travel and transport.
DataRec. no
Issue or recommendation Raised by / Date
Response or Action taken by applicant Response date
Resolved
Sign off by / Date
3a MRG recommended that investigations into whether case ascertainment is the same for different age and sex breakdowns as selection bias could affect the calculation of the indicators.
MRG
6/9/12
The age and sex breakdowns were investigated by the RCP in 2012 and found to be comparable with published literature and therefore not felt to represent a selection bias.
The SSNAP annual report provides some overall demographic details of patients included in the SSNAP https://www.strokeaudit.org/Documents/Newspress/SSNAP-Annual-Report-(April-2013-March-2014).pdf
Along with a host of other detailed audit information, the quarterly SSNAP public report provides specific details on the casemix breakdowns, including patient numbers, gender, age, co-morbidities, stroke type, Modified Rankin Scales scores, NIHSS and the onset of symptoms (Section 2: Casemix, p48) https://www.strokeaudit.org/Documents/Results/National/OctDec2014/OctDec2014-PublicReport.aspx
MRG10/09/15
3b The rationale for selecting the ICD-10 codes used to identify stroke patients should be clearly stated in the documentation for each indicator.
MRG
6/9/12
The SSNAP uses the following ICD-10 diagnosis codes to identify stroke patients:
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Update:There is a discrepancy between what SSNAP and the clinical classifications service consider a stroke, therefore further justification for the codes used is required and the definition should be updated (as stated in recommendation 2b).
MRG
10/09/15
I64 - Stroke, not specified as haemorrhage or infarction
The coding advice from the Clinical Classifications Service also includes I60 (Subarachnoid haemorrhage) and I62 (Other nontraumatic intracranial haemorrhage), however this advice would not be endorsed by the RCP as subarachnoid haemorrhage and other non-traumatic intracranial haemorrhage have a different care pathway and outcome.
Update:Subarachnoid haemorrhages and other non-traumatic intracranial haemorrhages are routinely and nearly always managed entirely outside of the stroke unit by neurosurgeons or by interventional neuroradiologists, which is what is recommended in national guidelines for these cases. The indicators need to reflect the care given on appropriate clinical pathways, not arbitrary groupings.
3c Peer review commented on whether there was uncertainty around data completeness in the audit collection as it is new.
Peer review
19/11/12
See section 6a 13/08/15 MRG10/09/15
3d Peer reviewer asked for clarification on how follow-ups, usually done in the community, would be captured as the peer reviewer didn’t have full understanding of how data will be completed in the audit.
Peer review
19/11/12
There is variation across the country in what processes are used for follow-up assessment with teams employing local recording practices. Some areas may use a shared or integrated care record. The data are submitted by providers via a secure web tool.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
3d The narrative around why SSNAP is being used as opposed to HES should be strengthened. The application states that over-coding occurs in HES, however the results in section 5.9 show that case “ascertainment” against HES is over 100%.
MRG
10/09/15
The application for this indicator did not state that over-coding occurs in HES. The application stated that HES does not contain the necessary detail required to measure this indicator.
3e The applicant should consider how useful it is to provide case ascertainment against HES data, since it is recognised that over-coding occurs in HES, making the figure hard to interpret. If the figure is to be presented, MRG recommend changing the name from “case ascertainment” to “case comparison” and to present bands above 90+%.
MRG
10/09/15
This contextual case ascertainment information aligns to the information and bandings presented in the RCP SSNAP publication. The RCP view is that it is not case comparison as it is not comparing the same year’s HES with SSNAP. Since the purpose of including case ascertainment is to highlight CCGs with low case ascertainment indicating that hospitals within the CCG have not been entering in all their patients onto SSNAP (and the results may therefore not reflect the care that all the CCGs patients received), having bands above 100% would not be useful. HES is not the ‘gold standard’, but it is a useful indication of case selection.
The HES case ascertainment figure (‘Estimated expected number of patients from HES’) is the number of patients who have been coded as a primary diagnosis of stroke during their admission in a year’s worth of HES, split by the patient’s CCG recorded in the HES record. The indicator is not reported for CCGs with less than 50% case ascertainment.
3f The data quality is currently low, however MRG endorse the approach taken by the
MRG If Audit Question 8.1 - Did this patient have a follow-up assessment at 6 months (plus or
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
applicant to include null responses in the denominator of the indicator to halp increase the quality. The quality of the data should be reassessed when the indicator is reviewed.
10/09/15 minus 2 months)? - is not completed, it is interpreted as an assessment did not take place and therefore included within the indicator denominator but not the numerator.
As SSNAP started data collection in January 2013, 6 month assessments did not start being undertaken until approximately June 2013 and even then the first quarter’s data collection was during a pilot phase and therefore low. This meant that not all patients were entered onto SSNAP to begin with, so they could not have their 6 month assessment recorded. Now that SSNAP records upwards of 95% of stroke admissions, this is no longer an issue for 2014/15 onwards; the denominator spans the whole year.
ConstructionRec. no
Issue or recommendation Raised by / Date
Response or Action taken by applicant Response date
Resolved
Sign off by / Date
4a MRG requested clarification on which patients were included and excluded from the indicator, as it is important to ensure that there isn’t a mismatch between the numerator and denominator. It was suggested that a useful way to do this would be to complete a matrix of possible scenarios (e.g. assessment occurring at 4, 5, 6, 7, 8 months on one axis and patient alive at 4, 5, 6, 7, 8 months on the other axis).
MRG
6/9/12
The original concern around a possible mismatch between the denominator and numerator was resolved. The RCP use a combination of SSNAP and ONS mortality data to determine applicability for the indicator.
In the first instance, patients that died whilst on the stroke care pathway are excluded from the indicator automatically, as it is not possible for the stroke team to complete Audit Question 8.1 (detailed below) on the web tool.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
The denominator includes records where Audit Question 8.1 (‘Did this patient have a follow up assessment at 6 months (plus or minus 2 months)’) is equal to ‘Yes’ or ‘No’. The question provides a ‘No, patient died within 6 months of admission’ option.
The numerator then includes patients where Audit Question 8.1 is equal to ‘Yes’.
A patient is included in both the denominator and numerator if they had a six month follow-up assessment (+/- 2 months), regardless of if they died after receiving it. A patient is excluded from the denominator if they did not have an assessment and died within 8 months of admission; identified using ONS data (Date of Death within 8 months of Admission Date).
Patients are only included once in the indicator and are only reported on after 8 months has elapsed. The median time from admission (or onset, if in hospital) is 6.4 months, based on 12,631 assessments completed in 2014/15.
4b The description of the denominator should state ‘following admission’ rather than ‘following discharge’, so that it is consistent with both the title of the indicator and the description of the numerator.
Following MRG’s suggestions, the indicator denominator is described in the indicator specification as:
The number of stroke patients entered into SSNAP excluding;
Patients who died within 6 months of initial admission for stroke
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
appointment offered Patients for whom an attempt is
made to offer an appointment but are untraceable as they are not registered with a GP
The indicator numerator is described in the indicator specification as:
Of the denominator, the number of patients who had a follow-up assessment between 4 and 8 months after initial admission for stroke.
4c The application provided should give additional information regarding what is meant by “Patients who decline an appointment offered”, as this group are currently excluded from the indicator.
MRG
10/09/15
Patients who decline an appointment offered are patients who are offered a follow-up assessment and decline or do not attend the assessment. These are identified in the SSNAP where Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘No but’ and therefore excluded from the denominator.
Presentation and InterpretationRec. no
Issue or recommendation Raised by / Date
Response or Action taken by applicant Response date
Resolved
Sign off by / Date
5a It is recommended that a break-down of response is given as contextual information, (i.e. “Yes, “No” and “Not completed”) so users can distinguish between cases when patients have notreceived the follow up and where the field hasn’t been filled in.
MRG
10/09/15
It is not proposed to provide a breakdown of responses for this indicator. Including patient records in the denominator where the follow-up question has not been completed should encourage an improvement in data quality for this indicator.
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
Rec. no
Issue or recommendation Raised by / Date
Response or Action taken by applicant Response date
Resolved
Sign off by / Date
6a MRG recommended that work on cross-validating the audit data set with HES data should continue, as the credibility of the indicators could be impacted by conflicting sources. MRG recommended including a contextual indicator on the relationship between the audit and HES data as this would also encourage improvements in data quality.
MRG
6/9/12
The indicator is published in the context of case ascertainment between SSNAP and HES. This is the percentage of patients with primary ICD-10 codes I61, I63 and I64 in HES who are included in SSNAP for the same time period.
The SSNAP is a mandatory collection and overall case ascertainment increased from 72% in Quarter 1 to 95% in Quarter 4, 2013/14 (Quarter 2: 83%, Quarter 3: 90%). It has further improved to 97% by Quarter 4, 2014/15. Case ascertainment is reported alongside the indicator for all CCGs in the published CCG OIS data files. Five CCGs (2.4%) had their percentages suppressed in the published 2013/14 data due to less than 50% case ascertainment with HES.
Patient records are only included in audit analyses if they include the minimum requirements of completion of mandatory fields. However, the follow-up assessment at six months (plus or minus two months) field is not mandatory, as it is not in the acute part of the data set. Case ascertainment is high, as the records have been submitted, but many do not include information on follow-up assessment.
There is very wide variation in 2013/14 data for this indicator. The subsequent RCP SSNAP Clinical audit October - December 2014 public report
NICE inherited this indicator and all its supporting documentation from NHS Digital on 1 April 2020
(https://www.strokeaudit.org/Documents/Results/National/OctDec2014/OctDec2014-PublicReport.aspx) provides an indication of how widely this section of the audit is being answered, rather than indicating the numbers of patients who had a six month assessment completed. In future quarters, if this question is not answered, it will be interpreted in the RCP SSNAP output as an assessment did not take place.
Between July and December 2014, 34,476 patient records should have had an answer. Of these, 10,709 (31.1%) did have an answer. The RCP feel it is extremely important that data regarding a patient’s six month follow-up is recorded in the SSNAP. This is regardless of whether or not the assessment was provided. While the vast majority of patients alive at this time after stroke are applicable to receive a six month review, this is only recorded to have happened in less than 20% of cases. It is proposed that in future data releases, if this question is not answered; it will be interpreted as an assessment did not take place. The relevant data filter has been amended to include records which are not completed in the denominator (see section 3.10 of the application form for the amendment to the data filter).
Any complaints or appeals against the decisions made during the assurance process should be made to the Indicator & Methodology Assurance Service (IMAS) Team at HSCIC. Likewise, if you are unclear regarding any of the recommendations in this report, or have any queries about the assurance process in general, please contact the IMAS team.
Indicator and Methodology Assurance ServiceHealth and Social Care Information Centre1 Trevelyan Square, Boar Lane,LEEDS LS1 6AE.
