i
Örebro University
School of Medicine
Degree project 15 ECTS
January 2017
Evaluation of the current
diagnostic tools for concussion
in sports
Version 2
Author: Johanna Wallinder
Supervisor: Pashtun Shahim MD PhD
Department of Neurochemistry
Sahlgrenska University Hospital, Mölndal,
Sweden
2
Abstract
Background: Concussion is a common injury in sports. The need for accurate diagnosis and
follow up is critical to avoid too early return to play and minimize the number of repeated
concussions obtained which can result in post concussions symptoms and chronic traumatic
encephalopathy.
Purpose: The objective of this study was to examine the support for 3 diagnostic tools used
for evaluation of sports-related concussion.
Methods: The tools investigated were Sports Associated Concussion Assessment Tool
(SCAT3), Rivermead Post Concussion Symptoms Questionnaire (RPQ) and Immediate Post-
Concussion Assessment and Cognitive Testing (ImPACT). This was done by searching three
databases, PubMED, PsychInfo and SportDiscus. The search words were the name of the
specific test, validity and reliability.
Results: A total of 18 articles were selected, 1 for SCAT 3, 3 for RPQ and 14 for ImPACT. Of
the studies evaluated in this study half (7/14) supported the reliability and validity of ImPACT
and the other approximate half (6/14) did not support the reliability and validity of the
ImPACT. SCAT 3 only yielded one article that supported the use of SCAT 3, primarily
within the first week of getting a concussion. Of the 3 articles yielded in RPQ search, two
questioned the construct validity and the third article, which contained two studies supported
it.
Conclusion: None of these diagnostic tests have an unquestionable support within current
literature. Further research regarding the validity and reliability is needed to further develop
the use and understanding of these tools.
3
Abbreviations:
mTBI-mild traumatic brain injury
SCAT3- Sport Concussion Assesment Tool 3
RPQ- Rivermead Post-Concussion Symptoms Questionnaire
ImPACT- Immediate Post-Concussion Assessment and Cognitive Testing
SAC- Standardized Assessment of Concussion
PCS- Post-Concussion Syndrome, GPA-grade point average
ADHD-attention deficit hyperactivity disorder
SBU-statens beredning för medicinsk och social utvärdering
4
CONTENT
Background ........................................................................................................................ iv
Material and Methods ......................................................................................................... iv
Diagnostic tests investigated .............................................................................................. iv
ImPact ................................................................................................................................ iv
SCAT 3 .............................................................................................................................. iv
RPQ .................................................................................................................................... v
Method................................................................................................................................. v
Inculsion and Exclusion Criteria ........................................................................................ v
Data extraction .................................................................................................................. vi
Results ................................................................................................................................ vi
Discussion ........................................................................................................................... x
Conclusion ......................................................................................................................... xii
References ........................................................................................................................ xiii
BACKGROUND
Concussion is a common injury in contact sports such as boxing, American football, rugby
and ice hockey. In the USA, the Centers for Disease Control estimate that 1,6-3,8 million
sports-related concussions occur annually in the United States alone1,2. This figure is,
iii
however, probably underestimated since not all who obtain a mild concussion seek medical
care3.
Concussion is a type of mild traumatic brain injury (mTBI) and is defined as a complex
pathophysiological disturbance of brain function induced by mechanical force or trauma2
4. When the brain is subjected to a rapid change of motion, as when receiving a tackle or hit
to the head, it starts a cascade in the brain cells which includes a change in ionic
concentrations in the cell, glutamate release, energy crisis, axonal damage, cytoskeletal
dysfunction and altered transmission. Inflammation and cell death has also been implicated as
part of the pathophysiology of concussion5.
The symptoms of a concussion include fatigue, headaches, visual disturbances, loss of
memory, trouble concentrating/paying attention and emotional disturbances as irritability and
depression.
At present, there are no valid blood biomarkers or imaging techniques for detection and
diagnosis of concussion. The current clinical imaging biomarkers, such as computed
tomography or magnetic resonance imaging may be performed to rule out intracranial
hemorrhage but contribute little to the diagnostic process. The concussion diagnosis is largely
based on clinical symptoms. Behavioral and cognitive tests may be used to aid the diagnosis,
but with limited specificity6.
Recent data suggest that athletes who have repeated concussions, especially unresolved
concussions with early return to play may be at risk of developing both short-term and long-
term neurological symptoms 2. Short-term risks include delayed reaction time which may
increase risk of further injury and second impact syndrome which includes swelling of the
brain and can therefore be lethal. It is, however, very uncommon7. Long-term risks are
those associated with cognitive impairment and long term neurological consequences,
however, more studies concerning this matter is needed.8
Considering the risks associated with concussions there is an urgent need for objective and
reliable diagnostic and prognostic tools. At present, there are many qualitative and semi-
qualitative tools used to aid in concussion diagnosis including Sports Associated Concussion
Assessment Tool (SCAT3), Rivermead Post Concussion Symptoms Questionnaire (RPQ) and
Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT).
iv
The purpose of this paper was to investigate the evidence of reliability and validity of these
three tests.
MATERIAL AND METHODS
DIAGNOSTIC TESTS INVESTIGATED
ImPact
ImPACT is a computerized concussion management tool and consists of 4 parts, verbal
memory, visual memory, visual motor speed and reaction time. There is also a questionnaire
covering self-reported symptoms. This tool is primarily utilized for baseline testing in the
beginning of the season and post-injury testing. ImPACT was developed from the Pittsburg
Steelers model which was a paper and pencil based test used to monitor Football players in
the National Football League 9. The Pittsburg Steelers model was used until 1998. ImPACT
baselines are usually administered during pre-season and follow up test 1-3 days after the
injury 10.
SCAT 3
SCAT is a standardized test for evaluating athletes with concussion. It was first developed in
2005 and has since been edited and updated to SCAT2 and now SCAT3. The idea for
developing a standardized assessment tool was developed during the consensus meetings held
2001 in Vienna for medical professionals organized by International Ice Hockey Federation,
Fédération Internationale de Football Association and the medical comission of the
International Olympic Committee. The latest version, SCAT3, was one of the outcomes from
the consensus meeting in Zurich 20124.
SCAT3 is mainly used as a sideline assessment tool. Baseline values may or may not be
obtained during preseason. It contains 8 major parts, Glasgow coma scale (testing alertness),
Maddocks score (testing short term memory), self-reported symptoms, Cognitive assessment
(using Standardized Assessment of Concussion, SAC.), neck examination, balance testing
v
(using modified Balance Error Scoring System), coordination and delayed recall. The SCAT 3
form is found in appendix no. 1.
RPQ
Rivermead is a symptom questionnaire where symptoms are self-assessed on a scale of 0-4.
The ratings are subjective comparisons of how the patient was feeling prior to the accident or
within the past 24 hours. The symptoms covered by the questionnaire are headache, dizziness,
nausea/vomiting, noise sensitivity, sleeping disturbance, fatigue, irritability, depression,
frustration/patients, memory, concentration, thought processing, vision and restlessness. It
was developed by N.S. King’s research group in 1995 and is designed to assess the presence
of post-concussion syndrome11. Post-concussion syndrome, PCS, is a when symptoms such
as headache, dizziness appear as a consequence of a mTBI. The RPQ questionnaire is found
in appendix no. 2.
METHOD
Databases PubMed, PsychInfo, SportDiscus were searched using the name of the test (i.e.
SCAT 3, Rivermead or ImPACT) together with realibility and validity.
Since this is a literature study there is no need for ethical considerations or statistical analysis.
Inculsion and Exclusion Criteria
1. Published Articles after 1990
2. Reported outcomes include an analysis of reliability and/or validity of SCAT 3, RPQ
or ImPACT
3. Individuals older than 13years of age
4. Data collected either at baseline or postconcussion
5. English language
6. SCAT, RPQ or ImPACT was used to evaluate sports related concussions
7. Articles must be original studies therefore review articles were rejected
The inclusion criteria no. 6 was disregarded for RPQ since the search yielded no results
related to concussions in sports.
vi
Data extraction
The articles were evaluated with help of the first section of “Mall för kvalitetsgranskning av
observationsstudier” (the GRADE scale was not used) from SBU’s website. The form is
found in appendix no. 3.
RESULTS
A total of 18 articles were selected, 1 for SCAT 3, 3 for RPQ and 14 for ImPACT. A table of
all included articles is found in appendix no. 4
Figure 1: Search for SCAT 3
Figure 2: Search for RPQ
vii
Figure 3: Search for ImPACT
viii
ImPACT
The search yielded 14 articles investigating the validity and reliability of the ImPACT. 10
were within subject cohort studies testing baseline re-testing reliability, three were cross
sectional studies which tested the subject only once and one was a cohort study investigating
differences between concussed and non-concussed athletes.
Of the studies evaluated in this study half (7/14) supported the reliability and validity of
ImPACT 12-17,24 and the other approximate half (6/14) did not support the reliability and
validity of the ImPACT 18-22,25. The reliability, that is that the test will score an athlete
with similar results when tested several times, is tested primarily by baseline testing and
retesting. The validity, meaning that the test measures what it is intended to measure, is harder
to conduct studies since there is no gold standard to compare with. There are studies
comparing ImPACT to other previously validated tests24,25, but as previously mentioned
none of the tests are considered gold standard for diagnosing concussion. The study
examining the rates and predictors for invalid baseline didn’t say whether it supported the test
or not but examined what factors can influence baseline results other than the construct of the
test19.
Baseline testing
There is a debate about whether comparison to baseline testing is the best way to evaluate the
scores of ImPACT or if it is better to compare the scores obtained post-injury to normative
data.
Many of the studies have investigated the test-retest reliability12-22, most of them only with
healthy participants. Study participants have both been athletes and non-athletes12,13, of
varying ages, from high school (13-18) and college students’ (18-24) to professional
players14-25. The results of these studies have, however, been inconclusive. Some studies
have shown high test-retest reliability12-17,24 while others have shown poor reliability18-
22,25. The author Schatz who has conducted several studies12,16 showing reliable test-
retesting within-subjects have criticized, primarily one study showing unreliable test-retesting
conducted by Broglio21, primarily because of the study method in investigating several
neurocognitive tests during the same session. That is, however, not the only study showing
varying results when it comes to testing the reliability of baseline testing 18-20, 22,25.
ix
For baseline-testing to be useful there is need for a valid test-result from the athlete taking the
baseline. ImPACT has as part of the data analysis a function for flagging tests as invalid
showing “++” on the scorecard. That is, if an athlete would for example score 30 or more on
Impulse Control composite their effort would be questioned25. Several predictors for
obtaining an invalid baseline have been identified by Nelson et. al. Several predictors were
studied such as lower GPA (grade point average), diagnosis such as ADHD (attention
deficiency hyperactivity disorder) or learning disability and age. The only predictor for
invalid baseline testing with ImPACT was lower age and that was only a modest association
which became non-significant after adjustment19. There have also been other questions
raised such as how primary language22, prior night’s sleep and so forth can affect the results
of not only baseline but also re-testing26, most of these studies did not, however, turn up in
the searches conducted in this study. Jared Bruce et. al22 raises the question of primary
language in his study of ImPACT’s test-retest reliability in professional hockey players. The
results indicate that the reliability may vary depending on language administration.
Follow up
The use of ImPACT as a tool to follow the concussed athlete’s recovery and as part of the
return to play decision is also not entirely supported in the literature. One study conducted by
L. D. Nelson et. al.23 where concussed athletes completed the ImPACT within 24 hours of
injury and 8, 15 and 45 days’ post-injury and were compared with non-concussed, matched
controls showed only significant differences within the first 24 hours’ and day 8 post-injury.
One study conducted by G. Iverson et. al. tested athletes within 21 days of sustaining a
concussion and found that the ImPACT composite socres testing Processing Speed and
Reaction Time did correlate with a traditional and previously validated test, the Symbol Digit
Modalities Test 24.
SCAT 3
The search for articles investigating the reliability and validity yielded only one article,
“Reliability and Validity of the Sport Concussion Assessment Tool -3 (SCAT3) in High
School and Collegiate Athletes by Esther Y. Chin et. al. In the study, concussed athletes were
x
matched with non-concussed controls and were evaluated 24 hours, 8, 15, and 45 days post-
injury (healthy controls being evaluated at similar times. The results indicated that significant
differences between concussed and healthy controls could be observed primarily 24 hours and
8 days post-injury27.
RPQ
The search for articles investigating the reliability and validity yielded three articles
investigating the construct validity and reliability of the RPQ.
The construct validity of the RPQ, that is if the RPQ is constructed so that it measures what it
is intended to, has been questioned by both S. Eyres et. al. and M. Lannsjö et. al. Both studies
suggest that instead of using one composite score the questions should be divided in to two or
more groups and these groups of questions be scored independent of each other since they
don’t measure the same thing28-29.
The reliability has been tested, as far as the results of this study, only by the developer, N.
King. There are two studies presented in one article, one where the RPQ was administered
twice, once by a professional and once self-administered and one study where the RPQ was
administered twice within one week as a 6 months’ post-injury follow up. Both studies were
considered proving the reliability of the RPQ11 it is, however, questioned by S. Eyres et. al
29.
DISCUSSION
ImPACT
ImPACT is one of the most widely used concussion diagnostic/follow-up tools used in the
USA. It is clear, however, that there is no consensus on the reliability and validity of this test.
Studies show varied results both within and between studies.
The advantages of ImPACT are, it is easy to administer on the computer, it comes in different
languages, and with the online versions scores can be accessed at any computer with access to
xi
internet (older desktop versions are, however, bound to the computer). The different
translations of ImPACT are, however, not studied in depth in terms of reliability. Of note is
also that most of the studies conducted have had any other primary language than English as
an exclusion criteria12-16,20,21,25. Further research of the translations of the ImPACT is
therefore warranted.
Another issue with baseline testing is the risk of some athletes intentionally trying to
underperform on their baseline, so called “sandbagging”19 to be allowed to return to play
sooner since their scores are seemingly not affected by the concussion. This search, however,
did not include those articles and are there -fore not further discussed.
It is a factor to keep in mind that this test is developed by a company and is therefore a
product made to be marketed and sold10. That can be both an advantage, they need a good
product to be able to sell it and remain a good reputation. It can also be a disadvantage in that
there is an interest in making the test look good and perhaps overstating its utility in order to
market it and sell more.
SCAT 3
Since the search only generated one article it is hard to make any conclusive statements.
SCAT 3 has many strengths such as being easy to administer on the side line by filling out the
form with paper and pencil. It doesn’t require a computer or tablet and it assesses more than
neurocognitive functions, for example consciousness and balance which are also affected by
concussions. It is also developed at a consensus meeting by medical professionals and is
available free on the internet, therefore there is no need to sell the test. It does, however, also
have weaknesses in not being evaluated for how all these different tests are to be interpreted
together. As far as this study could illustrate only one study has been made on the entire
SCAT 3, not only the individual parts. There is also no visual testing which is a major part of
other concussion diagnostic tests27.
Therefore, further research is needed of the whole SCAT 3.
xii
RPQ
The RPQ is not primarily used to diagnose concussion but is used to evaluate the presence of
post concussive symptoms. The positive aspects is that it is easy to administer and seems to
have a good reliability, whether self-administered or by a professional11. The reliability
and validity is, however questioned by Eyres et. al. 29 due to the low fit to the statistical
Rasch model used. It could also be questioned though because the studies with the greatest
support for the reliability were performed by the developer of the test and therefore it could
constitute a conflict of interest.
RPQ’s validity is however questionable since many post-concussive symptoms are vague and
includes common symptoms such as headache, nausea and fatigue. The studies support that
all these symptoms can’t be lumped together since the symptoms fit together in a more
complex way28,29.
Limitations
One limitation of this study is that only one person reviewed the articles. Further evaluating
the studies with traditional forms for evaluating diagnostic studies was not optimal since there
is no test that is considered gold standard for concussions. This makes it difficult to design
studies and to evaluate when you have nothing to compare the new diagnostic method to. The
“Mall för kvalitetsgranskning av observationsstudier” wasn’t optimal either since many of the
questions were not applicable or unclear, which made it hard to score the studies. Another
more specific weakness of this study is that this study only searched studies assessing the
whole SCAT 3 test. There are more studies1§ testing the individual parts of SCAT 3 which
could be useful, however that doesn’t mean that the way all these tests are interpreted together
as a whole needs to be further investigated. Also, SCAT 3 has been developed from the SCAT
2 which is similar to the SCAT 3
CONCLUSION
None of the tests discussed in this paper could be considered optimal for diagnosis and
prognosis of concussion in athletes. One test should not be the single base of clinical
decisions made. Consideration of the athletes’ background and circumstances during both
baseline testing and follow up should be part of the return to play decision. Concussions are
complex and the ramifications of too early return to play have not yet been fully understood.
xiii
Also, the determination of what is considered too early return to play is not fully determined.
Is it when symptoms have resolved or when the neurocognitive function is back to baseline
values? Further research in this area is needed to fully understand this problem. The
development and research has also focused a lot on development of imaging methods and
blood biomarkers which could also aid in diagnosis. Perhaps the best way for diagnosing is in
a future where imaging techniques and neurocognitive testing can be combined to avoid the
possible devastating ramifications of too many concussions or too early return to play.
ACKNOWLEDGEMENTS
Special thanks to my supervisor Pashtun Shahim for taking the time to help me write this
paper, also to Yelverton Tegner who helped me get in touch with Pashtun.
REFERENCES
1. Langlois JA, Rutland-Brown W, Wald MM. The epidemiology and impact of
traumatic brain injury: a brief overview. J Head Trauma Rehabil. 2006 Sep-
Oct;21(5):375–378.
2. Harmon KG, Drezner JA, Gammons M, Guskiewicz KM, Halstead M, Herring SA, et
al. American Medical Society for Sports Medicine position statement: concussion in
sport. Br J Sports Med. 2013 Jan;47(1):15-26
3. U.S DEPARTMENT OF HELATH AND HUMAN SERVICES, Centers for Disease
Control and Pervention, National Center for Injury Prevention and Control. Traumatic
Brain Injury in the United States Emergency Department Visits, Hospitalizations and
Deaths 2002-2006. Atlanta (GA): Centers for Disease Control and Prevention,
National Center for Injury Prevention and Control; 2010.
4. McCrory P, Meeuwisse WH, Aubry M, Cantu RC, Dvořák J, Echemendia RJ et al.
Consensus Statment on Concussion in Sport: the 4th Internation Concerence on
Concussion in Sport held in Zurich, November 2012. J Athl Train. 2013 Jul-
Aug;48(4):554-575
5. Giza CC, Hovda D. The New Neurometabolic Cascade of Concussion. Neurosurgery.
2014 Oct;75 (04):24-33
6. Svensson M, Bellander B-M, Undén J, Johnsson H. Traumatiska hjärnskador (TBI)
hos vuxna Internet… Utgivningsort: Internetmedicin 2016 Updated 2016-09-19;
cited 2016-11-30 Available from: http://www.internetmedicin.se/page.aspx?id=1178
xiv
7. Bey T, Ostick B. Second impact syndrome. West J Emerg Med. 2009;10(1):6–10)
8. Gaetz M, Weinberg H. Electrophyiological indices of persistent post-concussion
symptoms. Brain Inj. 2000 Sep; 14(9):815-832
9. Vanderbildt Medical School. The ImPACT Program: History and Development
Internet. Nashville TN 2013 Cited 2016-12-02 Available from:
https://medschool.vanderbilt.edu/vscc-research/files/vscc-
research/public_files/Lovell_Nashville_November2013_.pdf
10. ImPACT Applications, Inc. ImPACT Recourses for Parents and Athletes Internet.
Pittsburg PA 2016 Cited 2016-12-02. Availabel from:
https://www.impacttest.com/audience/?parents-3
11. King NS, Crawford s, Wenden FJ, Moss NE, Wade DT. The Rivermead Post
Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced
after head injury and its reliability. JNeurol. 1995 Sep; 242(9):587-592.
12. Schatz P, Ferris CS. One-Month Test—Retest Reliability of the ImPACT Test Battery.
Arch Clin Neruopsychol. 2013 Aug; 28(5):499-504.
13. Nakayama Y, Covassin T, Schatz P, Nogle S, Kovan J. Examination of the Test—
Retest Reliability of a Computerized Neurocognitive Test Battery. Am J Sports Med.
2015 Aug; 43(8):2018-26.
14. Tsushima WT, Siu AM, Pearce AM, Zhang G. Oshiro RS. Two-year Test—Retest
Reliability of ImPACT in High School Athletes. Arch Clin Neruospychol. 2016
Feb:31(1):105-111.
15. Brett BL, Solomon GS. The Influenve of validity criteria on Immediate Post-
Concussion Assessment and Covgnitive Testing (ImPACT) test—retest reliability
among high school athletes. J Clin Exp Neuropsychol. 2016 Sep 2:1-10
16. Elbin R.J, Schatz P, Covassin T. One-Year Test-Retest Reliability of the Online
Version of ImPACT in High School Athletes. Am J Sports Med. 2011 Nov; 39
(11):2319-2324.
17. Schatz P. Long-Term Test-Retest Reliability of Baseline Cognitive Assessments
Using ImPACT. Am J Sports Med. 2010 Jan; 38(1):47-53.
18. Resch J, Driscoll A, McCaffrey, Brown C, Ferrara MS, Macciocchi S et. al. ImPACT
Test-Retest Reliability: Reliably Unreliable? J Athl Train. 2013 Jul-Aug;48(4):506-
511
xv
19. Nelson LD, LaRoche AA, Pfaller AY, Lerner EB, Hammeke TA, Randolph C et. al.
Prospective, Head-toHead Study of Three Computerized Neurocognitive Assesment
Tools (CNTs): Reliability and Validity for the Assessment of Sport-Related
Concussion. J Int Neuropsychol Soc. 2016 Jan;22(1):24-37
20. Register.Mihalik JK, KontosDK, Guskiewicz KM, Mihalik JP, Conder R, Shields EW.
Age-Related Differences and Reliability on Computerized and Paper-and-Pencil
Neurocognitive Assessment Batteries. J Athl Train. 2012 May-Jun;47(3):297.305
21. Broglio SP, Ferrara MS, Macciocchi SN, Baumgartner TA, Elliot R. Test-Retest
Reliability of Computerized Concussion Assessment Programs. J Athl Train 2007
Oct-Dec;42(4)509-514
22. Bruce J, Echemendia R, Meeuwisse W, Compier P, Sisco A. 1 year test-retest
reliability of ImPACT in professional ice hockey players. Clin Neuropsychol. 2013
Dec;28(1):14-25
23. Nelson LD, Pfaller AY, Rein LE, McCrea MA. Rates and Predictors of Invalid
Baseline Test Performance in High School and Collegiate Athletes for 3
Computerized Neurocognitive Tests:ANAM, Axon Sports and ImPACT. Am J Sports
Med. 2015 Aug;43(8);2018-2026
24. Iverson GK, Lovell MR, Collins MW. Validity of ImPACT for Measuring Processing
Speed Following Sports-Related Concussion. J Clin Exp Neuropsycol. 2005
Aug;27(6):683-698.
25. Allen BJ, Gfeller JD. The Immediate Post-Concussion Assessment and Cognitive
Testing battery and traditional neuropsychological measures: A construct and
concurrent validity study. Brain Inj. 2011;25(2):179-191.
26. McClure DJ, Zuckerman SL, Kutscher SJ, Gregory AJ, Solomon GS. Baseline
neurocognitive testing in sports-related concussion: the importance of a prior night’s
sleep. Am J Sports Med. 2014 Feb; 42(2):472-478.
27. Chin EY, Nelson LD, Barr WB, McCrory P, McCrea MA. Reliability and Validity of
the Sport Concussion Assessment Tool—3 (SCAT3) in High School and Collegiate
Athletes. Am J Sports Med. 2016 Sep;44(9):2276-2285.
28. Lannsjö M, Borg J, Björklund G, Af Geijerstam JL, Lundgren.Nilsson A. J Rehabil
Med. 2011 Nov;43(11):997-1002.
29. Eyres S, Carey A, Gilworth G, Neumann V, Tennant A. Construct validity and
reliability of the Rivermead Post-Concussion Sumptoms Questionnaire. Clin Rehabil.
2005 Dec;19(8)878-887
xvi
Appendix no. 1.
xvii
xviii
xix
xx
Appendix no. 2.
xxi
Appendix no. 3.
xxii
xxiii
ii
Appendix no 4.
Publication
(Author, year,
referencenumber)
Type of
Study
Patient
characteristics
Exposure or
intervention
Compared to Measure of effect Results Quality
assessment
Schatz P, 2010,
17
Case series College
Athletes
Baseline
testing with
ImPACT
approximately
2 years apart
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC)
Baseline values
remain fairly stable
over a 2-year period
Risk for
systemic bias:
Moderately high
Nelson LD,
2015, 23
Prospective
head-to-
head
High School
and collegiate
athletes
Concussed
athletes
completing
ImPACT
Non-concussed,
matched controls
completing
ImPACT
Reliable change
indices (RCI)
Computerized
neurocognitive
testing may
contribute to
identification within
24h of injury but
little after symptom
reliefe
Risk for
systemic bias:
Moderately high
iii
Nelson LD,
2016, 19
Cross-
sectional
High School
and collegiate
athletes
Baseline
testing with
ImPACT
Number of
baselines classified
as invlaid
Validity criteria
may not be
sensitive to detect
the cause of invalid
baseline scores.
Risk for
systemic bias:
Moderately high
Iverson GL,
2005, 24
Cross-
sectional
Amature
athletes
ImPACT
testing post
Injury
Symbol Digit
Modalities Test
(SDMT)
Correlation between
the two test-scores
Speed and reaction
time composite
scores of ImPACT
seem to measure
similar construct as
SDMT
Risk for
systemic bias:
Moderately high
Broglio SP,
2007, 21
Repeated-
measure
University
students
Baseline Test
and Re-testing
of ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC)
Tests showed low
to moderate test-
retest reliability
Risk for
systemic bias:
Low
Bruce J, 2013, 22 Repeated-
measure
Professional
ice hockey
players
Baseline Test
and Re-testing
of ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC)
Mixed support for
visual motor and
reaction time
composite scores,
verbal and visual
composite scores
may not be
sensitive to clinical
change
Risk for
systemic bias:
Low
iv
Register-Mihalik,
2012, 20
Cohort
study
High school
and collegiate
athletes
Test and Re-
testing of
ImPACT
Differences in
composite scores
between age-groups
and between test
sessions
Performance may
vary between
sessions
Risk for
systemic bias:
Low
Resch J, 2013, 18 Cross-
sectional
Men and
women ages
18-24
Test and Re-
testing of
ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC)
Variable test-retest
reliability.
Risk for
systemic bias:
Moderately high
Brett BL, 2016,
15
Repeated
measure
High school
athletes
Test and re-
testing of
ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC)
Application of strict
validity criteria
does not affect rest-
retest reliability
Risk for
systemic bias:
Moderately High
Tsushima WT,
2016, 14
Repeated
measure
High school
athletes
Test and re-
testing of
ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC), Reliable
change index(RCI)
Study indicated that
baseline values
were stable within a
2 year interval
Risk for
systemic bias:
Moderately High
v
and regression
based method(BM)
Nakayama Y,
2014, 13
Repeated
measure
College
students
Test and re-
testing of
ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC),
Study indicated that
ImPACT has good
reliability
Risk for
systemic bias:
Moderately High
Allen BJ, 2011 Cross-
sectional
College
students
ImPACT
testing
NFL’s traditional
neurospychological
tests
Factor analysis
examined
component
structure.
Correlations
analyses examines
relationships
between variables
within and in-
between tests.
Study indicated that
there is both
overlapping in their
constructs as well
as notable
differences
Risk for
systemic bias:
Low
vi
Schatz P,
2013,12
Repeated
measure
College
students
Test and re-
testing of
ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC) and Pearson’s
correlation
coefficients
Repeated testing
may lead to
significant
improvement on
composites such as
visual motor speed
but does not lead to
practice effect on
memory or reaction
time.
Risk for
systemic bias:
Moderately High
Elbin R.J, 2011,
16
Repeated
measure
High school
athletes
Test and re-
testing of
ImPACT
Differences
between individuals
2 baseline scores
measured with Intra
Class Correlation
(ICC)
Reliability of the
online version of
ImPACT is stable
and better than the 2
year reported from
desktop version-
Risk for
systemic bias:
Low
Eyres, 2004, 29 Cross-
sectional
Adults
visiting the
emergency
room after
head trauma
Completing
RPQ post
injury
RPQ score analysed
with Rasch model.
RPQ is not up to
modern
psychometric
standards.
Risk for
systemic bias:
Moderately high
vii
Lannsjö, 2011,28 Cross
sectional
Patients with
mild
traumatic
brain injury
(mTBI)
visiting the
emergency
room, ages 6
years and
older
Completing
RPQ post-
injury
RPQ score analysed
with Rasch model
RPQ is not optimal
for patients with
mTBI. It is,
however,
unsensitive to age
and gender.
Risk of systemic
bias: Low
King NS, 1995,
11
Repeated
measure
Head injury
patients
Completing
RPQ twice
post-injury
Comparing the two
RPQ scores with
Spearman rank
correlation
RPQ shows good
reliability
Risk for
systemic bias:
Moderately high
Chin EY, 2016 Cohort
diagnostic
study
High school
and collegiat
athletes
Concussed
athletes
completing
SCAT 3 at
baseline and 4
times post
injury
Non-injured
matched controls at
same time as
injured athletes.
Comparing injured
with non-injured
matched controls
with RCI
When interpreting
the SCAT3 results,
individual
differences should
be considered
Risk for
systemic bias:
Moderately high