Clinical Evaluation Tools (2024)

Glasgow Coma Scale

Originally developed by Teasdale and Jennett (1974), the Glasgow Coma Scale (GCS) (see Table C-1) is a scoring scale for eye opening, motor, and verbal responses that can be administered to athletes on the field to objectively measure their level of consciousness. A score is assigned to each response type for a combined total score of 3 to 15 (with 15 being normal). An initial score of less than 5 is associated with an 80 percent chance of a lasting vegetative state or death. An initial score of greater than 11 is associated with a 90 percent chance of complete recovery (Teasdale and Jennett, 1974). Because most concussed individuals score 14 or 15 on the 15-point scale, its primary use in evaluating individuals for sports-related concussions is to rule out more severe brain injury and to help determine which athletes need immediate medical attention (Dziemianowicz et al., 2012).

Standardized Assessment of Concussion

The Standardized Assessment of Concussion (SAC) (see Figure C-1) provides immediate sideline mental status assessment of athletes who may have incurred a concussion (Barr and McCrea, 2001; McCrea et al., 1998, 2000). The test contains questions designed to assess athletes' orientation, immediate memory, concentration, and delayed memory. It also includes an exertion test and brief neurological evaluation. The SAC takes approximately 5 minutes to administer and does not require a neuropsychologist to evaluate test scores. The test is scored out of 30 with a mean score of 26.6 (McCrea et al., 1996).

Studies have found the SAC to have good sensitivity and specificity (McCrea, 2001; McCrea et al., 2003), making it a useful tool for identifying the presence of concussion (Giza et al., 2013). Significant differences in scores have been reported for males and females in healthy young athletes (9 to 14 years of age), suggesting the need for separate norms for males and females in this age group (Valovich McLeod et al., 2006) as well as in high school athletes (Barr, 2003).

Sport Concussion Assessment Tool 3

The Sport Concussion Assessment Tool 3 (SCAT3) is a concussion evaluation tool designed for individuals 13 years and older. Due to its demonstrated utility, the SAC has been incorporated into this tool, which also includes the GCS, modified Maddocks questions (Maddocks et al., 1995), a neck evaluation and balance assessment, and a yes/no symptom checklist as well as information on the mechanism of injury and background information, including learning disabilities, attention deficit hyperactivity disorder, and history of concussion, headaches, migraines, depression, and anxiety (McCrory et al., 2013c). The precursor SCAT2 had been standardized as an easy-to-use tool with adequate psychometric properties for identifying concussions within the first 7 days (Barr and McCrea, 2001). The SCAT3 was developed from the original SCAT to help in making return-to-play decisions (McCrory et al., 2009, 2013b). This concussion evaluation tool can be used on the sideline or in the health care provider's office. The SCAT3 takes approximately 15 to 20 minutes to complete.

Because the SCAT3 was recently published (McCrory et al., 2013a), normative data and concussion cutoff scores are not yet available. However, a recent study to determine baseline values of the SCAT2 in normal male and female high school athletes found a high error rate on the concentration portion of the assessment in non-concussed athletes, suggesting the need for baseline testing in order to understand post-injury results (Jinguji et al., 2012). The study also showed significant sex differences, with females scoring higher on the balance, immediate memory, and concentration components of the assessment.

Findings similar to those of Jinguji and colleagues (2012) were reported in a study of youth ice hockey players who demonstrated an average total score of 86.9 out of 100 points (Blake et al., 2012). In the largest assessment of the SAC/SCAT2, Valovich McLeod and colleagues (2012) assessed 1,134 high school students. Male high school athletes and male and female ninth graders were found to have significantly lower SAC and total SCAT2 scores than did female athletes and upperclassmen, respectively (Valovich McLeod et al., 2012). A self-reported history of previous concussion¹ did not have a significant effect on SAC scores, but it did affect the symptom and total SCAT2 scores. The authors recommended baseline assessments in order to understand post-injury results for individual athletes. Schneider and colleagues (2010) tested more than 4,000 youth hockey players with the original SCAT and reported baseline scores showing absolute differences with age and sex. However, because no parametric statistics were provided, the significance of the observed differences is not known.

The Child SCAT3 is a newly developed concussion evaluation tool designed for children ages 5 to 12 years (McCrory et al., 2013a). It is similar to the SCAT3 except that tests such as the SAC and Maddocks questions are age appropriate for younger children. The Child SCAT3 includes versions of the SAC and Maddocks questions, the GCS, a medical history completed by the parent, child and parent concussion symptom scales, neck evaluation, and balance assessment. As is the case with the SCAT3 for adults, the Child SCAT3 has yet to be validated, so no normative data are available, nor are there concussion cutoff scores.

Military Acute Concussion Evaluation

The military currently employs the Military Acute Concussion Evaluation (MACE) tool for concussion screening and initial evaluation (DVBIC, 2012). The first section of the MACE collects data regarding the nature of the concussive event and the signs and symptoms of concussion. The second “examination” portion of the MACE is a version of the SAC. The MACE was first employed in Iraq for determining concussions in theater (French et al., 2008). Coldren and colleagues (2010) examined concussed and control U.S. Army soldiers who were administered the MACE 12 hours after their injury. The researchers concluded that the MACE lacks the sensitivity and specificity necessary to determine a concussive event 12 hours post injury. However, a recent study by Kennedy and colleagues (2012) indicated that the MACE can be effective in serial concussion evaluation if originally administered within 6 hours of the injury.

King-Devick Test

The King-Devick test is designed to assess saccadic eye movements, measuring the speed of rapid number naming as well as errors made by the athlete, with the goal of detecting impairments of eye movement, attention, and language as well as impairments in other areas that would be indicative of suboptimal brain function (Galetta et al., 2011a). The King-Devick test includes a demonstration and three test cards with rows of single-digit numbers that are read aloud from left to right (see Figure C-2). The participant is asked to read the numbers as quickly as possible without making any errors. The administrator records the total time to complete the three cards and the total number of errors made during the test. The results are compared to a personal baseline. The King-Devick test usually takes approximately 2 minutes to complete (King-Devick, 2013).

Studies of the King-Devick test involve 10 or fewer concussed athletes (Galetta et al., 2011a,b, 2013; King et al., 2012, 2013), which is too small a sample size to determine the test's effectiveness in evaluating a concussion. Currently, there is not enough evidence to determine whether the test is effective in diagnosing or monitoring a concussion (Giza et al., 2013).

Clinical Reaction Time Test

Eckner and colleagues (2010) have developed a simple tool for measuring clinical reaction time (RT_clin). The test involves a systematic approach to dropping a weighted stick that is calibrated to reflect speed of reaction for catching it. The athlete holds his or her hand around, but not touching, a rubber piece at the bottom of the stick, then the test administrator drops the stick, and the athlete catches it on the way down. Several studies document the initial development of this tool and demonstrate its concurrent and predictive validity (Eckner et al., 2010, 2011a,b).

A 2010 pilot study established convergent validity of RT_clin with the CogSport simple reaction time measure (Eckner et al., 2010). The Pearson correlation for 68 of the 94 athletes who completed both RT_clin and Cog-Sport was .445 (p < .001). The other 26 athletes did not meet the validity criteria on CogSport. For those individuals, the correlations between the two tests were nonsignificant. A 2011 study looked at the test-retest reliability of RT_clin at a 1-year interval as well as the same reliability statistic for CogSport reaction time (Eckner et al., 2011a). The researchers used a two-way random effects analysis of variance model for intraclass correlation coefficient analysis for each test. This means that each subject was assessed by each rater, and the raters were randomly selected. The RT_clin intraclass correlation coefficient was above .60. One might argue that a two-way mixed effects analysis should be used, which would actually increase the coefficient. There was also significant improvement in the absolute reaction times from time one to time two for the RT_clin but not for CogSport (here the CogSport analysis only included valid responders).

Eckner and colleagues (2013) have also assessed the diagnostic utility of the RT_clin. They compared 28 concussed athletes to 28 controls. Concussed athletes were tested within 48 hours of injury and a control was selected at the same time interval. Post-injury tests were compared to baseline scores and reliable change indices were calculated using the control group means and standard error of difference from the two time-points. Using a 60 percent confidence interval (one-tailed significance), the authors calculated sensitivity at 79 percent and specificity at 61 percent for a score difference of –3. Thirty-three of the 56 athletes obtained this score; of the 33, 22 were concussed and 11 were not, and therefore were misidentified. The sensitivity and specificity were improved somewhat by adjusting the cutoff score to a difference of zero seconds (75 percent sensitivity; 68 percent specificity). This correlated with a 68 percent (one-sided) confidence interval. Of note, more stringent cutoff values lowered sensitivity but increased specificity, meaning that improvements in scores beyond 11 points increased the probability that the athlete did not have a concussion, which can be useful clinical information.

The RT_clin is a simple-to-use and low-cost test of reaction time. The initial test of reliability at 1-year intervals is promising, and the diagnostic statistics indicate adequate utility. Further independent validation is needed, and it would be valuable to determine the increase in diagnostic efficiency if the test were combined with other tools because multimodal diagnostic test batteries have been recommended in the literature.

Balance Error Scoring System

The Balance Error Scoring System (BESS) is a quantifiable version of a modified Romberg test for balance (Guskiewicz, 2001; Riemann et al., 1999). It measures postural stability or balance and consists of six stances, three on a firm surface and the same three stances on an unstable (medium density foam) surface (Guskiewicz, 2001; Riemann et al., 1999). All stances are done with the athlete's eyes closed and with his or her hands on the iliac crests for 20 seconds. The three stances are: feet shoulder width apart, a tandem stance (one foot in front of the other), and a single-leg stance on the person's nondominant leg (Guskiewicz, 2001; Riemann et al., 1999). For every error made—lifting hands off the iliac crests, opening the eyes, stepping, stumbling, or falling, moving the hip into more than a 30 degree of flexion or abduction, lifting the forefoot or heel, or remaining out of the testing position for more than 5 seconds—1 point is assessed. The higher the score, the worse the athlete has performed.

The BESS test has very good test-retest reliability (0.87 to 0.97 intraclass correlations) (Riemann et al., 1999). The test's sensitivity for diagnosis was 0.34 to 0.64, which is considered low to moderate, while specificity is high (0.91) (Giza et al., 2013). Using the BESS test in conjunction with the SAC and a graded symptom checklist increases the sensitivity (Giza et al., 2013). The BESS has only been found to be useful within the first 2 days following injury (Giza et al., 2013; McCrea et al., 2003).

Sensory Organization Test

The Sensory Organization Test (SOT) uses six sensory conditions to objectively identify abnormalities in the patient's use of somatosensory, visual, and vestibular systems to maintain postural control. The test conditions systematically eliminate useful visual and proprioceptive information in order to assess the patient's vestibular balance control and adaptive responses of the central nervous system (NeuroCom, 2013). Broglio and colleagues (2008) examined the sensitivity and specificity of the SOT using the reliable change technique. A baseline and one follow-up assessment were performed on healthy and concussed young adults. Post-injury assessment in the concussed group occurred within 24 hours of diagnosis. An evaluation for change on one or more SOT variables resulted in the highest combined sensitivity (57 percent) and specificity (80 percent) at the 75 percent confidence interval. The low sensitivity of the SOT suggests the need to use additional evaluation tools to improve identification of individuals with concussion.

Acute Concussion Evaluation

The Acute Concussion Evaluation (ACE) tool is a physician/clinician form used to evaluate individuals for a concussion (see Figure C-3; Gioia and Collins, 2006; Gioia et al., 2008a). The form consists of questions about the presence of concussion characteristics (i.e., loss of consciousness, amnesia), 22 concussion symptoms, and risk factors that might predict prolonged recovery (i.e., a history of concussion) (Gioia et al., 2008a). The ACE can be used serially to track symptom recovery over time to help inform clinical management decisions (Gioia et al., 2008a).

Concussion Symptom Inventory

The Concussion Symptom Inventory (CSI) (see Figure C-4) is a derived symptom scale designed specifically for tracking recovery. Randolph and colleagues (2009) analyzed a large set of data from existing scales obtained from three separate case-control studies. Through a series of analyses they eliminated overlapping items that were found to be insensitive to concussion. They collected baseline data from symptom checklists, including a total of 27 symptom variables from a total of 16,350 high school and college athletes. Follow-up data were obtained from 641 athletes who subsequently incurred a concussion. Symptom checklists were administered at baseline (pre-season), immediately post concussion, postgame, and at 1, 3, and 5 days following injury. Effect-size analyses resulted in the retention of only 12 of the 27 variables. Receiver-operating characteristic analyses (non-parametric approach) were used to confirm that the reduction in items did not reduce sensitivity or specificity (area under the curve at day 1 post injury=0.867). Because the inventory has a limited set of symptoms, Randolph and colleagues note the need for a complete symptom inventory for other problems associated with concussion.

Graded Symptom Checklist and Graded Symptom Scale

The Graded Symptom Checklist (GSC) (see Figure C-5) and Graded Symptom Scale (GSS) are self-report measures of concussion symptoms derived from the longer Head Injury Scale (Janusz et al., 2012). The symptoms are rated on their severity. The evidence is much stronger to support the use of such self-report symptom measures in youth ages 13 and older. Test-retest reliability has not been reported, but a three factor solution (cognitive, somatic, neurobehavioral) has been reported, although a better solution contained only nine items (Piland et al., 2006). Evidence of convergent validity includes parallel recovery on the GSC and measures of balance and neurocognitive function and correlation with the presence of posttraumatic headaches (McCrea et al., 2003; Register-Mihalik et al., 2007), and discriminant validity between higher and lower impact force (McCaffrey et al., 2007).

Health and Behavior Inventory

The Health and Behavior Inventory (HBI) is a 20-item instrument, with parent and child self-report forms, that has been validated on youth ages 8 to 15 years and their parents (Ayr et al., 2009; Janusz et al., 2012). Test-retest reliability has not been reported. A factor structure was reported (cognitive, somatic, and emotional) that is consistent across parent and child reports and both at baseline and at 3-month follow-up. Parent and child reports are moderately inter-correlated, internal consistency has been reported, and there is evidence of both convergent (correlation with quality of life, family burden, educational and social difficulties) and discriminant (from those with non-head orthopedic injuries, moderate to severe traumatic brain injury) validity. Finally, the measures shows reliable change in 8- to 15-year-olds with mild traumatic brain injury (mTBI) followed over 12 months, and these increases in symptomatology related to orthopedic controls were related to injury characteristics, abnormalities on neuroimaging, the need for educational intervention, and pediatric quality of life (Yeates et al., 2012).

Post-Concussion Symptom Inventory

The Post-Concussion Symptom Inventory (PCSI) has self-report forms for youth ages 5 to 7 years (13 items), 8 to 12 years (25 items), and 13 to 18 years (26 items) as well as reports for parents and teachers (26 items). The forms focus on symptoms in the cognitive, emotional, sleep, and physical domains. Interrater reliability on the child reports was moderate to high (r's ranging from 0.62 to 0.84) (Schneider and Gioia, 2007; Vaughan et al., 2008), as was internal consistency for all three reports (Gioia et al., 2008b; Vaughan et al., 2008). There was moderate agreement among the reporters (r=0.4 to r=0.5) in one of the two studies (see Gioia et al., 2009, Table 2); and evidence of predictive and discriminant validity (Diver et al., 2007; Vaughan et al., 2008). It would appear that the strongest data for adolescents supports the use of the Post-Concussion Symptom Scale (PCSS) (discussed in the following section) and, for children and adolescents, the HBI or the PCSI.

Post-Concussion Symptom Scale

The PCSS is a 21-item self-report measure that records symptom severity using a 7-point Likert scale of severity (see Figure C-6; Lovell and Collins, 1998). The measure has not been studied in youth under the age of 11. In contrast to several of the other measures, evidence of moderate test-retest reliability has been reported (pre- to post-season intraclass correlation was 0.55; test-retest, r=0.65) (Iverson et al., 2003), and the scale is able to detect reliable change (Iverson et al., 2003). Recently, a revised factor structure has been reported in adolescents ranging from ages 13 to 22 years, which showed, post concussion, a four factor solution—cognitive-fatigue-migraine, affective, somatic, and sleep—and higher scores in female than in male participants (Kontos et al., 2012). The PCSS is able to discriminate between concussed and non-concussed athletes (Echemendia et al., 2001; Field et al., 2003; Iverson et al., 2003; Lovell et al., 2006; Schatz et al., 2006); shows greater abnormalities in those with multiple concussions (Collins et al., 1999); and shows convergent validity with measures of neurocognitive performance and regional hyper-activation on functional magnetic resonance imaging during a working memory task (Collins et al., 2003; Pardini et al., 2010). Although a consistent relationship between neuropsychological outcomes and the PCSS has been reported, persistent neurocognitive abnormalities have been reported even in “asymptomatic” athletes who have recovered clinically and on the basis of their scores on the PCSS (Fazio et al., 2007).

Rivermead Post-Concussion Symptoms Questionnaire

The Rivermead Post-Concussion Symptoms Questionnaire (RPCSQ) is a 16-item self-report measure of symptom severity that asks individuals to compare the presence and severity of symptoms they have experienced within the past 24 hours relative to their experience of the same symptoms prior to the injury. One study showed discriminant validity between concussed and non-concussed youth ages 12 and under, but no reliability or factor structure was reported (Gagnon et al., 2005). In adolescents, high internal consistency but low test-retest reliability has been reported (Iverson and Gaetz, 2004) as well as strong discriminant validity between concussed and non-concussed youth (Wilde et al., 2008).

Automated Neuropsychological Assessment Metrics

Automated Neuropsychological Assessment Metrics (ANAM) is a computer-based neuropsychological assessment tool designed to detect the accuracy and efficiency of cognitive processing in a variety of situations (Levinson and Reeves, 1997). ANAM measures attention, concentration, reaction time, memory, processing speed, and decision making (Cernich et al., 2007). Test administration takes approximately 20 minutes.

Recently ANAM has published normative data for sex and age using more than 100,000 military service members 17 to 65 years of age (Vincent et al., 2012). Researchers have also investigated the sensitivity, validity, and reliability of the ANAM test battery. Levinson and Reeves (1997) investigated the sensitivity of ANAM in traumatic brain injury patients who were classified as marginally impaired (n=8), mildly impaired (n=7), or moderately impaired (n=7). The ANAM test battery was administered on two occasions separated by a 2- to 3-month interval in order to examine its ability to classify the patients by comparing accuracy and efficiency scores on the tests to appropriate normative data. At the first administration, the efficiency scores on the ANAM classified 91 percent of the patients into groups accurately; 100 percent classification was obtained in the second administration when the mild and moderately impaired patients were combined as one group. At the second administration, the efficiency scores accurately classified 86.36 percent of the patients. This study revealed the sensitivity of ANAM in distinguishing the severity of traumatic brain injury.

Bleiberg and colleagues (2000) investigated the construct validity of ANAM by examining the relationship between ANAM and a set of traditional clinical neuropsychological tests (Trail Making Test Part B, Consonant Trigrams total score, Paced Auditory Serial Addition test, the Hopkins Verbal Learning test, and the Stroop Color and Word test). The strongest correlation for mathematical processing, Sternberg memory procedure, and spatial processing were found with the Paced Auditory Serial Addition test (0.663, 0.447, and 0.327, respectively). The strongest correlation for matching to sample was found with Trail Making Test Part B (–0.497). This study indicated good construct validity between ANAM and traditional neuropsychological measures.

The ANAM sports medicine battery is a specialized subset designed for serial concussion testing. The ANAM sports medicine battery assesses attention, mental flexibility, cognitive processing efficiency, arousal and fatigue level, learning, and recall and memory (Reeves et al., 2007). Segalowitz and colleagues (2007) examined the test-retest reliability of ANAM sports medicine battery in a group of 29 adolescents. The researchers administered ANAM twice during the same time of day over a 1-week interval. The highest intra-class correlations were reported in matching to sample (0.72), followed by continuous performance test (0.65), mathematical processing (0.61), code substitution (0.58), simple reaction time 2 (0.47), and simple reaction time (0.44). The highest Pearson correlation coefficient was reported in code substitution (0.81), followed by matching to sample (0.72), continuous performance test (0.70), code substitution delayed (0.67), simple reaction time 2 (0.50), and simple reaction time (0.48). The results suggest that there is variability of test-retest reliability for individual subtests of ANAM.

Another study assessed the reliability of ANAM sports medicine battery through test-retest methods using a military sample (Cernich et al., 2007). The average test-retest interval for this study was 166.5 days. Results revealed a wide range of intra-class correlations for each subtest with the highest being in mathematical processing (0.87), followed by matching to sample (0.66), spatial processing (0.60), continuous performance test (0.58), Sternberg memory procedure (0.48), and simple reaction time (0.38). Both of these studies suggest that there are low intraclass correlations values for reaction time subtest scores (Cernich et al., 2007; Segalowitz et al., 2007).

As with all computerized neuropsychological test batteries, it is important to determine whether multiple test sessions result in practice effects. Eonta and colleagues (2011) examined practice effects of ANAM in two groups of military personnel. In the first study, 38 U.S. marines were administered four tests back-to-back on 1 day and on consecutive days. In the second study, 21 New Zealand military personnel were administered the ANAM test battery eight times across 5 days. Individuals demonstrated practice effects on five of six subtests in the two studies.

Bryan and Hernandez (2012) examined military service members who presented with mTBI to a military clinic in Iraq. The findings of the study indicated that on five out of the six ANAM subtests, a larger proportion of mTBI patients than control patients without TBI demonstrated significant declines in speed at 2 days post injury. The one exception was accuracy, which showed no difference between the mTBI and the control groups.

Finally, Coldren and colleagues (2012) examined concussed U.S. Army soldiers at 72 hours and 10 days post injury using the ANAM test battery. Concussed soldiers showed impairment at 72 hours compared to a control group; however, at 10 days the concussed group showed no significant impairments on the ANAM subtests when compared to the control group. Although the researchers concluded that the ANAM lacks utility as a diagnostic or screening tool 10 days following a concussion, they did not report whether the concussed soldiers were also asymptomatic and reporting a normal clinical exam. Thus, more research is warranted to determine whether the ANAM test battery is effective in the long-term assessment of concussed patients.

CogSport/AXON

CogSport is a computerized neuropsychological test battery developed by CogState that measures psychom*otor function, speed of processing, visual attention, vigilance, and verbal and visual learning and memory (Falleti et al., 2006). The battery employs a series of eight “card games” to examine cognitive function including simple reaction time, complex reaction time, and one-back and continuous learning. Axon Sports is a recently launched company within CogState that has developed the Computerized Cognitive Assessment Tool (CCAT). Like Cogsport, the CCAT employs a “card game” to evaluate cognitive domains, including processing speed, attention, learning, and working memory. This test specifically examines reaction time and accuracy.

Collie and colleagues (2003) determined the reliability of CogSport by calculating intraclass correlation coefficients on serial data collected in 60 healthy youth volunteers at intervals of 1 hour and 1 week. In the same study, construct validity was determined by calculating intraclass correlation coefficients between CogSport and performance on traditional paper-and-pencil assessment tools (the Digit Symbol Substitution test and the Trail Making Test Part B) in 240 professional athletes competing in the Australian Football League. The results indicated high to very high intraclass correlations in CogSport speed indices at intervals of 1 hour and 1 week (0.69 to 0.90), while CogSport accuracy indices displayed lower and more variable intraclass correlations (0.08 to 0.51). Construct validity between CogSport and the Digit Symbol Substitution test were found to be the highest in decision making (r=0.86), followed by working memory speed (0.76), psychom*otor speed (0.50), and learning speed (0.42). Smaller and less variable correlation coefficients between CogSport and the Trail Making Test Part B were observed with the highest in psychom*otor speed (0.44), decision making (0.34), working memory speed (0.33), and learning speed (0.23).

Broglio and colleagues (2007) examined the test-retest reliability of three computerized neuropsychological testing batteries: Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT), Concussion Resolution Index (CRI), and Concussion Sentinel, which is an earlier version of CogSport. In this study, these computerized neuropsychological test batteries were administered to 73 participants successively (the order of administration was not specified) on three occasions: at baseline, and at 45 days and 50 days after baseline. Based on seven neuropsychological tests, Concussion Sentinel develops five output scores: reaction time, decision making, matching, attention, and working memory. When baseline and day 45 were compared, the highest intraclass correlation was observed for working memory (0.65), followed by reaction time (0.60), decision making (0.56), attention (0.43), and matching (0.23). When days 45 and 50 were compared, the highest correlation was reported for matching (0.66), followed by working memory (0.64), decision making (0.62), reaction time (0.55), and attention (0.39). However, the effect on the participants' performance of successively administering three neuropsychological testing batteries was identified as a possible methodological flaw, potentially contributing to the low intraclass correlation scores.

Makdissi and colleagues (2010) conducted a prospective study that tracked the recovery of 78 concussed male Australian football players using the Axon Sport CCAT and traditional paper-and-pencil tests (the Digit Symbol Substitution Test and the Trail Making Test Part B). Although concussion-associated symptoms lasted an average of 48.6 hours (95 percent CI, 39.5-57.7 hours), and cognitive deficits on the traditional paper-and-pencil test had for the most part resolved at 7 days post injury, 17.9 percent of the athletes still demonstrated significant cognitive decline on the Axon sport CCAT. This study implied that the Axon sport CCAT has greater sensitivity to cognitive impairment following concussion than the Digit Symbol Substitution test and the Trail Making Test Part B. Because Axon Sport is a new computerized neuropsychological test battery, more research is warranted on this test battery to determine whether it is effective in assessing concussion outcomes.

Concussion Resolution Index

The CRI, developed by HeadMinder, Inc., is a Web-based computerized neuropsychological assessment battery composed of six subtests: reaction time, cued reaction time, visual recognition 1 and 2, animal decoding, and symbol scanning (Erlanger et al., 2003). Symbol scanning measures simple and complex reaction time, visual scanning, and psychom*otor speed. These six subtests form three CRI indices: Psychom*otor, Speed Index, Simple Reaction Time (Erlanger et al., 2003). In addition to cognitive testing, the CRI collects demographic information, medical history, concussion history, and symptom report.

Concurrent validity was established using traditional neuropsychological paper-and-pencil tests. Correlations for CRI Psychom*otor Speed Index were 0.66 for the Single Digit Modality Test, 0.60 and 0.57 for the Grooved Pegboard Test dominant and nondominant hand respectively, and 0.58 for the WAIS-III Symbol Search subtest (Erlanger et al., 2001). Correlations for CRI Complex Reaction Time Index were 0.59 and 0.70 for the Grooved Pegboard test for dominant and nondominant hand, respectively (Erlanger et al., 2001). Correlations for the CRI Simple Reaction Time Index were 0.56 for Trail Making Test Part A, and 0.46 and 0.60 for the Grooved Pegboard Test dominant and nondominant hand, respectively (Erlanger et al., 2001).

In the previously mentioned study by Broglio and colleagues (2007), test-retest reliability was also examined for the CRI at baseline and at 45 and 50 days post baseline test. The CRI demonstrated extremely low intraclass correlations (0.03) for Simple Reaction Time Error score when days 45 and 50 were compared, and 0.15 for baseline to day 45. The correlations for simple reaction time were 0.65 (baseline to 45 days) and 0.36 (day 45 to 50) and for complex reaction time were 0.43 (baseline to 45 days) and 0.66 (day 45 to 50); the complex reaction time error score was 0.26 (baseline to 45 days) and 0.46 (day 45 to 50), and the processing speed index was 0.66 (baseline to 45 days) and 0.58 (day 45 to 50). These low correlations may be due to the three neuropsychological testing batteries being administered during one session and also to the lack of counterbalance of these three computerized test batteries.

Erlanger and colleagues (2003) reported that test-retest reliability for a 2-week interval was 0.82 for psychom*otor speed, 0.70 for simple reaction time, and 0.68 for complex reaction time. Another study also examined the sensitivity of the CRI in detecting changes between baseline and post-injury testing, and it found that the CRI had a sensitivity of 77 percent in identifying a concussion (Erlanger et al., 2001). Thus, the CRI was found to be a valid method of identifying changes in psychom*otor speed, reaction time, and processing speed after a sports-related concussion.

Immediate Post-Concussion Assessment and Cognitive Testing

ImPACT is an online computerized neuropsychological test battery composed of three general sections. First, athletes input their demographic and descriptive information by following instructions on a series of screens. The demographic section includes sport participation history, history of alcohol and drug use, learning disabilities, attention deficit hyperactive disorders, major neurological disorders, and history of previous concussion. Next, the athletes self-report any of 22 listed concussion symptoms, which they rate using a 7-point Likert scale. The third section consists of six neuropsychological test modules that evaluate the subject's attention processes, verbal recognition memory, visual working memory, visual processing speed, reaction time, numerical sequencing ability, and learning.

Schatz and colleagues (2006) examined the diagnostic utility of the composite scores and the PCSS of the ImPACT in a group of 72 concussed athletes and 66 non-concussed athletes. All athletes were administered a baseline test and all concussed athletes were tested within 72 hours of incurring a concussion. Approximately 82 percent of the participants in the concussion group and 89 percent of the participants in the control group were correctly classified. This indicates that the sensitivity of ImPACT was 81.9 percent and the specificity was 89.4 percent.

To examine the construct validity of ImPACT, Maerlender and colleagues (2010) compared the scores on the ImPACT test battery to a neuropsychological test battery and experimental cognitive measures in 54 healthy male athletes. The neuropsychological test battery included the California Verbal Learning test, the Brief Visual Memory Test, the Delis Kaplan Executive Function system, the Grooved Pegboard, the Paced Auditory Serial Attention Test, the Beck Depression Inventory, the Speilberger State-Trait Anxiety Questionnaire, and the Word Memory Test. The experimental cognitive measures included the N-back and the verbal continuous memory task. The following scores were generated: neuropsychological verbal memory score, neuropsychological working memory score, neuropsychological visual memory score, neuropsychological processing speed score, neuropsychological attention score, neuropsychological reaction time score, neuropsychological motor score, and neuropsychological impulse control score. The results indicated significant correlations between neuropsychological domains and all ImPACT domain scores except the impulse control factor. The ImPACT verbal memory correlated with neuropsychological verbal (r=0.40, p=0.00) and visual memory (r=0.44, p=0.01); the ImPACT visual memory correlated with neuropsychological visual memory (r=0.59, p=0.00); and the ImPACT visual motor processing speed and reaction time score correlated with neuropsychological working memory (r=0.39, p=0.00 and r=–0.31, p=–.02), neuropsychological process speed (r=0.41, p=.00 and r=–0.37, p=0.01), and neuropsychological reaction time score (r=0.34, p=0.00 and r=–0.39, p=0.00). It must be noted that the neuropsychological domain scores for motor, attention, and impulse control were not correlated with any ImPACT composite scores. Overall the results suggest that the cognitive domains represented by ImPACT have good construct validity with standard neuropsychological tests that are sensitive to cognitive functions associated with mTBI.

Allen and Gfeller (2011) also found good concurrent validity between ImPACT scores and a battery of paper-and-pencil neuropsychological tests. Specifically, 100 college students completed the traditional paper-and-pencil test battery used by the National Football League and the ImPACT test in a counterbalanced order. Five factors explained 69 percent of variance with the ImPACT test battery with the authors suggesting that ImPACT has good concurrent validity.

Although ImPACT has been reported to have good sensitivity, specificity, and construct validity, its test-retest reliability has been shown to be somewhat inconsistent. Iverson and colleagues (2003) examined the test-retest reliability over a 7-day time span using a sample of 56 non-concussed adolescent and young adults (29 males and 27 females with an average age of 17.6 years). The Pearson test-retest correlation coefficients and probable ranges of measurement effort for the composite scores were: verbal memory=0.70 (6.83 pts), visual memory=0.67 (10.59 pts), reaction time=0.79 (0.05 sec), processing speed=0.89 (3.89 pts), and post-concussion scale=0.65 (7.17 pts). There was a significant difference between the first and 7-day retest on the processing speed composite (p < 0.003) with 68 percent of the sample performing better on the 7-day retest than at the first test session.

In the 2007 study by Broglio and colleagues, the ImPACT intraclass correlations ranged from 0.28 to 0.38 (baseline to day 45) and 0.39 to 0.61 (day 45 to day 50) (Broglio et al., 2007). The correlations for each composite score were: verbal memory (0.23 for baseline to day 45, and 0.40 for day 45 to day 50), visual memory (0.32 and 0.39, respectively), motor processing speed (0.38 and 0.61, respectively), and reaction time (0.39 and 0.51, respectively). However, it must be pointed out that this study was flawed by methodological errors which contributed to the low intraclass correlation values.

Miller and colleagues (2007) conducted a test-retest study over a longer time period (4 months) with in-season athletes. The researchers administered a series of ImPACT tests to 58 non-concussed Division III football players during pre-season (before the first full-pads practice), at midseason (6 weeks into the season), and during post-season (within 2 weeks of the last game). The results indicated no significant differences in verbal memory (p=0.06) or in processing speed (p=0.05) over the three testing occasions. However, the scores for visual memory (p=0.04) and reaction time showed significant improvement as the season progressed (p=0.04). Even though the statistical difference was found at the P level of 0.05, when an 80 percent confidence interval was used, the ImPACT results could be interpreted as stable over a 4-month time period in football players. The test-retest reliability of ImPACT has been examined with even longer time periods.

In response, Elbin and colleagues (2011) investigated a 1-year test-retest reliability of the online version of ImPACT using baseline data from 369 high school varsity athletes. The researchers administered the two ImPACT tests approximately 1 year apart, as required by the participants' respective athletic programs. Results indicated that motor processing speed was the most stable composite score with an intraclass correlation of 0.85, followed by reaction time (0.76), visual memory (0.70), verbal memory (0.62), and PCSS (0.57).

The test-retest study of ImPACT with the longest elapsed time was conducted by Schatz (2010), who tested 95 college athletes over a 2-year interval. Motor processing speed was the most stable composite score over those 2 years with an intraclass correlation of 0.74, followed by reaction time (0.68), visual memory (0.65), and verbal memory (0.46). Even though the correlation for verbal memory did not reach the “acceptable” threshold (0.60), with the use of regression-based methods none of the participants' scores showed significant change. Furthermore, reliable change indices revealed that only a small percentage of participants (0 to 3 percent) showed significant change. This study suggests that college athletes' cognitive performance remains stable over a 2-year time period. In addition, the ImPACT test battery has been shown to have good psychometric properties.

In a small sample of non-athlete college students (n=30), Schatz and Putz (2006) administered three computerized batteries along with select paper-and-pencil tests, counterbalanced over three 40-minute testing sessions. The results showed shared correlations between all the computer-based tests in the domain of processing speed, and between select tests in the domains of simple and choice reaction time. Little shared variance was seen in the domain of memory, although external criterion measures were lacking in this area. Of the test measures used, ImPACT shared the most consistent correlations with the other two computer-based measures as well as with all external criteria except for internal correlations in the domain of memory. However, the authors were clear about the limitations in sample size and the lack of a clinical population as well as the other limitations of the study, and they cautioned against considering this a complete evaluative study. The study does, however, provide some framework for understanding the concurrent validity of these tools.

Previous concurrent validity studies indicated good validity when compared to individual tests. The convergent construct validity of ImPACT was good compared to a full battery of neuropsychological tests (Maerlender et al., 2010). Using a factor analytic approach, Allen and Gfeller (2011) also found good concurrent validity between ImPACT scores and a battery of paper-and-pencil neuropsychological tests. However, there were differences in factor structure between the paper-and-pencil battery and the Im-PACT battery, suggesting differences in “coverage” of neuropsychological constructs.