The NIH Toolbox: Assessment of The NIH Toolbox: Assessment of function across the lifespanfunction across the lifespan

Nathan Fox, Ph.DUniversity of Maryland

This study is funded in whole or in part with Federal funds from the Blueprint for Neuroscience Research, National Institutes of Health under Contract No. HHS-N-260-2006-00007-C

PURPOSE OF THE NIH TOOLBOX

The NIH Toolbox, part of the NIH Neuroscience Blueprint initiative, seeks to develop brief yet comprehensive assessment tools measuring motor, cognitive, sensory, and emotional function. Upon completion, the Toolbox will be available for use in longitudinal epidemiologic studies and prevention or intervention trials for people ages 3-85.

ADDITIONAL OBJECTIVES OF THE TOOLBOX

• Enable cross-study comparisons and integration of data from multiple studies by providing a standard set of brief, well-validated measures• Dynamic/adaptable over time• Utilize state-of-the-art psychometric and technological approaches including computerized adaptive testing (CAT) and computer assisted evaluation

ADDITIONAL OBJECTIVES OF TOOLBOX

• Cover the full range of normal function (not disease states)• Be used as the basis for detecting at-risk populations• Be minimally burdensome to subjects and investigators

COGNITIVE DOMAIN

Executive Function Episodic MemoryWorking MemoryProcessing Speed LanguageAttention

SOCIO-EMOTIONAL HEALTH DOMAIN

Negative AffectPsychological Well-BeingStress & Self-EfficacySocial Relationships

MOTOR DOMAIN

LocomotionStrengthNon-Vestibular BalanceEnduranceDexterity

SENSORY DOMAIN

VisionAudition Vestibular BalanceSomatosensationTaste Olfaction

SUB-DOMAINS OF COGNITION AND INSTRUMENTS

Episodic Memory Picture Sequence MemoryExecutive Function- Cognitive Flexibility Dimensional Change Card SortExecutive Function –Inhibitory Control FlankerLanguage-Vocabulary Comprehension Picture Vocabulary Comp AdapProcessing Speed Pattern ComparisonWorking Memory List SortingLanguage-Reading Oral Reading Recognition

10

Working Memory: Working Memory: Complex Span TaskComplex Span Task

DOG HORSE

RABBIT

THREE ITEM FLASHING PRACTICE

RABBIT

SHEEP

ELEPHANT

Language: Reading

cucurbitaceous

forisfamiliate

prevarications

circumlocutory

imparidigitate

is

art

dodecahedral

Attention/Executive Function: Choice Reaction Time

(Flanker)

Look at all the fish!

Let’s practice!

Try to focus on the + sign in the middle of the screen.

Press the green light when you’re ready to start

Processing Speed: Pattern Comparison

28

Executive Function: Executive Function: Dimensional Change Card Sort Dimensional Change Card Sort

(DCCS)(DCCS)

Executive Function:Self-Ordered Pointing Task

Now look at these pictures. Press the one you want to pick

Now look at these pictures. Press the one you want to pick

36

Language: Language: VocabularyVocabulary

37

Executive Function: 3 to 6 YearsExecutive Function: 3 to 6 Years

38

Executive Function: 8 to 85 YearsExecutive Function: 8 to 85 Years

• The initial NIH toolbox contract included assessment of children (starting at age 3)

• Identification of domains and measures included those that could be used with children as young as age 3 (for the most part)

• NIH directors approved the Pediatric Option---additional funding for the norming and validation of measures with children

• Domain managers and consultants identified measures that can be used with young children (for the most part)

• Committee organized to identify principles for pediatric assessment (chaired by Nathan Fox)

Pediatric Option: A Brief Pediatric Option: A Brief ReviewReview

• Marilyn Jager Adams, Ph.D.Brown University

• Patricia Bauer, Ph.D.Emory University

• David Blitz, M.A.CORE

• Helena CorreiaCORE

• Scott Debb, M.A.CORE

• Nathan Fox, Ph.D.University of Maryland

• Richard Gershon, Ph.D.CORE

• Jean Berko Gleason, Ph.D.Boston University

• Roberta Golinkoff, Ph.D.University of Delaware

• Kathy Hirsh-Pasek, Ph.D.Temple University

• Jin-Shei Lai, Ph.D.CORE

• GiglianaMelzi, Ph.D.New York University

• Kathleen Wallner-Allen, Ph.D.Westat

• Phil Zelazo, Ph.D.University of Minnesota

• Nick Zill, Ph.D.Westat

Pediatric Working Group Pediatric Working Group MembersMembers

CORE: Center on Outcomes, Research and Education

• Importance of having an examiner present• Clarity of instructions• Importance of practice trials to insure that children

understand task• Importance of feedback after practice trials and during task

performance• Importance of engaging child and motivating performance• Pace of assessment to be monitored

Principles of Pediatric Principles of Pediatric AssessmentAssessment

• Age-Appropriateness of Instrument• Construct Appropriateness• Instructions• Equipment• Child’s Perspective• Flexibility

Cross-domain Issues with Cross-domain Issues with Pediatric Pediatric AssessmentAssessment

• Age-Appropriateness of Instrument– Can a 3-year-old do it? If not, are there minor adaptations to

make it appropriate?

• Construct Appropriateness– Will it be valid? Will it measure the same construct that the

task measures for an adult? Is it a meaningful construct for children?

• Instructions– Are the instructions clear for young children?– Can they be repeated? Should they be administered

automatically (via voice or on screen) or by examiner?

Cross-domain Issues with Cross-domain Issues with Pediatric Pediatric AssessmentAssessment

• Equipment– Appropriately-sized and comfortable to perform best– Child-size chairs, tables

• Feet flat on the ground when sitting back in chair• Can reach comfortably to respond

– Non-threatening

Cross-domain Issues in Cross-domain Issues in Pediatric Pediatric AssessmentAssessment

• Child’s Perspective– Will it be fun, scary?– Will I be able to do it?– Will it be boring?– What do I get out of it?

Cross-domain Issues in Cross-domain Issues in Pediatric Pediatric AssessmentAssessment

• Flexibility:Tasks, equipment, computer-interfaces must be flexible ---by design.

– One size does not fit all. Children are• Less predictable than adults• Less capable of regulating behavior and attention to task• Less able to remember or understand instructions

– Be prepared, anticipate behavior, but expect the unexpected. Schedule adaptations will be necessary.• Children may need to go back to a particular screen or task • They may need breaks during the testing session.

Cross-domain Issues in Cross-domain Issues in Pediatric Pediatric AssessmentAssessment

• Flexibility:Tasks, equipment, computer-interfaces must be flexible ---by design.

– To optimize a child’s performance• Must actively create an environment that supports the child• Must be sensitive to the needs of the child• Must keep the child engaged and motivated

Cross-domain Issues in Cross-domain Issues in Pediatric Pediatric AssessmentAssessment

– Flexibility

• Schedule, interface, and examiner need to be flexible to be sensitive to the child and to try to optimize performance

• Obviously, flexibility must occur within pre-defined boundaries

• Balance between child’s needs, practical considerations and needs for standardization of administration.

Flexibility in Flexibility in testingtesting

• Ergonomic Design– Type of equipment– “Workstation” (appropriately sized table, chair,

screen at comfortable angle, comfortable reach to respond)

Human-Computer Human-Computer InterfaceInterface

• Software Design -- Flexibility Important

– Even with (or especially with) extremely complex and expensive tasks there is a need to balance practical and child considerations.

– Need repeat or clarify instructions, pause in the middle of a task (as appropriate), easily move from one task to another, modify task order

Human-Computer Human-Computer InterfaceInterface

• Software Design -- Flexibility Important

– Consider: What is ideal on-task behavior? Vs. What is acceptable on-task behavior?

– Design should allow for both, if possible.• For example, ideally the subject should complete the

battery of tasks in one sitting but it should be acceptable to take breaks.• Children may be momentarily distracted---they may

need to be re-oriented to the task instructions. Administrators may need to skip ahead or go back. • Software should be flexible for these encounters.

Human-Computer Human-Computer InterfaceInterface

• Voice– Standardized, prerecorded voice with appropriate reflection

good, but children may need human interaction to keep them engaged and elicit optimal performance.

– Build system so both are possible. – If prerecorded is not working/holding attention/problem for

some other reason, then shift to human during administration.

– Instructions presented with prerecorded voice should be available to the examiner if the examiner needs to repeat/clarify.

• Pacing – Needs to be adjusted to the needs of the child. Some want

to move faster, others need slower, some need extra clarification in instructions

Human-Computer Human-Computer InterfaceInterface

• Role of the Examiner– Need clear guidelines for how to interact with the participant—

what kind of feedback/encouragement; when to intervene/not to intervene. • When examiner can say something, • what they can say, • how often can they say it.

– Trouble-shooting section to standardize the way the examiner interacts with the participant

Role of the Role of the ExaminerExaminer

• Training – Crucial for standardizing administration, minimizing across-site

differences in administration– Certification to ensure standardized implementation of

protocols.– What materials will be provided?

• Manuals?• Videos?• Web-based?

– Working with children is a skill.

Training ExaminersTraining Examiners

• Human interaction and eye contact• Establishing rapport• Smiling, enthusiasm, praise (appropriately placed)• Make it fun• Give the child some control• Be respectful of child• Provide breaks and snacks• Child should feel success, not failure• Incentives

– Type• Stickers, toys, visual and audio computer display

– Frequency• Along with way (points, stars)• After finishing a task• After finishing the visit

Ways to Engage and Ways to Engage and MotivateMotivate

– Must standardize assessment• Precisely define by task, how the examiner can interact

with the participant (what can say, when, how often)• Documentation, documentation, documentation (e.g.,

problem-solving or trouble-shooting section to provide guidance for handling common problems that arise).• Training – need to minimize across site variation in

administration. This is particularly vital while documentation is not fully developed.

Standardizing Standardizing AssessmentsAssessments

• Consent form need to be clear– Different versions for different ages

• IRB review– There will be differences in requirements across institutions

• Who gives consent?– Parent/guardian– At what age can a child?

• Assent --- verbal vs. written?– At what age is this required?– Is verbal assent allowed? Until what age?

Issues of Issues of Consent/AssentConsent/Assent

The End

• www.nihtoolbox.org