Combinatorics Meets Processing Power:Large-Scale Computational Resources for BRIMS
28 March 2007
Kevin Gluck AFRL/HEATMatthias Scheutz Notre DameGlenn Gunzelmann AFRL/HEATJack Harris AFRL/HEATJeff Kershner L3 at AFRL-Mesa
2
Outline
• Combinatorics in Cognitive Modeling
• Example from Fatigue Modeling
• High Performance Computing
• SWAGES
• Way Forward
3
Combinatorics in Computational Cognitive Modeling
Architecture + Knowledge = Model
All Knowledge is a ParameterPrevious ExperiencesLevels of ExpertiseStrategies
Dozens of Interacting ParametersSub-Component AssumptionsControl Structure AssumptionsNumerical moderators
•Infinite search space •Challenge is to identify invariants in the cognitive architecture and use them to make the search tractable
4
Human Representation Systems
Recent reviews (Gluck & Pew, 2005; Morrison, 2003; Pew & Mavor, 1998; Ritter et al., 2003) reveal there are many systems available for representing human behavior (in alphabetical order):
D-COGD-OMAREPAMEPICMicroPsiMicro Saint/HOS/IPMEMIDASPDP++SAMPLESoar
ACT-RAPEXARTBrahmsCHRESTC/IClarionCogAffCogentCOGNET/iGEN
They ALL involve:Structural assumptionsKnowledge representation assumptionsNumerical parameters that influence predictions
Each of these is just one of an infinite number of possible implementations
5
ACT-R (Anderson et al., 2004)
An embodied, hybrid cognitive architecture
Ai = Bi + Wj ⋅ Sji + σ Aj∑
Bi = ln tj−d
j∑
Ui = Pi ⋅G − Ci +σ U
Pi =Succi
Succi + Faili
iAi eFT −⋅=
Activation
Learning
Latency
Utility
Learning
Procedural Knowledge
EnvironmentPr
oduc
tions
(Bas
al G
angl
ia)
Retrieval Buffer (VLPFC)
Matching (Striatum)
Selection (Pallidum)
Execution (Thalamus)
Goal Buffer (DLPFC)
Visual Buffer (Parietal)
Manual Buffer (Motor)
Manual Module (Motor/Cerebellum)
Visual Module (Occipital/etc)
Intentional Module (not identified)
Declarative Module (Temporal/Hippocampus)Declarative Knowledge
6
Computational Theory of Degraded Cognition
Extending ACT-R to include the effects of factors such as sleep deprivation and circadian rhythm on cognitive functioning.
Neurobehavioral Assessment BatteryPsychomotor VigilanceSerial Addition SubtractionDigit-Symbol SubstitutionPaired Associate Learning
Experiment conducted at Penn (Van Dongen, Dinges):
88 hours total sleep deprivation
7
Psychomotor Vigilance Task (PVT)
• A popular task in sleep restriction research
– Sensitive to levels of sleep deprivation and circadian desynchrony
– No learning curve
• Very simple
– Wait for a stimulus to appear on the screen
• Delay varies from 2 to 12 seconds
– When stimulus appears
• Respond by pressing a button
8
Sample Trial
Slap your knee when the stimulus appears:
+NOW!!
9
Sample Trial (again)
Concentrate:
+NOW!!
10
Phenomena of Interest(and a model that accounts for them)
0.00
0.05
0.10
0.15
0.20
0.25
0.30
False S
tarts:
<170
:<1
90:
<210
:<2
30:
<250
:<2
70:
<290
:<3
10:
<330
:<3
50:
<370
:<3
90:
<410
:<4
30:
<450
:<4
70:
<490
:La
pses
:
BaselineDay 1 TSDDay 2 TSDDay 3 TSD
Sleep A
ttack
s
Prop
ortio
n of
Res
pons
es
• 2 global procedural parameters – Arousal (G)– Utility threshold (Ut)
• New “G decrement” mechanism
Human Data
0.00
0.05
0.10
0.15
0.20
0.25
0.30
False S
tarts:
<170
:<1
90:
<210:
<230
:<25
0:<2
70:
<290:
<310
:<33
0:<3
50:
<370:
<390
:<4
10:
<430
:<4
50:
<470
:<4
90:
Laps
es:
BaselineDay 1 TSDDay 2 TSDDay 3 TSD
Model Data
r = .989RMSD = 0.0049
Sleep A
ttack
s
# parameter values in search 16
X 21X 3612,096 nodes
Batch run took ONE MONTH to complete in our lab.
11
Realization …
• We want to generalize the account to 3 other tasks
– Batch run takes one month for one task
– Will take four months (or more) for four tasks
But Wait! It’s even worse …• The other tasks are more cognitively complex
– Declarative memory parameters
– Strategies
We’re gonna need more processors.
12
High Performance Computing
• Established account at Wright-Patterson HPC Center
• Developed initial batch run software
• Demonstrated ability to run ACT-R cognitive models
– 20,000 hours in FY06
– 90,000 hours in FY07 (so far)
• 1st “large” run in late spring 2006
– 9,600 parameter value combinations
– Serial Addition-Subtraction Task model
The HP XC (Falcon) Cluster•1024 XC Compute Nodes (2 proc’s / node) •2048 Processors (11.5 Peak TeraFLOPS) •2 Gigabyte Memory per Processor(4 GB/Node, 4 TB Total) •User accessible memory: 2.25 Gigabytes per 2-processor compute node. •Infiniband Interconnect •97 Terabyte Workspace •AMD Opteron (2.8 GHz) •HP SFS (Luster) Scalable File System •Operating System: Linux RedHat 2.4
13
Parameter Space in Serial Addition-Subtraction Task Model
Need to evaluate the range of human performance results which we can explain with new mechanisms and parameters
Parameter # ValuesArousal (G) 8Utility threshold (Tu) 5Base-level Activation (A) 5Retrieval Threshold (Tr) 8Activation Noise (σ) 6
9600 parameter value combinationsX 25 iterations at each combination (stochasticity)= 240,000 model runs
14
0
50
100
150
200
250
9,600 9,600 118,482 1,777,236
Total Parameter Value Combinations
Hou
rsReal Time (Submission to Completion)
CPU Time (1000's of Hours)
0
50
100
150
200
250
9,600 9,600 118,482 1,777,236
Total Parameter Value Combinations
Hou
rsReal Time (Submission to Completion)
CPU Time (1000's of Hours)
Growth in HPC use over past nine months
4.4 Billion Model Runs (Jan 2007)Used 25% of total capacity of Falcon cluster for 8 daysNB: Just one parameter search for one model
Huge gains in research efficiency
Enabling breadth and depth of exploration that was previously impossible.
15
More Intelligent Use of HPC
• We can not continue growth in full combinatorial search
• Point of research is NOT to consume processing resources, it is to make fastest possible progress in cognitive modeling and behavior representation
• Need a software infrastructure to support intelligent, efficient search
– SWAGES
16
SWAGES
SoftWare Architecture for Grid Experimentation System- manages scheduling, starting, and monitoring of distributed simulations and recovery from failures
Contact Matthias Scheutz for more on SWAGES
17
Many Fun Challenges Remain
• AI search methods
• Statistical sampling techniques
• Batch results databasing
• Visualization tools
• Alternative sources of CPU hours
– Volunteer Computing
• Hundreds of thousands of people around the world donate their idle computer processor time for scientific research
MindModeling@Home
18
Summary
• The combinatorial complexities inherent in computational human representation pose a challenge to the entire BRIMS community
• We are improving our pace of progress by scaling the computational resources available for the research to the combinatorial complexity of the scientific challenge
19
Questions?
