Post on 19-Dec-2015
transcript
Active Capture and Folk Active Capture and Folk ComputingComputing
Ana Ramírez and Marc Davis
ICME 2004 – Taipei, Taiwan29 June 2004
UC Berkeley - Garage Cinema Research - Group for User Interface Research
6/29/2004 2
Smart Multimedia Acquisition Smart Multimedia Acquisition SystemsSystems• First two papers – automatic camera
calibration Image Audio
• Third paper – understand structure of what is being captured to edit in real time
• Active capture - smart cameras that interactively guide and capture human action
6/29/2004 3
OutlineOutline
• Sample applications
• Active Capture
• Designing Active Capture algorithms
• Future work
6/29/2004 4
SampleSample ApplicationsApplicationsAutomatic Movie TrailersAutomatic Movie Trailers
Play Video
• Video of capture process
6/29/2004 5
SampleSample ApplicationsApplicationsAutomatic Movie TrailersAutomatic Movie Trailers
Play Video
• Video of automatically created movie trailer
6/29/2004 6
SampleSample ApplicationsApplicationsSports InstructionSports Instruction
6/29/2004 7
SampleSample ApplicationsApplicationsTelemedicineTelemedicine
Rural Town
Large City
leishmaniasis
6/29/2004 8
SampleSample ApplicationsApplicationsAutomated Health ScreeningAutomated Health Screening
Rural Town
Large City
leishmaniasis
6/29/2004 9
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
6/29/2004 10
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
6/29/2004 11
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
6/29/2004 12
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
6/29/2004 13
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
6/29/2004 14
Active CaptureActive Capture• Traditionally, signal processing
algorithms avoid interacting with the user
• Signal processing + interaction => more sophisticated recognizers
• How to design hybrid algorithms that involve capture, interaction, and processing
6/29/2004 15
Components of Active Capture Components of Active Capture AlgorithmsAlgorithms
• Simple computer vision and audition recognizers / sensors Motion Eyes Sound
• Desired action in terms of recognizers
• Interaction script
6/29/2004 16
Design ProcessDesign Process
• Input: Desired action = head turn Recognizers = motion, eyes
MotionEyes
time
6/29/2004 17
Design ProcessDesign Process
• Input: Desired action = head turn Recognizers = motion, eyes
• Step 1: Express desired action in terms of
recognizers
MotionEyes
time
No Motion
No Eyes
Motion
Eyes
No Motion
6/29/2004 18
Design ProcessDesign Process
• Input: Desired action = head turn Recognizers = motion, eyes
• Step 1: Express desired action in terms of
recognizers
• Step 2: Design interaction script
6/29/2004 19
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
6/29/2004 20
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
6/29/2004 21
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
Play Video
6/29/2004 22
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
6/29/2004 23
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
Play Video
6/29/2004 24
Design ChallengesDesign Challenges
Step I - Description of action Approximate timing Strict and non strict ordering
Step II – Interaction script What to do if something goes wrong –
mediation
Step I – Action DescriptionStep I – Action Description
6/29/2004 26
• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language
6/29/2004 27
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
6/29/2004 28
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
6/29/2004 29
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
6/29/2004 30
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
6/29/2004 31
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language
time
waving
noise
"Say your name and wave"
> 4.5 sec
capture
Waving
Noise
<1 sec <1 sec
• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
6/29/2004 32
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language
time
waving
noise
"Say your name and wave"
> 4.5 sec
capture
Waving
Noise
<1 sec <1 sec
• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
Step II – Interaction ScriptStep II – Interaction Script
6/29/2004 34
Step II – Interaction ScriptStep II – Interaction ScriptContextual InquiriesContextual Inquiries• Golf instructor• Aikido instructor• 911 emergency phone operator• Triage nurse• Children’s portrait photographer• Film and theatre directors
[Jeffrey Heer, Nathaniel S. Good, Ana Ramirez, Marc Davis, and Jennifer Mankoff. “Presiding Over Accidents: System Direction of Human Action.” In: Proceedings of the Conference on Human Factors in Computing Systems (CHI 2004) in Vienna, Austria. ACM Press, 463-470, 2004. ]
6/29/2004 35
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
• External aids
Play Video
6/29/2004 36
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
• Decomposition and “Show”
Play Video
6/29/2004 37
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
• Method shift from “Show” to “Tell”
Play Video
6/29/2004 38
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
6/29/2004 39
SummarySummary• Active Capture – smart cameras that
interactively guide and capture human action
• Sample applications Automated health screening Automated movie clips Sports trainer
• Design Challenges Description of action Interaction script
6/29/2004 40
Future WorkFuture Work
• Support design and implementation of Active Capture applications
• Evaluate the relative contribution of signal analysis and user interaction in these hybrid algorithms
6/29/2004 41
QuestionsQuestions
Ana Ramírezanar@cs.berkeley.edu
www.cs.berkeley.edu/~anar
Garage Cinema Researchhttp://garage.sims.berkeley.edu
Group for User Interface Researchhttp://guir.berkeley.edu