Personalized Modeling for Human-Robot Interaction

Ruzena BajcsyElectrical Engineering & Computer Sciences

University of California, BerkeleyOctober 23, 2015

Our Lab’s Motivation

We wish to understand the mechanics of human-robot interactions and to design algorithms for control-sharing between humans and autonomous systems.

Why is this problem important?

– A person is a complex kinematic/dynamic system with many degrees of freedom and parameters which vary from person to person

– Not all degrees of freedom are used in all activities

Human Model ClassesMusculoskeletal

Kinematic/

Dynamic

Kinematic

Agent Interaction

Micro

Macro

Human Model Classes: KinematicMusculoskeletal

Kinematic/

Dynamic

Kinematic

Agent Interaction

GoalModel human motion using a rigid-body kinematic model

Application

Automated Coaching/Quantitative Outcome Measures

Contributors: Qifei Wang, Gregorij Kurillo, and Ferda Ofli

Human Model Classes: Kinematic/DynamicMusculoskeletal

Kinematic/

Dynamic

Kinematic

Agent Interaction

GoalModel human motion using a rigid-body kinematic/dynamic model

Applications

Human-Robot Collaborative Manipulation

Human Dynamic Stability Analysis

Contributors: Aaron Bestick and Victor Shia

Application: Collaborative Manipulation

Goal: Enable intelligent control of robots providing direct physical assistance to humans

• Create unified model of the human-robot coupled mechanical system

• Predict intent of human operator based on physical cues

Application: Collaborative Manipulation

Personalized Human Mechanical Models

Coupled Human-Robot Dynamical

Models

Optimal Robot Control

Human Constraints

Robot Constraints

Task Constraints

Human Ergonomic

Cost

Key Neglected Aspect: Differences in constraints and cost functions between individual humans (e.g. age, disabilities, natural variation)Implicit Assumption: Differences between humans not a significant contributor to task variability Need personalized models

Human Model Classes: Musculoskeletal

Musculoskeletal

Kinematic/

Dynamic

Kinematic

Discrete States

GoalCombine a kinematic/dynamic model with a nonlinear model of muscle characteristics to predict biomechanical properties throughout the human’s workspace

ApplicationMedical Diagnostics

Dynamic Human Musculoskeletal Modeling• Data Acquisition– MRI Scans– DICOM Images with

Segmentation– Motion Capture– EMG Data

• Modeling– 2D/3D Visualization– Interactive Cleaning– Static, Kinematic, Dynamic

Scenarios– Physics based Dynamic

Deformation

MRI Data segmentation

Ultrasound for Muscle Observation

• External motion capture for pose of ultrasound

• Muscle and tendon outlines visible

Muscle

Ultrasound for Muscle Observation

• External motion capture for pose of ultrasound

• Muscle and tendon outlines visible

Muscle

Ultrasound for Muscle Observation

• External motion capture for pose of ultrasound

• Muscle and tendon outlines visible

Tendon

Ultrasound for Muscle Observation

• External motion capture for pose of ultrasound

• Muscle and tendon outlines visible

Tendon

Ultrasound for Muscle Observation

ConclusionsMusculoskeletal

Kinematic/

Dynamic

Kinematic

Agent Interaction

Micro

Macro

• Robotic technology has great utility for modeling and predicting human physical capabilities and limitations

• By modeling the human, we can improve shared control schemes for human-robot interaction