Unified Robotic Software Development using CLARAty Issa A.D. Nesnas Mobility and Robotic Systems...

Unified Robotic Software Development using CLARAty

Issa A.D. NesnasMobility and Robotic Systems Section

Autonomous Systems DivisionJuly 20, 2005

http://claraty.jpl.nasa.gov

Briefing to the Office of the Chief TechnologistSponsors: Mars Technology Program

Intelligent Systems Program

Theme: The Long Road to Technology Infusion

http://claraty.jpl.nasa.gov/





July 20, 2005 Briefing to the Chief Technologist - I.A.N. 2

Presentation Overview

• Historical Antecedents

• Problem and Objectives

• What is CLARAty?

• Interoperability Success Stories– Navigation

– Single-cycle Instrument Placement

• Challenges of Software Interoperability

• Challenges in Technology Infusion

• Architecture Overview

• Challenges Ahead


Some JPL Robots


Historical Antecedents

• Late 80’s - Early 90’s: parallel robotic developments– RSI, MOTES, Satellite Servicing, Robby, Mircorover– No shared hardware or software

• Mid 90’s: Mars rover research centralized with Rocky 7– First flight rover

• Late 90’s: Expansion and diversification of rover work– No software interoperability (Rocky 7, FIDO, Athena, DARPA)– Autonomy demonstration of Remote Agent Experiment (ARC and JPL)– MDS investigates reusable software for spacecraft control.

• ‘99-Early 00: Exploration Technology Program develops concept for a unifying autonomy architecture– Unifying autonomy and robotic control– Started the CLARAty task

• Today:– Unification of several robotic developments at JPL, ARC, and CMU– Two flight rovers with several new robotic capabilities


Problem and Approach

• Problem:– Difficult to share software/algorithms across systems

– Different hardware/software infrastructure

– No standard protocols and APIs

– No flexible code base of robotic capabilities

• Objectives– Unify robotic infrastructure and framework

– Capture and integrate legacy algorithms

– Simplify integration of new technology

– Operate heterogeneous robots


What is CLARAty?

CLARAty is a unified and reusable software that provides robotic functionality and simplifies the integration of new technologies on robotic platforms

A tool for technology development

and maturation


StereovisionJPL_STEREO

Interoperability: Software & Hardware

VxWorksLinux

QNX

CLARAty Reusable Software

Robot Adaptation

Pose EstimationMER_SAPPObstacle Avoidance

MORPHIN

Obstacle Avoidance

MORPHIN

Obstacle Avoidance

MORPHINGESTALT

Drivemaps StereovisionJPL_STEREOStereovision

JPL_STEREOPose EstimationMER_SAPP

Pose EstimationMER_SAPP

Pose EstimationMER_SAPP

SRI StereoARC Stereo Sojouner Pose

FIDO 3DEKF6D EKF

CAPABILITY: Navigation

Interoperability Success Stories

NavigationSingle-cycle Instrument Placement


Capturing Flight Algorithms: MER GESTALT on FIDO

QuickTime™ and a decompressor

are needed to see this picture.




Navigation Interoperability on FIDO and Rocky 8

Complex Algorithms on different Platforms

• I/O, motion control

• Trajectory Generation

• Rough Terrain Locomotion

• Odometry Pose Estimation

• Stereo Processing

• Visual Odometry

• Navigation (Morphin)– Obstacle avoidance

– Path Planning

Interoperability Success Stories

NavigationSingle-cycle Instrument Placement


Single-cycle Instrument Placement

Changes in FOV

(a)

Target

(b)

Designated Target

Target Tracking

time = t2(avoiding an obstacle)

time = t1

1st Frame 37th Frame after 10 m


QuickTime™ and aVideo decompressor


FALCON Visual Target Tracker on Rocky 8

Integration of Complex Algorithms

• I/O, motion control

• Trajectory Generation

• Rough Terrain Locomotion

• Odometry Pose Estimation

• Stereo Processing

• Visual Odometry

• Obstacle avoidance

• Mast Control

• Visual Tracking


Integrated Single-Cycle Instrument Placement




CLARAty Framework for Single-cycle Instrument Placement

2D/3D Visual Tracking

R8 Mast Pointing

Morphin Navigator

KIMHand-off

MSL Base Placement

HIP

S

Visual Odometry

MaestroGroundSystem

Visual Tracker

Visual Odometry

SIFTTracker

Gaze Pointing

K9 Mast Pointing

Locomotor

R8 Locomotor

Wheel Odometry

Pose Estimator

GESTALT

Drivemaps

FIDO EKF

6DOF EKF

Visual Odometry

Camera Hand-off

Mesh Registration

Bundle Adjustment

Base Placement

Haz Camera Tracking

2D/3D Visual Tracking

SIFTTracker

Obstacle Avoider

Possible Alternatives

Vis

ion

-gu

ide

d M

an

ipOn -board RoverSoftwareInfrastructure

Challenges in Interoperability

Mechanisms and Sensors

Hardware ArchitectureSoftware Algorithms


Different Mobility Mechanisms

Front

xy

z

C

(a)Skid Steering

(no steering wheels)

Frontx

y zC

(d)Partially Steerable

(e.g. MER, Sojourner,Rocky 7)

Front

x

y zC

(e)All wheel steering

(e.g. Rocky8, FIDO, K9)

Front

x

yz

C

(b)Two –wheel steering


Different Sensors and AppendagesCustom AnalogSun Sensor

3 Accelsz-axis gryo

6 DOF IMU

4 DOF Arm

4 DOF Mast

2 DOF Arm

3 DOF Mast

Camera Sun Sensor

QuickTime™ and aNone decompressor






Centralized Hardware Architecture

FIDO

Actuator/Encoders

Potentiometers

PID Control in Software

Video Switcher

IMU

RS232 Serial

PC104+x86 ArchFramegrabbersDigital I/OAnalog I/OWireless Ethernet


Distributed Hardware Architecture

Compact PCI- x86 Arch- Wireless E/net- 1394 FireWire- I2C Bus

Actuator/Encoders

Potentio-meters

I2C

1394 Bus

IMU

Sun Sensor

Rocky WidgetsSingle-axis controllersCurrent sensingDigital I/OAnalog I/O

RS232

Distributed MotionControl and Vision

Rocky 8





Software Challenges for Algorithm Infusion

The new algorithms to be integrated may:• Have architectural mismatches with the framework• Include multiple orthogonal functionalities• Make implicit assumptions about the platform• Duplicate functionality in the framework• Use incompatible data structures• Are complex and hard to tune • Require highly specialized domain expertise• Are poorly implemented

Architecture and Process

THE DECISION LAYER: Declarative model-based Global planning

CLARAty = Coupled Layer Architecture for Robotic Autonomy

INTERFACE:

Access to various levelsCommanding and updates

THE FUNCTIONAL LAYER: Object-oriented abstractionsAutonomous behaviorBasic system functionality

A Two-Layered Architecture

Adaptation to a system


Multi-level Abstraction Model

Use abstractionsInterface at different levels

Input / Output (AI, AO, DIO)

Motor

Motor Group

Wheeled Locomotor

Rover

Locomotor

Example of Levels of Abstraction

for locomotors

FIDO HW

Rocky 8Firmware/HW

ATRV SW/HW

Pluto SW/HW


Technology Tasks

Collaborations & Interactions

CLARAty

Jet Propulsion Lab

CMU

NASA ARC

U. Minnesota

R&TD, MDS, DRDF

Competed Mars TechnologyProgram

Other NASA Programs

Rover SimulationROAMS

Rover Hardware

JPL Internal Programs

Flight FocusedTechnology Programs

Science InstrumentsSimulation

Operator Interface

Legacy AlgorithmsFlight Algorithms

NASA Centers and

UniversitiesTechnology Tasks

NASA Centers and


NASA Centers and


NASA Centers and


TechnologyValidation Tasks

TechnologyValidation Tasks

Technology Tasks

Technology Tasks


CLARAty Test Bed for Regression Testing

ATRV Jr.

FIDO2 Stack

Dexter ManipulatorBench topRocky 8 PPC Bench top


Challenges Ahead

• Mature framework and robotic capabilities

• Investigate relevance to flight and define migration path

• Develop regression tests for long-term maintainability of robotic capabilities - very hard and open research topic

• Maintain current capabilities on existing platforms

• Develop new capabilities (e.g. continuous motion)

• Integrate new technologies from competed programs

• Develop a releasable version

• Develop formal documentation and tutorials

• Identify and deploy on low-cost rover platform

• Open source


Summary

• Developed a unified and reusable software framework• Deployed at multiple institutions• Deployed on multiple heterogeneous robots• Integrated multiple technologies from different institutions• Delivered algorithms for formal validation• Enabled new technology developments on multiple

platforms• Integrated flight algorithms for detailed performance

characterization and operation on research rovers.• Taking a technology from inception, to development in

CLARAty, to validation, and now to integration into flight


Current CLARAty Core Team

• NASA Ames Research Center– Clay Kunz– Eric Park– Susan Lee

• Carnegie Mellon University– David Apelfaum

– Nick Melchior

– Reid Simmons

• University of Minnesota– Stergios Roumeliotis

• Jet Propulsion Laboratory– Antonio Diaz Calderon– Tara Estlin– John Guineau– Won Soo Kim – Richard Madison– Michael McHenry– Mihail Pivtoraiko– Issa A.D. Nesnas– Babak Sapir– I-hsiang Shu

• OphirTech– Hari Das Nayar

Full Credits for all Developers and Contributors at:http://keuka.jpl.nasa.gov/main/project/team/index.shtml

Questions?


Rocky 7

Distributed Software Development

AFS Backbone

CMU JPL

CLARAty

3rd PartyWeb

UW...

VxWorks

ARC

FIDO

K9

Authentication

U. Minnesota

Rocky 8

0

10

20

30

40

50

60

70

FY00 FY01 FY02 FY03 FY04

Users/Collaborators

ExtendedTeam

JPL Team

Number of employees and not FTEs


Unify Mechanism Model

Ground_Body

Body4 Body5

Body Tree

Body0

Body2 Body3Body1

Mechanism Tree

Body Reference Frame

SensorMountFrame

Arm mountFrame

Center ofmass

Camera Mount Frame

Bodyi

B1

Jointi

Bounding Shape Tree

B2 B3 B3B4

B5

C1

B1

B2 B3

B4 C1 B5

Coarse Shape Finer Shape Finest Shape

Leaves of tree define finest shape

Relative to body reference frame

Bounding Shapes Resolution Levels

Bodies and Joints

Body1

Reference Frame

Body1CG

Body1

Upper arm link

RoverReference Frame

RoverCG

CameraFrame

Body0

Rover

Joint1

Body2

Lower arm link

Sensor Mount Frame

CameraMountFrame

Body2CG

Articulated Rotation

Body2 Reference Frame

Fixed Transform

Arm MountFrame

Shoulder Yaw

Joint2

Articulated Translation


Some CLARAty Statistics

• ~320 modules in repository (increase of 6% from FY03) – goal is to limit modules• ~60 modules are researched technology algorithms (~20%)• About 500,000 lines of C++ code – revise and reduce• Five adaptations: Rocky 8, FIDO, Rocky 7, ATRV, K9• Most technology modules are at Level III• None are at Level IV or Level V (formally reviewed, documented, and open source)

0

5

10

15

20

25

30

35

Number of Technology

Modules

Level I Level II Level III Level IV Level V

Technology Modules Only

CLARAty Integration Levels•Level I – Deposited•Level II – Encapsulated•Level III – Refactored•Level IV – Formally reviewed•Level V – Open source and fully documented


Relevance to the Missions

Why is this work relevant to the missions?

• Provides a common environment for development, test, and comparison of advanced robotic technologies

• Provides an infusion path for robotics technologies into flight missions

• Demonstrates technologies on relevant robotic systems

• Makes research rovers viable test platforms for flight algorithms (e.g. navigation)

• Is robust to changes in rover hardware designs

• Can be easily adapted to flight and new research rovers


Measuring Success or Failure

We succeed IF we:

• Significantly reduce integration time of new technology software onto real robotic systems

• Support multiple platforms with different hardware architectures

• Provide a service that is enabling for technologists

• Simplify the development/integrate/debug/test cycle for current and next generation NASA rovers

• Have people other than the developers using and “like” the system

Date post:	14-Jan-2016
Category:	Documents
Upload:	damon-owen
View:	212 times
Download:	0 times

Unified Robotic Software Development using CLARAty Issa A.D. Nesnas Mobility and Robotic Systems...

Documents