121 April, 2005

Flight Test Principles Applied to First Robotics: Simple Practices to Help Your Team Succeed

Steven R. JacobsonNASA Dryden Flight Research CenterFlight Controls and Dynamics Branch

steve.jacobson@nasa.gov

221 April, 2005

Steve Jacobson’s Background

Professional Background• BS Aeronautical Engineering Embry-

Riddle Aeronautical University 1992• MS Aeronautics and Astronautics

Purdue University 2000• Daedalus Research, Inc. 1993 – 1997

Design and flight test of small UAVs• NASA Dryden Flight Research Center

1997 – present. Development and flight test of control systems for vehicles ranging from 12’ UAVs to X-37

Autonomous Formation Flight

F-15 Inner Loop Thrust Vectoring

X-37 Approach and Landing Test

Vehicle

Networked UAV Teams (Aerial robots)Advanced Aeroelastic Wing

321 April, 2005

Similarities and Differences Between Coaching FLL and High School Robotics

Similarities between FIRST LEGO League (FLL) and First Robotics (FR) •Short time span•Student Mentor involvement•Coaching/Mentoring strategies very similar•Kids have energy that needs focused in the right direction•The more mentors, the merrier

Differences between FLL and HS• FLL is intended to let the

students do all of the work• First HS robotics is students

and mentors working side by side.

While most of my experience is with Mentoring FLL,

The principles in this presentation apply to both FIRST LEGO League and FIRST High School robotics

421 April, 2005

Steve Jacobson’s Background

FIRST Background• Mentor for Team 585, Tehachapi

High School – 2002, 2003, 2005

• First LEGO League– 2002 City Sights

• Golden Hills School Robotic Eagles

• 5th and 6th grade team• 2nd place at Coppertop

regional Qualifier• 6th place at LEGOLand

– 2003 Mission Mars• Valley Oaks Charter School• 4th, 5th and 6th grade team• 1st place at LA qualifier• 4th place at LEGOLand

– 2004 No Limits• Valley Oaks Charter School• 4th, 5th and 6th grade team• 1st place at Newhall qualifier• 2nd place at LEGOLand

521 April, 2005

Motivation for This Presentation

Each year – My teams have been mostly rookies– Most of my FLL teams have consisted of 4th and 5th grade

students– I start from scratch at the beginning of the season– I run my teams with the philosophy

• The kids come up with the ideas• The kids design, build and program• Let the kids make the mistakes and learn the lessons that are

obvious to the coaches

– Our robots are not very sophisticated– My teams have performed well in competition against older

teams. – I am amazed at what these kids can accomplish in FIRST.– There are accusations of coaches “doing all the work”

Each year is ask myself “What is different between my teams and other teams that makes us successful?”

621 April, 2005

Presentation Abstract

Many robotics teams are coached by people that have a passion for the sport but lack a technical background. Many coaches and mentors have a technical background, but may not have expertise in conducting systems integration, test and operations. Flight testing aircraft and preparing a robot for competition are very similar activities. Both require complex planning, testing, and tying up of loose ends. The flight test community has developed processes for accomplishing flight test and these processes can be applied to FIRST robotics. These include the systems engineering process, configuration management, risk/hazard analysis, system integration, system testing, and operations. When applied they can make an ordinary team into a great team. This paper is intended to help team leaders with technical and non-technical backgrounds prepare there teams at each step in the season. It maps the technical processes applied by the flight test community into simple concepts that can be understood by students and mentors alike. Most of the examples come from my FLL robotics experience but can be applied to high school robotics as well.

721 April, 2005

Johnson Space Center Training

• JSC is NASA’s Lead center for manned space flight• New project managers at JSC are assigned to

mentor a FIRST High School Robotics team• Why?

– Six week systems engineering cycle– Complex problem with infinite solutions– Limited budget– Untrained and unskilled workforce

• If a new project manager can pull this job off with these constraints, they can see what it takes to get any job done at NASA

• NASA Dryden analog: Working with first and applying simple engineering principles in a short time frame has helped me become a better engineer.

821 April, 2005

Phases of a Project

• Project Planning• Design/Building• Testing• Operations/Execution

Systems Engineering Vee

921 April, 2005

Phases of a Project

• Project Planning• Design/Building• Testing• Operations/Execution

1021 April, 2005

Project Planning: Team Leadership

NASA Flight Test leadership• Team leadership and management

skills are the most important skills for the leader

• Intimate knowledge of the technical details is required, but not the most important.

• Empower the technical leaders to get the job done

FIRST team Leaders• The RCA winners are the teams that

have strong student leadership. Team members that focus mostly on planning, delegating and making sure that the details get done.

• Require your parents to get involved

FIRST Mentors (Technical leaders)• Understand the technology

– FLL: LEGO structures and Programming

– FR: C, Fabrication, Pneumatics, etc… • Communicate this knowledge to the

students• Know the game• Know the rules

– Assign a mentor to rule checking

1121 April, 2005

Project Planning: Trained Workforce

FIRST Trained Workforce: Train the students in the technology• FR: Get your team members to trained before the season starts (C

programming, CAD, Pneumatics, drive trains etc.)– Use past years robots or fall to get the students comfortable with the technology – Find a school in your region that offers FIRST Robotics training in the fall– Start a robotics course at your high school– Teach your students how to use sensors

• Go over the game play and rules with your team so everyone understands them.

– Go for the awards not based on robot performance

1221 April, 2005

Project Planning: Trained Workforce

FIRST Trained Workforce: Train the students in the technology• FIRST LEGO League training: Go over the technology with

the students– Use the links on the FIRST web site

http://www.usfirst.org/jrobtcs/flg_tm.htm• Team building• Mindstorms and Robolab Programming

– Teach them how to build robust structures with LEGOs– Burn 10 copies of the programming CD and give it to each team

member• Sensors are your best friend in FLL

– Battery voltage and motor performance vary from hour to hour– Sensors increase repeatability of your robot– Rotation sensor is the most useful sensor in the kit.

• Rotation sensor for turns: Put the rotation sensor on the side that is on the outside turn most of the time

1321 April, 2005

Project Planning: Requirements development

Strategy For the Robot

Robot Base (Drive

System)

Mechan-isms

Software&

Sensors

Speedversus Control

Grasping orHandling

mechanism

TouchLight

Or rotation

Tank Drive Or

Steering

PneumaticsOr motors

Dead Reckoning

or feedback

“Begin with the end in mind”Stephen Covey

1421 April, 2005

Tools for developing requirements

Flight test tool FIRST Robotics First LEGO League

Determine the strategy for your game

•Stewbot•Risk analysis

•Drive a mock robot around the playing field•Look risks analysis associated with each mission

Develop a “Concept of Operations” for each phase of operation. Sketch it out

This will help identify synergy between the missions or modes of operation

•How will you score?•How will you defend?•What kind robustness will the design require?•Robot transportation•Sensor placement

•Go through each mission and strategize several ways on how you will accomplish •Determine what missions you may not want to attempt and focus on the others•Determine the missions that you can combine with others

1521 April, 2005

Tools for developing requirements

Flight test tool FIRST Robotics First LEGO League

Get all of the stakeholders involved

This will prevent you from forgetting important requirements and designing something unobtainable

• Drive system group• Mechanism group • electronics group• sensors group• programming group

• Builders• Programmers • Testers

Identify the interfaces

Defining interface requirements can allow for independent development of systems with integration towards the end of the project.

•Mechanism interface to robot base • Sensor locations•Allow for enough spikes and victors•Special software requirements•Don’t forget access to the robot controller

•Interchangeability of mechanisms•Sensor integration

–Touch sensor–Rotation sensor–Light sensor

•Software training on how to use the sensors

1621 April, 2005

Requirement example for FLL MissionsMechanisms Sensors Wheel or

trackCombination with other missions

CD Fork lift

Bumper

rake

Rotation Wheel or track Glasses

Food Delivery Forklift

Rack

Slide

Rotation

Touch

Wheel Chairs

Chairs Rack/bumper

Forklift

Rotation

Touch

Wheel or track Food

Glasses Hook

Forklift

Rotation Wheel or track CD

Bus Stop Hook or rake

Forklift

Rotation

Light

Touch

Wheel or track Gate, Chair

Pet food

1721 April, 2005

Requirements example: results

• Integrated forklift • Attachable rakes hooks and other

mechanisms• Integrated rotation sensor• Removable light sensor

• Robot required only removal of mechanisms during competition.

• Mechanisms were designed to used for multiple missions

The Requirements design took three practices. All requirements cameFrom the students and before we build the robot we knew exactly

what needed to go on it.

1821 April, 2005

Project Planning: Risk Analysis

Risk table High Risk Medium Risk Low Risk

High payoff Climb the stairs Basketball hoop

Medium payoff Food on table

Gate

Bus Stop

Glasses

Chairs

Low payoff Feed the pets CD

Risk Level

Pay

off

1921 April, 2005

Phases of a Project

• Project Planning

• Design/Building• Testing• Operations/Execution

2021 April, 2005

Design/building phase

• Subsystem Requirements Development• Design • Trade studies/prototyping• Configuration Management

2121 April, 2005

Design/Building: Subsystem Requirements

• Continue on with the requirements development as in the first phase for the subsystem

• Continually look at the interfaces for requirements changes• Have regular “standup” meetings to manage the

configuration• Trade studies:

– Allow competing designs and let the team members decide which is the best design

– Allow the team members to make mistakes. That is how they learn.

2221 April, 2005

Design/Building: Configuration Management

• Make it well understood at the beginning of the project that if you have any changes to your part of the design that you inform the rest of the team.

• Example: A change in a mechanism could effect the robot base, sensor requirements or a software change

• FIRST Robotics: – Hold a regular Configure Control Board (CCB): Discuss any

proposed configuration changes (Major or minor) at each meeting or each time a change is desired

• All groups (stakeholders should be represented)• FIRST LEGO League:

– Instruct team members that they should communicate changes to coaches and other members

– Use a configuration control box where you keep all of the mechanisms (Prevent unintentional disassembly)

– Take digital photographs of the robot and mechanisms to document the construction

– Backup programs before you make a change and after every practice

2321 April, 2005

Phases of a Project

• Project Planning• Design/Building

• Testing• Operations/Execution

Flight research separates “the real from the imagined,” and

makes known the “overlooked and unexpected.”

– Dr. Hugh L. Dryden

2421 April, 2005

Testing, Testing, Testing

• For every hour of flight test there are hundreds or thousands of ground tests

• Testing is where you learn the most about your strengths and weaknesses

• With enough time you can make modifications to improve your robot

• Testing increases the confidence of your team in themselves and their robot

• Plan to start testing with 1-2 weeks to spare• Go to a scrimmage match

Testing is probably one of the most important steps in development. It is usually overlooked and never

given enough time.

2521 April, 2005

Testing: Guidelines

• Write a test plan• Perform end-to-end testing of your robot • Robustness testing: Intentionally try to break the robot to

see if it will withstand the rigors of competition• Perform testing In the order you intend to operate it

– First Robotics: Test the autonomous mode followed by the manual mode

– FIRST LEGO League: Test in the order you intend to operate.• Statistical Process Control

– Run a mission 10-15 times without changing anything– Take notes on the successes and failures– Determine the failure modes based on observations – Modify the robot and repeat the tests

• Regression testing: If you change one thing, however minor, retest everything all over

2621 April, 2005

Phases of a Project

• Project Planning• Design/Building• Testing

• Operations/Execution

2721 April, 2005

Operations and Execution: Flight Cards

• Flight cards include– Checklists for setup before

competition– Steps in which to execute the

missions– Contingency plans

• Flight cards should be used– To prepare for a competition– During practice at the

competition– Only as a reference in case

you “Freeze”

2821 April, 2005

Operations and Execution: General Guidelines

• Practice, Practice, Practice – The flight test analogy is simulation

• Resist the urge to change spontaneously

• “Brief what you fly and fly what you brief”

2921 April, 2005

Conclusion

• Many analogies between getting an airplane ready to fly and a robot ready to play.

• Not every principle may be applicable to your team, but simple discipline in these areas will help your team.

• Most important areas (If you had to only do a few things)– Make sure you have mentors that can teach the

technology– Let the kids “stub their toes”– Spend adequate time on requirements development

early on– Configuration management– Test, Test, Test – Use “Flight cards”– Practice, Practice, Practice