1

Biomedical Engineering Key Content Survey - Results from Round One of a Delphi Study

David W. Gatchell and Robert A. LinsenmeierVaNTH ERC for Bioengineering Educational Technologies and Northwestern University

Whitaker Foundation Biomedical Engineering Educational SummitMarch, 2005

Supported by NSF EEC 9876363

2

Why conduct a BME key content survey? Motivation and potential benefits

Motivation Establish an identity for undergraduate Biomedical

Engineers Improve communication between academic BME programs

and industry Academia – Inform industry of the knowledge, skills and

training of BMEs Industry – Inform academia of the knowledge, skills and

training expected

Benefits More industrial positions for BMEs Each graduate does not have to explain curriculum Recognition that BME degree is ideal preparation for at least

some industrial positions.

3

In General: An iterative process for collecting knowledge from, and disseminating

results to, a group of experts Four steps (repeat steps #2 and #3 to attempt to reach consensus)

1. Develop a set of questions on a topic. 2. Experts give opinions on topics; suggest new ideas that were missed3. Explore and evaluate inconsistencies uncovered in step 24. Disseminate findings, or revise questions and go back to 2

Key point is that experts remain anonymous

Our Study: A set of three surveys Round 0: Select concepts from VaNTH taxonomies; reviewed by domain

experts Round 1: Survey BME industrial representatives and faculty. Asked

participants to rate relevance of concepts important for ALL undergrads in BME, and make suggestions of concepts missed

Round 2: Refine and update list of concepts and resubmit to the above groups for further evaluation

Round 3: Question proficiencies expected (e.g., using Bloom’s Taxonomy)

Delphi study - Overview

4

Utilized an online survey tool to query ~274 concepts: Eleven bioengineering domains (including design) Physiology, cellular biology, molecular biology and genetics,

biochemistry Mathematical modeling, statistics, general engineering skills (e.g.,

computer programming) Survey divided in two parts, each with half the domains:

Total number of participants, n = 136 Part one: Academia – 42, Industry – 25 Part two: Academia – 35, Industry – 23

Participants were asked to: Provide demographic information

Employer, Job Title, Responsibilities, Years of Experience Self-assess level of expertise in each domain (e.g., Biomechanics) Rate the importance/relevance of each concept to a BME core

curriculum Suggest concepts that had not been included

Overview of the key content survey, round one

5

Concepts rated on 5 point Likert Scale 1- very unimportant for all BMEs 5 – very important for all BMEs

Mean ratings across concepts similar for industry and academia Academia (n=77) mean and SD rating: 3.71 ± 0.52 Industry (n=48) mean and SD rating: 3.75 ± 0.41

Domains Investigated: Bioinformatics, bioinstrumentation, biomaterials, biomechanics, biooptics, biosignals and systems, medical

imaging, thermodynamics, transport (fluid, heat, mass) Cell biology, biochemistry, molecular biology and genetics, physiology Statistics, general engineering

Overview of the key content survey

6

Some concepts included as “Ringers” -Expected to have low rating

Concept Rating (Academia)

Rating (Industry)

Statistical Physics (e.g., Bose-Einstein statistics; Fermi-Dirac statistics)

2.32 2.58

Statistical Physics (e.g., Partition function; statistical representation of entropy;

population of states)

2.82 2.58

Comparative Genomics (e.g., ortholog and paralog genes; gene fusion events)

2.50 2.94

Dynamical Instability and Chaos 2.59 3.11

Unsteady state mass diffusion equation (e.g. Fick’s second law; production and

consumption; boundary conditions; different geometries; multiple layers)

3.42 3.29

All except unsteady state mass diffusion equation met expectations

7

Some concepts included in more than one domain to check consistency of response

Two values shown are ratings when concepts are included in different domains

Generally good agreement, but rating sometimes depended on context

Concept Ratings (Academia)

Ratings (Industry)

Databases - Interfaces and Structures (e.g., MySQL, relational tables, simple queries, PERL,

CGI, DBI)

2.29/2.66 3.22/3.68

Signal Processing to Reduce Noise (e.g., signal-to-noise ratio; signal averaging)

4.24/3.83 4.17/4.14

Properties of Systems (e.g., boundary, surroundings, universe)

3.88/3.97 3.63/3.88

Electrochemical Potential, Nernst Potential, Fick's Law

4.09/4.23 3.54/4.00

8

Concept RatingHypothesis Testing (e.g., paired and un-paired t-tests; chi-square test) 4.69

Principles of Statics (e.g., forces; moments; couples; torques; free-body diagrams)

4.68

Descriptive Statistics (e.g., mean, median, variance, std deviation) 4.63

Circuit Elements (e.g., resistors, capacitors, sources, diodes, transistors, integrated circuits)

4.56

DC and AC circuit analyses (e.g., Ohm's and Kirchoff's laws) 4.56

Mathematical Descriptions of Physical Systems (e.g., functional relationships, logarithmic, exponential, power-law; ODEs; PDEs)

4.54

Strength of Materials (e.g., stress, strain; models of material behavior) 4.53

Pressure-Flow Relations in Tubes and Networks (e.g., flow rate = [change in pressure]/resistance; Poiseiulle relation; Starling resistor) 4.51

Measurement concepts (e.g. accuracy, precision, … 4.50

Regression analysis 4.49

Forces and pressures in fluids (e.g. shear, normal, surface tension… 4.49

Results: Highest rated eng’g concepts – AcademiaOrange concepts are from statistics and general engineering

9

Results: Highest rated eng’g concepts – IndustryOrange concepts are from statistics and general engineering

Concept RatingDescriptive Statistics (e.g., mean, median, variance, standard

deviation)4.76

Measurement Concepts (e.g., accuracy, precision, sensitivity; error analysis - sources, propagation of error)

4.71

Hypothesis Testing (e.g., paired and un-paired t-tests; chi-squared) 4.65

Probability Distributions (e.g., normal, Poisson, binomial) 4.62

Strength of Materials (e.g., stress, strain; models of material behavior)4.57

Fundamental Properties of Polymers, Metals and Ceramics 4.50

Product Specification (e.g., requirements, design, reliability, evolution/tracking of the product)

4.45

Principles of Statics (e.g., forces; moments; couples; torques; free-body diagrams)

4.43

Mechanical Properties of Biological Tissues (e.g., elastic; viscoelastic, hysteresis, creep, stress relaxation)

4.43

Data Acquisition (e.g., sampling rates and analog-digital conversion; Nyquist criterion; aliasing) 4.39

10

Results: lowest rated concepts some from “Ringers”

Academia

Databases - Interfaces and Structures (e.g., MySQL, relational tables, simple queries, PERL, CGI, DBI)

2.29

Statistical Physics (e.g., Bose-Einstein and Fermi-Dirac statistics) 2.32

Artificial Intelligence (e.g., artificial neural networks, fuzzy logic) 2.33

Analysis of Phylogenetic Trees, Molecular Evolution 2.47

Comparative Genomics (e.g., ortholog and paralog genes; gene fusion events)

2.50

Structural Prediction and Molecular Design 2.53

Industry

Statistical Physics (e.g., Partition function; statistical representation of entropy; population of states)

2.58

Statistical Physics (e.g., Bose-Einstein; Fermi-Dirac statistics) 2.58

Artificial Intelligence (e.g., artificial neural networks, fuzzy logic) 2.78

Storage Instruments and their properties (e.g., tape, disk, memory) 2.89

Comparative Genomics 2.94

Root Locus Plots (e.g., definition, properties, sketching) 2.95

11

Results: Industry - Academia agreementDistribution of mean ratings of all concepts

2.00

3.00

4.00

5.00

2.00 2.50 3.00 3.50 4.00 4.50 5.00

Academia

Ind

us

try

Concept Ratings

Most concepts rated highly. Few ringers in survey. All traditional domains had some highly rated concepts. Cutoff level for inclusion in recommended undergrad curriculum

still to be determined on basis of further analysis and round two.

12

Results: Industry – Academia AgreementDifferences in means (A-I)

-1.25

-1.00

-0.75

-0.50

-0.25

0.00

0.25

0.50

0.75

0 50 100 150 200 250Concept #

Dif

fere

nce

s in

Mea

n R

esp

on

ses

Design

13

Results: Discrepancies in design concepts

Rankings - BME Design Concepts (A comparison of opinions from Academia and Industry)

1.00 2.00 3.00 4.00 5.00

Human Factors Issues/FDA

Computer-Aided Design Considerations

Risk Analysis/Hazard Analysis

Software and Process Design Considerations

Software for Design and Project Management (e.g.,flowcharting; Gannt and PERT charts)

Design for Manufacturing and Assembly

Decision Matrix Approaches to Initial Design

Mean Ranking (all participants)

Industry

Academia

14

Results: A comparison of general engineering concepts

1.00 2.00 3.00 4.00 5.00

Measurement Concepts (e.g., accuracy, precision, sensitivity;error analysis - sources, propagation of error)

Estimation and Order of Magnitude Calculations

Competency with (at least) One Programming Environment(e.g., Matlab, Mathematica, C, C++, FORTRAN)

Generalized Ohm's Law (i.e., driving force-flow-resistanceconcept)

Numerical Differentiation and Integration

Scaling and Dimensional Analysis

Familiarity with Multiple Computing Platforms (e.g., Windows,Macintosh, LINUX, UNIX)

Artificial Intelligence (e.g., artificial neural networks, fuzzylogic, etc.)

Databases - Interfaces and Structures (e.g., MySQL, relationaltables, simple queries, PERL, CGI, DBI)

Mean Rating (all participants)

Industry

Academia

Ringer

Significant Deltas

15

2.5

3.0

3.5

4.0

4.5

5.0

2.5 3.0 3.5 4.0 4.5 5.0

Rating of Concept - Academia

Ra

tin

g o

f C

on

ce

pt

- In

du

str

yBiochemistry

Cell Biology

Molecular Biol.

Bioinformatics

Unity slope

Results: Biology Domains

Good agreement on the whole

All biology areas important, but industry sees molecular biology as being more important than academia

Bioinformatics generally scored low, but industry feels that it is more important than academia does

16

Concepts Academia Industry Academia - Industry

Flow of Genetic Information (i.e., DNA to RNA to Protein)

4.5 4.1 0.44

Methods for Determining Macromolecular Structure (e.g., NMR...)

3.5 4.2 -0.70

DNA Microarrays 3.4 3.8 -0.42

Biological Networks (e.g., genetic networks...)

3.2 3.7 -0.46

Structural Prediction and Molecular Design (e.g., homology modeling and

prediction of macromolecular structures and interactions)

2.5 3.3 -0.72

Results: Largest biology discrepancies

17

2.5

3.0

3.5

4.0

4.5

5.0

2.5 3.0 3.5 4.0 4.5 5.0

Rating of Concept - Academia

Rat

ing

of

Co

nce

pt

- In

du

stry

Physiology

Unity slope

Results: Physiology (82 concepts)

Very large span within domain

Generally good agreement

Cardiovascular, neural, cellular physiology concepts rated highly

Digestive, renal, parts of endocrine rated low

18

Results: Largest physiology discrepancies between academia and

industryConcepts Academia Industry Academia

- Industry

Cellular Anatomy (e.g...) 4.56 4.14 0.41

Membrane Dynamics (e.g....) 4.44 3.95 0.49

Processes of the Kidney (e.g....) 4.26 3.85 0.41

Renal Filtration (e.g...) 4.03 3.60 0.43

Homeostasis of Volume and Osmolarity

4.03 3.50 0.53

Water Balance and Urine Concentration 3.73 3.25 0.48

Platelets and Coagulation (e.g....) 3.21 3.75 -0.54

Sodium Balance and the Regulation of ECF Volume (e.g....)

3.76 3.15 0.61

19

Results: Should the following foundational courses be required?

Comparison of Responses - Industry and Academia

Calculus - Differential, Integral and Multivariate

Vector Calculus

Linear Algebra

Ordinary Differential Equations

Chemistry - General

Chemistry - Organic (Semester One)

Chemistry - Organic (Semester Two)

Physics - Mechanics

Physics - Electricity and Magnetism

Physics - Waves and Optics

Industry

Academia

"NO" "YES""UNSURE"

Agreement that second semester organic chemistry is not universally required; some uncertainty about one semester

20

Universities represented in round one of the survey

1. Arizona State University*2. Binghampton University3. Boston University*4. Columbia University5. Devry Institute of Tech6. Duke University*7. Florida International University8. IIT9. Johns Hopkins University*10. Marquette University*11. Milwaukee SOE*12. MIT13. NJIT14. NC State University*15. Northwestern University*16. RPI*17. RHIT18. Stanford University19. Syracuse University*

20. SUNY – Stony Brook

21. Tulane University*

22. University of Akron*

23. University of Cincinnati

24. University of Illinois – UC*

25. University of Iowa*

26. University of Memphis

27. University of Michigan

28. University of Minnesota*

29. University of Pittsburgh*

30. University of Rochester*

31. University of Texas – Austin*

32. University of Toledo*

33. Vanderbilt University*

34. VCU*

*ABET Accredited – 21 of 37 Accredited Programs Participated

21

Companies and industrial expertise represented in round one of the survey

Companies Represented Abbott Laboratories AstraZeneca Baxter Healthcare Boston Scientific Cardiodynamics Cleveland Medical Devices Datasciences, International Dentigenix, Inc. Depuy, a Johnson and Johnson Co. ESTECH Least Invasive Cardiac

Surgery GE Healthcare Intel, Corp. Materialise, Inc. Medtronic, Inc. Tyco Healthcare Underwriter Laboratories

Areas of Expertise Biomaterials Biomechanics Bioinformatics Bioinstrumentation BioMEMS Biotransport Cellular Biomechanics Computational Modeling Control Systems Engineering Fluid Mechanics Medical Devices Medical Imaging Medical Optics Signal Processing

22

Conclusions More analysis is required to:

Investigate variation of opinions for individual topics Correlate ratings with expertise levels Eliminate contextual bias Incorporate concepts omitted from first round

BUT, preliminary results have shown that: “Consistency checks” validate data Generally good agreement between industry and academia Industry and academia disagree on a significant number of

Design concepts Industry highly values knowledge of statistics and

probability Core biology should include all domains assessed

23

Conclusions

Remaining issues Determine level of significance for deciding what concepts

can be dropped from core curriculum Determine significance of differences between industry

and academia Launch second round – by summer

Full matrix of results by concept will be posted on www.vanth.org/curriculum