+ All Categories
Home > Documents > How CIFE Does Translational Research for Industrial … CIFE Does Translational Research for...

How CIFE Does Translational Research for Industrial … CIFE Does Translational Research for...

Date post: 29-Apr-2018
Category:
Upload: dangduong
View: 251 times
Download: 4 times
Share this document with a friend
58
CIFE Research Questions and Methods How CIFE Does Translational Research for Industrial Sponsors 10/1/2009 Research Methods 1 John Kunz, CIFE
Transcript

CIFE Research Questions and Methods

How CIFE Does Translational Research for Industrial Sponsors

10/1/2009 Research Methods 1

John Kunz, CIFE

Construction Management Today

• Large unique projectsprojects

• Lots of– stakeholders – different types

of infoNeed to relate– Need to relate data

– Paper– Time for

coordination• Time and cost

Needs: implemented, tested theory w/• Power

10/1/2009 Research Methods 2

• Time and cost pressures • Generality

Research question at CIFE

• What is a theory, based on fundamental y,principles, that can be used to describe and explain or predict elements of (sustainable)

?design, construction and operations?• What is a way to operationalize a new theory

i t th d ?using computer methods?• How well does the theory describe and

predict reality; how general is it?predict reality; how general is it?

10/1/2009 Research Methods 3

Research methods

• Careful observation of practice– Embedded participation

Ethnographic interviews– Ethnographic interviews• Formal models of project elements, i.e., products,

organizations and processes ( ) ( G– Usually symbolic (non-numeric) + graphical (4D, Gantt, tables,

many specialized views)– Often computational

C f ll d i d lid ti t di t h id• Carefully designed validation studies to show evidence of power and generality– Yes: Study real projects (n); compare performance of innovative

d b li tiand baseline practices– No:

• Solve hard problems: every good engineer does it; not a “mouse”D th t d t b k i t

10/1/2009 Research Methods 4

• Demo that does not break is not a mouse• “One mouse is no mice”

Objective for Ph.D. research

• Make a believable claim that an innovative theorycontributes to knowledge in a particular subarea of AEC research

• Theory (scientific sense): analytic structure to that• Theory (scientific sense): analytic structure to that can be used to describe and explain or predict empirical observations– Universal requirement of the Ph.D. in all fields, all countries

10/1/2009 Research Methods 5

Translational Medicine

Phases:• Observe and prototype: explore needs, develops

potential treatments in basic laboratory research, and tests safety and efficacy, principally in randomized

CIF y y p p y

clinical trials. • Field intervention: examine how findings from clinical

science shown to be efficacious and safe treatments

FE Focu science, shown to be efficacious and safe treatments

established in phase 1 translational research, function when are applied in routine practice.

• Implement broadly: add necessary information to

us

• Implement broadly: add necessary information to convert treatments and prevention strategies, shown to be effective and cost-effective in Phase 2 Translational Research into sustainable solutions

IndustrFocus

10/1/2009 Research Methods 6

Translational Research, into sustainable solutions.– Based on Wikipedia

ry s

Translational research at CIFE

• Integrate: g– Baseline observations of practice– Theory that integrates fundamentals of

engineering, basic sciences, social and political sciences

– Results of field intervention studiesResults of field intervention studies • Goal: improve project (virtual) design and

construction in ways that measurably improveconstruction in ways that measurably improve project success and life cycle quality.

10/1/2009 Research Methods 7

CIFE “Horseshoe” Research Method

Metrics (performance),Doable?

Research M th d

Observed Problem

Intuition

scope (domain)

Generality

Power Testable?

MethodsTheoretical POD

Research Ei t i Questions

R Tasks: • Theory

Einstein: The only real

valuable thing is intuition.

ValidationClaimed

Evidence?• Model• Test

Legend:

ResultsClaimed

ContributionsPredicted Impact

10/1/2009 Research Methods 8

Leads toCompare

IntellectualMeritImpact

Example of Horseshoe in UseRyan Orr

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

y

Observed Problem Theoretical POD/ Gap RQs Data Source

Lack an integrated theoretical

It takes firms many years to evolve a good strategy. What

are they learning?

- Lack an integrated theoretical framework for why firms adopt

different entry & staffing strategies.

- Literature tells us strategy selection is discretionary.

(1) What strategies do firms use?

(2) How do they select these t t i ?

(1) Case studies w. 3 firms:Kone, Fluor, Hines

(2) Asia data(Melin quote about deepare they learning? selection is discretionary.

- Classic mgmt theory has been overlooked.

strategies? (Melin quote about deep understanding)

R d t OD'<- Identifies framework to <- Actors, scripts, set, props>

(1) GTRecommends a concrete OD strategy & show that not all strategies are appropriate for

all firms.

describe environment>-(1) Recognize set of strategies

that firms adopt- (2) Implies a KBV contingency

approach to strategy

, p , , p p- (1) Strategies: incr. KS, decr.

KD, incr. except handling, knowl. integration

- (2) Strategy = F(EMB, KD)

(1) GT(2) Inference

<copy the paper b/w US & Canada>

Predicted Impact Contribution Findings Data Analysis/Presentation

10/1/2009 Research Methods 9

Example of Horseshoe in UseClaudio Mourgues

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

g

10/1/2009 Research Methods 10

AgendaObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Motivating Engineering • ExamplesMotivating Engineering Problem

• Intuitions

p

• Properties• What: Research

Questions

Properties

• How: – Research Methods

Conclusions• Conclusions

10/1/2009 Research Methods 11

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Observed problem: – Some problem faced in practice– Motivates your research, not just another problem

t lto solve– May relate to one or more of product (technical),

organization and process limitso ga a o a d p ocess s

10/1/2009 Research Methods 12

Observed Motivating ProblemObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Embedded Measure of successparticipation method

• Spend time with practitioners (days to

• You find some theory to address fundamental moti ating iss es (laterpractitioners (days to

many months)• Identify a recurring

problem

motivating issues (later in intuition)

problem• Identify (complex)

problem context: staffing, no time, bad site conditions, …

• Identify fundamental

10/1/2009 Research Methods 13

ymotivating issue(s)

Example-1: ObservationSheryl Staub-French

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

y

Hard to (re)estimate costsW ll H i h 8’ 10 5’Wall Height: 8’ → 10.5’ Wall Rating: Unrated → Fire-Rated

D i i QTY Res. TotalProd.

Description QTY uom Res. Costs

Total Cost

Metal Stud Wall, Type A (using Crew C-1)Install Metal Stud 1550 LF 6.9 lf/hr $3,125 $3,513

Prod. Rate

Description QTY uom Res. Costs

Total Cost

Metal Stud Wall (Crew C-1 & Rolling Scaffolding)Install Metal Stud 1895 LF 4.3 lf/hr $6,000 $6,474Hang Drywall 5040 SF 65 sf/hr $6,978 $8,037Apply Tape 5040 SF 45 sf/hr $8,960 $9,464

Prod. Rate

10/1/2009 Research Methods 14

Hang Drywall 3840 SF 65 sf/hr $5,317 $6,123Apply Tape 3840 SF 45 sf/hr $6,827 $7,211Install Insulation 1920 SF 100 sf/hr $1,440 $2,304Framing for Openings 6 EA 75 sf/hr $450 $476

Install Insulation 2520 SF 100 sf/hr $1,890 $3,024Framing for Openings 6 EA 75 sf/hr $450 $476Frame Wall-Beam Intersec 12 EA 1 hr/ea $540 $552Apply Fire Caulking 48 LF 40 lf/hr $54 $222Cut Drywall 240 LF 60 lf/hr $180 $180

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Intuition

• Intuition:– How you want to approach the problem … your

focus on one or more of product (technical), organization and process limitsorganization and process limits

– Motivates and guides your library and internet search for enabling theoretical methods

– No need to justify it -- if it helps you make progress

10/1/2009 Research Methods 15

Intuition: Features drive costsObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Resources>Features <Objects Actions Cost

C t I t ti M

-- Property Set

Component Features

Intersection Features

Macro Features

• Fire-rating• Fire-rated• Height

Length

• Area• Bending Radius• Bottom Elevation

Curved

• Thickness• Top Elevation• Width

-- Feature Set

• Penetrations• Wall-Beam Intersection

• Dimension Change • Direction Change

• Length• Curved

10/1/2009 Research Methods 16

Wall Beam Intersection• Wall-Countertop Connections

Direction Change• Grouping(Similarity)• Openings

Type OfInstance OfImpacts

Legend

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Theoretical Point of Departure (POD): – Literature that relates to the motivating problem in

the focus of interest (based on intuition)C id ll th l t CIFE lit t– Consider all the relevant CIFE literature

– Be scholarly: consider all the relevant literature from journals, potential reviewers of your worko jou a s, po e a e e e s o you o

– Distinguish • Fundamental: a few theoretical constructs on which you

b ildbuild• Courtesy: many theoretical and practical references of

broad interest to you or potential readers

10/1/2009 Research Methods 17

Example POD: Work InstructionsObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Job Assignment Sheets Design principles for assembly instructions

Project’s 3D Model Company’s Process ModelsKey plan

• Step-by-step, one for each major step

(Oglesby et al., 1989)assembly instructions (Agrawala et al., 2003)

j Company s Process Models

Work area

Model view

view

Color coding

• Clear and explicit order

• Parts added in each step should be visible

• Mode of attachment

Activity Work InstructionKey planMethod to Generate Key plan

Details view

Color coding view

Mode of attachment should be visible

• Action diagrams rather than structural

A d id liWork area

Model view

Details view

Key plan view

Color coding view

Method to Generate Work Instructions from Product and

Work area

Model view

Details view

Key plan view

Color coding view

• Arrows and guidelines to indicate attachment

Work Packaging

18

Process Models(missing reference)

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Research Questions

• Research questions: specific questions that the• Research questions: specific questions that the research intends to (or did) address:– Answer gives you a contribution no matter what it

b h t h t b tmay be, e.g., how, not what or best– Relate to motivating problem and theoretical POD– Typically involveTypically involve

• A model, or “ontology,” which is a vocabulary in the computerReasoning method to anal e the inp t data• Reasoning method to analyze the input data given the model

• Framework that links the model and reasoning

10/1/2009 Research Methods 19

method

Example: Research questions, theoryObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

1. How do product features in a building product model1. How do product features in a building product model affect construction costs?

2 How can the relationship between product features and2. How can the relationship between product features and construction cost be formalized in computer models?

3 How can reasoning mechanisms leverage the3. How can reasoning mechanisms leverage the representation in (2) to automatically generate cost-loaded and resource-loaded activities?

10/1/2009 Research Methods 20Sheryl Staub-French, 19 December 2001

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research MethodsResearch Methods

• Research methods: Broad approaches to research t ktasks, e.g., – Hypothesis testing using statistical comparison of

a performance of baseline and innovative methodsp– Grounded theory to develop a conceptual

framework when none existsComputer demonstration of an existence proof– Computer demonstration of an existence proof that a conceptual model and analysis of it can be successfully implemented I f l th i t l– Informal, e.g., synthesize an ontology

– Computational modeling: develop a method

10/1/2009 Research Methods 21

Research Methods, TasksObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

How Research Methods:How – Research Methods:– Observe: e.g., Hathaway project – Computational model and statistical analysis of results p y

of use – Validate by analysis of charrette, retrospective or

prospective applicationprospective application– n(cases) ~100 Ideally; at least >= ~10

• What: Staub-French created– Feature and cost ontology (i.e., vocabulary in

computer)Model of effects of feature changes on costs

10/1/2009 Research Methods 22

– Model of effects of feature changes on costs

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

R Tasks: •Theory•Model• Test

• Research tasks: specific segments of the work to address the research questionsaddress the research questions

• Operationalize research methods, e.g.,– Collect survey data test a hypothesisy yp– Observe how practitioners use a traditional and

innovative method test the hypothesis that the two are the sameare the same

– Do retrospective or (ideally) prospective testing statistical evidence for power and generality of the underlying model reasoning and frameworkunderlying model, reasoning and framework

– Modeling or theory formation– Implement reasoning method and Framework

10/1/2009 Research Methods 23

g– Initial testing to validate the concept

Research tasks: Example-1

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Build

Research tasks: Example 1

• Build– Graphical product

model– Symbolic product

model– Symbolic process

ComponentFeatures

<F> and/or <O>

IntersectionFeatures

<F> and/or <O>Actions

<A>Symbolic process model (Cost analysis procedure)

P f t ti

Subcomp-onents<O>

Resource Activity• Perform retrospective & charrette studies

Activities

Resources Costs

ResourceSpecification

ActivitySpecification

10/1/2009 Research Methods 24

Resources<R>

Costs<C>

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

ValidationResults

• Validation results: quantitative data that you interpret as• Validation results: quantitative data that you interpret as evidence for the power or generality of the innovative method in comparison with traditional methods

10/1/2009 Research Methods 25

Validation methodsObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

ValidationResults

E id f d lit f• Evidence for power and generality of theory: “milk-stool” consistency

10/1/2009 Research Methods 26

TheoryObservation

Model Analysis (Simulation)

Validation methodsObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

ValidationResults

E id f d lit f• Evidence for power and generality of theory: “milk-stool” consistency

Swiss milk stool

10/1/2009 Research Methods 27

TheoryObservation

Model Analysis (Simulation)

Swiss milk stool

Validation study results: Example-1Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

10/1/2009 Research Methods 28Δ Theoretical Ideal – ACE Process Δ ACE Process – Traditional Process

Validation Method: Charrette studyObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

ValidationResults

• Realistic engineering problem: 1-2 hours for titi t “ l ”practitioner to “solve”

• Two studies, same testers– Conventional methodConventional method– Experimental method

• Measured performance metrics: time, cost, accuracy, l t i tcompleteness, consistency, …

• Null hypothesis: Performance same for two methods• Test results provide evidence of power and generalityTest results provide evidence of power and generality

of the feature and relationship representation and associated reasoning method claim that theory of each is a contribution to knowledge

10/1/2009 Research Methods 29

each is a contribution to knowledge

Form of a Claim to make a contribution to k l d

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

knowledge

• Based on theseBased on these• empirical data (test results) that• provide (verb)• evidence (information that an independent reviewer can

believe that helps confirm or deny a hypothesis or elucidate mechanisms) of)

• power (ability to solve a particular problem) and• generality (ability to solve a range of problems) of the

th ti l f k ( t l d l) d• theoretical framework (ontology or model) and • method (reasoning or analysis process), • I claim that <my> theory is a contribution to the knowledge of

10/1/2009 Research Methods 30

I claim that my theory is a contribution to the knowledge of <some subfield of AEC>

Example claimObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Claimed Contributions

• Claimed contribution: Theory that adds to the POD with evidence– Theory that adds to the POD with evidence

– Not data (normally)– Universal requirement for the Ph.D.

• Form of a “claim” for a contribution:– Data: validation data using a (Prospective) method show

improved performance of the innovative method in p pcomparison with traditional methods.

– Interpretation: I interpret these data as evidence of the power and generality of the innovative method. p g y

– Claim: Based on the evidence, I claim the underlying theory (innovative ontology, reasoning method and framework) as contributions to knowledge in the field of

10/1/2009 Research Methods 31

) gVirtual Design and Construction.

Steps in the Horseshoe Research MethodObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Practical contribution:• Practical contribution: – Your speculation about impact on practice of broad

application of the innovative methodpp– Normal and appropriate for engineering– Good for your career and job prospects – Not required for the Ph.D.

10/1/2009 Research Methods 32

Example-2: ObservationChuck Han

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

It is hard to ask:D thi d i• Does this design comply with building code?code?

• How do you know?

10/1/2009 Research Methods 33

IntuitionObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Intent is a crucial issue for compliance– Designer: rationale

f ffor forms– Code: requirements

of formsof forms

10/1/2009 Research Methods 34

Research questions, theory

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Research questions, theory

1. Can a generalized analysis framework be built to support automated architectural building code checking?

based on the design intent of building code– based on the design-intent of building code 2. What are the product model issues to support (1)

– Considering “IFC” standardCo s de g C s a da d– Reformulate IFCs to represent intent

10/1/2009 Research Methods 35Chuck Han, 5 October 1998

Research methods: Example-2

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Research methods: Example 2

• Build and testBuild and test– Graphical product

modelS b li d t– Symbolic product model

– Symbolic process model (Analysis procedure)

– GUIGUI• Retrospective,

Intervention studies

10/1/2009 Research Methods 36

Research Tasks

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Research Tasks

Research tasks – specifically how:Research tasks specifically, how:– Observe: Sunnyvale building permit group – Model and analyze (simulation) results y ( )– Validate

• Many what-if animations• Stanford career center (intervention study)

Research Tasks: Han created computer models ofP f k– Process – framework

– Product – building; wheelchair

10/1/2009 Research Methods 37

Intervention Study Method

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Intervention Study Method

• Tasks– Observe engineering case– Predict project performance: time, cost, quality, …

• With current plan• Following possible intervention (s)

– Intervene - at management discretion• Intervention ⇒ Evidence for “believability”• Small sample statistics w/o control• Small-sample statistics w/o control

10/1/2009 Research Methods 38

Research methods and tasks: Example-3John Haymaker

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Build

John Haymaker

• Build– Graphical product

model (CAD)– Formal Symbolic

Project Model as a Directed Acyclic Graph of dependencies between geometric views

• Perform retrospective validation studies (of WDCH)

10/1/2009 Research Methods 39

WDCH)

Example-3: WDCH Concert Hall Cantilever Test CaseObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

10/1/2009 Research Methods 40

Cantilevered Ceiling Panels Perspector GraphObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

10/1/2009 Research Methods 41

Task: Develop and test RepresentationObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Perspector

Perspective

Perspective

10/1/2009 Research Methods 42

Task: ReasoningObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

A Generic Perspector:- Analyzes the source Features in the source Perspective(s)

10/1/2009 Research Methods 43

y p ( )- Constructs dependent Feature(s) in the dependent Perspective- Relates the dependent Feature(s) and source Features(s)

Evidence For PowerObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Deck attachments

Interpretation• Interpretation by staff

Perspector• Perspector interpretation

10/1/2009 Research Methods 44

Evidence for Power of the Perspective Approach: Deck Attachment Retrospective Test Case

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

field welds 5 Perspective

Deck Attachment Retrospective Test Case

~ total attachments 100 WDCH Approach CommentsQuality Accuracy

- Found in test case 0 (shop weld)* 114Significant improvement over current practice is possible.

___________ ___

Found in test case 0 (shop weld) 114 - Missed in test case 86 (field weld)* 2 Completeness - Amount of detail

p pFurther improvement possible.

Automation could make creating more detail cost-effective.

Time to Specify - To conceptualize 0 sec. 10 hrs

To code Perspectors 0 sec 200 hrs

Re-use of Perspectors reduces programming time. Practice can

- To code Perspectors 0 sec. 200 hrs - To assemble graph 0 sec. 120 sec to Construct and Integrate - For project 140 hours no data

F t t d t 56

Once graph is specificed, representations can be constructed quickly

reduce conceptualization time.

10/1/2009 Research Methods 45

- For test case no data 56 secs constructed quickly.

Evidence for generalityObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

PerspectorPerspector method for

• Deck• Deck Attachments

• Cantilever Deck Attachment Perspector

conditions

Cantilevered Ceiling Panel Perspector

10/1/2009 Research Methods 46

Cantilevered Ceiling Panel Perspector

Types of Validation studiesObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Existence demonstration; stress test: not evidence!• Retrospective: Look at past cases; adjust theory to p p ; j y

get good results; analyze to identify– Quality of results: power– Context in which quality is good: generality

Mor Context in which quality is good: generality

– N >= 10 (hopefully)• Charrette: N >= 10 (hopefully)

Prospective:

re

• Prospective: – Look at current cases; do not adjust theory; analyze power

and generalityN 100 (hopefully)

Pow – N ~ 100 (hopefully)

• Intervention: – Look at a case; do not adjust theory; sponsor intervenes

b d l i

er

10/1/2009 Research Methods 47

based on analysis– N ~ 1 (typically)

(Empirical) Validation methods

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

(Empirical) Validation methods

Validation study methods: 1 2 3

• Retrospective test case(s)Retrospective test case(s)

• “Stress” test

• Engineering test cases

• Prospective (engineering) test case(s)Prospective (engineering) test case(s)

• Laboratory study - Charrette method

10/1/2009 Research Methods 48

• Field intervention study

Conclusions about Validation

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Conclusions about Validation

• Serious validation studies are a realistic objective for university research

• Validation motivates studentsS• Skills– Define measurable performance objectives

Observe engineering practice– Observe engineering practice– Collect data: Formal instrument– Analyze data– Interpret results

10/1/2009 Research Methods 49

Contributions:Observed Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

Based on evidence of generality and power, Haymaker claimsthat the underlying mechanismsthat the underlying mechanisms of his method are a contribution to knowledge, specifically

1. Project Model Ontology: dependencies, views, Features, relationshipsrelationships

2. Perspector: reasoning to formally construct a newformally construct a new Perspective from other Perspectives

10/1/2009 Research Methods 50

3. Graph Manager:

SummaryObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Motivating Observations: – Engineering practice still has many aspects of a craftg g p y p– Careful observation of practice can inform and test research

• Intuitions:Engineering Principles can guide practice– Engineering Principles can guide practice

– Implementing principles in computer can lead to testable theoretical models

• Research Methods: Horseshoe method• Research Methods: Horseshoe method• Research questions:

– What fundamental principles characterize civil engineering design d t?and management?

– How can we operationalize new ones in the computer– How how good are they?

10/1/2009 Research Methods 51

SummaryObserved Problem

Theoretical POD

Research Questions

ValidationResultsClaimed

ContributionsPredicted

Impact

Intuition

R Tasks: •Theory•Model• Test

Research Methods

• Build Theory: – Processes (e.g., estimate costs, check ADA, plan, create ( g , , , p ,

reasoning, …)– Products (Buildings, components, systems),

• Model: Object-oriented symbolic and graphical models ofModel: Object oriented symbolic and graphical models of products and processes

• Validation to show evidence of power and generality: Many CIFE Ph D projects– Many CIFE Ph.D. projects

– Integration into CEE curriculum: DCI, CEE 100, 111, 215, 222, 241, 242, VDC internshipsOther Stanford departments: CS Medicine Symbolic Systems– Other Stanford departments: CS, Medicine, Symbolic Systems

10/1/2009 Research Methods 52

Skills for success

• Good engineer: design analyze manage• Good engineer: design, analyze, manage• Research methods• Communication

– Written/Oral– Programming

Colleagues sponsors stakeholders– Colleagues, sponsors, stakeholders• Ability to identify, find examples in the chaos of

practice, propose “gold standard” methods• Integrated use of quick-response, careful analysis,

reflection

10/1/2009 Research Methods 53

Research: Benefits for Students

• Real engineering problems• Synthesize academic, work experience• Methods apply to big questions:

– What fundamental principles characterize civil engineeringWhat fundamental principles characterize civil engineering design and management?

– How can we operationalize new ones in the computer– How how good are they?How how good are they?

• Research methods necessary for careers in practice & researchInteresting• Interesting

• Support

10/1/2009 Research Methods 54

Research: Benefits to Industry

• Industry suggests problems– Helps identify big, pre-competitive problems

• University perspective adds value – Broad (“integrated”)– Theoretical: consistent, repeatable, exportable processes– Modern methods

• Unique opportunity to address hard problems– Focused effort of gifted graduate students working on industrial-class

problemsproblems• Results have been of interest

– Emergence of VDC and BIM in practice now big!– Applications (e g org analysis 4D)Applications (e.g., org analysis, 4D)– Methods (e.g., VDC, OOP, 4D, testing, POP, ICE)– Successful evangelists within companies– (Informal) benchmark evaluation of company processes wrt global best

10/1/2009 Research Methods 55

( ) p y p gpractices

Research Program Status

• Research questionsWhat fundamental principles characterize civil engineering design and management?How can we operationalize new ones in theHow can we operationalize new ones in the computerHow how good are they?

• Research methodsCareful observation of practiceSymbolic (non numeric) computer models ofSymbolic (non-numeric) computer models of products and processesEvidence of power and generality

10/1/2009 Research Methods 56

Research Program Status

Progress P ti l ( VDC lt )• Practice: real (see VDC survey results)

• Academic: – Questions and methods now well-validated by CIFE

research since 1988, especially last 10 years– CIFE now 20! New competition emergingCIFE now 20! New competition emerging– CIFE methods now part of Stanford CEE culture:

Integration requirement; DCI, CEE 100, 111, 115, 222, 241, 242, 243, …

– External recognition: “CIFE is the premier academic center for VDC research”

10/1/2009 Research Methods 57

center for VDC research– External influence: VDC, validation now real issues

Comments, Concerns

+ Δ• •

10/1/2009 Research Methods 58


Recommended