NDA FORUM “Enabling the Potential of Non-Deterministic Approaches ” A Joint Industry/Academia/Government Workshop Atlanta, April 2, 2000 Recommendations of the AIAA Non-Deterministic Approaches Working Technical Group Edited by Dr. S. Singhal and Dr. D.M. Ghiocel TECHNOLOGY BASE: Non-Deterministic Approaches Non-Deterministic Approaches (NDA) are predictive tools which simulate naturally-occurring uncertainties inherent in any process, hierarchically coupled with the physics, and integrated with existing deterministic models resulting in real-life modeling along with all the pertinent information necessary for realistic decision making. At this time, NDA offers to our engineering community a strong technical basis for developing a new, cost-effective, high-performance, robust industrial design. Using NDA the key aspects related to technical performance, safety and cost of a design can be rationally integrated. Faster, better, cheaper and safer, in a word a more affordable engineering design can be developed. The new NDA-based engineering design addresses all the performance related aspects of a product design, including manufacturing and maintenance. Such a NDA-based technology will largely increase the cost-effectiveness of product design, especially by reducing drastically the amount of lab and field experiments. The use of NDA greatly enhance the engineering prediction capabilities when significant uncertainties are present in the operational environment and/or material complex behavior. For the new advanced high-tech designs, with new materials, higher performances and harsher environments, for which no past engineering experience exist, the use of a NDA technology for investigating the design robustness, reliability and cost-effectiveness represents a key factor of the US high-tech development for successfully competing on the international market in the new created condition of world economy globalization using the internet technology. BACKGROUND: NDA-WTG Identity and Planned Activities The NDA-WTG is a multi-organizational working group comprised of representatives from all professional organizations with interest in NDA. The AIAA (American Institute of Aeronautics
Transcript
NDA FORUM
“Enabling the Potential of Non-Deterministic Approaches ”
A Joint Industry/Academia/Government Workshop
Atlanta, April 2, 2000
Recommendations of the AIAA Non-Deterministic Approaches Working
Technical Group Edited by Dr. S. Singhal and Dr. D.M. Ghiocel TECHNOLOGY BASE: Non-Deterministic Approaches Non-Deterministic Approaches (NDA) are predictive tools which simulate naturally-occurring uncertainties inherent in any process, hierarchically coupled with the physics, and integrated with existing deterministic models resulting in real-life modeling along with all the pertinent information necessary for realistic decision making. At this time, NDA offers to our engineering community a strong technical basis for developing a new, cost-effective, high-performance, robust industrial design. Using NDA the key aspects related to technical performance, safety and cost of a design can be rationally integrated. Faster, better, cheaper and safer, in a word a more affordable engineering design can be developed. The new NDA-based engineering design addresses all the performance related aspects of a product design, including manufacturing and maintenance. Such a NDA-based technology will largely increase the cost-effectiveness of product design, especially by reducing drastically the amount of lab and field experiments. The use of NDA greatly enhance the engineering prediction capabilities when significant uncertainties are present in the operational environment and/or material complex behavior. For the new advanced high-tech designs, with new materials, higher performances and harsher environments, for which no past engineering experience exist, the use of a NDA technology for investigating the design robustness, reliability and cost-effectiveness represents a key factor of the US high-tech development for successfully competing on the international market in the new created condition of world economy globalization using the internet technology. BACKGROUND: NDA-WTG Identity and Planned Activities The NDA-WTG is a multi-organizational working group comprised of representatives from all professional organizations with interest in NDA. The AIAA (American Institute of Aeronautics
& Astronautics) Structures Technical Committee will be the primary home of NDA-WTG. The SAE (Society of Mobility Engineers) and ASCE (American Society of Civil Engineers) will have strong ties to NDA-WTG. Other engineering and non-engineering organizations will be tapped for liaison with NDA-WTG. The mission of NDA-WTG is to encourage and sustain the development and application of non-Deterministic Approaches within the industry, government, and academia and to serve as the platform for information exchange for all aspects of non-deterministic approaches. The goals of WTG-NDA to accomplish its mission are: -To reach out to all those interested in non-deterministic approaches in all walks of life, with initial focus on aerospace structural systems. -To reach out to all those who may not be aware of non-deterministic approaches but can benefit from them. -To identify and put together a core group of people who are technical experts, those who are experienced practioners, and those who have demonstrated orders of magnitude of time and cost savings by using non-deterministic approaches. - To enable a dialogue toward the understanding and commonalities of the various statistical, stochastic, and other pertinent non-deterministic (probabilistic approaches. - To facilitate collection of information on past successful and unsuccessful uses of non-deterministic approaches and their correspondent perceived and/or real lessons and benefits. - To develop a sound approach for rapid, credible, and results-oriented education and implementation of non-deterministic approaches to highly visible product development cycles and decision making processes. - To facilitate the demonstration of the orders of magnitude savings for the selected highly visible products and/or processes, beyond any doubt. - To facilitate the establishment of procedures, guidelines, and lessons learned. - To facilitate dissemination of relevant information from the demonstration cases to other products and/or processes. - To facilitate education and implementation of proven non-deterministic approaches in all walks of life. - To facilitate the establishment of standards to enable the use of non-deterministic approaches as the routine practice. The NDA-WTG’s target is to become a nationally/internationally recognized technical group comprised of world-renowned experts knowledgeable in state-of-the-art tools of non-deterministic approaches. The NDA-WTG will be an invaluable human resource for in-depth understanding of all NDA aspects, will be the path-finder for futuristic thinking and endeavors, and will be available and instrumental in holding hands of those who need to understand and implement NDA for the benefit of human kind. The focus will be on simulating the effects of real-life uncertainties by the most appropriate techniques that will provide tangible benefits to our society, for both the public and private sectors. The AIAA NDA-WTG was initiated in June 1998, by Dr. Suren Singhal, Dynacs and Dr. Dan M. Ghiocel, STI Technologies. By the end of February 1999 the total of NDA-WTG members was 23 including mostly NDA researchers from university, industry and national labs. Today, after two years from the initiation date, the total number of members of the AIAA WTG-NDA
has increased to double, to more than 40. The current NDA-WTG composition includes top university professors, industry and government technical managers, and national labs researchers. We plan to increase the number of AIAA NDA-WTG members to a number of 50 during the year 2000. Now, we try to bring new members who are high-level technical managers from industry with a NDA exposure and understanding and top government representatives from key research forums, such as NSF and NAE. We have focused on large businesses and governmental agencies with an important say for promoting NDA for aerospace industry. After membership full completion by the end of the year 2000, we will send to all a list of the Founding Members of AIAA NDA-WTG. After this, more members can be added only by invitation from Founding Members. The AIAA NDA-WTG membership will be by invitation from Founding Members, subject to peer review by all Founding Members and approval by the Membership Chair of NDA-WTG. Invitations will be extended to those NDA specialists with outstanding credentials for developing, implementing, and communicating NDA. Paid membership in a specific professional society will not be a requirement for membership in NDA-WTG. Member’s responsibilities are stated in the Charter of the NDA-WTG issued in March 1999. The Agenda of the NDA-WTG Workshop is shown in Attachment A. DEBATE: NDA-WTG Workshop Presentations and Recommendations The starting activity of the NDA-WTG has been on creating an agenda for the future NDA activities. The main focus of the NDA-WTG for the year 2000 has been to organize a Joint Industry/Government/Academia Workshop on “Enabling the Potential of Non-Deterministic Approaches”, in conjunction with the AIAA NDA Forum at the 41th AIAA/ASCE/ASME/AHS/ ASC SDM Conference, Atlanta, April 2, 2000. The NDA-WTG Workshop was intended to bring together government, industry, and academia for much-needed dialogue on NDA-based technology and serve as a common platform for sharing the related efforts and experiences throughout the aerospace community. The NDA-WTG Workshop has exposed critical issues for NDA implementation in industry and discussed directions for continuing research for NDA development. The Agenda of the NDA-WTG Workshop is shown in Attachment B. The NDA-WTG Workshps addresses the following questions (without being limited to them):
(1) What should be the AIAA NDA-WTG role for promoting probabilistic and other non-deterministic approaches and for implementing them in aerospace industry ?
(2) What are the needs and goals of industry for probabilistic/non-deterministic technology ?
(3) What needs to be done to promote probabilistic/non-deterministic technology in the future ? What should be done in short-term and in long-term ? What can we do ?
(4) What are the major technical challenges and milestones for implementing probabilistic/non-deterministic technology ? What can we do to be most efficient for overcoming the implementation difficulties for industry ? What are the main points of our focus ?
(5) What are the resources needed and available ? (6) Other questions which you think that are important for the NDA-WTG activity and
for promoting probabilistic/non-deterministic know-how for a full industrial use ?
The Mission of the NDA-WTG Workshop was to create Recommendations and a White Paper on “Enabling the Potential of Non-Deterministic Approaches” for aerospace industry. The NDA-WTG White Paper which will represent an useful, relevant, authoritative document on promoting NDA-based technology for the benefit of our nation. Brief Summary of Workshop Presentations: The full-length slide presentations at the workshop are included in Attachment C (pdf file). Here, in this section only a brief description of the content of generic ideas on NDA is given. The order of the presentation follows the Agenda. Opening Remarks: Dr. Suren Singhal, Dynacs, Cleveland, OH, Chair of the NDA-WTG Dr. Singhal presented the “Enabling the Potential of Non-Deterministic Approaches” Workshop Agenda and Mission. He referred that using NDA for accounting uncertainties shall be done BY: understanding, quantifying and managing; IN: design, manufacturing, maintenance, customer service, decision-making, human factors; FOR: affordable reliability and safety, risk versus cost trade-offs, aging, liability, warranty cost, etc., significant reduction in expensive testing. “Variability is a fact of life, yet most engineering is based on deterministic methods”. Industry Reports: (1) Dr. Norman R. Kuchar, GE Corporate Research and Development, Schenectady, NY. Dr. Kuchar discussed the “Design for Six Sigma: Non-Deterministic Design at GE”. The Six-Sigma goals imply a nearly flawless performance in GE product design. The Six-Sigma is “the most important initiative the (GE) Company has ever undertaken. (It) will fundamentally change our Company forever” (Welch’s letter to GE Officers, May, 1996). He referred to the benefits and implementation of Six-Sigma Quality Program in GE design and manufacturing.
Figure 1
Dr. Kuchar discussed key issues on the NDA design implementation in practice (Figure 1): - Computation and analytical methods
- Statistical design and optimization require consideration of a much larger design space than deterministic design, so computational requirements can be much larger - Today, this limits the use of statistical design and optimization to components, subsystems or low-complexity systems - The benefits of statistical design need to be extended to larger, complex systems
This will require improved statistical design and optimization methods and faster computing: hardware, parallel algorithms, etc.
- Data and models for statistical design, reliability and life prediction - Standard deviations or other variability measures are generally not available (and probably not known) for many sourced components - Mfg process capability data from suppliers are sparse - Even when known, mfg process capability data are not generally easy to use by designers, e.g., are not related to design features such as holes, flanges, … - Data on life and reliability are not available for many sourced components - Predictive, physics-based models for component life and reliability are not generally available (particularly for electronic components)
- Standards and design practices - As product development increasingly becomes an effort of global or “virtual” teams, the need for national/international statistical design standards and disciplined design processes/practices becomes mandatory to achieve product quality - Statistical design standards and practices (and standards for reliability testing and data) are
generally lacking, both nationally and internationally - Engineering education
- Statistical design is not generally included in design courses - Tolerancing is not often taught as a “science,” and is rarely related to mfg process capability - Design is not generally taught as a disciplined process, nor as an element of a customer-to-customer business process
- Legal liability - Statistical design always yields a non-zero defect level or probability of failure. When is a low probability of failure low enough from a liability standpoint? Are our predictive capabilities good enough to provide high confidence in defect and failure rate predictions? Are they good enough to be accepted by the legal system?
(2) Dr. Gene Rogers, Boeing Company, Space System Division, Downey, CA. Dr. Rogers discussed the “Non-Deterministic Approaches at Boeing Company”. Dr. Rogers showed that the use of Probabilistic Analysis and Design System (PADS) reduced the costs and time of Shuttle/Mir analysis by a factor of 10. Large savings due to 20% average weight reduction using PADS are reported for the Evolved Expendable Launch Vehicle Upper Stage Design (Figure 2).
The Boeing CompanySpace Systems Engineering Page 7
PADS Identified Optimal Design & Reduced OverallDeterministic Design Weight by More than 20%Initial deterministic design
LH2 Tank Weight = 1237 lbs
tcylinder = 0.063 inch5 rings
LH2 Tank Weight = 1521 lbs
tcylinder = 0.1 inch
LO2 Tank Weight = 647 lbs
tcylinder = 0.1 inch
w = 120.5 inch
LO2 Tank Weight = 362 lbs
Spin formed domewith integrally machined joints
w = 140.2 inch
PADS-optimized design
Reduced 284 lbs (18%)
Reduced 285 lbs (44%)
2 rings
2 rings
tcylinder = 0.08 inch
Figure 2
The PADS design strategy considers: - Manufacturing Process Alternatives - Stock Size Alternatives - Alternate Failure Modes, e.g., buckling, crippling, and yielding - Design features for cost-effective manufacturing & easy assembly , e.g., spin formed domes using integrally machined joints & conventional welds
The PADS design results includes:
- Searched over one million design configurations and identified Optimal EELV upper stage vehicle design - Weight reduced more than 20%, compared to initial designs using deterministic approach - Computed reliability of the design - Computed probabilistic sensitivity factors to identify key design drivers for special attention
(3) Dr. Mohammad Khalessi, Unipass Technologies, Irvine, CA. Dr. Khalessi ‘s presentation title was “Probabilistic Technology”. He discussed the implementation aspects of Probabilistic Technology. He referred to present status, short-term and long-term status of Probabilistic Technology. Present Status:
Baseline Technology Development: Has Been Developed - Academia, Governments, Industry End-user, Software Vendors
Commercialization: Is Slowly Starting - Government and Industry End-user: No Commitment
- Some Training Through Academia, Workshops, and Vendors - Some Low Visibility R&D Pilot Projects Are Underway - No Full Implementation Commitment
- Software Vendors - Several Startups Have Been Formed - Some Existing Software Vendors Exploring Potential Business Opportunities
- Academia - No Course Requirements Exist for Graduation - Limited Number Graduates Have Any Training in PT - Some Research Activity
Short-Term: - Technology Demonstration: Will Continue - Baseline Technology Development: Will Continue - Commercialization
- Some Non-engineering Industries Will Be Fully Committed - Limited Commitment in Some Engineering Disciplines Is Expected - Software Vendors
- Rate of Growth Will Depend on End-user Commitment to Technology, However, Significant Growth Is Expected
- Academia - Number of Universities Providing Courses Will Increase - Number of Optional Courses Will Increase - Probabilistic Courses Will Not Be Requirement - Research Will Continue
Long-Term: - Technology Demonstration: Will Not Be Needed Anymore - Baseline Technology Development: Will Not Be Needed Anymore - Commercialization
- Government and All Industries Will Be Fully Committed - Probabilistic Software Industry Will Be Very Strong and Stable - Academia
- All Universities Will Provide Probabilistic Courses - Probabilistic Courses Will Be Required - Research Will Reduce
(4) Dr. Kadampi Rajagopal, Boeing, Rocketdyne Propulsion and Power, Canooga Park, CA Dr. Rajagopal discussed key aspects of the implementation of the NDA technology in the design process at Rocketdyne. He addressed issues related to the availability and adequacy of the in-house software infrastructure, the design requirements, training and design paradigm shift-culture from the deterministic the non-deterministic engineering philosophy. He mentioned the important role of using friendly GUI and internet-based application software.
A typical NDA-based design integration at Rocketdyne are shown in Figures 3 and 4.
9
Dynamic Analysis
Cost AnalysisMechanical Design
Logistics & Field Support
Aerodynamics Stress AnalysisRisk/Life
Management
PARAMETRIC MATH MODEL
Manufacturing
Design Process Director
DeterministicOptimization
ProbabilisticAnalysis
ProbabilisticSensitivities& Scans
TaguchiDesignScans
ProbabilisticOptimization
SensitivityAnalysisDeterministic
Design
Typical CaseWorst Case
SensitivityVariable Ranking
Design Space ExplorationResponse Surface
RobustnessNominal Design Point
Min cost, WeightMax Performance
RiskReliability
Reliability BasedRanking
Min Cost, WeightMax Reliability
Modern Design Framework
Figure 3.
10
Multi-Disciplinary Models Linked Together
Geometry Variables
Heat Transfer Variables
Mechanical LoadVariables
Material PropertyVariables
GeometryEngine
MechanicalStress
Analyzer
Life (F.S.)Analyzer
Links between Codes
Response Variables (R’s)
Mass Max.Temperature Max.Combined Stress F.S.
Input Variables (X’s)
Thermal Stress + Mechanical Stress
ThermalAnalyzer
Figure 4.
An important statement on design requirements was made by Dr. Rajagopal from the customer voice point of view. He said that while using NDA we should not put emphasis on the accurate estimations of the “absolute” very-low failure probabilities and their confidence intervals for a given, but to put emphasis on the safety comparisons for different designs so that we can find the most robust and safest one. Dr. Rajagopal suggested that reliability index or number of standard deviations from the means to define a safety margin is a better measure for real applications than reliability numbers given in terms of probabilities.
He also mentioned that non-deterministic analyses are to requested by any of the Rocketdyne’s customers. Contracts should impose a clause of variability assessment including design sensitivity evaluation to random variabilities. On training aspects, Dr. Rajagopal highlighted:
- Needs teach Ph.D. students to design using NDA tools such as probabilistic analysis - Only limited publications available on using the tools to improve the design - More design tutorials are needed to use the non-deterministic results to improve the design. Specifically, text-books on integrating sensitivity analysis, reliability analysis and cost models to improve the design will be extremely useful.
(5) Dr. Stu Roth, Raytheon Electronic, Missile Systems Dr. Roth discussed the evolution on design methodology for missile systems including the role of NDA. Typically the NDA-based design calculations consists in Monte-Carlo simulation using uniform pararater distributions. He mentioned that today the characteristics of NDA applications at Raytheon are based on decentralization of computing puts the power of simulation in the hands of each designer. The engineering workforce is being trained in each of the above tools/ processes with an emphasis on understanding and reducing variability. Six Sigma design scorecards are being used to predict performance prior to fabrication, assembly and testing. Many elements of the predictive scorecard much of which is compiled using NDA techniques. Dr. Roth presented several success stories of great savings for missile systems developed by Raytheon. Dr. Roth made some remarks on barriers and short-term goals to widespread NDA technology in the engineering community: Barriers:
- Customer understanding and acceptance of the use of statistical methods in lieu of worst case deteministic approaches (rice bowls). - Lack of component models to be used in simulations - Little parametric variable data available - Overuse of simulations to validate all permutations and combinations of all variables
Short term:
Focus on Education - Customer/consumer education on the understanding and impact of variability - Mandatory engineering college curriculum focused on variability and uncertainty - Supplier education on the need to supply variables parametric data - Evolve to standardized models for easy use in analytical modeling tools
Measurable plans and goals:
Develop awareness training - SAE G-11 Probabilistic Methods Committee by end of year 2000 Deploy to major corporations and government agencies
- Utilize AIAA and SAE forums to educate Develop college and university curriculum guidelines on the importance and NDA application - Joint representation from academia and industry representatives by end of 2000
(6) Dr. Jim Norton, Lockheed-Martin Aeronautics Company, Forth-Worth, TX Dr. Norton discussed the use of NDA during conceptual design. The NDA provides a much better alternative to use to the deterministic “worst-case” approach (Figure 7).
Lockheed Martin Aeronautics Company - Fort Worth Figure 7
Dr. Norton summarized the NDA status and future needs as follows: Current Status:
- Many companies utilize both NDA and deterministic methods - The number of NDA applications continues to increase - Integral tool for disciplines such as reliability, safety, … - The number of consulting groups with NDA capability is increasing - Executive level awareness/engagement continues to be a key factor that determines extent of NDA use
- NDA training is generally available, but not required for Engineering degrees - Training within companies is likely to be OJT or ad hoc courses
- Software tools that aid performance of NDA are becoming more sophisticated and affordable - Success stories are often not documented, particularly with respect to cost savings - Advocacy groups such as in AIAA and SAE now exist, seeking to advance the NDA use
Short-term perspective (2-5 years): - More companies using more often - Quality, productivity initiatives - Models for more diverse applications/validated - Cost effectiveness more widely appreciated
Long-term perspective: (15-20 years): - Integral part of engineering analyses - Large shared data bases exist - Large shared collections of validated models exist - Cost effectiveness universally accepted
Recommended Actions for NDA promotion: - Invite/encourage executive attendance at NDA working group sessions - Expand training - Executives can encourage use by asking the right questions - Include NDA unit in systems engineering curriculum - Improve and deploy tools - Showcase success stories - particularly affordability aspects - Incorporate NDA into standard procedures - Hiring employees with NDA skills
Specific Recommendations: - Require NDA for Engineering degree - ABET endorses (2000) - DOD fund applications illustrating benefits of NDA (2000) - Establish a web listing of available NDA tools with brief descriptions and sources for more detailed information - AIAA, SAE (2000)
(7) Dr. Dave Robinson, Sandia National Laboratories, Albuquerque, NM Dr. Robinson presented “Non-Deterministic Methods at Sandia National Laboratory: Dealing with Uncertainty in an Evolving Stockpile”. He discussed that SNL developed CASSANDRA code for uncertainty analysis in response to:
– Need by engineers to address reliability and aging effects for stockpile safety assessment
– Need to test and validate new methods for structural reliability and uncertainty analysis methods avoid ‘re-inventing the wheel’ for each new reliability problem
Dr. Robinson discussed different areas of the NDA applications. At the SNL the current effort is to develop integrated NDA systems from molecular to system level as suggested in Figure 8.
AIAA NDA-WTG Presentation5
How? Reliability Characterization Cycle:Molecular Level to System Level Impact
RBP =d ∆R / R0( )
dt= kPCl2
x 1− exp −RHη
β
exp −EaRT
Experimental characterization of degradation
Identification of important parameters
Statistical characterization of uncertainty
Reliability characterization of component
Impact on system level reliability
Figure 8
Short-term (2-5 years): Technical Issues
- Reduced number of function evaluations - New SNL unique algorithms being developed and tested - Improved sensitivity analysis - Improved communication between software components SNL Corrosion Initiative - system level impact of microscale evolution of corrosion by- products -Time dependent behavior Aging analyses National power grid vulnerability - Multiple failure modes System level reliability vs component reliability
Educational issues - Lack of qualified researchers and engineers Longer-term (5-10 years): Technical Issues
- Increased use of NDA methods by engineers - Improved communication between software components - Time dependent behavior - Multiple failure modes
Educational Issues - Continued lack of support from educational institutions for qualified graduates
Recommendations: Need a national focal point for NDA research that is not tied to a specific technology, business group or commercial product
- NSF ? - National Lab ?
Need government to reward reliability-based design efforts - Subsidize parallel design efforts to minimize risk
Encourage graduate programs in NDA research - Where are the jobs? Where are the students? (chicken/egg) e.g. SNL has openings for US citizens but can not find anyone to fill positions…
(8) Dr. Dan M. Ghiocel, STI Technologies, Rochester, NY. Dr. Ghiocel envisioned from STI research experience the industry, university and government perspectives on probabilistic/NDA technology implementation in short-term and long-term. He also made specific recommendations for future actions in industry/university/government for enabling NDA. Vision for Industry: Short-term (2-5 years):
- Create the first-generation of guidelines, best practices, standards for NDA/probabilistic technology implementation - Refine, develop and validate probabilistic/NDA models and methods for real-life, complex engineering applications - Train technical stuff to understand the basics of probabilistic engineering philosophy and its practices - Perform comparative deterministic vs. probabilistic analyses and designs for selected components/systems Long-term (10-20years): - Create complex, user friendly, graphical-computational systems for an integrated risk-based cost-optimized industrial design engineering approach - Probabilistic/NDA engineering to be currently applied in high-tech product design for all aspects implied - design, manufacturing, maintenance, cost-modeling as shown. Recommendations: - Further develop the internal applied research on engineering application by performing comparative studies - deterministic vs. non-deterministic predictions - Select highly-qualified and experienced engineers to lead the NDA effort in the direction of probabilistic engineering - need both skills and vision ! - Train the technical stuff with basics aspects of structural safety theory/concepts - Collaborate with universities and small businesses in joint research projects - Organize/participate in workshops on probabilistic/NDA engineering aspects - Develop/participate together with universities and government agencies to produce technical documents, guidelines for probabilistic engineering implementation - Develop short-term exchange programs with universities, research institutes and labs - Participate in small businesses research programs, such as SBIRs and STTRs - Continue to improve and reorganize the existent statistical databases for computer integration - Perform new experiments and develop guidelines for the validation of NDA results - Perform risk assessment studies to compare designs and to investigate field failures in operation - Perform in-depth comparative studies for different probabilistic/NDA models and methods to understand their limitations and practicality Vision for Universities: Short-term (2-5 years) - Revise the curriculum to include appropriate probabilistic/NDA engineering courses to and a mandatory curriculum - Revise the existent courses so they include at least a section on probabilistic/NDA engineering - Request university professors to understand and apply probabilistic/NDA engineering tools - Requirement to publish on probabilistics/NDA research - Organize/Participate in industry workshops, meetings, panels Long-term (10-20 years) - The engineering courses will include to a large extent probabilistic/NDA engineering concepts - The students will be taught how to handle uncertainties in engineering design, manufacturing and maintenance using adequate probabilistic/NDA models - The engineering calculus will be largely on a probabilistic/NDA-basis
Vision for Government: Short-term (2-5 years): - Use the necessary funds to support long-term research programs under umbrellas of GUIde consortiums - Revise the university curriculum to include appropriate probabilistic engineering courses and establish a mandatory curriculum - Participate/jointly organize NDA workshops, meetings and develop technical guidelines - Prepare society reaction and legal aspects for probabilistic/NDA technology implementation Long-term (10-20 years): - Develop a competitive, high-performance, risk-based, affordable high-technology - Maintain the US high-tech dominance on international market in the actual economic globalization process Government Reports: (1) Dr. Chris Chamis, NASA Glenn Research Center, Cleveland, OH. Dr. Chamis presented a “Limited Government Perspective” on NDA technology. Dr. Chamis indicated that for NDA to be enabling must be versatile and inclusive and have most, if not all, of the following features: - Represent uncertainties as they naturally occur - Physics-based
- Bottoms-up - Formulation at lowest tractable level - Formalism to propagate uncertainties from the bottom to the top
- Computationally efficient - Easily used - Generate useful information at all propagated scales Present Status:
- Familiarity and contractual requirements determine usage - Bottom line billions saved for new system development - Training is limited but available - Sensitivities calculable - 90% time reduction; minimize, eliminate reworking - not used as yet Recommendations for Short-term and Longer-term are given in the Table shown below.
Dr. J. Housner, NASA Langley Research Center, Langley, NC Dr. Housner made a relatively detailed presentation on “Revolutionizing NASA’s Engineering and Science Processes” with the focus on the NASA Inteligent Synthesis Environment (ISE) program vision and goals. NDA plays a key role in the NASA ISE program. The NASA ISE approach is based on a large extent on the NDA as shown in Figures 9 and 10.
Figure 9
NDA
Figure 10
The ISE program will enable NASA to advance engineering and science practices by developing - Prototype user-intuitive engineering and science life-cycle software system - Intelligent networking and integration technologies - Collaboration and knowledge sharing capabilities - Virtual prototyping and life-cycle simulations quantifying performance,
Cost and risk/reliability Large-scale applications are being used as mission pull and NDA are being used as technology push. R&D laboratories are being established to develop, evaluate and validate ISE technologies. (3)Dr. C. Hedges, Federal Aviation Administration (FAA), Safety Systems Dr. Hedges discussed the current status of NDA use, goals and needs at FAA on safety and risk assessment related issues. Current Status: Safety and risk management are critical components of our day-to-day operations - Air carrier Oversight ( Operations and Certification)) - Employs risk management
- Looks at processes and outputs - Airworthiness - Certification of new designs - Ensuring safety of aircraft systems as they age - Air Traffic Services
- Safety, capacity, and efficiency of system - Developing new technologies and acquisitions
- End-to-end System Safety - Processes used to analyse data, evaluate options
- Commercial Aviation Safety Team (CAST) -Weighted scoring technique
- Global Aviation Safety Team (CAST) - Data, analysis, information to identify accident precursors - Future hazards Working Group (FHWG) - Expert panel and Analytical Hierarchy Process - Other - Cost benefit analysis - Investment analysis - Safety risk analysis under FAA Order 8040.4 Goals: - Short term - Achieving 80% reduction in accident rate by 2007 - Improved surveillance and certification - Increased level of safety built into equipment - GAIN prototype
- Long term - Changing Airspace system - Integration of space and terrestrial air traffic - More demand, same infrastructure
Needs: - How should resources (dollars/skills) be allocated to achieve the optimum level of safety? - Increased inspectors? - New technology? - Research and development? - Existing infrastructure upkeep? - How do you identify, prioritize GAIN information to achieve the greatest reduction of accidents and incidents? - Turn information into knowledge and understanding - Increase confidence in decision –making - How can the FAA meet the increasing demand for services (more flights, more passengers, more destinations), given a fixed infrastructure, and still maintain the safest airspace system in the world? - Larger capacity aircraft, more night flights, smaller craft- more traffic into smaller airports- older population? - Can the FAA anticipate future accidents rather than continue the traditional practice of responding to the last one? Anticipate risks associated with changing conditions - Increased operations
- New Procedures (e. g., Land and Hold Short Operations) - Changing environment Anticipate risks associated with new technologies - New designs (very large aircraft, blended wing body) - Commercial space operations - New Navigation systems - New flight control concepts (glass cockpit) (4) Dr. Paul Hoffman, NAVAIR Systems, Structures Division Dr. Hoffman referred in his presentation on the NDA implementation, status and future at NAVAIR Systems. The present status is based on technical risk assessment guidelines (NAVAIRINST 5100.11). In short-term the expectations are to approach structural component reliability for crack initiation and aging effects. In long-term the air vehicle reliability will be approached as complex systems and flight certification design criteria will be based of reliability computations. Present Status: Currently the risk assessment is performed using a subjective definition of probability of occurrence of the loading events (or frequency) and categories of severity for failure consequences: Probability of occurrence for discreet events may replace Frequency based upon the Table: Severity is the worst credible consequence of a hazard in terms of degree of injury, property damage or effect on mission defined below.
Frequent
10-3
Remote
10-6
Occasional
10-5
Probable
10-4
Improbable
Catastrophic - Class A (damage > $1M / fatality / permanent total disability) Critical - Class B ($200K <damage<$1M / permanent partial
disability/hospitalization of 5 or more personnel) Marginal - Class C (10K<damage<$200K / injury results in 1 or more loss
workdays)
Negligible - All other injury / damage less than Class C. Based on NAVAIR Instruction 5100.11 the review of risk process is defined by as shown in the Table below:
IPT / FST / SSW G Acceptance6-10 MEDIUM SAFETY RISK 18-20 VERY LOW SAFETY RISK
S E V E R I T Y
FREQUENCY
HAZARD CATEGORIZATION
UNACCEPTABLE ACCEPTABLE WITH REVIEW
CNO / TYCOM / Fleet Acceptance
UNDESIRABLE ACCEPTABLE WITHOUT REVIEW
1-5 HIGH SAFETY RISK
PEO / AIR-1.0 Acceptance
Short-term at NAVAIR:
Organization’s Technical Challenges - Limited/censored data analysis - Incorporating corporate- historical data
- Modeling - Engineering versus statistics Personnel Needs:
- Academics - fundamentals of probability concepts (theorems, simulation, etc.)
- Training - engineering judgement Long-term at NAVAIR: - A current S&T is focused on design/specifications:
- Given a US Navy service requirements (mission, service life, etc.) - What limits or constraints does one impose for a specified reliability?
- UAV/UCAV Tech Development - Radical departure from traditional
- Extensive risk analysis studies - Maximum return on investment
Recommendations (AIR 4.3 NDA Needs/Recommendations): - Many mathematical constructs have been developed. (i.e., the civil experiences as a basis! Many reliability analysis methodologies exist but there has been little in the way of mainstream aerospace applications.) - The need is in successful applications/demonstrations. (Success breeds imitation! There is no substitute for success in justifying a methodology.) - Success means benefit(s) heretofore not attainable.
(Benefit: More informed-decision-making; potential cost reduction/cost avoidance; and in new areas of engineering, e.g., UCAV a cost reliability
tradeoff, risk taking analysis capability) -Visibility at Conferences (focus on both Management and Engineers) (Get management to embrace non-deterministic methods! Recruit engineers with interest and background in NDA!)
(5) Dr. Rob Sues, Applied Research Associates, Raleigh, NC. Dr. Sues discussed the NDA implementation status, solutions and barriers, short-term and long-term and recommendation and needs. Barriers and solutions: - Requires specialized expertise Need for more academic and professional training - Too difficult to implement better integration with existing CAE tools - Too time consuming to model standardized procedures, more demonstration problems - Too time consuming to compute numerical methods R&D, parallel processing hardware - Immature technology prone to numerical and accuracy problems numerical methods R&D, guidelines for application Short-term (2-5): - Continued specialized applications, continued R&D, continued integration into existing software packages - Technology maturation - standardization of procedures for particular applications (e.g., design of aircraft lapjoints for fatigue, design of jet engine rotors, design of jet engine fan blades), standardization of distributions, standardization of procedures to develop distributions Long term (15-20 years) - Ubiquitous use of NDA - all engineering simulation results will be provided with some type of uncertainty bounds (intervals, probabilities, qualitative or quantitative confidence levels) - Use of RBMDO to find robust optimal designs, optimal levels for quality control - Pay for performance will be commonplace (e.g., pavements, etc.) Recommendations/Needs: - Integration of NDA by CAE companies - requires government and industry requests and perhaps financial support - Demonstration problems leading to standardization - requires shared industry/government leadership and support - Training - continue to expand university and professional opportunities - Computational methods R&D - Fast sensitivity analysis -- direct differentiation, auto-differentiation, special purpose iterative solvers, parallel processing, variable fidelity methods, successive approximations - Probabilistic analysis/optimization analysis synergy - Smarter search algorithms - Heuristic methods (6). Dr. Justin Wu, Southwest Research Institute, San Antonio, TX
Dr.Wu’s presentation addressed “Probabilistic Mechanics and Reliability Technology, Status and Challenges Based on Application Experience at SwRI”. He discusses two important aspects of probabilistic/NDA implementation: (i) Probabilistic Analysis and Design Process and (ii) Data for Probabilistic Analysis and Design. These aspects are summarized below: (i) Probabilistic Analysis and Design: Status: - Many commercial analysis tools available - Analysis speed no longer a major problem - Need significant training and experience
- Example: - Took one trained engineer 30 days to complete an analysis - Took an expert 3 days to complete the sample analysis - Users have problems collecting/developing input Challenges/Needs/Recommendations: - Tools need to be intelligent and user-friendly to relieve the users from needing to be an expert to properly use the methods - Engineers need to be well-trained in basic probability and statistics (ii) Data for Probabilistic Analysis and Design: Status: - Data collection may be the most time-consuming and costly task Example: The development of the titanium hard-alpha defect exceedance curve used for DARWIN analysis took several years in a joint industry effort - Data collection may require iterative procedure - Lack of data often a reason to use other simpler approach Challenges/Needs/Recommendations: - Need data banks, probabilisic design handbooks, and recommended practices for collecting/analyzing data - Need methods to develop input distributions from physics-based models and not from tests - Need joint effort from Industry/Government/Academia
Academia Reports: (1) Professor A. Noor, University of Virginia, VA Professor A. Noor addressed in his presentation the “Future Role of Non-Deterministic Approaches”. He showed an animated presentation on the NASA Intelligent Aerospace Vehicle programs with special emphasis on NDA (Figure 11).
Figure 11
The future computational prediction systems include the properties:
Some important ideas of Dr. Noor’s presentation are summarized below: Future aerospace systems will include:
- Complex systems with different interconnected/interwoven components – Interrelationships among components far exceed their number
- Many uncertainties which are quantifiable at best as engineering estimates - Synthesis begins before requirements are firmed up Qualitative techniques for synthesis are:
- Metaphors - Heuristics
- Progressive modeling Non-traditional methods include key aspects (Figure 12):
- Multiscale - Multiphysics - Uncertainties
Figure 12 NDA include: - Probability theory and random processes
- Fuzzy sets - Set-theoretical anti-optimization approach Development of learning modules;
- Current deterministic thinking aims at finding “the Answer”. - Real-world engineering problems mostly do not have an unique answer - Statistical ensembles of results
- Broader and deeper understanding of uncertainty is needed in core engineering curriculum
- Probability, stochastic and statistical inferences should be cast in the framework of engineering practice
- Need of new mathematics and metaphysics for dealing with complexity and chaos, e.g. Predictive Deduction and Uncertainty Calculus
Recommendations for the NDA implementation: - Procedure developed in the form of pre- and post-processors
- Can be attached to any deterministic analysis program - Generates ranges of variation of selected response quantities governing failure initiation
(2) Professor S.S. Rao, University of Miami, FL Dr. Rao had a presentation on fuzzy set implementation for engineering applications. He addressed the future of NDA in (i) simulation of design and (ii) product design. Simulation of Design: - Random variability - Modeling uncertainty - Statistical uncertainty Product Design: - Uncertainty - Attitude toward risk
- Value of information
(3) Professor Mircea Grigoriu, Cornell University, Ithaca, NY Professor M. Grigoriu ‘s presentations was on “Some Comments on NDA”. He presented a brief history of NDA, current status and difficulties of NDA implementation, and potential solutions for future. A brief history of NDA is illustrated in Figure 13.
A Brief History of NDA
1920 30 40 50 60 70 80 90 2000
Weibull -extremes -size effects
Danielssystems
Randomvibration
Reliability Measures -2nd moment -FORM/SORM
Systemreliability
Stochastic FE/FD
Time-dependentreliability
Non-Gaussianmodels
Generalstochasticmechanics
Time
Size Effect(Experiments)
LRFD standards(performance design)
PROBANNESSUS
Probabilisticfracturemechanics
MC Simulation
Optimizationwith reliabilityconstraint
Compositematerials
Non-Gaussianmodels for loadsand materialproperties
Evolution ofmicrocracks(Digitalmaterial)
Figure 13
Current status on NDA include difficulties are due to weak links within the triangle Education- Industry-Government/Funding Agencies. Current difficulties in: Universities:
- Few field-related courses on NDA - Traditional courses do not use NDA
- Limited time - Most faculty lack expertise in NDA - Lack of perceived need for NDA - Text books
- NDA requires knowledge of - Mechanics/materials/design
- Probabilistic/statistical thinking Industry: - NDA viewed as the last resort
- Lack of perceived need
- University graduates lack expertise in NDA - DA seem to work Notable exceptions: - GE, NASA, … - Insurance/risk analysis - Banking
Government/Funding Agencies: - Insufficient funding for NDA - Random walk approach to funding Potential Solutions: Universities:
- Curriculum changes: - Require an undergraduate foundation course on NDA - Follow up by using NDA in other courses - Do not wait until graduate studies
- Drivers: - Professors: If trained can use NDA in their own course
Industry: - Short courses/distance learning - Support M.Eng/MS/PhD students working on NDA - Joint industry-university projects on NDA
Government/Funding Agencies: - Industry recommendation
(4) Professor Dario Gasparini, Case Western Reserve University, Cleveland, OH Professor Gasparini discussed the role of NDA in the Design and Manufacturing processes and focused on the NDA motivation and the NDA mathematical tools. NDA mathematical tools and theories:
- Probability theory, random variables and processes/fields - Fuzzy theory, fuzzy sets, fuzzy inference - Stochastic calculus, stochastic differential equations - Reliability/decision theory - Stochastic analyses, stochastic finite elements, random vibration theory - Risk analysis - Expert systems, decision theory - Simulation - Reliability-based optimization
(5) Professor Roger Ghanem, Johns Hopkins University, Baltimore, MA Professor Ghanem presented “Perspectives on NDA”. He mentioned about the status on NDA at Johns Hopkins University. He discussed the “Questions that NDA should address” and “Wish list for NDA technology”.
Questions that NDA should address:
- What Confidence to assign to predictions given the current Knowledge ? - How to achieve Target Confidence (through a combination of Data and Mesh
refinement) -
- A fresh look at the Problem is motivated by recent developments in Sensing and Computing Technologies - Ability to approximate Probabilities of failure is not enough - Require ability to Propagate and Manage Uncertainties - Most engineering components are a part of a System; there is a need to Propagate Uncertainty through a cascaded system with Minimum Loss of Information. Wish list for NDA technology: - Rigorous foundation for credible predictions - Accurate propagation of uncertainty (error estimation) - Ability to provide feedback for experiment design - Ability to enhance numerical refinement (mesh refinenment) with data refinement (6) Professor S. Mahadevan, Vanderbilt University, Nashville, TN Professor Mahadevan’s presentation focused on the NDA status at Vanderbilt, short-term and long-term expectations, plus recommendations. Short-term: - Extent of use: - Aerospace/mechanical engineering: More data and demo problems - Universities: - More engineering schools with course offerings in NDA - More textbooks and teaching aids - More engineers trained in NDA - Industry/Government: - Increased training - Demo problems – experience gathering, new challenges - Methods Development:
- Lack of data – non-probabilistic methods. Bayesian methods - Model uncertainty, human error, system level decision methods
- Revise curriculum to include NDA in design courses 1 yr - Educate practitioners in non-deterministic methods 2-5 yrs - Research in technology transfer, outstanding problems 2-10 yrs
- Industry/Government: - Fund research in demo problems, document experience 1-5 yrs - Fund software development and methods research 1-5 yrs - Provide increased training to engineers 1-5 yrs - Lobby top management for resources, new hires etc. 1-2 yrs - Communicate with legal and financial experts 2-5 yrs - Develop targeted partnerships with academia 1-2 yrs
- Professional Societies: - Workshops, white papers, publicity, pro-active lobbying 1-5 yrs - Training, reports, remove cultural & legal barriers 2-10 yrs - Develop NDA-based design specifications 5-10 yrs
(7) Professor R. M. Mullen, Case Western University, Cleveland, OH Professor Mullen discussed in his presentation on “Non-Deterministic Analysis and Design” the use of interval arithmetic in conjunction with FEM and the use of micro-mechanical models for generating probability distributions of material properties. He also discussed on the barriers to the incorporation on NDA in design:
- Development of tools in still in progress - Education in the NDA field - Availability of data
(8) Professor E. Nikolaidis, University of Toledo, Toledo, OH Professor Nikolaidis discussed the NDA research needs. In his presentation he focused on the fuzzy-set based design decision making process. Research needs: - Simulation-based design - Estimate total uncertainty
- Random uncertainty - Modeling uncertainty
- Design decision-making - Uncertainty
- Preferences - Need tools for making decisions that account for uncertainties and the decision maker’s attitude towards risk
Dr. Nikolaidis showed that an important factor influencing the decision outcome is the attitude of the decision maker toward risk (included in the utility measures). He showed that the decision outcome is highly dependent on decision maker attitude (Figure 14).
10
Design using utility• Best design maximizes expected utility
∑= iiPUUE )(
E(U)
Risk proverse decision-maker
E(U)
Risk averse decision-maker
Figure 14
Recommendations During the Discussions
1) Establish benchmark problems to judge the usefulness of non-deterministic vs. deterministic approaches. Include cases for:
- Systems with thousands of parts vs. those with very few parts - Mass-production vs. one-of-a-kind product
- No or few data versus lots of data - Static versus time-variant - Linear versus non-linear - Component versus system
Include cases in between extremes indicated above. Address: - Failure consequences - What non-deterministic method is good for what case?
2) Compile success stories about specific case studies and associated benefits.
3) Develop strategy to communicate with management and lawyers. Address commonalities between ours and theirs approach.
4) Tie predictions from engineering analysis to economy. Demonstrate use of NDA to
economic analysis. Link cost models to engineering models. Address:
- Fidelity - Cost of improved design - Placing your bet fairly early in the process, i.e., start NDA during conceptual design phase - Reliability versus cost - Cost of reworking - Cost of new ways of doing business
- Small versus multi-billion dollar projects - Function of management culture.
Revisit the GE example of composite blade of 1960s.
5) Couple NDA to risk analysis for cases with mass-produced vs. one-of-a-kind products.
6) Translate the voice of customer to: - Engineering - Flow down requirements
7) Get a handle on the treatment of various types of uncertainties. Provide consistency in
interpreting input and output.
8) How can we affect prescription of design both at component and system level, and of design environment from deterministic to non-deterministic? Address:
- Vendor attitude on non-deterministic performance requirements on their parts? - Acceptance Criteria – some of the criteria used by GE includes mean and standard deviation - Culture change? - Specifications on component and on systems level
9) Address NDA for special techniques when change occurs from traditional methods, e.g.
micro structural characterization.
10) How do we use NDA when we don’t have a physical model? How do we depict reality?
Address: - Bayesian approach - DOE (Design of Experiment) - QFD (Quality Function Deployment)
Evaluate effects of errors in the model.
11) Need methods to calibrate the model and characterize input parameters, such as that for material properties.
12) How do we validate - model, data?
13) To what load should we test components when we design using NDA?
- Check predictive scatter band. - How many 9’s to test to? - How do we validate deterministic design? - How do we test so we are confident? - Test for one-of-a-kind vs. mass-produced products? - If we have funds only for, say, 200 tests, how do we structure the test program for maximum confidence?
14) Investigate the following:
- Is NDA keeping pace with the deterministic approach for new developments? - Reliability testing, accelerated testing - Generic method vs. specific to one individual case
15) Address simulation based design – the design of future. It should:
- Transcend static/dynamic/linear/nonlinear, etc. - Emphasize collaboration for a standardized design (Monte Carlo Simulation?) - Be autonomous - Be intelligent - Balance simulation vs. testing (also simulation means different things to different people)
16) How do we address NDA in wake of current Information Technology revolution (billions of dollars of e-commerce)?
- Do XML specs for NDA
17) How do university professors get steady stream of support for students? 18) Include an executive summary in the proposed White Paper on NDA.
- Include roadmap, milestones, etc. - Infuse published NDA papers into the White Paper to go to the mainstream. - Address specialized vs. general situation.
Closing Statements from Attendees
1) We need to focus on how to get the attention of decision makers, e.g. Program Managers, Directors, etc.
2) It is important to integrate NDA results with deterministic results. 3) Consider space/time variability and loading history. 4) Use NDA for manufacturing.
5) I am one of those who didn't want to hear much about NDA a few years ago and now I do
research on NDA. We need to give NDA more visibility such as via this meeting. The approach should not be the shopkeeper approach - I can solve your problem if you come to me. The approach should be the yellow pages type - you will be out of business if you don’t do it.
We need an evangelical approach to promoting NDA or it will take too long. Use Motorola, GE, and Allied Signal as examples. Our data is public information that can be used for influencing people. Be proactive on issues such as legal implications, what methods, lack of data.
Find ways to address these issues. Grab the bull and run with it. Started at GE in July, 1997.Today, products using NDA are out of the door to the
customer and benefits are obvious.
6) There ought to be industry commitment to share the successes stories, experience, methods. 7) Industry should get together and ask for standardization. 8) Begin to think about prescribing non-deterministic performance criteria and corresponding
actions in the design environment. 9) It is important to share lessons learned and stories that didn't lead to success. 10) Need national resources.
Collect as much data as possible for easier access to others Make SAE G–11 Committee's Applications Document as part of the national
resource Conduct quantative risk assessment Standardize tools to integrate NDA into current processes
11) Get NDA examples from other disciplines. Capture their success stories.
12) Need support for Ph.D. students
13) Need a video on NDA that everyone can use as a success story. Suren said, he is already
planning to work on it with Dr. Pickard of Allison Engine Company.
14) Need to document experiences of people in applying different methods.
15) We have a lot of applications for probabilistic analysis but not on probabilistic design. We need to work on more design cases.
16) We need to follow the model of the SAE G-11 PMC and address multiple aspects such as: Public relations Liaisons to advertise success stories on the Web Magazine articles Editorials Need for evangelical type approach in communicating NDA
success stories
17) Get other professors (who are not here today) involved in this group's future meetings and action items.
18) Industry people need to give seminar to universities.
19) We must work on developing a unified NDA approach with the various methods including
fuzzy logic.
20) One goal of using NDA is to properly convey confidence in results.
21) Verify, validate, and get the right physics.
22) Find how other groups have identified with NDA techniques and use those as examples or models.
23) It is important to identify non-deterministic approach for deterministic recommendations. We
need to identify a list of potential questions in this regard.
24) I see decisions on point estimates. Too much information confuses them. Because of too much turnover, we can’t educate them.
25) Try to keep the executive summary in the proposed White Paper on NDA in an agricultural
term.
26) There is a need for semi-analytical non-deterministic method. 27) The White Paper should portray realistic expectations of NDA.
28) Look at today and determine what is needed to accelerate use of NDA tomorrow.
29) Use the SAE G–11 Probabilistic Methods Committee web site as a focal point. Suren needs to
coordinate that between AIAA, SAE, and other professional engineering organizations.
30) Program training materials so as to fill the needs of various levels of personnel converging to brief training for managers.
31) Need to create a consortium of faculty from different universities. Suren is working on that.
32) To be successful, the impetus for using NDA must come from the top such as that at GE. Need a national coordinator.
33) We have no say in the acquisition reform, we can’t specify test load anymore. Simulation based
design is the way to go.
34) Need to establish a national NDA Center to share data, publish recommended practices, and provide regular training. Suren indicated, he has been working on conceptualizing such a Center.
35) This workshop today is the right way to address the critical NDA needs. Doing the right thing.
