Internet Internet Aided Aided SoftwarevalidationSoftwarevalidation · 2011. 4. 19. · Sphere (1/7)...

NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________

1

M. Franke, A. M. Franke, A. WodrichWodrich

Internet Internet AidedAided SoftwarevalidationSoftwarevalidation

Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop

Sou

rce:

ZE

ISS

, Klin

geln

berg

, Hex

agon

, Jen

optik

, Mah

rSoftware in Software in coordinatecoordinate metrologymetrology


2


Sou

rce:

ZE

ISS

, Klin

geln

berg

, Hex

agon

, Jen

optik

, Mah

r

Evaluation Evaluation softwaresoftware

MeasurandMeasurand

Software in Software in coordinatecoordinate metrologymetrology

�� HardwareHardware�� MachineMachine--oriented softwareoriented software

��Physical standardsPhysical standards

�� AlgorithmsAlgorithms

��Numerical standardsNumerical standards

Point Point cloudcloud


Software Software bugsbugs

The first one?The first one?

Found in the electromechanical computer Mark IIFound in the electromechanical computer Mark II


3


Software Software bugsbugs

�� On 4 June 1996 the maiden flight of the On 4 June 1996 the maiden flight of the ArianeAriane 5 5 launcher ended in a failurelauncher ended in a failure

�� Reason: software bugReason: software bug

�� Conversion from float to integer failedConversion from float to integer failed

�� CostsCosts: : ~125 000 000~125 000 000€€ startingstarting costscosts~425 000 000~425 000 000€€ destroyeddestroyed cargocargo

loadload (4 (4 satelitessatelites))~600 000 000~600 000 000€€ followfollow--upup costscosts

horizontal_veloc_bias := integer(horizontal_veloc_sensor);

�� Only a single line of code leads to a catastrophe!Only a single line of code leads to a catastrophe!

One of the most expensive:One of the most expensive:


MotivationMotivation

Software is an essential component in coordinate Software is an essential component in coordinate metrologymetrology

�� Existing solutions (Softwaretest for Gaussian assoc iated features PTB, Existing solutions (Softwaretest for Gaussian assoc iated features PTB, NIST, NPL …, ISO 10360NIST, NPL …, ISO 10360--6, B89.4) cover only a very small part of modern 6, B89.4) cover only a very small part of modern software software

�� Existing process is manually and therefore prone to errorExisting process is manually and therefore prone to error

�� Test not reproducible for end users, e.g. after sof tware updatesTest not reproducible for end users, e.g. after sof tware updates

The correctness of the evaluation The correctness of the evaluation algorithms usually not been provedalgorithms usually not been proved


4


EU Project „Testing Coordinate Measuring Machine Al gorithms, Phase II“, 1991EU Project „Testing Coordinate Measuring Machine Al gorithms, Phase II“, 1991

�� GaußianGaußian best fitbest fit�� Geometrical elements:Geometrical elements:

Straight line, plane, circle, cylinder, cone, sphereStraight line, plane, circle, cylinder, cone, sphere�� Extend: 44 datasetsExtend: 44 datasets�� 17 participants17 participants�� 37%37% of the submitted solutions are acceptable of the submitted solutions are acceptable �� 18% 18% are questionableare questionable�� 45%45% are not tolerableare not tolerable�� only 2 participants delivered completely acceptable solutions for all datasetsonly 2 participants delivered completely acceptable solutions for all datasets

Regular test service by PTB: Regular test service by PTB: SoftwaretestSoftwaretest based on Phase II based on Phase II (Figures from 2007(Figures from 2007--2010)2010)

�� 20 Tests performed20 Tests performed�� 7 not passed at the first try7 not passed at the first try�� Problems experienced with each element typeProblems experienced with each element type

Failures / Type of element

Cone (6/7) IIIII I

Cylinder (5/7) IIIII

Plane (2/7) II

Line (4/7) IIII

3D-Circle (3/7) III

Sphere (1/7) ISubject matter is still relevant!Subject matter is still relevant!

Experience Experience ofof PTBPTB


SolutionSolution

IdeaIdea�� Supply a service for Supply a service for online validation online validation of metrological softwareof metrological software

VisionVision�� Test the correctness of mathematical algorithms integrated in Test the correctness of mathematical algorithms integrated in

metrological software metrological software –– at any time at any time and and any placeany place

PrinciplePrinciple�� A A central servercentral server holds or generates validated reference dataholds or generates validated reference data�� By By well defined protocols well defined protocols the software under test requests the the software under test requests the

data and submits test resultsdata and submits test results�� The server generates a The server generates a digital certificatedigital certificate , confirming the , confirming the

succesfulsuccesful testtest�� Possible access: online (via Possible access: online (via webservicewebservice) or offline (via ) or offline (via webformwebform))


5


AdvantagesAdvantages

�� AutomatedAutomated softwaresoftware testingtesting forfor algorithmsalgorithms

�� ImplementationImplementation accordingaccording toto international international standardsstandards andand guidelinesguidelines::

�� ISO 10360ISO 10360--6, ASME B89.4,…6, ASME B89.4,…

�� ExtensibleExtensible

�� GearGear softwaresoftware (GDE(GDE--REL)REL)

�� FreeformFreeform

�� Form Form measurementmeasurement

�� Filter Filter algorithmalgorithm

�� DirectDirect accessaccess toto validatedvalidated referencereference datadata

�� ForFor softwaresoftware manufacturersmanufacturers duringduring developmentdevelopment

�� Quality Quality assuranceassurance

�� ForFor end end usersusers after after softwaresoftware updatesupdates

�� ConfidenceConfidence buildingbuilding


CMM software

Sample pointsBest-fit geometriclal elements

Combine Report

Testing body

Reference data Test parameter

Reference parameter

Comparison & judgement

Converted test parameter

certificate

ManuellManuell workflowworkflow

Manual action needed!


6


CMM software

Sample pointsBest fit geometrical elements

Combine Report

IASV*

Reference data Test parameter

Reference parameter

Comparison & judgement

Converted test parameter

Digital certificate

Digital certificate

* Internet-Aided Software Validation system

Interfaces to be defined

Workflow using internet based serviceWorkflow using internet based service


NMIsNMIs ServerServer

Overview on the infrastructureOverview on the infrastructure

ReferenceReference--softwaresoftware

Data generatorData generator

CustomerCustomer

�� Expert knowledgeExpert knowledge�� Cross checking of Cross checking of

different reference different reference softwaresoftware�� validated test datavalidated test data

�� Test tasksTest tasks�� Reference dataReference data�� Reference parameterReference parameter�� Permissible deviationPermissible deviation

�� User of measuring User of measuring equipmentequipment

�� Vendor of evaluation Vendor of evaluation softwaresoftware


7


Webservice-Client Software under testApplication serverXML-Webservice

Data base

Referencedata

Parameter:Reference +

Test

Administrativedata

App

licat

ion

logi

c

ApplicationSoftware I

Stand-alone-webclient

Webform

ApplicationSoftware III

ApplicationSoftware II

Web

serv

ice

System layoutSystem layout

IASV-components◄ ► vendor / customer


Validation processValidation process

ComponentsComponents

Administrative:Administrative:�� Type Type andand scopescope ofof thethe testtest ((productproduct: PTB Gauß, : PTB Gauß, GearGear, ISO 10360, ISO 10360--6, 6,

ASME B89.4)ASME B89.4)�� Customer Customer datadata ((namename, , adressadress, , softwaresoftware, ….), ….)�� Order Order datadata ((trackingtracking numbernumber, …), …)

Technical:Technical:�� DeliverDeliver testtest datadata ((cloudcloud ofof pointspoints))�� Software Software calculatescalculates thethe requestedrequested testtest parameterparameter andand sendssends resultsresults

back back toto thethe serverserver�� Test Test resultsresults areare comparedcompared withwith referencereference datadata on on thethe serverserver


8


Content of data structureContent of data structure

Test task

Reference parameter

Test parameter

Performance parameter

Test set

1..n

Test data

(e.g. 3D koordinates)

1..n

Identification of test task,

Versioning

Defintion of task

(computational aim)

= input for software under test = Response of the IASV-Server= output of software under test

Identification of client

PTB Phase II, Best-Fit Gauß

Sphere, Best-Fit Gauß

Sphere3D.PI44.L2.XXX-YYY-ASD

Centre ( -39.850, 4.140, 8.180) Radius: 74.8830

Centre ( -39.8593, 4.1413, 8.1878) Radius: 74.8838

Deviation:Centre: 0.7764±0.0001Radius: 0.0008±0.0001

CloudFit 4.0.1.02.2006

-43.0804 -0.5598 82.7332-17.3040 24.4077 76.5901-29.5830 42.7305 71.5822-56.2379 18.3333 79.9582-10.4244 48.2501 60.9937

: : :

To

m

iib Rzyxbd ),,,(,min 00

2

2 =∑=


4.0022.9990.0

0.00.01.0

548.010

coordinate

normal vector

radius

84.70472075.0215 994.4039

0.00.00.0

….

…

coordinate

normal vector

ExampleExample ofof XML XML structurestructure


9


Screenshot example programScreenshot example program


WebWeb--frontend of the IASV test serverfrontend of the IASV test server

URI of IASV server

OrderID

File to upload


10


Actual stateActual state

Realised components Realised components �� Transmission protocol (XML) Transmission protocol (XML) �� Database (Database (mySQLmySQL))�� Test serverTest server�� Test data for the current PTB test (Phase II)Test data for the current PTB test (Phase II)�� Some test data for the test according ISO 10360Some test data for the test according ISO 10360--66�� Example program (demonstrator) can be downloaded at:Example program (demonstrator) can be downloaded at:

http://iasv.ptb.de:8080/index.html http://iasv.ptb.de:8080/index.html

�� MAHR, Hexagon, MAHR, Hexagon, MitutoyoMitutoyo, , FrencoFrenco, ZEISS are testing the , ZEISS are testing the new servicenew service


Thank you for your attention.Thank you for your attention.

Date post:	20-Oct-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Internet Internet Aided Aided SoftwarevalidationSoftwarevalidation · 2011. 4. 19. · Sphere (1/7)...

Documents