Quibim Precision® Validation Methodology and Technical...

Page 1 | 33

QUIBIM S.L – Quantitative Imaging Biomarkers in Medicine

Av. Fernando Abril Martorell 106 Torre A - Biopolo La Fe - 46026, Valencia (SPAIN)

[email protected]

+34 961 243 225

www.quibim.com

Quantitative Imaging Biomarkers in Medicine

Quibim Precision® Validation Methodology and Technical

Datasheet

Module: Bone Morphometry and Mechanical properties

Version:

1.0

mailto:[email protected]

http://www.quibim.com/

Page 2 | 33



[email protected]

+34 961 243 225

www.quibim.com


Quibim Precision® Validation Methodology and Technical Datasheet

Module: Bone Morphometry and Mechanical properties v.1.0.

Validation actions: Imaging biomarkers

validated: Version of validation:

Submission date:

Precision

• Image acquisition -Intra-centre -Inter-centre

• Methodology -Algorithm -Human interaction Accuracy

• Phantom Clinical value

• Short term

BV/TV

Tb.Th-mean Tb.Sp-mean

Tb.N D2D D3D

QTS value EappX EappY EappZ

v.1 28-02-2018



Page 3 | 33



[email protected]

+34 961 243 225

www.quibim.com


EXECUTIVE SUMMARY (1/2)

Performance Indicator

Assays Complete Verified Total

Precision

Image Acquisition

variability

Intra - Equipment

✓

2/2

Different Centres

Different vendors

Methodology

variability

Algorithm

✓

Human interaction

Accuracy

Phantom

✓ 1/2 Current Gold Standard (E.g. phatology biopsy)

Clinical

Endpoints

Short term (diagnostic and therapeutic values) ✓ 1/2

Long term (prognostic value)

Imaging Biomarker Full Validation checklist

Analysis Module: Bone Morphometry and Mechanical properties v.1.0



Page 4 | 33



[email protected]

+34 961 243 225

www.quibim.com


EXECUTIVE SUMMARY (2/2)

PERFORMANCE INDICATOR

ASSAYS STATISTICS VARIABLES RESULTS ACCEPTABLE VALIDATED

Precision

Image Acquisition

Coefficient of Variation

CoV (%)

BV/TV 2.83

< 15%*

2/2

Tb.Th 3.79

Tb.Sp 1.44

Tb.N 2.44

D2D 1.71

D3D 2.85

QTS value 9.21

Methodology


CoV (%)

BV/TV 2.93

< 15%*

Tb.Th-mean 2.32

Tb.Sp-mean 0.49

Tb.N 0.81

D2D 1.57

D3D 2.22

QTS value 6.53

Accuracy

Phantom/Digital

Reference Objects (DRO)/

Open Data Challenge

Relative error ɛ (%)

BV/TV 4.82 < 15%*

1/2

Tb.Sp 0.05

Pathology (biopsy)


-

- < 15%*

Clinical Endpoints

Short term

Statistical difference (t-Student or ANOVA)

BV/TV p=0.45

p < 0.05

1/2

Tb.Th p=0.03

Tb.Sp p=0.05

Tb.N p=0.26

D2D p=0.04

D3D p=0.001

θa p=0.57

DA p=0.001

EappX p=0.01

EappY p=0.43

EappZ p=0.04

Long term


- - p < 0.05

✓ Validated Imaging Biomarker 4/6

* The CoV and ɛ should be below 15%, which is the Lowest Limit Of Quantification (LLOQ), considering a maximum limit of 20%.

Following the validation methodology of imaging biomarkers proposed in section 3, Quibim Precision® Trabecular Bone Morphometry and Mechanical Analysis has been validated by fulfilling the established requirements.



Page 5 | 33



[email protected]

+34 961 243 225

www.quibim.com


Table of Contents

1. PURPOSE .................................................................. 6

2. TRABECULAR BONE ANALYSIS .................................. 6

3. IMAGE ACQUISITION REQUIREMENTS ....................... 7

4. IMAGES PREPARATION .............................................. 7

5. IMAGES ANALYSIS .................................................... 8

6. QUIBIM VALIDATION METHODOLOGY ..................... 10

7. BONE VALIDATION .................................................. 13

a. PRECISION .............................................................................................................................. 16

b. ACCURACY ............................................................................................................................. 17

c. CLINICAL ENDPOINTS ............................................................................................................ 18

8. RESULTS ................................................................. 19

a. PRECISION .............................................................................................................................. 19

b. ACCURACY ............................................................................................................................. 23

c. CLINICAL ENDPOINTS ............................................................................................................ 25

9. CONCLUSIONS ........................................................ 27

10. REFERENCES ........................................................... 28

11. ANNEXES ................................................................ 31



Page 6 | 33



[email protected]

+34 961 243 225

www.quibim.com


1. PURPOSE To describe the validation test performed to approve the analysis module “Trabecular

Bone Morphometry and Mechanical Analysis” version 1.0 as a plug-in of QUIBIM

Precision® platform.

To evaluate the precision, accuracy and clinical value of “Trabecular Bone

Morphometry and Mechanical Analysis”version 1.0, by the analysis of brain Magnetic

Resonance (MR) examinationsand Computed Tomography (CT), using bone synthetic

phantoms as reference standard.

2. TRABECULAR BONE ANALYSIS Several pathological conditions like osteoporosis or multiple myeloma carry a high risk

of bone fracture due to bone loss. Bone Mineral Density (BMD) measured by Dual-

energy X-ray absorptiometry (DEXA) is considered the main measure to predict the risk

of fracture, sometimes in combination with more complex assessments like FRAX

(Fracture Risk Assessment tool) [1] or TBS (Trabecular Bone Score) (Medimaps Group

SA, Plan les-Ouates Geneva, Switzerland) [2]. DEXA explains adequately nearly 60% of

all fractures [3]. Because osteodegenerative diseases affect not only the BMD but also

the trabecular bone microarchitecture [4], BMD presents limitations for a complete

characterization of trabecular bone deterioration [5].

The architectural abnormalities associated with osteoporosis produce an alteration of

the specific structural organization and mechanical parameters of cancellous bone. On

the other hand, microarchitecture deterioration induced by osteoporosis increases the

fracture risk due to trabecular loss. Therefore, the characterization of bone

microarchitecture is relevant to deeply understand and predict the risk of bone fracture

in patients and directly assess the clinical endpoint of time to fracture.

QUIBIM has developed a fully automated tool to asses the diagnosis of osteoporotic

disease, both in the clinical setting and in clinical trials: “Bone Morphometry and

Mechanical properties v.1.0”.

The aim of Quibim Precision® Bone morphometry and mechanical properties is to

analyze and quantify morphologic, irregularity and mechanical elastic properties of the

bone using 3D MR or CT images. .



Page 7 | 33



[email protected]

+34 961 243 225

www.quibim.com


3. IMAGE ACQUISITION REQUIREMENTS The main parameters of the image acquisition protocol required for the proper module

performance can be appreciated in figure 1:

Figure 1. Image acquisition protocol requirements for Bone morphometry and mechanical properties v1.0.

The protocol is designed to provide a high spatial resolution and contrast between gray

and white matter, which are the most relevant features for the analysis.

4. IMAGES PREPARATION

Firstly, image segmentation was performed by placing a 3D prism as a region of interest

(ROI), verifying that the ROI only contained trabecular bone and bone marrow,

excluding cortical bone. Then a correction algorithm was applied to remove

heterogeneities of local intensities of the segmented ROI. If necessary, an interpolation

algorithm is applied to improve image resolution. To obtain the final image

binarization, the Otsu algorithm is used, dividing the image in to classes: trabecular

bone (1) and bone marrow (0). The whole process can be seen in figure 1.

Modality: MR Sequence: 3D T1 – weighted Gradient Echo TE/TR: Approx. 5-16ms Flip angle (α): 25º Plane: transverse Spatial resolution:

– In-plane: pixel < 220mm, matrix 512x512

– Slice-thickness: < 2mm Mandatory ROI: Yes Modality: CT Spatial resolution:

– plane: Axial – In-plane: pixel<150µm,matrix

512x512 – Slice-thickness: < 1mm

Voltage: 120 kVp Tube current: 250mAs to 50mAs Mandatory ROI: Yes

Acquisition Protocol

M R

C T



Page 8 | 33



[email protected]

+34 961 243 225

www.quibim.com


Figure 2. Image processing in QUIBIM Precision Bone analysis. (1) Image interpolation. (2) Otsu binarization. (3

left) 3D reconstruction. (3 down) structural analysis.

5. IMAGES ANALYSIS

QUIBIM Precision® Bone analysis is based on the extraction of different imaging

biomarkers of bone morphology, connectivity, complexity and mechanical properties

are extracted:

• Bone Volume to Total Volume (BV/TV), (%); measures the total volume of bone contained in the delimited region of interest (ROI).

• Trabecular thickness (Tb.Th), (µm); measures the average thickness of the trabecular contained in the delimited region of interest (ROI).

• Trabecular separation (Tb.Sp), (µm); measures the average size of the pores.

• Trabecular Index (Tb,N), (µm-1); measures the relation between (BV/TV) and the trabecular thickness (Tb.Th).

• Fractal Dimension (D2D, D3D), (a.u.); measures the complexity of the trabecular structure. The higher these indices are, the more complex the structure and, a priori, the better the quality and strength of the bone.

• The apparent elastic module of the whole structure could be obtained for each one of the directions (Eappx, Eappy, and Eappz).

In (Figure 3) the whole pipeline performed in the QUIBIM’s “Bone morphometry and

mechanical properties v1.0” method can be observed.



Page 9 | 33



[email protected]

+34 961 243 225

www.quibim.com


Figure 3. QUIBIM “Bone morphometry and mechanical properties v1.0” pipeline.



Page 10 | 33



[email protected]

+34 961 243 225

www.quibim.com


6. QUIBIM VALIDATION METHODOLOGY

There are still no official requirements for the validation of imaging biomarkers. In

QUIBIM we have decided to adopt the “Guideline on bioanalytical method

validation” from the European Medicines Agency – EMA, July 2011. These guidelines

provide the reference precision and accuracy indicators for laboratory techniques in the

analysis of biological samples. We have considered that a process with medical images

in the input (as sample) and results in the output (as biomarkers extracted) can be

considered a bioanalytical method.

The QUIBIM Standard Operating Procedure (SOP) process for Imaging Biomarker

Validation is defined in the pipeline shown in Figure 5, which considers an integral

evaluation in 3 main steps: precision, accuracy and clinical value. The validation

pipeline takes into account potential influences that can introduce uncertainty in the

measurements.

Figure 4. Validation process of Imaging Biomarkers.

Precision

Precision can be evaluated for all imaging biomarkers and obtaining a high precision is

considered mandatory for the validation of an imaging biomarker (unlike accuracy).



Page 11 | 33



[email protected]

+34 961 243 225

www.quibim.com


For the evaluation of precision, the coefficients of variation (CoV) of the imaging

biomarkers obtained repeatedly with the variation of different factors are calculated.

The factors can be related either to the acquisition or to the methodology.

For the evaluation of the image acquisition influence in the uncertainty of the

measurements, the imaging biomarkers should be calculated ideally for the same

subjects testing the following varying conditions:

• Imaging centres

• Equipment

• Vendors

• Acquisition protocol parameters

• Patient preparationand different patient preparation)

For the evaluation of the methodology influence in the uncertainty of the

measurements, the imaging biomarkers should be calculated ideally for the same

subjects and acquisitions testing the following varying conditions:

• Operator influence (intra-operator variability, inter-operator variability

• Algorithm variability

The global CoV should be below 15%, which is the Lowest Limit Of Quantification

(LLOQ), considering acceptable a maximum CoV of 20%.

Accuracy

The evaluation of accuracy is not mandatory for the validation of imaging biomarkers.

Knowing that we have a precise method, the lack of knowledge in the real accuracy can

be compensated by the demonstration of a clinical sensitivity and specificity of the

imaging biomarker calculated (i.e.: we do not know how accurate we are, but we know

the imaging biomarker is related to the disease clinical endpoints).

Accuracy of the method can be evaluated by comparing the calculated results to a

reference pattern in which the real value of the biomarker is known. The reference

pattern can be based on information extracted from a pathological sample after biopsy

or from synthetic phantoms with different compounds and known properties that

emulate the characteristics of the biological tissue. For the evaluation of the accuracy,

the relative error of the imaging biomarker compared to the real value must be

calculated. The relative error must be below 15%, considering a maximum error of 20%

as the Maximum Limit Of Quantification (MLOQ). In some cases there is no reference



Page 12 | 33



[email protected]

+34 961 243 225

www.quibim.com


pattern availability, either because the synthesis of a stable phantom is a complex

process or because the considered reference pattern has also a high variability and a

coarser category-based analysis (e.g. steatosis grades in pathology vs. fat fraction

quantification from MR) than the continuous domain of imaging biomarkers.

Clinical value

The purpose of this step is to evaluate the relationship between the imaging biomarker

extracted and the clinical endpoints of the disease. The imaging biomarker must have

either a short term value (assessing detection, diagnosis and evaluation of treatment

response) or long term (prognostic value). The type and degree of relationship between

the imaging biomarkers and clinical variables will be analyzed through sensitivity,

specificity, statistical difference between clinical groups and correlation studies.



Page 13 | 33



[email protected]

+34 961 243 225

www.quibim.com


7. BONE VALIDATION

Performance Indicator

Assays Complete Verified Total

Precision

Image Acquisition

variability

Intra - Equipment

✓

2/2

Different Centres

Different vendors

Methodology

variability

Algorithm

✓

Human interaction

Accuracy

Phantom

✓ 1/2 Current Gold Standard (E.g. phatology biopsy)

Clinical

Endpoints

Short term (diagnostic and therapeutic values) ✓ 1/2

Long term (prognostic value)

Imaging Biomarker Full Validation checklist

Analysis Module: Bone Morphometry and Mechanical properties



Page 14 | 33



[email protected]

+34 961 243 225

www.quibim.com


Bone Phantoms

A total of 10 samples of synthetic bones from Sawbone® (SAWBONES EUROPE AB,

Malmö, Sweden) with two different densities (table 1) were used as the reference

pattern. The samples were built using an open cell rigid foam that resembles cancellous

human bone with known density, mass, and cell size.

Samples were classified in two groups depending of their density and all of them

feature 1.5 to 2.5 mm of pore size [9]. Densities from studied samples (figure 3) were of

15 and 30 PCF (Pounds per cubic foot), respectively. These densities correspond to a

15% and 30% of material percentage within the volume.

15 PCF 30 PCF

Density [g/cc] 0.24 0.48

Volume Fraction [%] 0.15 0.31

Compression strength [MPa] 0.67 3.2

Compression Module [MPa] 53 270 Table 1 Morphometry features of 15 PCF and 30 PCF bone Phantoms.

Figure 3. Samples of 15PCF (Left) and 30PCF (Right). Images from Sawbone® (SAWBONES EUROPE AB, Malmö,

Sweden).

The samples were acquired in two different 3 Tesla MR Phillips systems (figure 4), one

Achieva-TX 3.0T, Best System locates in the Hospital Universitario y Politécnico La Fe,

in Valencia, Spain and the second locates in the Hospital PTS, in Granada, Spain.

A methacrylate box was built in order to introduce the samples and fill them with water

to obtain MR signals from the marrow-like region. The image acquisition protocol was

based on a 3D T2-Weighted Turbo Spin Echo (3D T2W TSE) sequence implemented

with 0.1s and 1.5s for echo time and repetition time respectively, slice

thickness=0.5mm, spacing between slices=0.25mm and pixel spacing=0.146mm. The



Page 15 | 33



[email protected]

+34 961 243 225

www.quibim.com


coil used was an 8 channel sense head coil achieving a high-resolution reconstruction

of 0.15 mm x 0.15 mm x 0.25 mm.

The imaging studies were analyzed through the QUIBIM Precision® Morphometry &

Mechanical Analysis tool. The bone microarchitecture analysis is implemented in a

pipeline to measure bone properties that can be extracted from MR and CT images.

Figure 4. Preparation and acquisition of the bone phantom in the 3T MR Phillips System.



Page 16 | 33



[email protected]

+34 961 243 225

www.quibim.com


a. PRECISION

• Image Acquisition variability

Intra- and Inter- Centre:

Two different phantoms (15PCF and 30PCF) were acquired in both centres (Hospital la

Fe de Valencia and the Hospital PTS de Granada), following the same MR acquisition

protocol.

For each image acquired of the phantom, the image analysis scientist selected 3

different ROIs and performed the bone quantitative analysis of each ROI through the

QUIBIM Precision® Bone Morphometry and Mechanical Analysis software.

Each hospital performed 6 bone acquisitions; 3 of them on the 15 PCF phantom and the

other 3 on the 30 PCF. The inter-centre reproducibility test was implemented by

comparing the results obtained in Hospital La Fe to the ones obtained in the Hospital

PTS de Granada. The intra-centre reproducibility was established by comparing the

results obtained within each hospital.

• Methodology variability

Human Interaction:

The same studies were analyzed by two different users of the QUIBIM Precision® Bone

morphology and mechanical analysis tool: an experienced radiologist with more than

10 years of professional career and a 4th year resident of radiology.

The image studies acquired in the Hospital La Fe de Valencia were sent to the person in

charge of the analysis in the Hospital PTS in Granada, and vice versa. The objective is

that the image studies acquired from each hospital were analyzed by the analysis

expert of the other Hospital by trying to select similar 3 ROI's and using the QUIBIM

Precision® Bone morphology and mechanical analysis tool for the analysis.

Each phantom was analyzed 6 times, 3 of them by the analysis expert from the same

hospital where the samples were acquired, and the remaining 3 by the expert from the

other hospital. Consequently, the image acquisitions were distributed as shown in

(Table 2).



Page 17 | 33



[email protected]

+34 961 243 225

www.quibim.com


PHANTOM HOSPITAL ANALYSIS OBSERVER

15PCF Granada- GR 1;2;3 Granada - OBS1

Valencia- VLC Valencia - OBS2

30 PCF Granada- GR 1;2;3 Granada - OBS1

Valencia- VLC Valencia - OBS2 Table 2. Identification codes of each image acquisition of the bone phantoms 15PCF and 30 PCF.

Algorithm:

The same study was analyzed 5 times with the same ROI by the QUIBIM Precision® Bone Morphometry & Mechanical Analysis v1.0 moduleto evaluate the intrinsic variability of the QUIBIM’s trabecular bone analysis algorithm.

b. ACCURACY

• Phantom

Two different image modalities, MDCT and MR, were used to acquire 5 datasets of the

30 PCF. For each modality an acquisition protocol was defined and the acquisition was

performed once for each sample. The acquisition was performed by the same

technician who took care of placing and orienting the samples following established

guidelines. For each acquisition (5 acquisitions from MDCT and 5 acquisitions from MR),

the QUIBIM Precision® Bone Morphometry and Mechanical Analysis tool was applied.

The BVTV and Tb.Sp are the most relevant and accurate features offered by the

manufacturer, so these properties were chosen to validate the accuracy.

The criteria selected for the bone imaging biomarkers accuracy validation required a

relative error ɛ < 20% across the structural features measured by QUIBIM Precision®

Bone Morphometry and Mechanical Analysis tool and the ones known from synthetic

bones phantoms.

• Biopsy

As of now, no validation accuracy study of QUIBIM “Bone morphometry and

mechanical properties v1.0” analysis algorithm has been carried out in bone disections.



Page 18 | 33



[email protected]

+34 961 243 225

www.quibim.com


c. CLINICAL ENDPOINTS

• Short Term

In order to evaluate the clinical validity of the QUIBIM Precision® Bone Morphometry

and Mechanical Analysis tool, a cohort consisting on 24 subjects [6], all female,

including healthy volunteers and osteoporotic patients, was evaluated. The analysis

purpose was to find significant differences on the fractal imaging biomarkers (D2D and

D3D) between healthy subjects and osteoporotic patients and also differences in the

morphometry and mechanical results between both groups [10].

This cohort included 24 female subjects, with ages ranging from 54 to 80 years old

(mean [± SD] = 67 ± 7 years). The cohort was evenly split in 12 healthy volunteers,

without clinical history of bone diseases or fractures, and 12 post-menopausal female

patients with osteoporosis which met the World Health Organization (WHO) criteria to

be diagnosed as osteoporotic and who had suffered a bone fracture in the past 5 years.

Both healthy and osteoporotic populations were comparable regarding age (p = 0.06,

Student’s t test; 64 ± 7 vs 69 ± 7 years for healthy subjects and osteoporotic patients

respectively).

Images were acquired using a 3 Tesla (3-T) MR Achieva scanner (Philips Healthcare,

Best, The Netherlands). The sequence was a 3D spoiled T1-weighted gradient echo

(TR/TE, 16/5 ms; flip angle, 25º), voxel size of 180 × 180 × 180 μm and an acquired matrix

of 512 x 512, with the highest possible signal to noise ratio (SNR) and contrast between

bone marrow and trabecular bone. The osteoporotic patients, as per clinical protocol,

also underwent a DXA scan (Norland XR-46, Norland Corp, Fort Atkinson, USA). Areal

BMD of lumbar spine was obtained at the L2-L3-L4 vertebrae.

• Long Term

As of now, no long-term studies have been carried out to validate that QUIBIM “Bone morphometry and mechanical properties v1.0” imaging biomarkers serve as a long-term prognostic value of the considered pathologies.



Page 19 | 33



[email protected]

+34 961 243 225

www.quibim.com


8. RESULTS

a. PRECISION

• Image Acquisition

Intra- Centre Table 3 shows the results of the intra-centre reproducibility studies performed in the

Hospital La Fe of Valencia and the Hospital PTS of Granada. Both phantoms, 15 PCF

and 30 PCF, were acquired in each hospital and were analyzed through the QUIBIM

Precision® Morphometry & Mechanical Analysis tool. CoV for each test was between

0.41% and 11.43 %. These CoV results are all below 15%, which is the acceptable

threshold fixed by the European Medicines Agency (EMA) recommendations for the

validation of new analytical methods.

ACQ.

CENTRE

Co

V

ANALYSIS BV/TV Tb.Th Tb.Sp Tb.N D2D D3D QTS

Granada 15GR 1.42 1.02 0.23 0.41 0.51 3.39 4.25

Granada 30GR 6.84 5.67 0.61 1.17 4.05 4.06 11.44

Valencia 15GR 0.49 1.10 0.40 0.72 0.62 1.10 4.46

Valencia 30GR 1.17 1.96 0.37 0.79 1.72 1.73 4.79 Table 3. Results of the Intra-Laboratory reproducibility study of the QUIBIM Precision® Bone tool having bone

phantoms as reference standard.

In (Table 15) of the ANNEXES, it can be appreciated the breakdown of the Intra-Centre

variability results of the study.

Inter-Centre Table 4 shows the results of the inter-centre Reproducibility Studies, which is based on

the comparison of the quantitative results obtained in the Hospital La Fe with the ones

obtained in the Hospital PTS usingthe QUIBIM Precision® Morphometry & Mechanical

Analysis solution. Most of the quantitative bone parameters have a CoV below 15%, the

accepted threshold of the European Medicines Agency (EMA) recommendations for

the validation of new analytical methods. These results confirm the excellent

reproducibility of QUIBIM Bone methodology among acquisitions of different

hospitals. Only the CoV of the Quality of Trabecular Structure (QTS) is 18,17%, higher

than 15% but under the Maximum Limit Of Quantification (MLOQ) of 20%. This



Page 20 | 33



[email protected]

+34 961 243 225

www.quibim.com


coefficient, being a multivariate combination of all the other parameters, is more

influenced by acquisition variability.

ACQ.

CENTRE

Co

V


Granada -

Valencia 15GR 2.37 6.02 2.00 3.83 0.52 3.49 18.17

Granada -

Valencia 30GR 4.72 6.98 5.01 7.73 2.83 3.30 12.16

Table 4. Results of the Inter-Laboratory reproducibility study of the QUIBIM Precision® Bone between the Hospital La

Fe and the Hospital PTS.

Table 16 of the ANNEXES shows the breakdown of the Inter-Centre variability results

of the study.

• Methodology

Human interaction:

Table 5 shows the results of the intra-observer reproducibility studies performed in the Hospital La Fe of Valencia and in the Hospital PTS of Granada. Both phantoms, 15 PCF and 30 PCF, were acquired in each hospital and 3 ROIs were analyzed through QUIBIM Precision® Bone Morphometry and Mechanical Analysis software. Every observer reproducibility test has a CoV below 15%, the accepted threshold of the European Medicines Agency (EMA) recommendations for the validation of new analytical methods. These results show an excellent reproducibility of QUIBIM Bone methodology among different observers, confirming that the QUIBIM bone analysis is not hindered by human interaction.

ACQ.

CENTRE

Co

V


Granada 15GR 2.72 2.31 0.28 0.54 1.11 3.25 8.09

Granada 30GR 6.73 5.43 0.59 1.30 2.81 3.26 11.48

ACQ.

CENTRE

Co

V

ANALYSIS BV/TV Tb.Th Tb.Th Tb.N D2D D3D QTS

Valencia 15GR 2.59 1.46 0.31 1.36 1.63 1.15 7.51

Valencia 30GR 2.63 2.40 1.28 0.83 2.33 3.46 5.57



Page 21 | 33



[email protected]

+34 961 243 225

www.quibim.com


Table 5. Results of the Intra-Observer reproducibility study of the QUIBIM Precision® Bone solution in the Hospital PTS

of Granada (up) and in the Hospital La Fe of Valencia (down).

Tables 17 and 18 of the ANNEXES show the breakdown of the Intra-Observer variability

results of the study in the Hospital PTS of Granada and in the Hospital La Fe of Valencia.

Algorithm:

Table 6 shows the results of the algorithm repeatability study of the QUIBIM Precision® Trabecular Bone Morphometry and Mechanical Analysis tool, performed in the QUIBIM Precision® platform. The CoV is 0% because in the trabecular bone analysis method there are no iterative or probabilistic algorithms, thus eliminating the intrinsic variability that these procedures may introduce in the results.

ANALYSIS

Co

V BV/TV Tb.Th Tb.Sp Tb.N D2D D3D QTS

P1-ROI1 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Table 6. Results of the Algorithm Repeatability study of the QUIBIM Precision® Bone solution carried out in the

QUIBIM Platform.

Table 19 of the ANNEXES shows the breakdown of the the QUIBIM Precision®

Trabecular Bone Morphometry and Mechanical Analysis algorithm repeatability results

of the study.

Table 20 of the ANNEXES shows the breakdown of global reproducibility study of the

the QUIBIM Precision® Trabecular Bone Morphometry and Mechanical Analysis

solution.

The most relevant results of the global precision study of the the QUIBIM Precision®

Trabecular Bone Morphometry and Mechanical Analysis tool are shown in table 7. This

global study is based on the comparison of all the bone imaging biomarkers obtained

for the 15 PCF phantom to the ones obtained for the 30 PCF phantom. Table 7 shows

the average CoV of the extracted parameters of the analysis for the case of variability

in the acquisition of the images and for the case of variability in the methodology.

The CoV for each bone imaging biomarker extracted (BV/TV, Tb.Th (mean), Tb.Sp (mean), Tb.Sp (SD), Tb.N, D2D, D3D) is below 15%. These results prove the excellent precision of QUIBIM bone imaging biomarkers among different observers and



Page 22 | 33



[email protected]

+34 961 243 225

www.quibim.com


laboratories, confirming the robustness of the methodology of the the QUIBIM Precision® Trabecular Bone Morphometry and Mechanical Analysis tool.



Precision

Image Acquisition

CoV global (%)

BV/TV 2.83

< 15%*

2/2

Tb.Th 3.79

Tb.Sp 1.44

Tb.N 2.44

D2D 1.71

D3D 2.85

QTS value 9.21

Methodology

CoV global (%)

BV/TV 2.932

< 15%*

Tb.Th-mean 2.32

Tb.Sp-mean 0.49

Tb.N 0.81

D2D 1.57

D3D 2.22

QTS value 6.53 Table 7. Global Precision Study of the QUIBIM Precision® Bone solution.



Page 23 | 33



[email protected]

+34 961 243 225

www.quibim.com


b. ACCURACY

• Phantom

QUIBIM has performed an internal accuracy validation project with phantoms

(scientific publication pending). The aim of this study is to evaluate the accuracy of the

QUIBIM Precision® Bone Morphometry and Mechanical Analysis tool measurements,

comparing them to the known BVTV and Tb.Sp from the reference standard synthetic

bones.

Only the measurements of BV/TV and Tb.Sp parameters can be validated as they are

the only two parameters of the synthetic bones with known values (ground truth).

Table 8 shows the BV/TV results obtained for the 15 PCF phantom and for the 30 PCF

phantom and the comparison to the known ground truth (phantom). As it can be seen,

the standard error is less than 15% in all cases.

BV/TV (%) Truth Average Standard

Deviation

Standard

Error Means

15PCF MDCT

15±1.5 16.42 1.69 0.75

MR 19.19 1.83 0.82

30PCF MDCT

30±3 27.63 4.83 2.16

MR 27.98 2.91 1.30 Table 8. Results of the BT/TV Accuracy study calculated by QUIBIM versus ground truth (bone phantom) extracted

from MR and MDCT Imaging.

Table 9 shows the Tb.Sp results obtained for the 15 PCF phantom and the ones for the

30 PCF phantom and the comparison to the known ground truth (phantom). As it can

be seen, the standard error is less than 15% in all cases.

Tb.Sp (mm) Truth Average Standard

Deviation

Standard

Error Means

15PCF MDCT

2±0.5 2.09 0.14 0.06

MR 1.32 0.08 0.04

30PCF MDCT

2±0.5 1.82 0.13 0.06

MR 0.96 0.07 0.03

Table 9. Results of the Tb.Sp Accuracy study calculated by QUIBIM versus ground truth (bone phantom) extracted from MR and MDCT Imaging.



Page 24 | 33



[email protected]

+34 961 243 225

www.quibim.com


In this case the analysis was performed for the MDCT and MR acquisitions of the 5

samples for the 30 PCF bone phantom. Table 10 shows that the analysis performed on

both modalities, MR and MDCT, yield accurate measurements. The standard error is

less than 15% in all cases.

Table 10. Bone Volume to Total Volume measurements extracted from MR and MDCT Imaging by QUIBIM Precision®

Software.

Table 11 shows the results of the global accuracy study of the QUIBIM Precision® Bone

solution. This global study is based on the comparison of the mean bone imaging

biomarkers (BV/TV and Tb.Sp) obtained for the 15 PCF phantom and the 30 PCF

phantom versus the known value for each parameter of the bone phantom.

BV/TV and Tb.Sp (mean) have a relative error below 15%. These results show the

excellent accuracy of QUIBIM Bone methodology versus the ground truth (bone

phantom).



Accuracy Phantom ɛ BV/TV 4.82

< 15%*

1/2 Tb.Sp 0.05

Pathology (biopsy)

ɛ -

- < 15%*

Table 11. Global Accuracy Study of the QUIBIM Precision® Bone solution.

PHANTOM MODALITYTruth

(%)

BVTV

(%)

Absolut

Error (%)

Relative

Error

(%)

BVTV

Mean (%)

Relative

Error

Mean SAMPLE30_1 CT 30 31,32 1,32 4,39

SAMPLE30_2 CT 30 32,73 2,73 9,10

SAMPLE30_3 CT 30 24,32 5,68 18,95

SAMPLE30_4 CT 30 21,18 8,82 29,41

SAMPLE30_5 CT 30 28,60 1,40 4,66

SAMPLE30_1 MR 30 32,88 2,88 9,60

SAMPLE30_2 MR 30 27,22 2,78 9,28

SAMPLE30_3 MR 30 25,12 4,88 16,28

SAMPLE30_4 MR 30 27,73 2,27 7,58

SAMPLE30_5 MR 30 26,94 3,06 10,20

13,30

27,98 10,59

27,63



Page 25 | 33



[email protected]

+34 961 243 225

www.quibim.com


c. CLINICAL ENDPOINTS

• Short Term

PARAMETER HEALTHY OSTEOPOROSIS p-value

Morphology

BV/TV [x100 %] 0.22 ± 0.03 0.21 ± 0.03 0.45

Tb.Th [µm] 192.90 ± 4.40 188.50 ± 4.70 0.03

Tb.Sp [µm] 998.00 ± 106.70 1086.80 ± 94.30 0.05

Tb.N [10-3 µm-1] 1.11 ± 0.14 1.19 ± 0.16 0.26

Complexity D2D 1.55 ± 0.03 1.50 ± 0.06 0.04

D3D 2.33 ± 0.04 2.27 ± 0.03 0.001

Anisotropy θa 68.51 ± 14.58 71.26 ± 13.16 0.57

DA 1.11 ± 0.02 1.14 ± 0.02 0.001

Mechanical

EappX[MPa] 56.03 ± 21.54 20.55 ± 24.66 0.01

EappY[MPa] 54.22 ± 21.77 39.99 ± 35.59 0.43

EappZ[MPa] 114.72 ± 106.94 59.12 ± 40.58 0.04 Table 12. Results of the Clinica Value study calculated by QUIBIM in a healty and a pathological (osteoporosis) cohort. Table 13 shows the results of the global clinical value study of the QUIBIM Precision® bone solution. This study aimed to find significant differences (p<0.05) between the values of bone imaging biomarkers of a healthy cohort and a pathological one.

Most of the bone imaging biomarkers evaluated present statistically significant

differences between healthy and pathological (osteoporosis) cohorts (p<0,05). In the

case of BV/TV, Tb.N, θa, EappY, there are no differences between both groups. The

most significant imaging biomarker for the differentiation of groups is D3D.



Page 26 | 33



[email protected]

+34 961 243 225

www.quibim.com




Clinical Endpoints

Short term

Correlation

BV/TV 0.45

p < 0.05 1/2

Tb.Th 0.03

Tb.Sp 0.05

Tb.N 0.26

D2D 0.04

D3D 0.001

θa 0.57

DA 0.001

EappX 0.01

EappY 0.43

EappZ 0.04

Long term Correlation - - p < 0.05

Table 13. Global Clinical Value Study of the QUIBIM Precision® Bone solution.



Page 27 | 33



[email protected]

+34 961 243 225

www.quibim.com


9. CONCLUSIONS

PERFORMANCE INDICATOR ASSAYS STATISTICS VARIABLES RESULTS ACCEPTABLE VALIDATED

Precision

Image Acquisition


CoV (%)

BV/TV 2.83

< 15%*

2/2

Tb.Th 3.79

Tb.Sp 1.44

Tb.N 2.44

D2D 1.71

D3D 2.85

QTS value 9.21

Methodology Coefficient of Variation

CoV (%)

BV/TV 2.93

< 15%*

Tb.Th-mean 2.32

Tb.Sp-mean 0.49

Tb.N 0.81

D2D 1.57

D3D 2.22

QTS value 6.53

Accuracy

Phantom Relative

error ɛ (%)

BV/TV 4.82 < 15%*

1/2

Tb.Sp 0.05

Pathology (biopsy)


-

- < 15%*

Clinical Endpoints

Short term


BV/TV 0.45

p < 0.05

1/2

Tb.Th 0.03

Tb.Sp 0.05

Tb.N 0.26

D2D 0.04

D3D 0.001

θa 0.57

DA 0.001

EappX 0.01

EappY 0.43

EappZ 0.04

Long term


- - p < 0.05

✓ Validated Imaging Biomarker 4/6

* The CoV should be below 15%, which is the Lowest Limit Of Quantification (LLOQ), considering a maximum CoV of 20%.

Table 14. Global Validation Study of the QUIBIM Precision® Bone solution Following the validation methodology of imaging biomarkers, Quibim Precision® Bone Morphometry and Mechanical properties Analysis has been validated by fulfilling the established requirements.



Page 28 | 33



[email protected]

+34 961 243 225

www.quibim.com


10. REFERENCES

[1] Rationale of the Spanish FRAX model in decision-making for predicting osteoporotic

fractures: an update of FRIDEX cohort of Spanish women [Internet]. [cited 2016 Jul 6].

Available from:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4912785/?report=classic

[2] Silva BC, Leslie WD, Resch H, Lamy O, Lesnyak O, Binkley N, et al. Trabecular bone

score: a non-invasive analytical method based upon the DXA image. J Bone Miner Res.

2014 Mar;29(3):518–30.

[3] Wehrli FW, Saha PK, Gomberg BR, Song HK, Snyder PJ, Benito M, et al. Role of

magnetic resonance for assessing structure and function of trabecular bone. Top Magn

Reson Imaging. 2002;13:335-55.

[4] Newitt DC, van Rietbergen B, Majumdar S. Processing and analysis of in vivo high-

resolution MR images of trabecular bone for longitudinal studies: reproducibility of

structural measures and micro-finite element analysis derived mechanical properties.

Osteoporosis Int 2002; 13:278–287

[5] Carballido-Gamio J, Majumdar S. Clinical utility of microarchitecture measurements

of trabecular bone. Curr Osteoporos Rep 2006; 4:64–70

[6] Alberich-Bayarri A, Marti-Bonmati L, Angeles Pérez M, Sanz-Requena R, Lerma-

Garrido JJ, García-Martí G, et al. Assessment of 2D and 3D fractal dimension

measurements of trabecular bone from high-spatial resolution magnetic resonance

images at 3 T. Med Phys. 2010 Sep;37(9):4930–7.

[7] Alberich-Bayarri A, Marti-Bonmati L, Sanz-Requena R, Belloch E, Moratal D. In vivo

trabecular bone morphologic and mechanical relationship using high-resolution 3-T

MRI. AJR Am J Roentgenol. 2008 Sep;191(3):721–6.

[8] Alberich-Bayarri A, Martí-Bonmatí L, Sanz-Requena R, Sánchez-González J, Hervás

Briz V, García-Martí G, et al. [Reproducibility and accuracy in the morphometric and

mechanical quantification of trabecular bone from 3 Tesla magnetic resonance

images]. Radiologia. 2014 Feb;56(1):27–34.

[9] Sawbones | Open Cell Block 15 PCF [Internet]. [cited 2016 Sep 29].



Page 29 | 33



[email protected]

+34 961 243 225

www.quibim.com


[10] Alberich-Bayarri, A. In vivo morphometric and mechanical characterization of trabecular bone from high resolution magnetic resonance imaging. PhD Dissertation. 2010. [11] Wehrli FW. Structural and functional assessment of trabecular and cortical bone by

micro magnetic resonance imaging. J Magn Reson Imaging. 2007;25:390-09.

[12] Gullberg B, Johnell O, Kanis JA. World-wide projections for hip fracture.

Osteoporos Int. 1997;7:407-13.

[13] Alberich-Bayarri A, Marti-Bonmati L, Sanz-Requena R, Belloch E, Moratal D. In vivo

trabecular bone morphologic and mechanical relationship using high-resolution 3-T

MRI. AJR Am J Roentgenol. 2008;191:721-6.

[14] Nieto L, Moratal D, Marti-Bonmati L, Alberich-Bayarri A, Galant J. Morphological

characterization of trabecular bone structure using high resolution magnetic resonance

imaging. Radiología. 2008;50:401-8.

[15] Newitt DC, Majumdar S, van Rietbergen B, von Ingersleben G, Harris ST, Genant

HK, et al. In vivo assessment of architecture and micro-finite element analysis derived

indices of mechanical properties of trabecular bone in the radius. Osteoporos Int.

2002;13:6-17.

[16] Whitehouse WJ. The quantitative morphology of anisotropic trabecular bone. J

Microsc. 1974;101:153-68.

[17] Gomberg BR, Wehrli FW, Vasili´c B, Weening RH, Saha PK, Song HK, et al. Reproducibility and error sources of micro-MRI-based trabecular bone structural parameters of the distal radius and tibia. Bone. 2004;35:266-76. [18] Sell CA, Masi JN, Burghardt A, Newitt D, Link TM, Majumdar S. Quantification of trabecular bone structure using magnetic resonance imaging at 3 Tesla - calibration studies using microcomputed tomography as a standard of reference. Calcif Tissue Int. 2005;76:355-64. [19] Majumdar S, Genant HK, Grampp S, Newitt DC, Truong VH, Lin JC, et al. Correlation of trabecular bone structure with age, bone, mineral density, and osteoporotic status: in vivo studies in the distal radius using high-resolution magnetic resonance imaging. J Bone Miner Res. 1997;12: 111-8.



Page 30 | 33



[email protected]

+34 961 243 225

www.quibim.com


[20] Alberich-Bayarri A, Marti-Bonmati L, Pérez MA, Sanz-Requena R, Lerma-Garrido JJ, García-Martí G, et al. Assessment of 2 D and 3 D fractal dimension measurements of trabecular bone from high-spatial resolution magnetic resonance images at 3 Tesla. Med Phys. 2010;37:4930-7. [21] Ito M, Ikeda K, Nishiguchi M, Shindo H, Uetani M, Hosoi T, et al. Multi-detector row CT imaging of vertebral microstructure for evaluation of fracture risk. J Bone Miner Res. 2005;20:1828-36. [22] Geraets WG, van der Stelt PF, Lips P, Elders PJ, van Ginkel FC, Burger EH. Orientation of the trabecular pattern of the distal radius around the menopause. J Biomech. 1997;30:363-70.



Page 31 | 33



[email protected]

+34 961 243 225

www.quibim.com


11. ANNEXES

Table 15. Results of the Intra-Laboratory reproducibility study of the QUIBIM Precision® Bone tool having bone phantoms as reference standard.

Table 16. Results of the Inter-Laboratory reproducibility study of the QUIBIM Precision® Bone between the Hospital La

Fe and the Hospital PTS.



Page 32 | 33



[email protected]

+34 961 243 225

www.quibim.com


Table 17. Results of the Intra-Observer reproducibility study of the QUIBIM Precision® Bone solution in the Hospital

PTS of Granada.

Table 18. Results of the Intra-Observer reproducibility study of the QUIBIM Precision® Bone solution in the Hospital La

Fe of Valencia.



Page 33 | 33



[email protected]

+34 961 243 225

www.quibim.com


ANALYSIS

Co

V

BV/TV Tb.Th Tb.Sp Tb.N D2D D3D QTS

P1-ROI1 43.15 527.72 652.04 0.82 1.70 2.39 5.61

P1-ROI1 43.15 527.72 652.04 0.82 1.70 2.39 5.61

P1-ROI1 43.15 527.72 652.04 0.82 1.70 2.39 5.61

P1-ROI1 43.15 527.72 652.04 0.82 1.70 2.39 5.61

P1-ROI1 43.15 527.72 652.04 0.82 1.70 2.39 5.61

Table 19. The results of the algorithm variability study of the QUIBM Precision® Bone morphometry and mechanical properties solution.

Table 20. The Results of the Global reproducibility study of the QUIBM Precision® Bone solution.



Date post:	25-Jul-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Quibim Precision® Validation Methodology and Technical...

Documents