+ All Categories
Home > Documents > Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in...

Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in...

Date post: 27-Mar-2015
Category:
Upload: samantha-mcnulty
View: 213 times
Download: 1 times
Share this document with a friend
Popular Tags:
17
Group 5: Historical control data • Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as an important topic, but was not discussed in depth because of lack of time. • A short session took place in Basel to further discussed this topic. 5th IWGT, Basel August 17-19, 2009 Group 5: Historical control data
Transcript
Page 1: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

Group 5: Historical control data

• Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as an important topic, but was not discussed in depth because of lack of time.

• A short session took place in Basel to further discussed this topic.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 2: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

Assigned Participants

Kerry Dearfield

Makoto Hayashi (Chair)

David Jacobson-Kram: not able to attend

Peter Kasper

David Lovell (Co-chair)

Hans-Joerg Martus

Veronique Thybaud (Rapporteur)

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 3: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

Objectives• The group mainly discussed negative control data. • Historical control data:

- Until now used for acceptance criteria. - More and more used also for interpretation of results.

• Up to now there is no clear guidance.• Objectives of the group are:

- To provide generic advice for historical control data that could be applied to all assays, not to describe assay specific recommendations. - To make consensus on the following questions: What are the historical control data? How to make the historical control data? How to use the historical control data?

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 4: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

• Quite helpful, formed the basis of initial discussion• Statements still acceptable, but needed to be widened to includefull range of assays including in vitro ones.

An example of use for in vivo assays, i.e. data distribution and the use of quality control charts to explore changes over in negative control data in a large set of in vivo MNT data generated in a same laboratory.

Page 5: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

What are the historical control data? -1-

• It is the complete set of data from previous experiments.• It includes both individual values and the means of the

negative control groups of that experiment.• Individual values:

For in vivo assays: animalsFor in vitro assays: cultures, plates

• All experiments must have a concurrent negative control and all concurrent negative controls should be added to derive at a representative negative historical control.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 6: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to build up the historical control data? -1-

• When you are building up historical control data you put in all data that have been accumulated over the past.

• Ideally data should be accumulated in order to visualize the distribution for individuals and the group means, and to calculate the key parameters to be used for data analysis.

• Graphical control charts may be useful for exploring the data.• Data should only be excluded from the data set when there is

a clear and valid reason to reject it. • Reporting only a range (i.e., minimum, maximum) of data

from the historical studies is not adequate.• The data set should be managed so that more recent data

carry more weight that older data.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 7: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to build up the historical control data? -2-• The choice of the specific statistics such as mean, median

etc. for the end-points to be assessed in historical control data should be determined by the assay community, based upon their experience on the most appropriate parameters.

• Historical control data set should ideally be as big as possible but probably comprise at least 10 separate experiments, but preferably 20.

• However, this may depend upon the use of historical data (e.g., acceptance criteria, distribution, internal quality control), on the variability within an assay, and also on the practicability (number of assays per year, etc).

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 8: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to build up the historical control data? -3-• The experimental protocol must have remained fixedthroughout the period covered by the historical data andthe current experiment, unless it can be demonstrated thatthe change does not impact the values.- The method of scoring must be unchanged during the relevant period.

- The experimental units must be comparable throughout the period.

- There must exist no known systematic differences between the various control groups, current and historical, that would produce systematic differences in response, e.g., the data must have been gathered by the same investigators within the same laboratory.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 9: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to build up the historical control data? -4-

Outliers: May need more work. • Criteria for exclusion:

- Should not be excluded just because unusual. - Should only be excluded if scientifically justified, as could be part of assay variability.

• If decisions are only based on “ranges”, outliers have disproportionate effect, that is one reason why the use of ranges is not recommended, and confidence interval type approaches are preferred.

• Values should be consistent with data reported in literature. • Acceptable “ranges” should be defined for each assay by expert

group, e.g., as already done for MLA.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 10: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to use the historical control data? -1-• Two distinct approaches to addressing negative historical

control data: - To assess whether the concurrent control data is consistentwith previous negative control data collected by that laboratory.Each assay will have its own specific acceptance criteria.- To assess within a laboratory whether the techniquesused are still in “control” based upon a series of negative

controls results. This can be assessed by formal quality control.

• The historical control is very important for the technical establishment and familiarization of the assay at the laboratory.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 11: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to use the historical control data? -2-

• Concurrent negative controls are essential:- As part of the acceptance criteria for the evaluation of the technical acceptance of the study.- For appropriate statistical treatment of the data.- To accept a test (by comparing concurrent negative control values to the historical negative control).

- To help assess the biological relevance of experimental data (e.g., statistically significant increases detected, but values comparable to historical control data).

• Nevertheless, it can be considered as a two step process: - First compare the experimental data with the concurrent negative

control,- then consider if meaningful as compared to historical data

distribution.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 12: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to use the historical control data? -3-

To design an assay (1):

• Assay/studies should be designed (number of duplicates, number of cells scored, etc.):

- To detect a biologically important increase over the background.- To detect a predetermined effect size/ difference from background (test specific) of the observed endpoint at an a-level of 0.05 and a power of 80% was considered to be an easily achievable minimum standard. - Based on the a-level (type I error) for recognizing a predetermined increment over the spontaneous level with an acceptable statistical power (1-type II error).

• In some assays doubling is used but this may not mean the same thing biologically for all assays and depends on the background.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 13: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to use the historical control data? -4-

To design an assay (2):

• It should be generally required that a clear description of the design of the experiment including the power of the test, the statistical methods used to analyze the data and the level of significance actually achieved is given in every data report.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 14: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to use historical control data? -5-• How do you reject a control group, and therefore the experiment,

from an experiment because it is unacceptable: - The group advice against the use of range for this acceptance

limit criteria (see earlier). - These could be based upon either the observed spread of

data or on calculated from the distribution values using (say) 2 or 3 standard deviations or upper 99% percentile from the mean of that which was actually observed.

- The choice of criteria has to be determined for the individual assay based upon the specific characteristics of the assay. Each assay determines its own acceptance/limit criteria which is consistent with historical data generated by that scientific community. For example, could be useful to define performance standard and acceptable “range” or interval confidence for all assays as done before for MLA, or 5% for CA.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 15: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to use of historical control data? -6-• If a negative control group in an experiment is outside the

acceptance criteria the experiment is generally no longer valid.

• How far outside the acceptance criteria is still valid should be managed for each assay and should be consistent with values reported in scientific community.

• However some use may be made with the information of the experiment if for instance the treated group show a clear increase. But care is needed in case such an increase is also a result of artifact such as problem with the culture in an in vitro study that could have impacted all cultures and the whole experiment.

• In case of negative results you would have to repeat the experiment.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 16: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

How to use of historical control data? -7-

• Historical control data may have a role in defining the size of effect that is considered biologically important.

• Such considerations may have a role in the future in putting dose-response relationship seen in experimental studies into perspective in the context of the identification of “practical or pragmatic threshold”.

• Considerable care would be needed in the choice of data to use in such assessment and more work is probably needed to assess the potential of this approach.

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data

Page 17: Group 5: Historical control data Follow-up of 2005 IWGT where the use of historical control data in interpretation of in vitro results was identified as.

Thank you

5th IWGT, BaselAugust 17-19, 2009Group 5: Historical control data


Recommended