Hernani Cualing MD 1
Slide Based Cytometry of
Immunostained Tissue:
"Virtual Flow Cytometry" on a
Slide
Hernani Cualing MD 2
Background
• The standard of clinical practice is to estimate the percentage of immunohistochemically (IHC) stained cells
• This practice is subjective and often gives a wide range of results that depends on the level of the microscopists’ skill. – limitation of our visual system
– difficulty in counting positive(or negative) cells • overlapped stained nuclei
• variability of immunostaining presence of other irrelevant objects
• Takes time to manually count these cells
• Wide intra- and interobserver results because of subjectivity
• Developing a cytometry tool that promises similar objectivity as flow cytometry, will only decrease the incidence of errors in mensuration and diagnosis.
Hernani Cualing MD 3
Quantitative IHC Report-All iHC are
estimates
Above example is the semiquantitative
report by estimate- subjective, may be
inaccurate,may contribute to error
propagation in diagnosis and prognosis
Hernani Cualing MD 4
Although the hue value of the neck
of the hourglass remains the same,
we do not see them as equal.
Although we excell in pattern recognition
our color visual system is influenced by
the saturation and intensity components of color;
we are also poor judge of intensity differences
Human perception of color is limited.
(hue = 0)
Hernani Cualing MD 5
Computerized algorithms could separate color
into its components for analysis- left panel,magnified view
of immunohistologic stained cells- negative nuclei and positive nuclei, right panel are
the hue, saturation and intensity components of the pixels
Hernani Cualing MD 6
Overlapping and variability of
staining and presence of other
cells of different class(type)
Positive
lymphoid cells
Negative
lymphoidcells
Non
lymphoid
negative
tumor cells
Irrelevant
objects
Hernani Cualing MD 7
Stained and unstained cells of the same class, ie, lymphocytes
unstained
Hernani Cualing MD 8
Flow cytometry, as a model for this virtual flow cytometry, classifies cells of
the same class as positive and negative on a defined set of marker/s
Hernani Cualing MD 9
Flow cytometry uses fresh cell suspension to give objective results but the observer
loses visual confirmation. In contrast, immunohistochemistry using fixed tissue,
allows observer to see but loses the objectivity in the counting of these cells. The
flow cytometry also loses the location information-location cytomics is not
recovered.MARKERS
– IMMUNOPHENOTYPE
• flow
cytometry
• Immunohisto-
chemistry
Hernani Cualing MD 10
Hernani Cualing MD 11
Applications of virtual flow cytometry
• Research applications– Tissue culture biomarkers analysis; location cytomics
• Prognostic markers– tumor infiltrating lymphocytes (CD8 TILs in B-cell
lymphomas, sarcomas, carcinomas, and others)
• Diagnostic uses
– CD4/CD8 ratio
– Kappa Lambda ratio
– Proliferative Index ( Ki-67, Mib1)
• Therapeautic applications
– Monitoring targeted cell antigens pre- and post-therapy in PTCLs-AntiCD2,anti CD30, etc
– Monitoring CD20 post-Rituxan therapy
Hernani Cualing MD 12
Challenges of Slide based virtual Flow Cytometry
1)Thick tissue sections with overlapping of cells,
2) Variability of technics and platforms in immunostaining of tissues,
3) Optimal sampling resolution for obtaining size and staining data per cell object ( objective magnification 10x vs 20x vs 40x),
4) Issues of optical output properties of image acquisition hardware/s,
5 ) Lack of cell based algorithms able to detect dual populations (positive immunostained cells and non-immunostained relevant cells of the same class) and,
6) Arriving at an automated and adaptive thresholding algorithm based on the characteristics of the objects and the background non objects,
7) Determining image specific parameters that optimize detection of percentage positive cells
Hernani Cualing MD 13
Solutions• 1. Quality Histology ( 2 micron sections and good hematoxylin countestain)
• 2. Using an automated industry standard autostainer
• 3. At 20x magnification, sampling is optimal ; 10x too pixelated, 40x less cells per frame
• 4. Built in contrast enhancement provided by the three chip CCD blue, green and red channels
• 5. Enhanced image lends easily to “dual thresholding” of positive and negative cells of the same class; thresholding divided the image pixels into positive and negative cell objects; here we threshold for brown and then blue pixels
• 6. Adaptive automated histogram based thresholding algorithm combined with percentile thresholding; the latter to discard the darkest and brightest pixels
.
• 7. We discovered the parameters obtained from the image itself to automate the thresholding function using the entropy algorithm. We found a good correlation between the ratio of positive cell objects with negative cell objects and the value of this parameter entropy.
Hernani Cualing MD 14
• Image segmentation- is the process of dividing an image into regions;
in digital images based on pixels (picture elements) properties into
objects and into background. Most use either thresholding of pixel
histogram or localizing object boundaries using edge detection.
• The simplest and most widely used segmentation method is
thresholding- where all the pixels below a certain darker threshold is
assigned the object and above, the background –grayscale and color
• Cellular logic operations- to further define objects
Image analysis components for slide based cytometry
1. Segmentation- need dual mode for population of positive and
negative
2. Feature extraction-size, perimeter, staining density, etc.
3. Classification- positive and negative, percent positive, stain
density of positive
Definitions
Hernani Cualing MD 15
Edge vs Thresholding
Two major approaches in segmentation use the property of color and gray
level values of objects: discontinuity and similarity.
Objects with well-defined discontinuities usually benefit from edge-based
detection and
those with poorly defined discontinuities may benefit from similarity
approach such as thresholding. Accurate segmentation remains one
of the problems in pixel based image ratio with up to 15% error on both
falsely positive and falsely negative results (USPTO patent #
6,553,135).
Because of the complex, discontinuous, multicolor, large scale
images in immunohistochemically stained biologic cells, We
implemented cell-based thresholding algorithm.
Hernani Cualing MD 16
The generic method in extracting information from an image by
segmentation is by thresholding the brightness of pixels-
divides the image into component objects or regions based on
their distribution in the brightness scale; Objects==below
threshold and background==above threshold, that is for gray
scale images. For colored images, the issue is more complex.
Hernani Cualing MD 17
Approaches:
1) simple gray scale thresholding?
Hernani Cualing MD 18
Raw color frame for manual interpretation
Tumor nuclei
CD8 positive
T-lymphocytes
CD8 negative
lymphocytes
Hernani Cualing MD 19
corresponding computer generated
interpretation
Tumor nuclei
CD8 positive
T-lymphocytes
CD8 negative
lymphocytes
Hernani Cualing MD 20
Hernani Cualing MD 21
Strategy for reducing the theory of a slide
cytometer or virtual flow cytometer to practice
1. Count positive and negative cells of the
same cell type-phase I
2. Extract the size and color intensity
information and generate a two parameter
histogram distribution-phase II
3. Validate the results in relation to
manual count and flow cytometry results on a
defined cancer type-phase III.
Hernani Cualing MD 22
Results of Phase I Study : CD8 Tumor
infiltrating Lymphocytes with Positive and
Negative Cells
Case x Immunostai
nDensity Lymphocyte Number
Lymphocytes (+) stain
Lymphocyte stain (+) (%)
1 4 58 23 39.655174 2 4 52 22 42.307693 3 4 57 43 75.438599 4 4 56 22 39.285713 5 4 61 44 72.131149 6 4 106 48 45.28302 7 4 54 25 46.296295 8 4 74 13 17.567568 9 4 43 33 76.744186
10 4 45 22 48.888889 11 4 36 18 50 12 4 73 8 10.958904 13 4 131 13 9.923664 14 4 86 24 27.906977 15 4 42 18 42.857143
Min 4 36 8 9.923664 Max 4 131 48 76.744186 Median 4 57 22 42.857143 Total 974 376 Average 4 64.933334 25.066668 38.603695
Hernani Cualing MD 23
Phase II extraction of cell
objects data• The formula below was used to convert pixel area to cell diameter in
microns ( Y data), for a 20x magnified image:
Cell diameter = 2 * (sqrt of (Area in pixels / )) / 1.5
= sqrt of (Area in pixels * P) ;where diameter is in microns, is conversion factor (
0.56588424212) for 20x
• The optical density data per cell object was given by the sum of optical density
of the colored pixels comprising the previously labeled Cell objects ( X data).
• Each cell object then was labeled and contained size and density numbers
which were used for the dot plot display. The final resulting data then consisted
of an array of labeled objects with size and corresponding stain density.
• These two parameters defined the frequency distribution of the dual cell objects
in the two parameterl dot plot: a display commonly used in flow cytometry.
Hernani Cualing MD 24
Flow cytometry immunophenotypic analysis is applied to cell suspension to
give the percentage values of stained and unstained cells based on their
fluorescence intensities.
Hernani Cualing MD 25
iHCFlow ™ Tissue Cytometry
• A method and approach is developed-preservation of location cytomics.
• The conversion of immunohistochemistry (IHC) data to a flow cytometry-like two parameter dot-plot display, is calledIHCFLOW technic and because the dot plot is obtained from a slide using single cell analysis of the same population; is also called a virtual flow cytometry.
Hernani Cualing MD 26
How we differ from previous studies
– Other systems use multispectral instrumentation and adaptive wavelength filters
– confocal immunofluorescent microscopy to obtain separation of positive stain from nonstained pixels.
– few hybrid microscope-flow cytometry were developed to obtain single cell information to mimic flow cytometry.
– Immunofluorescent technics were applied on slide substrate to obtain cell populations using laser scanning instrumentation
• The IHCFLOW technic does not need complex, fluorescent and expensive instrumentation. It needs a 3 chip RGB CCD equipped microscope
• Some studies looked at counting immunohistochemistry stained slides, but none converted the results to a flow cytometry paradigm.
Hernani Cualing MD 27
Virtual Flow cytometry
Hernani Cualing MD 28
Segmentation and Result
Hernani Cualing MD 29
Results of Flow Cytometry
and Virtual Flow Cytometry
6 % CD8 +5.08 % CD8
+
Hernani Cualing MD 30
CD3
CD3 = 24%
Hernani Cualing MD 31
BCL-1
85%
Hernani Cualing MD 32
Ki-67
28%
Hernani Cualing MD 33
CD5
55%
Hernani Cualing MD 34
Slide Based image cytometry
Slide based cytometry promises to be a tool to complement flow cytometry
and enhance objectivity in histologic analysis
Niche: For tissue and cell-based quantitative analysis of
immunohistochemistry stained cells
Other applications:
Quantitative tissue microarrays
Stromal marker studies
Quantitative biomarker profile of epithelia
Tumor infiltrating lymphocytes in mantle cell lymphomas
Tumor infiltrating lymphocytes in colonic carcinomas
Hernani Cualing MD 35
Tissue MoAb Microarray ( bcl-1 in mantle cell
lymphoma)
Hernani Cualing MD 36
Tissue Microarray Slide Based Flow Cytometry
Hernani Cualing MD 37
Hernani Cualing MD 38
Area X Y Perim
.
Angle Circ. Feret IntDe
n
Skew %Are
a
XStart YStart
1 51 253 6 25 0 -1 9 0 0 100 250 2
2 118 503 7 48 174 1 18 0 0 100 505 2
3 173 468 10 47 172 1 17 10455 1 76 464 3
4 73 56 12 32 56 1 13 0 0 100 55 7
5 167 220 22 49 66 1 19 1530 5 96 221 13
6 299 128 29 68 69 1 24 0 0 100 123 19
7 54 145 26 66 20 0 16 0 0 100 145 20
8 830 191 38 129 33 1 50 85425 0 60 193 21
9 227 233 30 57 74 1 20 16065 1 72 230 21
Hernani Cualing MD 39
Ki67 in tonsilar epithelium- 31 %
Hernani Cualing MD 40
Ki67 in tonsilar epithelium- 18 %
Hernani Cualing MD 41
VALIDATION OR PHASE III: PROJECT on
MANTLE CELL LYMPHOMAS AND Tumor
Infiltrating lymphocytes ( TIL’S)
Hernani Cualing MD 42
DESIGN AND RESULTS
• Segmentation of a 512 x 474 RGB image and tabular display of statistical results table took 2-3 seconds using proprietary developed algorithms ( by IHCFLOW, Inc).
• We used a panel of 7 antibodies for validation on 14 cases of mantle cell lymphoma giving percentage positive, total lymphocytes, and staining density. A total of 2,027 image frames with 810,800 cell objects (COBs) were evaluated. Antibodies to CD3, CD4, CD8, Bcl-1, Ki-67, CD20, CD5 were subjected to virtual flow cytometry on tissue. The results of Tissue Cytometry were compared with manual counts of expert observers and with the results of flow cytometric immunophenotyping of the same specimen.
• The correlation coefficient and 95 % confidence interval by linear regression analysis yielded a high concordance between manual human results (M), flow cytometry results (FC), and Tissue Cytometry (TC) results per antibody, (r =0.9365 manual vs TC, r =0.9537 FC vs TC). The technical issues were resolved and the solutions and results were evaluated and presented
• Reference: Cualing HD,Moscinski LC, Zhong E. “Virtual Flow Cytometry” of Immunostained Lymphocytes on Microscopic Tissue Slides, Clinical Cytometry Jan 15, 2007.
Hernani Cualing MD 43
Correlation
r = 0.95
r = 0.93
Hernani Cualing MD 44
Mantle cell lymphoma and TIL’s
Hernani Cualing MD 45
S04-A Y a S04-c y c
TC manual flow TC manual flow TC
bcl1 94 95 81 85 79
cd3 36 30 34 24 25 29 31
cd4 19 20 16 19 20 21 25
cd5 86 50 89 80 85 75 99
cd8 14 20 15 15 9 8 7
cd20 96 90 86 84 90 76 94
ki67 8 10 10 12 56
Hernani Cualing MD 46
CD8 and CD20 tumor infiltrating lymphocytes (TILs)
and prognosis of adenocarcinoma of the colon
CD8 CD20
carcinoma
Hernani Cualing MD 47
Hernani Cualing MD 48
Quantitative microscopy redefined
Quantitative microscopy is increasingly used in both pathology and biomedical research applications
Applications needing morphometry values from digital images of microscopic sections of tissue-”Tissue Cytomics or Tissomics”
Where morphometry is defined beyond the commonly used pixel area of the target objects per image frame(“pixel based analysis”) ;
But that of quantitative information obtained per cell-“Cell-based analysis or Cytomics”
Delineation of % of cells carrying the targeted biomarker in a population of the same class of cells “ Class targeted cytomics”
Hernani Cualing MD 49
Tissomics
• Combining diverse data streams across
different levels of observations-such as
molecular, cellular, clinical data-is a system
wide approach
• Tissue cytomics –high throughput high
content phenotyping methodology provides
data rich profile of cellular heterogeneity in
tissue enabling correlative statistics over
biologics targets
Hernani Cualing MD 50
Class-targeted Cytomics
• Need for slide based:
– Non consumptive analysis= re-analysis
– Allows visual validation
– Cell parameters could be correlated
– Fixed tissue
– Amenable to routine and experimental
setup
Hernani Cualing MD 51
Virtual flow cytometry applied to an
immunostained tissue analysed on a slide
matrix
• Reduced to practice, patent pending
• Not the same as antigen quantitation
• Has real world applications in daily
pathology practice
• CPT code ready- ie.88361- computer
assisted software analysis of IHC
• Similar to flow cytometry but performed
on paraffin fixed histologic sections
Hernani Cualing MD 52
Cytomics tool
• The iHCFlow ™ Tissue Cytometry fulfills the major Cytomics criteria of
• 1) relating multiple parameters to each other,
• 2) within large population of cells,
• 3) on a single-cell basis,
• 4) on a quantitative and observer-independent manner.
• But unlike the other cytomics systems which use immunofluorescent tissue stains, the system differs in using routine immunohistochemistry on tissue generating flow cytometry-like results.
• Ecker R C,Steiner GE. Microscopy-based multicolor tissue cytometry at the single-cell level. Cytometry A
2004;59:182-190.
Hernani Cualing MD 53
summarypopulation statistics and dot plot two-dimensional histogram results show a
reliable, robust, accurate reporting similar to that of a flow cytometer.
We differ from the current methods of image analysis systems by focusing on cell-based population statistics instead of pixel-area data.
The correlation between each case run in flow cytometry and estimated by experts and by the advanced image Tissue Cytometry is high and suggest a valid approach to objectively quantifying immunostaining
REFERENCES
Cualing HD,Moscinski LC, Zhong E. “Virtual Flow Cytometry” of Immunostained Lymphocytes on Microscopic Tissue Slides, Clinical Cytometry Jan 15,
2007.
Khalil F*, Heekin J, Lili Miles, Cualing H.The rate and morphometric pattern of bone marrow recovery post-myelosuppressive therapy for acute leukemia: a
quantitative study-submited to Archives of Pathology and Lab Medicine Aug 2007
Cualing H. Automated Analysis in Flow Cytometry. Cytometry, 2000 , 42:p.110-113.
Young IT, Quantitative Microscopy. IEEE Engineering in Medicine and Biology, 1996, 15(1): p.59-66.
Ridler TW ,Calvard S. Picture Thresholding Using Iterative Selection Method. IEEE Trans. On Systems, Man, and Cybernetics, 1978. SMC-8(8):p 630-632.
Cualing H. Kothari R, Balachander T. Immunophenotypic Diagnosis of Acute Leukemia Using Decision Tree Induction. Lab Investigation, 1999, 79:p.205-212.
Hernani Cualing MD 54
Collaborators
• IHCFLOW, Inc
• GreenGreat, Inc.
• E Zhong
• Marshall Kadin
• L Moscinski