Post on 02-Jan-2016
transcript
1
ITK Lecture 8 - NeighborhoodsITK Lecture 8 - Neighborhoods
Methods in Image AnalysisCMU Robotics Institute 16-725U. Pitt Bioengineering 2630
Spring Term, 2006
Methods in Image AnalysisCMU Robotics Institute 16-725U. Pitt Bioengineering 2630
Spring Term, 2006
Damion Shelton
2
Goals for this lectureGoals for this lecture
Understand what a neighborhood is and and the different ways of accessing pixels using one
Use neighborhoods to implement a convolution/correlation filter
Understand what a neighborhood is and and the different ways of accessing pixels using one
Use neighborhoods to implement a convolution/correlation filter
3
What is a neighborhood?What is a neighborhood? You may already be familiar with the concept of pixels having neighbors
Standard terminology in 2D image processing will refer to the 4 neighborhood (N,E,S,W) and the 8 neighborhood (4 neighborhood + NE, SE, SW, NW)
You may already be familiar with the concept of pixels having neighbors
Standard terminology in 2D image processing will refer to the 4 neighborhood (N,E,S,W) and the 8 neighborhood (4 neighborhood + NE, SE, SW, NW)
4
Neighborhoods in ITKNeighborhoods in ITK
ITK carries this concept a bit further
A neighborhood can be any collection of pixels that have a fixed relationship to the “center” based on offsets in data space
See 11.4 in the ITK Software Guide
ITK carries this concept a bit further
A neighborhood can be any collection of pixels that have a fixed relationship to the “center” based on offsets in data space
See 11.4 in the ITK Software Guide
5
Neighborhoods in ITK, cont.Neighborhoods in ITK, cont. In general, the neighborhood is not completely arbitrary Neighborhoods are rectangular, defined by a “radius” in N-dimensions
ShapedNeighborhoods are arbitrary, defined by a list of offsets from the center
The first form is most useful for mathematical morphology kinds of operations, convolution, etc.
In general, the neighborhood is not completely arbitrary Neighborhoods are rectangular, defined by a “radius” in N-dimensions
ShapedNeighborhoods are arbitrary, defined by a list of offsets from the center
The first form is most useful for mathematical morphology kinds of operations, convolution, etc.
6
Neighborhood iteratorsNeighborhood iterators
The cool & useful thing about neighborhoods is that they can be used with neighborhood iterators to allow efficient access to pixels “around” a target pixel in an image
The cool & useful thing about neighborhoods is that they can be used with neighborhood iterators to allow efficient access to pixels “around” a target pixel in an image
7
Neighborhood iteratorsNeighborhood iterators
Remember that I said access via pixel indices was slow? Get current index = I Upper left pixel index IUL = I - (1,1)
Get pixel at index IUL
Neighborhood iterators solve this problem by doing pointer arithmetic based on offsets
Remember that I said access via pixel indices was slow? Get current index = I Upper left pixel index IUL = I - (1,1)
Get pixel at index IUL
Neighborhood iterators solve this problem by doing pointer arithmetic based on offsets
8
Neighborhood layoutNeighborhood layout
Neighborhoods have one primary parameter, their “radius” in N-dimensions
The side length along a particular dimension i is 2*radiusi + 1
Note that the side length is always odd because the center pixel always exists
Neighborhoods have one primary parameter, their “radius” in N-dimensions
The side length along a particular dimension i is 2*radiusi + 1
Note that the side length is always odd because the center pixel always exists
9
A 2x1 neighborhood in 2DA 2x1 neighborhood in 2D
0 1 2 3 4
5 6 7 8 9
10 11 12 13 14
10
StrideStride
Neighborhoods have another parameter called stride which is the spacing (in data space) along a particular axis between adjacent pixels in the neighborhood
In the previous numbering scheme, stride in Y is amount then index value changes when you move in Y
In our example, Stridex = 1, Stridey = 5
Neighborhoods have another parameter called stride which is the spacing (in data space) along a particular axis between adjacent pixels in the neighborhood
In the previous numbering scheme, stride in Y is amount then index value changes when you move in Y
In our example, Stridex = 1, Stridey = 5
11
Neighborhood pixel accessNeighborhood pixel access The numbering on the previous page is important! It’s how you access that particular pixel when using a neighborhood iterator
This will be clarified in a few slides...
The numbering on the previous page is important! It’s how you access that particular pixel when using a neighborhood iterator
This will be clarified in a few slides...
12
NeighborhoodIterator accessNeighborhoodIterator access Neighborhood iterators are created using: The radius of the neighborhood The image that will be traversed The region of the image to be traversed
Their syntax largely follows that of other iterators (++, IsAtEnd(), etc.)
Neighborhood iterators are created using: The radius of the neighborhood The image that will be traversed The region of the image to be traversed
Their syntax largely follows that of other iterators (++, IsAtEnd(), etc.)
13
Neighborhood pixel access, cont.Neighborhood pixel access, cont.
1.2 1.3 1.8 1.4 1.1
1.8 1.1 0.7 1.0 1.0
2.1 1.9 1.7 1.4 2.0
Let’s say there’s some region of an image that hasthe following pixel values
14
Pixel access, cont.Pixel access, cont.
Now assume that we place the neighborhood iterator over this region and start accessing pixels
What happens?
Now assume that we place the neighborhood iterator over this region and start accessing pixels
What happens?
15
Pixel access, cont.Pixel access, cont.
1.20
1.31
1.82
1.43
1.14
1.85
1.16
0.77
1.08
1.09
2.110
1.911
1.712
1.413
2.014
myNeigh.GetPixel(7) returns 0.7so does myNeigh.GetCenterPixel()
16
Pixel access, cont.Pixel access, cont.
Next, let’s get the length of the iterator and the stride length
Size() returns the #pixels in the neighborhood
unsigned int c = iterator. Size () / 2;
GetStride returns the stride of dimension N
unsigned int s = iterator. GetStride(1);
Next, let’s get the length of the iterator and the stride length
Size() returns the #pixels in the neighborhood
unsigned int c = iterator. Size () / 2;
GetStride returns the stride of dimension N
unsigned int s = iterator. GetStride(1);
17
Pixel access, cont.
1.20
1.31
1.82
1.43
1.14
1.85
1.16
0.77
1.08
1.09
2.110
1.911
1.712
1.413
2.014
myNeigh.GetPixel(c) returns 0.7myNeigh.GetPixel(c-1) returns 1.1
18
Pixel access, cont.
1.20
1.31
1.82
1.43
1.14
1.85
1.16
0.77
1.08
1.09
2.110
1.911
1.712
1.413
2.014
myNeigh.GetPixel(c-s) returns 1.8myNeigh.GetPixel(c-s-1) returns 1.3
19
The ++ methodThe ++ method
In ImageRegionIterators, the ++ method moves the focus of the iterator on a per pixel basis
In NeighborhoodIterators, the ++ method moves the center pixel of the neighborhood and therefore implicitly shifts the entire neighborhood
In ImageRegionIterators, the ++ method moves the focus of the iterator on a per pixel basis
In NeighborhoodIterators, the ++ method moves the center pixel of the neighborhood and therefore implicitly shifts the entire neighborhood
20
Does this sound familiar?Does this sound familiar? If I say:
I have a region of interest defined by a certain radius around a center pixel
The ROI is symmetric I move it around an image
What does this sound like?
If I say: I have a region of interest defined by a certain radius around a center pixel
The ROI is symmetric I move it around an image
What does this sound like?
21
Convolution (ahem, correlation)!Convolution (ahem, correlation)!To do convolution we need 3
things:1. A kernel2. A way to access a region of an
image the same size as the kernel
3. A way to compute the inner product between the kernel and the image region
To do convolution we need 3 things:1. A kernel2. A way to access a region of an
image the same size as the kernel
3. A way to compute the inner product between the kernel and the image region
22
Item 1 - The kernelItem 1 - The kernel
A NeighborhoodOperator is a set of pixel values that can be applied to a Neighborhood to perform a user-defined operation (i.e. convolution kernel, morphological structuring element)
NeighborhoodOperator is derived from Neighborhood
A NeighborhoodOperator is a set of pixel values that can be applied to a Neighborhood to perform a user-defined operation (i.e. convolution kernel, morphological structuring element)
NeighborhoodOperator is derived from Neighborhood
23
Item 2 - Image access methodItem 2 - Image access method We already showed that this is possible using the neighborhood iterator
Just be careful setting it up so that it’s the same size as your kernel
We already showed that this is possible using the neighborhood iterator
Just be careful setting it up so that it’s the same size as your kernel
24
Item 3 - Inner product methodItem 3 - Inner product method The NeighborhoodInnerProduct computes the inner product between two neighborhoods
Since NeighborhoodOperator is derived from Neighborhood, we can compute the IP of the kernel and the image region
The NeighborhoodInnerProduct computes the inner product between two neighborhoods
Since NeighborhoodOperator is derived from Neighborhood, we can compute the IP of the kernel and the image region
25
Good to go?Good to go?
1. Create an interesting operator to form a kernel
2. Move a neighborhood through an image
3. Compute the IP of the operator and the neighborhood at each pixel in the image
Voila - convolution in N-dimensions
1. Create an interesting operator to form a kernel
2. Move a neighborhood through an image
3. Compute the IP of the operator and the neighborhood at each pixel in the image
Voila - convolution in N-dimensions
26
Inner product exampleInner product example
itk::NeighborhoodInnerProduct<ImageType> IP; itk::DerivativeOperator<ImageType> operator ;
operator->SetOrder(1); operator->SetDirection(0); operator->CreateDirectional();
itk::NeighborhoodIterator<ImageType> iterator(operator->GetRadius(), myImage, myImage->GetRequestedRegion());
itk::NeighborhoodInnerProduct<ImageType> IP; itk::DerivativeOperator<ImageType> operator ;
operator->SetOrder(1); operator->SetDirection(0); operator->CreateDirectional();
itk::NeighborhoodIterator<ImageType> iterator(operator->GetRadius(), myImage, myImage->GetRequestedRegion());
27
Inner product example, cont.Inner product example, cont.iterator.SetToBegin();while ( ! iterator. IsAtEnd () ) { std::cout << "Derivative at index " <<
iterator.GetIndex () << is << IP(iterator, operator) << std::endl; ++iterator; }
iterator.SetToBegin();while ( ! iterator. IsAtEnd () ) { std::cout << "Derivative at index " <<
iterator.GetIndex () << is << IP(iterator, operator) << std::endl; ++iterator; }
28
NoteNote
No explicit reference to dimensionality in neighborhood iterator
easy to make N-d
No explicit reference to dimensionality in neighborhood iterator
easy to make N-d
29
This suggests a filter...This suggests a filter... NeighborhoodOperatorImageFilter wraps this procedure into a filter that operates on an input image
So, if the main challenge is coming up with an interesting neighborhood operator, ITK can do the rest
NeighborhoodOperatorImageFilter wraps this procedure into a filter that operates on an input image
So, if the main challenge is coming up with an interesting neighborhood operator, ITK can do the rest
30
Your arch-nemesis... image boundariesYour arch-nemesis... image boundaries One obvious problem with inner product techniques is what to do when you reach the edge of your image
Is the operation undefined? Does the image wrap? Should we assume the rest of the world is empty/full/something else?
One obvious problem with inner product techniques is what to do when you reach the edge of your image
Is the operation undefined? Does the image wrap? Should we assume the rest of the world is empty/full/something else?
31
ImageBoundaryConditionImageBoundaryCondition
Subclasses of itk::ImageBoundaryCondition can be used to tell neighborhood iterators what to do if part of the neighborhood is not in the image
Subclasses of itk::ImageBoundaryCondition can be used to tell neighborhood iterators what to do if part of the neighborhood is not in the image
32
ConstantBoundaryConditionConstantBoundaryCondition The rest of the world is filled with some constant value of your choice
The default is 0 Be careful with the value you choose - you can (for example) detect edges that aren’t really there
The rest of the world is filled with some constant value of your choice
The default is 0 Be careful with the value you choose - you can (for example) detect edges that aren’t really there
33
PeriodicBoundaryConditionPeriodicBoundaryCondition The image wraps, so that if I exceed the length of a particular axis, I wrap back to 0 and start over again
If you enjoy headaches, imagine this in 3D
This isn’t a bad idea, but most medical images are not actually periodic
The image wraps, so that if I exceed the length of a particular axis, I wrap back to 0 and start over again
If you enjoy headaches, imagine this in 3D
This isn’t a bad idea, but most medical images are not actually periodic
34
ZeroFluxNeumannBoundaryConditionZeroFluxNeumannBoundaryCondition I am not familiar with how this functions
The documentation states that it’s useful for solving certain classes of differential equations
A quick look online suggests a thermodynamic motivation
I am not familiar with how this functions
The documentation states that it’s useful for solving certain classes of differential equations
A quick look online suggests a thermodynamic motivation
35
Using boundary conditionsUsing boundary conditions With NeighborhoodOperatorImageFilter, you can call OverrideBoundaryCondition
With NeighborhoodOperatorImageFilter, you can call OverrideBoundaryCondition
36
SmartNeighborhoodIteratorSmartNeighborhoodIterator This is the iterator that’s being used internally by the previous filter; you can specify its boundary behavior using OverrideBoundaryCondition too
In general, I would suggest using the “smart” version - bounds checking is good!
This is the iterator that’s being used internally by the previous filter; you can specify its boundary behavior using OverrideBoundaryCondition too
In general, I would suggest using the “smart” version - bounds checking is good!
37
An aside: numeric traitsAn aside: numeric traits This has nothing to do with Neighborhoods but is good to know
Question: given some arbitrary pixel type, what do we know about it from a numerics perspective?
This has nothing to do with Neighborhoods but is good to know
Question: given some arbitrary pixel type, what do we know about it from a numerics perspective?
38
itk::NumericTraitsitk::NumericTraits
NumericTraits is class that’s specialized to provide information about pixel types
Examples include: min and max values IsPositive(), IsNegative() Definitions of Zero and One
NumericTraits is class that’s specialized to provide information about pixel types
Examples include: min and max values IsPositive(), IsNegative() Definitions of Zero and One
39
Using traitsUsing traits
What’s the maximum value that can be represented by an unsigned char?
itk::NumericTraits<unsigned char>::max()
Look at vnl_numeric_limits for more data that can be provided
What’s the maximum value that can be represented by an unsigned char?
itk::NumericTraits<unsigned char>::max()
Look at vnl_numeric_limits for more data that can be provided