DESIGN OF GUIDED FILTER FOR IMAGE FILTERING

i

DESIGN OF GUIDED FILTER

FOR

IMAGE FILTERING

A PROJECT REPORT

Submitted by

ANITHA.P

Register No: 14MAE001

in partial fulfillment for the requirement of award of the degree

of

MASTER OF ENGINEERING

in

APPLIED ELECTRONICS

Department of Electronics and Communication Engineering

KUMARAGURU COLLEGE OF TECHNOLOGY

(An autonomous institution affiliated to Anna University, Chennai)

COIMBATORE - 641 049

ANNA UNIVERSITY: CHENNAI 600 025

APRIL - 2016

ii

BONAFIDE CERTIFICATE

Certified that this project report titled “Design of Guided Filter for Image Filtering”

is the bonafide work of ANITHA.P (Reg. No. 14MAE001) who carried out the project

under my supervision. Certified further, that to the best of my knowledge the work

reported here in does not form part of any other project or dissertation on the basis of

which a degree or award was conferred on an earlier occasion on this or any other

candidate.

The candidate with Register No. 14MAE001 was examined by us in the project

viva-voce examination held on ……………..

INTERNAL EXAMINER EXTERNAL

EXAMINER

SIGNATURE

Dr.A.VASUKI

PROFESSOR AND HEAD

Department of ECE

Kumaraguru College of Technology

Coimbatore-641 049

SIGNATURE

Dr.G.AMIRTHA GOWRI

PROJECT SUPERVISOR

Associate Professor

Department of ECE

Kumaraguru College of Technology

Coimbatore-641 049

iii

ACKNOWLEDGEMENT

First, I would like to express my praise and gratitude to the Lord, who has

showered his grace and blessings enabling me to complete this project in an excellent

manner.

I express my sincere thanks to the management of Kumaraguru College of

Technology and Joint Correspondent Shri Shankar Vanavarayar for his kind support

and for providing necessary facilities to carry out the work.

I would like to express my sincere thanks to our beloved Principal

Dr.R.S.Kumar M.E., Ph.D., Kumaraguru College of Technology, who encouraged

me with his valuable thoughts.

I would like to thank Dr.A.Vasuki M.E., Ph.D., Head of the Department,

Electronics and Communication Engineering, for her kind support and for providing

necessary facilities to carry out the project work.

In particular, I wish to thank with everlasting gratitude to the project

coordinator Ms.S.Umamaheswari M.E.,(Ph.D) Associate Professor, Department of

Electronics and Communication Engineering, throughout the course of this project

work.

I am greatly privileged to express my heartfelt thanks to my project guide

Dr.G.Amirtha Gowri M.E., Ph.D., Associate Professor, Department of Electronics

and communication Engineering, who encouraged me in each and every step of the

project work and I wish to convey my deep sense of gratitude to all teaching and non-

teaching staff of ECE Department for their help and cooperation.

Finally, I thank my parents and my family members for giving me the moral

support and abundant blessings in all of my activities and my dear friends who helped

me to endure my difficult times with their unfailing support and warm wishes.

iv

ABSTRACT

Image processing plays a major role in many applications like medical, satellite

communication, multimedia, etc. The images taken in real time will have

additional disturbance like noise. So, it is necessary to filter these parameters.

The filters used for this purpose are sobel filter, bilateral filter, joint bilateral

filter, etc. Among these filters, guided filter plays a major role due to its better

performance around the edges. Guided image filtering is used to smooth the

result of transferred colors. Guided filter has the edge-preserving smoothing

property but does not suffer from the gradient reversal artifacts near the edges. A

reformation of guided filter formula is used to prevent the error resulted from the

truncation. The guided filter has an O(N) time complexity (in the number of

pixels) exact algorithm for both gray-scale and color images. Guided filter has the

nonapproximation characteristic and offers an ideal option for real-time filter

applications. Guided filter can be embedded in mobile devices. Guided filter has

better performance near the edges than bilateral filter. Guided filter is applied to

the different images to remove the disturbances and obtained results are

compared with the results of the bilateral filter and the performance of guided

filter is analyzed for noise removal.

v

TABLE OF CONTENTS

CHAPTER

NO.

TITLE PAGE

NO.

ABSTRACT iv

LIST OF FIGURES vi

LIST OF NOMENCLATURE

LIST OF TABLES

Viii

x

1 INTRODUCTION 1

1.1 DEFINITION OF A

DIGITAL IMAGE

1

1.2 CHARACTERISTICS OF

IAMGE

1

1.3 IMAGE DENOISING 2

1.4 GENERAL

CLASSIFICATION OF IMAGE

FILTERING TECHNIQUES

2

1.4.1Spatial domain filtering

1.4.1.1 Linear Filters

1.4.1.2 Non-Linear Filters

1.4.2 Transform domain filtering

1.4.2.1 Data adaptive transform

1.4.2.2 Non-data adaptive transform

2

3

3

3

3

4

2 LITERATURE SURVEY 6

vi

3 DENOISING USING GUIDED FILTER 9

3.1 BILATERAL FILTERING 9

3.1.1 Functional Units of Bilateral Filter 10

3.1.2 Merits of bilateral filter 11

3.1.3 Limitations of bilateral filter 11

3.1.4 Applications of bilateral filter 12

3.2 GUIDED FILTER

3.2.1 Algorithm

3.2.2 Merits of Guided filter

3.2.3Applications of guided filter

3.2.4 Methodology

12

14

15

15

15

4 RESULTS AND DISCUSSION

4.1 INTRODUCTION

4.2PERFORMANCE METRICS

4.3SIMULATION RESULTS

4.3.1Results of Guided filter

4.3.2 Results of Bilateral filter

4.3.3 SCHEMATIC OF GUIDED FILTER

18

18

18

19

19

23

28

5 CONCLUSION 36

REFERENCES 37

vii

LIST OF FIGURES

FIGURE NO. NAME OF FIGURE PAGE NO.

1.1 Basic model of image donoising 3

3.1 Illustration of bilateral filter 10

3.3 Example for Guided Filter

13

3.4 Work flow 17

4.1 – 4.8 Results of Guided Filter 19 - 22

4.9 – 4.16 Results of Bilateral Filter 23 - 26

4.17 Simulink Model of Guided Filter 28

4.18 Simulink Model for calculating mean to

Guided Image

29

4.19 Simulink Model for calculating mean to Input

Image

29

4.20 Simulink Model for calculating mean to

correlation of Input and guided image

30

4.21 Simulink model for calculating mean to

correlation of Guided image and Guided image

30

4.22 Simulink model for calculating co-efficient

‘a’ and ‘b’

31

4.23 Simulink Model for viewing filtered output

image

32

4.24 RTL Schematic for subsystem 33

viii

4.25 Guided Image 34

4.26 Input image 34

4.27 Output image 35

ix

LIST OF NOMENCLATURE

BF Bilateral filtering

DSP Digital Signal Processors

CCD Charge Coupled Device

CWM Central Weighted Median

SFR Special Function Register

BF Bilateral Filter

RGB Red Green Blue

PSNR Peak Signal to Noise Ratio

SDK Software Development Kit

FPGA Field Programmable Gate Array

MSE Mean Square Error

x

LIST OF TABLES

TABLE NO. CAPTION PAGE NO.

5.1 Parameters Used 28

5.2 Comparison of Performance

Metrics of Guided Filter and

Bilateral Filter

28

1

CHAPTER 1

INTRODUCTION

The main aim of this project is to design the guided filter for image filtering.

Filtering is widely used in many applications. Image processing is one the main

application of image filtering. More specifically, filtering can be applied in other

applications such as noise reduction, texture editing, detail smoothing/enhancement,

colorization, relighting tone mapping, haze/rain removal, and joint upsampling. The

Guided filter has the good edge preservation quality and the noise removal property.

The most popular technique is the edge-preserving bilateral filter for image filtering

and image noise reduction. Bilateral filter can be used on high dynamic range (HDR)

images. Based on bilateral filter, joint bilateral filter is developed and used in flash/no-

flash denoising. Joint bilateral filter can be used for upsampling problems. Although a

bilateral filter has a good edge-preserving characteristic, it has been noticed that it may

have artifacts in detail decomposition and HDR compression. Artifacts are resulted

from those pixels around the edge that may have an unstable Gaussian weighted sum.

To overcome this problem a guided filter is designed which can filter output by

considering the content of the guiding image. Compared to a bilateral filter, the guided

filter can perform better at the pixels near edges. Moreover, the guided filter is a non-

approximate linear-time algorithm, which is a very important strength for real-time

applications.

1.1 Definition of a Digital Image

A digital image (also called a discrete image) is obtained from an analogue image

by sampling and quantization. This process depends on the acquisition device and

depends, for instance, on CCDs for digital cameras. Basically, the idea is to

superimpose a regular grid on an analogue image and to assign a digital number to

each square of the grid, for example the average brightness in the square. Each square

is called a pixel, for picture element, and its value is the gray-level brightness.

2

1.2 Characteristics of Image

The space domain S, which is the set of possible positions in an image. This is

related to the resolution, i.e., the number of rows and columns in the image.

Consumer-grade cameras now give images with several megapixels (i.e. millions of

pixels), typically between 5 and 10, professional cameras provided up to 16

megapixels, and some prototypes reach several hundreds of megapixels or even a few

gig pixels.The range domain R, which is the set of possible pixel values. The number

of bits used to represent the pixel value may vary. Common pixel representations are

unsigned bytes (0 to 255) and floating point. To describe a pixel, one may also need

several channels (or bands): for example, a vector field has two components; a color

image is described with three channels, red, green and blue (or any other color space

such as hue, saturation, value, namely HSV).

1.3 Image denoising

One of the fundamental challenges in the field of image processing and computer

vision is image denoising, where the underlying goal is to estimate the original image

by suppressing noise from a noise-contaminated version of the image. Image noise

may be caused by different intrinsic (i.e., sensor) and extrinsic (i.e., environment)

conditions which are often not possible to avoid in practical situations. Therefore,

image denoising plays an important role in a wide of applications such as image

restoration, visual tracking, image registration, image segmentation, and image

classification, where obtaining the original image content is crucial for strong

performance. While many algorithms have been proposed for the purpose of image

denoising, the problem of image noise suppression remains an open challenge,

especially in situations where the images are acquired under poor conditions where the

noise level is very high. So, the challenge of good image denoising model is that it has

to remove noise while preserving edges.

3

Figure 1.1.Basic model of image denoising

1.4 General classification of image filtering techniques

There are two basic approaches of the image filtering: spatial domain filtering and

transform domain filtering.

1.4.1 Spatial domain filtering

A traditional way to remove noise from image data is to employ spatial filters.

Spatial domain filtering methods take original noisy image into consideration and

apply filtering processing on it. Spatial filters are high speed processing tools.

1.4.1.1 Linear Filters

Linear filters like mean filter, wiener filter too tend to blur sharp edges, destroy

lines and other fine image details, and perform poorly in the presence of signal-

dependent noise.

Mean filtering is simple, intuitive and easy to implement method of reducing

noise in images. The idea of mean filtering is simply to replace each pixel value in an

image with the mean (‘average’) value of its neighbors, including itself. The Wiener

filtering method requires the information about the spectra of the noise and original

signal and it works well only if the underlying signal is smooth.

1.4.1.2 Non-Linear Filters

With non-linear filters, the noise is removed without any attempts to explicitly

identify it. To resolve the issues raised with linear filters, a variety of non-linear filters

such as median, weighted median, rank conditioned rank selection, and relaxed

median have been developed.

(Input Image +

Noise)= noisy

image

Apply wavelet

Transform to

decompose image

Apply filter &

threshold to shrink

Denoised image Inverse Wavelet

Transform

4

1.4.2 Transform domain Filtering

In contrast with spatial domain filtering methods, transform domain filtering

methods first obtain some transform of given noisy image and then apply denoising

procedure on transformed image. The transform domain filtering methods were

subdivided according to the choice of the basis transform functions which may be data

adaptive or non-data adaptive.

1.4.2.1 Data adaptive transform

The transform domain filtering methods that made choice of data adaptive

transform functions inside a popular example of Independent component analysis

(ICA) method. This method is successfully implemented for denoising for non-

Gaussian data. This method assumes the signal should be non-Gaussian. This

assumption helps to denoising images with non-Gaussian as well as Gaussian

distribution. The main drawback with ICA method is its computational cost because it

uses a sliding window and requires sample of noise free data or at least two image

frames of the same scene. But in some applications, it might be difficult to obtain the

noise free data.

1.4.2.2 Non-data adaptive transform

The transform domain filtering methods that made choice of non-data adaptive

transform functions were further subdivided into two domains namely spatial-

frequency domain and wavelet domain.

Spatial frequency domain

Filtering methods in spatial-frequency domain refer use of low pass filtering

by designing a frequency domain filter that passes all the frequencies lower than and

attenuates all frequencies greater than a cut-off frequency. Before applying the

filtering method, domain of given noisy image is changed from spatial to frequency

using Fast Fourier Transform (FFT). These methods are time consuming and depend

on the cut-off frequency and the filter function behavior. Furthermore, they may

produce artificial frequencies in the processed image.

5

Wavelet transform

For denoising in wavelet transform, various algorithms based on wavelet

transform have been developed. The focus was shifted from the spatial and Fourier

transform domain to the wavelet transform domain. It has been proved that the use of

wavelets successfully removes noise while preserving the signal characteristics,

regardless of its frequency content.

Similar to spatial domain filtering, filtering operations in the wavelet domain can

also be subdivided into linear and non-linear methods. Linear wavelet transform

methods include the most popular example of Wiener filters while non-linear wavelet

transform methods include coefficient thresholding based methods.

6

CHAPTER 2

LITERATURE SURVEY

A temporal information considerably improves the frame-by-frame approach of

[3] for both stereo and optical flow estimation and outperform the current state-of-the-

art in local space-time stereo matching. However, it is not applicable for CPU

computation. Although GPU provides an alternative solution to a high-throughput

guided filter, it has higher cost and power demand that is not suitable for mobile

devices like digital cameras or mobile phones. Quantitative and qualitative results

demonstrate that the approach (i) considerably improves over frame-by-frame

methods for both stereo and optical flow; and (ii) outperforms the state-of- the- art for

local space-time stereo approaches.

Discrete label-based approaches have been successfully applied to many

computer vision problems such as stereo, optical flow, interactive image segmentation

or object recognition. In a typical labeling approach, the input data is used to construct

a three-dimensional cost volume, which stores the costs for choosing a label at image

co-ordinates. Because of the degradation in quality caused by fast approximation

bilateral filter, a guided filter was used for fast cost volume filtering. To achieve this (i) disparity maps in real-time, whose quality exceeds those of all other fast (local)

approaches on the Middlebury stereo benchmark, and (ii) optical flow fields with very

fine structures as well as large displacements. [4]

An efficient and scalable design for histogram-based bilateral filtering (BF)

and joint BF (JBF) by memory reduction methods [17] and architecture design

techniques to solve the problems of high memory cost, high computational

complexity, high bandwidth, and large range table. The presented memory reduction

methods exploit the progressive computing characteristics to reduce the memory cost

to 0.003%– 0.020%, as compared with the original approach. The architecture design

techniques adopt range domain parallelism and take advantage of the computing order

and the numerical properties to solve the complexity, bandwidth, and range-table

problems.

7

A novel filtering-based technique to tackle this issue, called “importance

filtering”. It uses a guided filter to filter out the image saliency under the guidance of

the original image [16]. It avoids undesired distortion such as pixel swap that occurs

to many earlier methods. The importance filtering operations are highly efficient and

ready for real-time applications. The simple nature of filter operations allows highly

efficient implementation for real-time applications and easy extension to video

retargeting, as the structural constraints from the original image naturally convey the

temporal coherence between frames.

An optical degradation model which enables us to adopt a point operation

scheme to realize image multi-focusing [15]. It can effectively reduce halo artifacts in

the refocused image and greatly improve the computational efficiency. A two-step

approach is applied to estimate the blur map of the input image. i) A sparse blur map

is obtained by estimating the amount of defocus blur at edge locations. ii) The guided

image filtering method is applied to propagate the value from edge locations into the

unknown regions. A simple geometry prior of photograph to eliminate the ambiguity

over the focal plane. Based on the obtained depth map, we can directly produce

different styles of images by multi-focusing with the adjustment to the camera

parameters.

The VLSI architecture to achieve high-throughput and improved-quality stereo

vision for real applications. The stereo vision processor generates gray-scale output

images with depth information from input images taken by two CMOS Image Sensors

(CIS). The depth estimator using the sum of absolute differences (SAD) algorithm as

stereo matching technique is implemented on hardware by exploiting pipelining and

parallelism. To produce depth maps with improved-quality at real-time, pre- and post-

processing units are adopted, and to enhance the adaptability of the system to real

environments, special function registers (SFRs) are assigned to vision parameters

[14].

Automatically generated depth maps from video are usually not aligned with the

objects in the original image and produced at lower resolutions. In order to apply a

joint-bilateral filter to smoothen the depth map within the objects and up sample it to

the original image resolution while keeping object edges in the depth map aligned

8

with the original image. The performed algorithmic and DSP specific optimizations to

achieve the real-time implementation on an embedded DSP processor, TM3270, while

preserving high quality results [13].

A novel explicit image filter called guided filter. Derived from a local linear

model, the guided filter computes the filtering output by considering the content of a

guidance image, which can be the input image itself or another different image. The

guided filter can be used as an edge-preserving smoothing operator like the popular

bilateral filter [12], but it has better behaviors near edges. The guided filter is also a

more generic concept beyond smoothing: It can transfer the structures of the guidance

image to the filtering output, enabling new filtering applications like dehazing and

guided feathering. Moreover, the guided filter naturally has a fast and no approximate

linear time algorithm, regardless of the kernel size and the intensity range. The guided

filter is both effective and efficient in a great variety of computer vision and computer

graphics applications, including edge-aware smoothing, detail enhancement, HDR

compression, image matting/feathering, dehazing, joint up sampling .

Bilateral filtering smooth's images while preserving edges, by means of a

nonlinear combination of nearby image values. The method is non iterative, local, and

simple. It combines gray levels or colors based on both their geometric closeness and

their photometric similarity, and prefers near values to distant values in both domain

and range. In contrast with filters that operate on the three bands of a colour image

separately, a bilateral filter can enforce the perceptual metric underlying the CIE-Lab

color space, and smooth colors and preserve edges in a way that is tuned to human

perception [5].

Digital photography has made it possible to quickly and easily take a pair of

images of low-light environments: one with flash to capture detail and one without

flash to capture ambient illumination. The variety of applications is presented that

analyze and combine the strengths of such flash/no-flash image pairs. The

applications include denoising and detail transfer (to merge the ambient qualities of

the no-flash image with the high-frequency flash detail), white-balancing (to change

the color tone of the ambient image), continuous flash (to interactively adjust flash

intensity), and red-eye removal (to repair artifacts in the flash image) [7].

9

CHAPTER 3

DENOISING USING GUIDED FILTER

Digital images can be corrupted by noise during the process of the acquisition

and transmission, degrading their quality. Image denoising is one of the fundamental

challenges in the field of image processing and computer vision, where the underlying

goal is to estimate the original image by removing noise from a noisy version of the

image. A major challenge is to remove noise as much as possible without eliminating

the most representative characteristics of the image, such as edges, corners and other

sharp structures.

Ideally denoising is all about filtering noise from degraded image while

keeping other details unchanged. Indeed, filtering is the most fundamental operation

of image processing and computer vision and it is used extensively in a wide range of

applications, including image smoothing and sharpening, noise removal, resolution

enhancement and reduction, feature extraction and edge detection. In the broadest

sense of the term "filtering", the value of the filtered image at a given location is a

function of the values of the input image in a small neighborhood of the same location.

Filtering is an image processing technique widely adopted in computer vision,

computer graphics, computational photography, etc. More specifically, filtering can be

applied in many applications such as noise reduction, texture editing, detail

smoothing/enhancement, colorization, relighting tone mapping, haze/rain removal,

and joint up sampling. The most popular technique is the edge-preserving bilateral

filter. A bilateral filter has a good edge-preserving characteristic, it has been noticed

that it may have artifacts in detail decomposition and HDR compression. Artifacts are

resulted from those pixels around the edge that may have an unstable Gaussian

weighted sum. To overcome this problem, a guided filter is designed, which can filter

output by considering the content of the guiding image. Compared to a bilateral filter,

the guided filter can perform better at the pixels near edges. Moreover, the guided

filter is a non-approximate linear-time algorithm, which is a very important strength

10

for real-time applications. Guided filter can prevent the error resulted from truncation.

It can be embedded in mobile devices to achieve real-time HD applications. The

designed guided filter architecture greatly reduces the usage not only in equivalent

gate counts but also in on chip memory. Therefore, the architecture can achieve lower

implementation costs.

3.1 BILATERAL FILTERING

Figure 3.1. Illustration of bilateral filter

Bilateral filtering (BF) is widely adopted in image and video processing such as

denoising, texture editing and relighting tone management stylization, and optical flow

estimation due to its texture preserving capabilities during processing. Figure 3.1.

shows the illustration of bilateral filter. The intensity value at each pixel in an image is

replaced by a weighted average of intensity values from nearby pixels. This weight

can be based on a Gaussian distribution. Crucially, the weights depend not only on

Euclidean distance of pixels, but also on the radiometric differences (e.g. range

differences, such as color intensity, depth distance, etc.). This preserves sharp edges

by systematically looping through each pixel and adjusting weights to the adjacent

pixels accordingly.

3.1.1 Functional Units of Bilateral Filter

The image data, as well as all constants and coefficients used in the following

design concept, are integer numbers. There is no need to implement floating-point

computation. With the aid of the presented design concept, the bilateral filter can be

realized as a highly parallelized pipeline structure giving great importance to the

11

effective re-source utilization. In this project, the data paths are detailed. The

description of the control signals is not addressed here.

Figure 3.2. Order of the functional units of the bilateral filter

For the design description, a window size of 5 × 5 is chosen. This window size

is the tradeoff between high noise reduction and low blurring effect. The design

concept for the implementation of the bilateral filter is subdivided into three functional

blocks. The block-based design approach reduces design complexity and simplifies

validation. Figure 3.2. presents these units and their order in the concept. The input

data marked by “Data in” are read line by line and arranged for further processing in

the register matrix. The second unit is the photometric filter which weights the input

data according to the intensity of the processed pixels. The filtering is completed by

the geometric filter, and the filtered data are marked by “Data out”.

3.1.2 Merits of bilateral filter

1. In contrast with filters that operate on the three bands of a color image

separately, a bilateral filter can smooth colors and preserve edges in a way

that is tunes to human perception.

2. Bilateral filtering produces no phantom colors along edges in color images,

and reduces phantom colors where they appear in the original image.

3.1.3 Limitations of bilateral filter

1. Staircase effect - intensity plateaus that lead to images appearing like

cartoons.

2. Gradient reversal – introduction of false edges in the image.

There exist several extensions to the filter that deal with these artifacts.

Alternative filters, like the guided filter, have also been designed as an efficient

alternative without these limitations.

12

3.1.4 Applications of bilateral filter

1. Denoising - This is of course the primary goal of bilateral filter, and it has

been used in several applications such as medical images, movie restoration,

etc. Some fields of applications are described. An extension of the bilateral

filter will be presented in the cross bilateral filter.

2. Contrast Management – Bilateral filtering has been particularly successful as

a tool for contrast management tasks such as detail enhancement or

reduction. The bilateral filter is used to separate an image into a large-scale

component and a small-scale component by subtracting filtered results.

3. Data Fusion – These applications use bilateral filtering to decompose several

source images into components and then recombine them as a single output

image that inherits selected.

4. Texture and Illumination Separation, Tone Mapping, Retinex, and Tone

Management - Based on a large-scale / small-scale decomposition of images,

these applications edit texture and manipulate the tonal distribution of an

image to match the capacities of a given display or achieve photographic

stylization.

5. Three – dimensional Fairing – This is the counterpart of image denoising for

three-dimensional meshes and point clouds. Noise is removed from these data

sets.

3.2 GUIDED FILTER A novel explicit image filter called guided filter. Derived from a local linear

model, the guided filter computes the filtering output by considering the content of a

guidance image, which can be the input image itself or another different image. The

Figure 3.3. shows that the process of guided filter. It can be used as an edge-

preserving smoothing operator like the popular bilateral filter, but it has better

behaviors near edges. The guided filter is also a more generic concept beyond

smoothing: It can transfer the structures of the guidance image to the filtering output,

enabling new filtering applications like dehazing and guided feathering. Moreover, the

guided filter naturally has a fast and non-approximate linear time algorithm, regardless

of the kernel size and the intensity range. Currently, it is one of the fastest edge-

preserving filters. Experiments show that the guided filter is both effective and

efficient in a great variety of computer vision and computer graphics applications,

13

including edge-aware smoothing, detail enhancement, HDR compression, image

matting/feathering, dehazing, joint up sampling.

Guided filter has the non-approximation characteristic and offers an ideal

option for real-time filter applications on HD videos. Recently, many applications

adopted a guided filter as the filtering method. Because of the degradation in quality

caused by fast approximation bilateral filter, a guided filter was used for fast cost

volume filtering. In order to suppress color noise while preserving color structures,

guided image filtering was used to smooth the result of transferred colors.

Figure 3.3. Example for Guided Filter

In guided filter the input is represented as p and the output is represented as q

and I is nothing but the guided image representation.

Compared to a bilateral filter, the guided filter can perform better at the pixels

near edges. Compared with joint bilateral filter, a guided filter has the edge-preserving

smoothing property but does not suffer from the gradient reversal artifacts near edges.

First define a general linear translation-variant filtering process, which

involves a guidance image I, an filtering input image p, and an output image q. Both I

and p are given beforehand according to the application, and they can be identical.

The filtering output at a pixel is expressed as weighted average:

Qi =ΣjWij (I) Pj [3.1]

Where i and j are pixel indexes. The filter kernel Wij is a function of the

guidance image I and independent of p. This filter is linear with respect to p.

The key assumption of the guided filter is a local linear model between the

guidance I and the filtering output q. Assume that q is a linear transform of I in a

14

window centered at the pixel k:

Where ( , ) are some linear coefficients. To determine the linear coefficients

( , ) we need constraints from the filtering input p. Then model the output q as the

input p subtracting some unwanted components n like noise/textures.

iii npq [3.3]

So after computing coefficients for all windows kw in the image, then compute the

filtering output by ,

kwi

kiki bIaw

q1

[3.4]

3.2.1 Algorithm

Input: Filtering input image p, guidance image I, radius r, regularization

Output: Filtering output q.

1. Compute following mean values by applying averaging filter ‘fmean’:

meanI = fmean(I)

meanp = fmean(p)

meanIp = fmean(I .* p)

meanII = fmean(I .* I)

2. Compute covariance of (I,p) using formula: covIp = meanIp –meanI.* meanp

3. Compute variance using formula: varI = meanII –meanI.* meanI

7. Compute linear coefficients a and b as:

a = covIp/ (varI + ε )

b = meanp – a .* meanI

8. Compute mean of a and b as:

meana = fmean(a)

meanb = fmean(b)

kwikiki bIaq

kbka

[3.2]

kbka

15

9. Compute filtered output as: q = meana .* I + meanb

3.2.2 Merits of Guided Filter

1. This filter has the edge-preserving smoothing property like the bilateral filter,

but does not suffer from the gradient reversal artifacts.

2. The guided filter has an O(N) time (in the number of pixels) exact algorithm

for both gray-scale and color images.

3. The guided filter performs very well in terms of both quality and efficiency.

3.2.3 Applications of guided filter

1. Image smoothing / enhancement.

2. HDR compression – The HDR compression is done in a similar way, but

compressing the base layer instead of magnifying the detail layer.

3. Flash/no-flash imaging – Denoise a no-flash image under the guidance of its

flash version.

4. Matting/feathering – combined with the global sampling, the guided filter is

the best performing filtering based matting method in the alphamatting

benchmark.

5. Dehazing.

6. Joint upsampling - joint upsampling is to upsample an image under the

guidance of another image.

3.2.4 Methodology

The main aim of this project is to design guided filter for image filtering. In

order to design the guided filter, the graphical block diagramming tool Simulink is

used. Based on the algorithm the guided filter is designed by using the blocks

provided by the Simulink. In the algorithm of guided filter the mean, the correlation

and covariance of both the guided image and input image are calculated. The

mathematical blocks in Simulink are used to calculate these parameters. There is a

inbuilt block to calculate the ‘mean’. For calculating the mean the image is converted

from 2D to 1D. And then the mean is calculated. After that the value is displayed by

16

using frame rate display. The HDL code will be generated directly from the Simulink

by using ‘HDL Generation’ option.

But the mean block doesn’t support for the HDL generation. So, the

Simulink model for designing the guided filter is subdivided into three stages. In the

first stage the mean of the guided image and input image is calculated. The

correlation of input image and guided image is calculated. Also, the correlation

guided image and guided image is calculated. The mean block is also take place in

these calculations also. So, these models are doesn’t converted to HDL code directly

from Simulink. The values of these parameters are noted.

In the second stage, the coefficients ‘a’ and ‘b’ are calculated. In this stage,

there is no mean block. By using all the values which is obtained from the first stage

the coefficients ‘a’ and ‘b’ are calculated. After that the Simulink model for

calculating coefficients is put into a ‘subsystem’. The subsystem is then converted as

HDL code by using the option ‘HDL Generation’.

In the third stage, the mean for coefficients of ‘a’ and ‘b’ are calculated and by

using that values the output filtered image is obtained. Guided filter is the one, which

reduces noise, preserves edges in image and videos. Here, the image is taken as a

input to the guided filter to obtain the high quality output image.

By analysis, it is known that an image cannot be given as an input in Xilinx

directly. Hence, the process is carried out with the use of Simulink. Simulink is an

integrated tool with the MATLAB software. For testing purpose, different images are

given directly in Simulink and received the higher quality image. The sub-system is

converted HDL code by Simulink. The obtained HDL code is given to Xilinx to

convert it into RTL model.

17

Figure 3.4. Work flow

The Figure 3.4. Shows the methodology used to design the guided filter. The Simulink

and system generator is available in MATLAB environment. After converting the

Simulink model to HDL the code is processed in Xilinx and the RTL schematic will

be generated.

MATLAB ENVIRONMENT

Simulink model

Code Generation

System Generator

Xilinx Implmentation Flow

RTL Schematic

18

CHAPTER 4

RESULTS AND DISCUSSION

4.1 INTRODUCTION

The filtered output of Guided filter and RTL schematic of subsystem for

generating coefficient ‘a’ and ‘b’ are presented in this chapter. The performance

measure that has been used to analyze results is Peak Signal to Noise Ratio (PSNR)

and Mean Square Error (MSE).

The Simulink tool is used to view the result and the Xilinx tool is used to generate

the RTL model.

4.2 PERFORMANCE METRICS

The results are analyzed using different quality metrics which are detailed below

Mean Square Error

Mean Square Error is the average squared difference between a reference

image and reconstructed image. For a m x n reference image I and

reconstructed image K, the MSE is given by

1

0

1

0

2)],(),[(1 m

i

n

j

jiKjimn

MSE

Peak Signal to Noise Ratio

Peak Signal to Noise Ratio is the ratio between the reference image and the

reconstructed image, given in decibels. The higher the PSNR value, the closer

the reconstructed image is to the reference image .

MSE

MAXPSNR I

2

10log10

19

4.3 SIMULATION RESULTS

4.3.1 RESULTS OF GUIDED FILTER

A. Visual Results

Test Image: Tulip Image.

a) Guided Image b) Input image

Figure 4.1. Result of Guided Filter

Figure 4.1 b) Input image (Noisy Image) has the Salt and Pepper Noise with noise

density 0.02(PSNR : 27.47).

Figure 4.1 c) shows the output image after applying Guided Filter (PSNR : 42.01 )

a) Guided image b) Input image c) After Applying Guided Filter



density 0.04 (PSNR : 23.21).

Figure 4.2 c) shows the output image after applying Guided Filter ( PSNR : 41.73).

c) After applying guided filter

20

Test Image: Academy

a) Guided image b) Input image c) After applying Guided Filter



density 0.02. (PSNR : 20.84)

Figure 4.3 c) shows the output image after applying Guided Filter (PSNR : 32.65).




density 0.04 (PSNR : 18.96).


21

Test image : Mandrill




density 0.02 (PSNR : 25.53).





density 0.04 (PSNR : 18.71).


22

Test image : Einstein




density 0.02. (PSNR : 22.50)

Figure 4.7 c) shows the output image after applying Guided Filter (PSNR : 38.31)




density 0.04 (PSNR : 19.75).


23

5.3.2RESULTS OF BILATERAL FILTER

Test Image: Tulip Image.

a) Input image b) Noisy image c) After applying Bilateral Filter

Figure 4.9. Result of Bilateral Filter

Figure 4.9 b) Noisy Image has the Salt and Pepper Noise with noise density 0.02

(PSNR: 27.47).

Figure 4.9 c) Shows the output image after applying Bilateral Filter (PSNR: 26.71).




(PSNR: 23.21).

Figure 4.10 c) Shows the output image after applying Bilateral Filter (PSNR : 24.43)

24

Test Image: Academy




(PSNR: 20.84).

Figure 4.11 c) Shows the output image after applying Bilateral Filter (PSNR: 21.65).




(PSNR: 18.96).

Figure 4.12 c) Shows the output image after applying Bilateral Filter (PSNR : 22.75).

Test image: Mandrill

25




(PSNR: 25.53).


b) Input image b) Noisy image c) After applying Bilateral Filter



(PSNR: 18.71).


26

Test image: Einstein




(PSNR: 22.50)

Figure 4.15 c) Shows the output image after applying Bilateral Filter (PSNR: 27.61)

c) Input image b) Noisy image c) After applying Bilateral Filter



(PSNR: 19.75)

Figure 4.16 c) Shows the output image after applying Bilateral Filter (PSNR: 29.81)

27

The visual results obtained for the test images indicate that the proposed

Guided Filter which makes use of guided image is better compared with other filters

like bilateral filter.

Table 5.1: Parameters Used

Component Parameter Value of Parameter

Image Image size 1920 x 1200

256 x 256

Type Gray and color Image

Noise Type Salt and Pepper Noise

Variance of Noise 0.02 - 0.04

Software MATLAB R2012a

VIVADO

Table 5.2: Comparison of Performance Metrics of Guided Filter and Bilateral

Filter (Salt and Pepper Noise with Noise Density 0.02 and 0.04)

TECHNIQUE NOISE

DENSITY

METRIC IMAGES

Tulip Academy Mandrill Einstein

Guided Filter

0.02 PSNR 42.01 32.65 38.31 38.31

0.04 PSNR 41.73 30.75 34.92 33.69

Bilateral

Filter

0.02 PSNR 26.71 21.65 27.38 27.61

0.04 PSNR 24.43 22.75 25.75 29.81


The RTL Schematic is obtained by using Simulink and Xilinx tool. In Simulink,

the Guided Filter based on its algorithm is designed. In this design of guided filter, the

mean for input image and

calculated. The ‘mean’ block doesn’t support to generate the HDL code directly from

the Simulink blocks. For this reason, the architecture is subdivided into three parts:

The ‘subsystem’ which is u

HDL code and RTL Schematic also obtained for the same.

Figure 4.

The design of guided filter using Simulink blocks is depicted in

The mean of guided image and input image and the correlation of input image and

guided image , correlation of guided image and guided image will be given as the

input to the subsystem. The coefficients ‘a’ and ‘b’ are generated by the subsystem.

Again, the mean for coeffi

filtered output image will be obtained.

28




guided image as well as co-efficient ‘a’ and ‘b’ has been



The ‘subsystem’ which is used for generating the co-effeicients only converted into

HDL code and RTL Schematic also obtained for the same.

Figure 4.17. Simulink Model of Guided filter

The design of guided filter using Simulink blocks is depicted in

ed image and input image and the correlation of input image and



Again, the mean for coefficient 'a' and 'b' is calculated and by using that value the

filtered output image will be obtained.



efficient ‘a’ and ‘b’ has been



effeicients only converted into

The design of guided filter using Simulink blocks is depicted in Figure 4.17.

ed image and input image and the correlation of input image and



cient 'a' and 'b' is calculated and by using that value the

29

Figure 4.18. Simulink model for calculating mean to Guided image

The Figure 4.18 shows the Simulink model for calculating the mean for guided image.

The image is taken by using ‘Image from file’ block. Then the image is converted

from 2D to 1D by using ‘Convert 2-D to 1-D’ block. After that by using the ‘Mean’

block the mean will be calculated and the value displayed using ‘Frame Rate Display’

block. The mean of guided image is 5.9312.

Figure 4.19. Simulink model for calculating mean to Input image

The Figure 4.19 shows the Simulink model for calculating the mean for input image.

The image is taken by using ‘Image from file’ block. Then the image is converted

from 2D to 1D by using ‘Convert 2-D to 1-D’ block. After that by using the ‘Mean’

block the mean will be calculated and the value displayed using ‘Frame Rate Display’

block. The mean of input image is 5.322.

30

Figure 4.20. Simulink model for calculating correlation of Input image and

Guided image

The Figure 4.20.Shows the Simulink model for calculating correlation of Input

image and Guided image. The input image and guided image is taken by using ‘Image

From File’ block. After that the ‘product’ block is used to multiply the two images.

After that the mean is calculated by using ‘Mean’ block. The value is displayed by

Frame Rate Display. The correlation value is 1.6316.

Figure 4.21. Simulink model for calculating mean to correlation of Guided image

and Guided image

The Figure 4.21. Shows the Simulink model for calculating correlation of

Guided image and Guided image. The input image and guided image is taken by using

‘Image From File’ block. After that the ‘product’ block is used to multiply the two

31

images. After that the mean is calculated by using ‘Mean’ block. The value is

displayed by Frame Rate Display. The correlation value is 1.5918.

Figure 4.22. Simulink model for calculating co-efficient ‘a’ and ‘b’

The coefficient 'a' and 'b' are calculated by Figure 4.22. This Simulink model

has been inserted in subsystem. The subsystem is converted as HDL code and the RTL

Schematic is generating for that subsystem. The mean of input image and guided

image is used as the input to this subsystem. Also the correlation of input image and

32

guided image is given as the input to the subsystem. The correlation of guided image

and guided image also given as the input to the subsystem.

The values of coefficients ‘a’ and ‘b’ are 0.005 and 0.010.

This subsystem is directly converted to the HDL code by using ‘HDL Generation’

option.

Figure 4.23. Simulink model for viewing filtered output image

The Figure 4.23. shows the Simulink model for viewing filtered output image.

The ‘Image From File’ block is used to take the guided image. And the values of

coefficients ‘a’ and ‘b’ are given by using ‘constant’ block. After that these values are

multiplied by using ‘Product’ block. The product of guided image I and coefficient

‘a’ is added with coefficient ‘b’ by using ‘Add’ block. The filtered output image is

viewed from ‘Video Viewer’ block.

33

Figure 4.24. RTL Schematic for Subsystem

The HDL code which is automatically generated for subsystem in the simulink

is then given to the xilinx software. The RTL Schematic which is shown in Figure

4.24. is generated from VHDL code through xilinx software.

The RTL schematic has eight inputs and two outputs. The two outputs are

coefficients ‘a’ and ‘b’. The Inputs will be given by as the values which is obtained

from the first stage (i.e.) The mean of Input image , mean of guided image, correlation

of input and guided image and also the correlation of guided image and guided image.

34

Figure 4.25. Guided Image

Figure 4.26. Input Image

35

Figure 4.27. Output Image

The guided image which is shown in Figure 4.25. and the input image which is

shown in Figure 4.26 are given as the inputs to the guided filter and the output image

which is shown in the Figure 4.27. is obtained as the filtered output image which has

the sharp edges than the input image is obtained from the guided filter.

36

CHAPTER 5

CONCLUSION

In the field of image processing, the filters with high performance and low

hardware cost plays a vital role. In this project, the guided filter is designed using

Simulink block set and it is converted into VHDL code for Schematic generation. Two

images are given as input which obtains highly sharp edged output image with

absence of noise and improved quality. The guided filter performs edge preserving,

smoothening on an image using the content of the second image called a guidance

image to influence the filtering. The guidance image can be an input image or a

different version of an input image or a completely different image. The performance

of guided filter is compared with bilateral filter and it is observed that the guided filter

image has high PSNR value. Finally the RTL schematic for the subsystem is

generated by using the HDL code which is generated from the Simulink using the

Xilinx tool. Also, the same process can be applied for HD videos.

37

REFERENCES

[1] A. Y.-S. Chia, S. Zhuo, R. K. Gupta, Y.-W. Tai, S.-Y. Cho, P. Tan, and S. Lin,

“Semantic colorization with internet images,” ACM Trans.Graph., vol. 30, no. 6, pp.

156:1–156:8, Dec. 2011.

[2] C. Liu, W. Freeman, R. Szeliski, and S. B. Kang, “Noise estimation from a single

image,” in Proc. IEEE CVPR, vol. 1, 2006, pp. 901–908.

[3] .C. Rhemann, M. Bleyer, and M. Gelautz, “Temporally consis-tent disparity and

optical flow via efficient spatio-temporal filtering,” in Advances in Image and Video

Technology, LNCS. vol. 7087, Y.-S. Ho,Ed. Berlin, Germany: Springer, 2012, pp.

165–177.

[4] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz, “Fast cost-volume

filtering for visual correspondence and beyond,” in Proc.IEEE Conf. CVPR, 2011,

pp. 3017–3024.

[5] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc.

IEEE 6th ICCV, 1998, pp. 839–846.

[6] F. Durand and J. Dorsey, “Fast bilateral filtering for the display of high-dynamic-

range images,” ACM Trans. Graph., vol. 21, no. 3, pp. 257–266, Jul. 2002.

[7] G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama,

“Digital photography with flash and no-flash image pairs,” ACM Trans. Graph., vol.

23, no. 3, pp. 664–672, Aug. 2004.

[8] J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele, “Joint bilateral

upsampling,” ACM Trans. Graph., vol. 26, no. 3, article no. 96, Jul. 2007.

[9] J.-W. Van de Waerdt, S. Vassiliadis, S. Das, S. Mirolo, C. Yen, B. Zhong, C. Basto,

J.-P. van Itegem, D. Amirtharaj, K. Kalra, P. Rodriguez, and H. van Antwerpen, “The

TM3270 media-processor,” in Proc. 38th Annu.IEEE/ACM Int. Symp. MICRO-38,

2005, pp. 331–342.

[10] J. Zhang, L. Li, Y. Zhang, G. Yang, X. Cao, and J. Sun, “Video dehazing with spatial

and temporal coherence,” Vis. Comput., vol. 27, pp. 749–757, Jun. 2011.

[11] K. He, C. Rhemann, C. Rother, X. Tang, and J. Sun, “A global sampling method for

38

alpha matting,” in Proc. IEEE Conf. CVPR, 2011, pp. 2049–2056.

[12] K. He, J. Sun, and X. Tang, “Guided image filtering,” in Proc. 11thECCV: Part I,

2010, pp. 1–14.

[13] O. Gangwal, E. Coezijn, and R.-P. Berretty, “Real-time implementation of depth map

post-processing for 3D-TV on a programmable DSP (TriMedia),” in Proc. Dig. Tech.

Papers ICCE, 2009, pp. 1–2.

[14] S.-K. Han, “An architecture for high-throughput and improved-quality stereo vision

processor,” Master’s Thesis, Dept. Electr. Eng., Univ. of Maryland, College Park,

MD, USA, 2010.

[15] Y. Cao, S. Fang, and F. Wang, “Single image multi-focusing based on local blur

estimation,” in Proc. 6th ICIG, 2011, pp. 168–175.

[16] Y. Ding, J. Xiao, and J. Yu, “Importance filtering for image retargeting,” in Proc.

IEEE Conf. CVPR, 2011, pp. 89–96.

[17] Y.-C. Tseng, P.-H. Hsu, and T.-S. Chang, “A 124 Mpixels/s VLSI design for

histogram-based joint bilateral filtering,” IEEE Trans. ImageProcess., vol. 20, no. 11,

pp. 3231–3241, Nov. 2011.

[18] Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, “Edge-preserving

decompositions for multiscale tone and detail manipulation,” ACMTrans. Graph., vol.

27, no. 3, pp. 67:1–67:10, Aug. 2008.

39

LIST OF PUBLICATIONS

[1]. Anitha P, Amirthagowri G, “Design of Guided Filter using Double Integral

Image Architecture for Full – HD Video”, on CITEL 2016 in Kumaraguru

College of Technology, 30th March 2016.

40

41

Date post:	16-Jan-2022
Category:	Documents
Upload:	others
View:	6 times
Download:	0 times

DESIGN OF GUIDED FILTER FOR IMAGE FILTERING

Documents