A Parallel Algorithm A Parallel Algorithm for Hardware for Hardware Implementation of Implementation of Inverse Halftoning Inverse Halftoning Umair F. Siddiqi Umair F. Siddiqi 1 , Sadiq M. Sait , Sadiq M. Sait 1 & Aamir A. Farooqui & Aamir A. Farooqui 2 1 Department of Computer Engineering Department of Computer Engineering King Fahd University of Petroleum & King Fahd University of Petroleum & Minerals, Dhahran 31261, Saudi Minerals, Dhahran 31261, Saudi Arabia Arabia 2 Synopsys Inc. Synopsys Inc. Synopsys Module Compiler, Mountain Synopsys Module Compiler, Mountain View View California, USA California, USA
Transcript
A Parallel Algorithm for A Parallel Algorithm for Hardware Hardware
Implementation of Implementation of Inverse HalftoningInverse Halftoning
Umair F. SiddiqiUmair F. Siddiqi11, Sadiq M. Sait, Sadiq M. Sait11 & & Aamir A. FarooquiAamir A. Farooqui22
11Department of Computer EngineeringDepartment of Computer EngineeringKing Fahd University of Petroleum & King Fahd University of Petroleum & Minerals, Dhahran 31261, Saudi ArabiaMinerals, Dhahran 31261, Saudi Arabia
22Synopsys Inc. Synopsys Inc. Synopsys Module Compiler, Mountain ViewSynopsys Module Compiler, Mountain ViewCalifornia, USACalifornia, USA
Analog halftoning Analog halftoning
►The process of rendition of continuous The process of rendition of continuous tone pictures on media on which only tone pictures on media on which only two levels can be displayed. two levels can be displayed.
►The size of dots are adjusted according The size of dots are adjusted according to the local print intensity. to the local print intensity.
►When looked at a distance it gives the When looked at a distance it gives the impression of the original picture. impression of the original picture.
Digital halftoningDigital halftoning
► In digital halftoning the input of the In digital halftoning the input of the system is a grey-level image having system is a grey-level image having more than two levels for example, 256 more than two levels for example, 256 levels and the resulting image has levels and the resulting image has only two levels. only two levels.
► The halftone image is comprised of The halftone image is comprised of zeros and ones but gives the zeros and ones but gives the impression of the original image from impression of the original image from a distance. a distance.
Inverse halftoningInverse halftoning► Inverse halftoning is the reconstruction of continuous Inverse halftoning is the reconstruction of continuous
tone picture (e.g. 256 levels) from its halftoned version. tone picture (e.g. 256 levels) from its halftoned version.
► The input to an inverse halftoning system in an image The input to an inverse halftoning system in an image that consists of zeros and ones and output is an image that consists of zeros and ones and output is an image in which each pixel have value from 256 gray-levels. in which each pixel have value from 256 gray-levels.
► Inverse Halftoning finds application in image Inverse Halftoning finds application in image compression, printed image processing, scaling, compression, printed image processing, scaling, enhancement, etc. enhancement, etc.
► Inverse halftoning can be for color images but we are Inverse halftoning can be for color images but we are concerned with gray-level images and their halftonesconcerned with gray-level images and their halftones. .
Example of Inverse HalftoningExample of Inverse Halftoning
Halftone Image Inverse Halftone or grey-level image
Demonstration of our Inverse Demonstration of our Inverse halftoning algorithmhalftoning algorithm
►The next few slides show how inverse The next few slides show how inverse halftone operation is performed in halftone operation is performed in our algorithm. our algorithm.
Lookup Table (LUT) based Lookup Table (LUT) based Inverse Halftone operationInverse Halftone operation
► The Lookup Table (LUT) method proposed by The Lookup Table (LUT) method proposed by Mese and Vaidyanathan is used for inverse Mese and Vaidyanathan is used for inverse halftone operation. halftone operation.
► The LUT method uses a template “19pels” to The LUT method uses a template “19pels” to select pixels from the neighborhood of the select pixels from the neighborhood of the pixel that is going to be inverse halftone. pixel that is going to be inverse halftone.
► This “19pels” then goes into a LUT which This “19pels” then goes into a LUT which compares the “19pels” with its stored values compares the “19pels” with its stored values and returns a gray-level for the input “19pels”. and returns a gray-level for the input “19pels”.
““19pels” Template19pels” Template
11 22 33 44 55
66 77 88 99 1010
1111 1212 00 1313 1414
1515 1616 1717
1818
The pixel numbered 0 is the one going to be inverse halftoned
This pattern is associated with each pixel that is to be inverse halftoned
Demonstration of LUT inverse Demonstration of LUT inverse halftoninghalftoning
This is the first “19pels” selected
This is the second “19pels” selected
This is the third “19pels” selected
This is the fourth “19pels” selected
Our modification to LUT based Our modification to LUT based Inverse HalftoningInverse Halftoning
Problem of parallel LUT inverse Problem of parallel LUT inverse halftone operationhalftone operation
►The LUT method uses one Lookup table The LUT method uses one Lookup table that contains inverse halftone values for that contains inverse halftone values for all “19pels” that are obtained through all “19pels” that are obtained through training set of halftones of standard training set of halftones of standard images. images.
►To fetch parallel inverse halftone values To fetch parallel inverse halftone values of more than one 19pels we need to of more than one 19pels we need to implement implement multiplemultiple copies of the LUT ! copies of the LUT !
Our approach to parallel LUT Our approach to parallel LUT inverse halftoninginverse halftoning
►The single large LUT has been divided The single large LUT has been divided into many Smaller LUTs (SLUTs). into many Smaller LUTs (SLUTs).
►Now more than one 19pels can fetch its Now more than one 19pels can fetch its inverse halftone value from a separate inverse halftone value from a separate SLUT independent to other parallel SLUT independent to other parallel 19pels. 19pels.
►Next problem is to develop a method to Next problem is to develop a method to send incoming 19pels to separate SLUTs. send incoming 19pels to separate SLUTs.
Method to distinguish 19pels Method to distinguish 19pels from each otherfrom each other
► The task to send many incoming 19pels to their The task to send many incoming 19pels to their separate SLUTs is accomplished by defining an separate SLUTs is accomplished by defining an operator over 19pels. operator over 19pels.
► This operator is called Relative XOR Change (RXC). This operator is called Relative XOR Change (RXC).
► When all incoming 19pels are operated through this When all incoming 19pels are operated through this operator they convert into distinguished values in operator they convert into distinguished values in the range of the range of –t –t to to +t+t, where , where tt = 19 in our case, but = 19 in our case, but it could be any random integer within a suitable it could be any random integer within a suitable range with respect to total number of SLUTs and range with respect to total number of SLUTs and hardware complexity. hardware complexity.
Demonstration of RXC Demonstration of RXC operationoperation
RXC Operator for PRXC Operator for Pnn
1.1. PPn-1n-1= “19pels” with the pixel 0 at position (row,col-1);= “19pels” with the pixel 0 at position (row,col-1);
2.2. PPnn= “19pels” with pixel 0 at position (row,col);= “19pels” with pixel 0 at position (row,col);
3.3. xor_1= XOR(Pxor_1= XOR(Pn-1n-1, P, Pnn ); );
4.4. Magnitude of RXC= |RXC|= Number of Ones(xor_1);Magnitude of RXC= |RXC|= Number of Ones(xor_1);
5.5. Sign of RXC= sgn(RXC)= + when |PSign of RXC= sgn(RXC)= + when |Pnn| > |P| > |Pn-1n-1|| - when |P- when |Pnn| < |P| < |Pn-1n-1||
Note: pixel 0 is the one that is to be inverse halftonedNote: pixel 0 is the one that is to be inverse halftoned
RXC over gray-level halftones RXC over gray-level halftones II
Gray-level 230 Corresponding halftone obtained through Floyd and Steinberg Error Diffusion Method
RXC over gray-level halftones RXC over gray-level halftones IIII
Gray-level 130 Corresponding halftone obtained through Floyd and Steinberg Error Diffusion Method
Magnified look at the Magnified look at the halftones Ihalftones I
Halftone shows no column-wise periodicity among dots over
small 19pels regions
Halftone shows column-wise periodicity among dots over small 19pels regions
Gray-level 210 Gray-level 130
Magnified look at the Magnified look at the halftones IIhalftones II
Halftone shows no periodicity among dots over small 1D
19pels regions
Gray-level 120
Halftone shows no periodicity among dots over small 1D
19pels regions
Portion of the halftone from image Boat
NON Periodic Vibratory RXC NON Periodic Vibratory RXC Operator Operator
► The operator RXC has been defined that is simple to The operator RXC has been defined that is simple to implement in hardware as well as gives NON implement in hardware as well as gives NON periodic vibratory response over most of the gray periodic vibratory response over most of the gray levels from 0 to 255. levels from 0 to 255.
► We have assumed that a gray level image is a We have assumed that a gray level image is a composition of many gray levels and obtaining the composition of many gray levels and obtaining the performance of RXC over individual gray levels can performance of RXC over individual gray levels can give a clue about its performance on images. give a clue about its performance on images.
► This assumption is found to be correct in simulation This assumption is found to be correct in simulation results. results.
Parallel application of RXCParallel application of RXC
Development of parallel table Development of parallel table access algorithm with RXCaccess algorithm with RXC
The addition of Slut values
from previous pixels
simplifies the hardware
design
Formal Algorithm Formal Algorithm
SimulationSimulation
► The algorithm is implemented in MATLAB the The algorithm is implemented in MATLAB the performance and quality of inverse halftoning is performance and quality of inverse halftoning is estimated. estimated.
► We assumed LUT inverse halftone operation to be We assumed LUT inverse halftone operation to be ideal. ideal.
► The simulation results show the quality loss with The simulation results show the quality loss with respect to original image that occurred in distribution respect to original image that occurred in distribution of parallel “19pels” to different SLUTs through RXC. of parallel “19pels” to different SLUTs through RXC.
► This pixel loss is compensated through replicating This pixel loss is compensated through replicating gray level values from the neighbors. gray level values from the neighbors.
Sample Image ISample Image I
PSNR= 34.7880
peppers
Sample Image IISample Image II
PSNR= 32.5685
lena
Sample Image IIISample Image III
PSNR= 28.1264
mandrill
Hardware ImplementationHardware Implementation
► This section shows the hardware This section shows the hardware implementation of the proposed parallel implementation of the proposed parallel algorithm in terms of block diagrams. algorithm in terms of block diagrams.
► The specification of the hardware design The specification of the hardware design is:is:
1.1. Parallel Pixels to be inverse halftone= Parallel Pixels to be inverse halftone= nn= 15= 15
2.2. Number of SLUTs= 19Number of SLUTs= 19
Two Blocks of hardware Two Blocks of hardware ImplementationImplementation
► The hardware system can be divided The hardware system can be divided into two blocks: into two blocks:
1.1. RXC and modulus operatorsRXC and modulus operators
2.2. 19pels to gray-level decoders19pels to gray-level decoders
RXC and modulus operatorsRXC and modulus operators► RXC and modulus operators components are responsible for the following RXC and modulus operators components are responsible for the following
tasks:tasks:
Input:Input: 19pels 19pels Output:Output: SLUT numbers SLUT numbers SlutSlut
1.1. Accept 19pels from the halftone image and assign a sequence number to Accept 19pels from the halftone image and assign a sequence number to each entered 19pels. each entered 19pels.
2.2. Perform RXC operation on all 19pels. Perform RXC operation on all 19pels.
3.3. Add the Add the SlutSlut value of the 19pels that has preceding sequence number to value of the 19pels that has preceding sequence number to the current result. the current result.
4.4. Then take mod of the current result with a fixed number i.e. 19 in our case Then take mod of the current result with a fixed number i.e. 19 in our case to obtain to obtain SlutSlut value for the current 19pels. value for the current 19pels.
5.5. The above three steps are pipelined so new 19pels are coming in while the The above three steps are pipelined so new 19pels are coming in while the current 19pels are in process. current 19pels are in process.
RXC and modulus Block RXC and modulus Block DiagramDiagramRXC calculation for 19pels Pn
Pn-1 and Pn are two 19pels among all 19pels to be inverse halftoned in parallel.
Slut is the Smaller LUT number where the concerned 19pels should go to fetch its inverse halftone value.
Hardware Design of RXC and Hardware Design of RXC and modulus Operatormodulus Operator
►The next slides can show the hardware The next slides can show the hardware design of RXC operator for a 19pels design of RXC operator for a 19pels pattern named Ppattern named Pnn with the following with the following parameters:parameters:
►Parallel pixels to be inverse halftoned at a Parallel pixels to be inverse halftoned at a time= time= 1515
►Total number of SLUTs= 19Total number of SLUTs= 19, , therefore, therefore, SlutSlut is from 0 to 19. is from 0 to 19.
Determination of Determination of SlutSlut from from RXCRXC
PSNR with PSNR with pixel pixel compensationcompensation
BoatBoat FS EDFS ED 65.086465.0864 30.374930.3749
ClockClock FS EDFS ED 70.666770.6667 30.167130.1671
PeppersPeppers FS EDFS ED 68.943368.9433 28.548428.5484
BoatBoat GN EDGN ED 63.753163.7531 28.713928.7139
ClockClock GN EDGN ED 69.876569.8765 31.255431.2554
PeppersPeppers GN EDGN ED 68.950968.9509 29.007729.0077
BoatBoat EG EDEG ED 67.308667.3086 32.137032.1370
ClockClock EG EDEG ED 68.592668.5926 29.928929.9289
PeppersPeppers EG EDEG ED 69.990569.9905 28.548328.5483
Comparison to halftone Comparison to halftone 256*256256*256
Algorithm in [7] Proposed Algorithm
Cycles/pixel 1 0.066 LUT size 5.1 K entries 19 K entries Latency 4 clock cycles 17 clock cycles
Time taken 691.3502 ms 45.6389 ms
Conclusion and Future WorkConclusion and Future Work
►A parallel implementation for inverse A parallel implementation for inverse halftone has been presented. halftone has been presented.
►Results can be improved by improving Results can be improved by improving the operators and training.the operators and training.
►Results obtained are encouraging.Results obtained are encouraging.
Method to generate contents of Method to generate contents of SLUTSLUT
►The algorithm is applied on images in The algorithm is applied on images in a training set and Sluts values are a training set and Sluts values are obtained. obtained.
►The 19pels then placed in the SLUT The 19pels then placed in the SLUT given by the corresponding Slut value. given by the corresponding Slut value.
Properties of SLUTs Properties of SLUTs ► The SLUTs were developed using training set composed of FS The SLUTs were developed using training set composed of FS
ED halftone images of Boat and Peppers of size 256x256-ED halftone images of Boat and Peppers of size 256x256-pixels. pixels.
► The size of one SLUT is found to be 2.5K entries . The size of one SLUT is found to be 2.5K entries .
► The summation of entries in all 19 SLUTs comes to be 42.6K. The summation of entries in all 19 SLUTs comes to be 42.6K.
► The size of LUT in single LUT method is 9.86K entries, The size of LUT in single LUT method is 9.86K entries, however, if the single LUT method is implemented multiple however, if the single LUT method is implemented multiple times for 15 parallel pixels the total size could become 148K times for 15 parallel pixels the total size could become 148K entries. entries.
► In this way, our method can provide 3.5 times decrease in In this way, our method can provide 3.5 times decrease in lookup table size over single LUT based method. lookup table size over single LUT based method.
Behavior of RXC over Grey-level Behavior of RXC over Grey-level halftoneshalftones
NON Periodic Vibratory ResponsePeriodic Vibratory Response
Gray level 130Gray level 210
Halftones obtained through Floyd & Steinberg Error Diffusion Method
Representation of RXC values Representation of RXC values on number lineon number linePeriodic Vibratory Values
RXC values to be used in SLUT access are calculated by adding the RXC to the RXC of the previous “19pels”
That is:
RXC for SLUT of Pn (Slut)=
RXC of Pn-1 +
RXC of Pn-2(n)
From the number line we can see that adding RXC over previous values gives zero or constant result, therefore, we need NOT periodic vibratory response from RXC operator.
Modified RXC IModified RXC I
► At present, RXC is a comparative operator At present, RXC is a comparative operator that it gives values in comparison to the that it gives values in comparison to the previous 19pels. previous 19pels.
► This behavior of RXC can give different Slut This behavior of RXC can give different Slut values if some 19pels are replaced from the values if some 19pels are replaced from the image. image.
► Therefore, we are required to store same Therefore, we are required to store same 19pels in more than one table. 19pels in more than one table.
Modified RXC IIModified RXC II
► Let us define a standard value for RXC=Let us define a standard value for RXC= The value of 19pels at which the The value of 19pels at which the
histogram of 19pels present in training set histogram of 19pels present in training set images can be divided into two portions. images can be divided into two portions.
► We find Slut value of each 19pels with We find Slut value of each 19pels with respect to this standard value. respect to this standard value.
► That way we can have almost uniform table That way we can have almost uniform table size with no repetition of same value in size with no repetition of same value in different tables. different tables.
