Received: 24 August 2012 Revised: 25 October 2012 Accepted: 9 November 2012 Published online in Wiley Online Library: 25 January 2013
(wileyonlinelibrary.com) DOI 10.1002/jrs.4232
608
A new paradigm for signal processing ofRaman spectra using a smoothing freealgorithm: Coupling continuous wavelettransform with signal removal methodAhmad Esmaielzadeh Kandjani,a,b Matthew J. Griffin,b Rajesh Ramanathan,a,b
Samuel J. Ippolito,b Suresh K. Bhargavab and Vipul Bansala,b*
Noise removal is considered a primary and inevitable step for background correction in experimentally obtained Ramanspectra. Employing an appropriate algorithm for a smoothing-free background correction technique not only increases the
* Correspondence to: Vipul Bansal, School of Applied Sciences, RMIT University,GPO Box 2476 V, Melbourne, VIC 3000 Australia.Email: [email protected]
a NanoBiotechnology Research Lab (NBRL), School of Applied Sciences RMIT Uni-versity, GPO Box 2476 V, Melbourne, VIC 3000, Australia
b Centre for Advanced Materials & Industrial Chemistry (CAMIC), School of AppliedSciences, RMIT University, GPO Box 2476 V, Melbourne, VIC 3000 Australia
Introduction
Raman spectroscopy, due to its ability to provide information aboutthe physical and chemical characteristics of materials, finds itsapplication in many different branches of science from biology tochemistry and materials science.[1,2] It is a non-destructive tech-nique, which is routinely used to qualitatively and quantitativelyanalyse materials by identifying their native structures and struc-tural impurities.[3–5] Raman spectroscopy can also be used for inves-tigating the thermodynamical aspects and phase equilibrium ofdifferent materials.[6] Similarly, Raman spectroscopy has attractedsignificant attention in bio-imaging wherein different biologicalcomponents could be differentiated due to the differences in theresonance nature, especially by the position of peaks in Ramanspectra as well as their relative intensities, which is related to thequantity of each molecular structure, thus making Raman spectros-copy a strong candidate for mapping biological objects.[7]
One of the major problems associated with employing Ramanspectroscopy, especially for biological samples, is that due to thesensitivity of biological samples to the incident wavelength ofthe laser, Raman signals from biological samples typically havelower signal to noise ratios (SNRs) and high magnitude of back-ground which arises mainly from auto-fluorescence. Since theexistence of the background suppresses the main spectrum, theinterpretation becomes very difficult. Therefore, it is essentialfor a background correction method (BCM) to be applied to thespectrum before performing detailed analysis of the spectraobtained from Raman spectroscopy.
J. Raman Spectrosc. 2013, 44, 608–621
Given the significant scope of different spectroscopic techniquesin analysing biological, chemical and materials sciences samples,extracting meaningful information from spurious spectra is essen-tial. Hence, it is of utmost importance for signal processing softwareto be able to distinguish noise and background from the originalsignal. From a mathematical point of view, a sampled signal canbe considered as an array (S) that can be given as:
S ¼ Ps þ Bð Þ þ N (1)
where, Ps, B and N are related to the noiseless signal without back-ground (pure spectrum), background and noise, respectively.Hence, to separate spurious features from spectroscopic data, it isof utmost importance to remove noise and background signalsfrom the experimental spectra. The reasons required for theremoval of noise and background from an experimentally obtainedspectrum may be diverse and application dependent; however, inmost of the cases, it is essential to apply a proper backgroundcorrection algorithm for increasing the effective resolution for
New paradigm for signal processing of Raman spectra
609
quantitative analyses. Therefore, significant efforts have beendirected towards determining new approaches for the removal ofbackground from signals with higher accuracy, independent fromsignal nature and human error. [8]
To this end, in most of the BCMs, some information related tothe signal or background is required before applying these BCmethods. Based on the type of information that needs to beextracted from signals, BCMs can be categorized into two majorgroups. The first group of BCMs include methods requiringknowledge about background, blurring effect and noise thatpredominantly deal with signals by utilizing knowledge aboutthe signal components such as background shape, position andSNR. Some of the well-known examples in this category includethe noise median method,[9] signal removal method (SRM)[10]
and threshold-based classification (TBC).[11] On the other hand,the second group of BCMs include those requiring knowledgeabout frequency of signal components, i.e. if a signal isdecomposed based on frequencies, the noise and backgroundwould have completely different characteristics because noise isgenerally a high-frequency phenomenon, while background canbe considered as a low-frequency component of a signal. Thissuggests that if we decompose signals based on their frequen-cies and filter the noise and background components, a purenoiseless and background corrected signal can be obtained. Thistype of signal processing forms the base for the morecommonly employed Fourier transform (FT) [12] and wavelettransform (WT) methods.[13]
However, each of the aforementioned categories has someassociated disadvantages. One of the major limitations with thefirst group of BCMs is that the existing noise in the signal makesde-noising and smoothing an inevitable process before engagingin any background removal. This is because most methods in thiscategory (e.g. SRM or TBC) essentially employ the derivative ofthe signal that is estimated numerically, which may be ratheroverwhelming to calculate without employing a smoothingprocess in advance. Additionally, applying a smoothing proce-dure in the early stages can inject unwanted errors into thesignals depending on the form of de-noising methodology. Anerroneous de-noising process can further result in peak shifts oreven peak suppression in the case of low SNRs.[14] On the otherhand, for the second category, the frequency-based methodsdivide the signal into components based on their frequencies,which can sometimes be rather daunting. This is generally dueto the fact that components in real signals do not have constantfrequencies, i.e. the frequency component is not a single rigidvalue but consists of a range of frequencies, making selectionof thresholds a challenging endeavour. Therefore, decomposingthe spectrum into different frequencies followed by rebuildingthe final spectrum could leave traces of noise or background.Cai et al. highlighted some of these issues in their study usingdifferent discrete WT (DWT) as well as FT methods, and readersare encouraged to refer to this paper and the references within.[15]
If a technique could be used in background subtraction inwhich the derivative of a spectrum could be detected withoutsmoothing procedures, SRM could be used to estimate thebackground of the spectrum. As mentioned before, smoothingof the spectra could inject some unwanted errors to the system,but based on the authors’ best knowledge, there are limitedreports wherein the background correction is carried out withoutsmoothing[16]. As mentioned before, in SRM methodology, peaksare generally removed from the spectrum using derivative of thespectrum to understand the position, starting and finishing
points of the those peaks. Continuous WT (CWT) and DiscreteWT (DWT) can be used as an alternative approach for procuringderivatives of noisy signals.[17–19]
In the current work, we provide a new algorithm for a highlyefficient background correction of Raman spectra, which is basedon combining certain strengths of SRM and CWT methodologies.Specifically, in the current study, initially, CWT is employed tocalculate the approximate second derivative of the noisy signalwhich leads to the identification of signal peak positions in theexperimental spectrum. This is followed by using SRM to removethe signal peak component of the spectrum and fitting thereminiscent spectrum to find the background, which is furthersubtracted from the original spectrum to obtain a background-corrected signal. It is notable that CWT was chosen over DWT inthe current study as the former is a tool for analysis, featuredetermination and approximate derivative calculation, while thelatter is the preferred technique for data compression anddenoising. Application of the current algorithm for backgroundcorrection of four different noisy experimental systems (L-serine,rhodamine, methyl red and crystal violet) showed that a goodperformance of this algorithm, wherein the end effect errors werefound considerably less than the commonly reported studies.Notably, the major strength of the current algorithm is that itdoes not involve any smoothing step, avoiding which is a majorchallenge in obtaining background-corrected spectra.
Basics and methods
Basics
WT like FT are a convolution between a wavelet function (c(t))and signal (x(t)). The major difference between FT and WT is thatin FT, the wavelet has a sine or cosine form that specificallyprovides information in the frequency domain, while in WT,the mother wavelet could have any function if it has a zero-meanoscillation behaviour. A mother wavelet could produce families ofwaves through:
ca;p tð Þ ¼ 1ffiffiffia
p ct � b
a
� �(2)
where b is the parameter for transition and a represents dilation(that is always a positive integer). If we consider 1/a representingan average frequency, then b indicates the position of waveletwindow. Hence, in employing WT, information on both timeand frequency can be extracted from a spectrum. It is essentialto note that although, both WT and FT provide information onthe frequency, they are not a replacement for each other.
WT can be divided into two main categories viz. CWT and DWTand can be defined as:
Wf a; bð Þ ¼Zþ1
�1f tð Þ�c�
a;b tð Þdt (3)
where the asterisk (*) represents complex conjugation. This equationcan also be given as:
Wf a; bð Þ ¼ f bð Þ�c�a bð Þ (4)
where � represents convolution.Although wavelet transformation has been studied extensively
for processing spectroscopic data,[20,21] only in the recent past, thistechnique was used for calculating the approximate derivative of a
signal. This was substantiated by showing that the nth orderderivative of a signal could be achieved in a dilation (scale) ofa by applying an appropriate mother wavelet. Furthermore, themother wavelet was chosen in a way that its derivative still hada wavelet nature.[17] For instance, if the Gaussian function is con-sidered as a mother wavelet, its second derivative commonlyreferred to as ‘Mexican Hat’ or ‘Marr’ with a minus sign can alsobe used as a wavelet.[22] In the present case, the nth derivativeof a signal can be estimated using ‘Gaussian’ wavelet applied ntimes to the spectrum or a proper nth derivative of the Gaussianfunction. As outlined before, noise and spectrum have differentfrequencies where lower frequency components are related tothe higher dilation (scale) numbers. Thus, we can approximatelysuppress the side effects of noise of the transformed signal byincreasing the dilation. This gives us the ability to find theapproximate derivative of a noisy spectrum by reducing theinfluence of noise.Spectroscopic techniques commonly employ derivative calcu-
lations as a resolution enhancement technique,[23,24] especiallythe second order derivatives for extracting peak characteristicssuch as position and starting–finishing point from a signal.[25]
For high-resolution enhancement, higher order derivativescan be used for locating and de-convoluting overlapping peaks.With the second-order derivative, peaks can be mined from aspectrum where the remaining points of the spectra would bethe representative segments of background that can be furtherused for background estimations. The presence of noise in thesesignals can be a serious drawback in finding peaks and calculat-ing derivatives of experimental signals. One of the most commontechniques employed for calculating the derivative is ‘NumericalCalculation’. Due to the random nature of noise in these spectra,numerical derivative would result in noisy signals especiallyin those with low SNRs, which makes spectral smoothing anessential process.
Figure 1. (a) Simulated spectrum with ten peaks without noise and withou(c) Linear background, (d) Spectrum generated from adding linear backgrogenerated from adding sigmoidal background shown in e to spectrum b, (g) Sbackground shown in g to spectrum b.
All experimental steps in the current study were investigatedutilizing MATLAB for MS Windows, version 7.11(R2010b). In thisstudy, Raman peaks were simulated using a Gaussian functiongiven by:
f xð Þ ¼ a�e �0:5 x�cð Þ2s2
� �(5)
where a is the intensity controller, c and s are median and vari-ance, respectively, of the Gaussian peak. The program createssimulated Gaussian peaks of variable quantity with random posi-tions, intensity and width distributed in the spectrum with threemain backgrounds as linear, sigmoid and sinusoid forms (Fig. 1)and variable background constants. The general formulae forbackgrounds are:Linear background:
Background ¼ a�x þ b (6)
where a and b are the slope and scope of line, respectively.Sigmoidal background:
Background ¼ 1
1þ exp �a x � cð Þð Þ I þ O (7)
where a is the gradient at the inflection point, c is the location ofthe inflection point, I is the intensity controller (since sigmoidfunction results in numbers between 0 and 1) and O is an offset.Sinusoidal background:
Background ¼ x1:5SinðxaÞ�I þ O (8)
where a is the frequency controller, I is the intensity and O is theoffset. Noise is considered as white Gaussian noise and addedbased on calculated SNRdB.
t background, (b) Same spectrum after adding white noise with SNR=25,und shown in c to spectrum b, (e) Sigmoidal background, (f) Spectruminusoidal background and (h) Spectrum generated from adding sinusoidal
Wiley & Sons, Ltd. J. Raman Spectrosc. 2013, 44, 608–621
Figure 2. General flowchart for estimating background.
New paradigm for signal processing of Raman spectra
611
Generating calibration curve
As outlined in the introduction, the second derivative of a spec-trum can be estimated with CWT using ‘Mexican hat’ as a motherwavelet.[17] This can be followed by choosing the appropriatescale to reduce the effects of noise in the second derivative. Anincrease in the scale (i.e. towards lower frequencies) results in adecrease in the influence of noise. This increase also results inbroadening of the wavelet (i.e. widening of transformed peaks),which in turn reduces the resolution of the derivative spectrumdue to merging of peaks at higher dilation numbers. Therefore,to overcome this issue and to select the Best-Scale, which isrepresentative for the derivative of noiseless spectrum, SNR ofthe spectrum must be considered during calculations. For thisreason, correlation coefficient (r) was used as a factor to selectthe Best-Scale:
where xi and yi represent the ith element of vectors X and Y,respectively, where (r) could have values between 0 and 1, andif r=1, vectors X and Y would be completely similar to each other.
If the spectrum has a signal without noise, its second derivativecan be easily calculated numerically. Now, if noise with a knownSNR is added to this signal, by using the CWT method, its trans-formed spectrum can be calculated at different scales. Thereafter,by comparing the resultant spectrum of each scale with noiselesssecond derivative of the signal and calculating the respectivecorrelation coefficient values, one can find the variation of corre-lation coefficient with increase in SNR.
In this study, several spectrawith each containing a single Gaussianpeak of similar intensities and positions, however, with differentwidths and SNRs were synthesized. The numerical second deriva-tives of these noiseless spectra were determined before adding awhite Gaussian noise to these spectra. Thereafter, the correlationcoefficients between the numerically derived second derivativeand wavelet transformed spectra with different SNRs at differentscales were calculated. Transformed spectra in scales with highestcorrelation coefficients were chosen for each SNR and named asthe Best-Scales. Following the determination of the Best-Scales forsignals with different SNRs, these parameters were plotted for eachof the signal widths and the respective calibration curves wereestimated by fitting a function to these points. This allowed us toestimate the Best-Scale values by knowing SNR and the width of asignal. A flowchart outlining the estimation of the Best-Scale isshown in the supporting information (Fig. S1).
BCM
The underlying principles for background correction employed inour method are as follows:
• Estimating SNR• Calculating second derivative of the spectrum using CWT in
the Best-Scale using estimated SNR• Finding the starting and finishing points of peaks in the
spectrum through estimated second derivative• Finding points related to background by removing peaks from
• Fitting a ninth-order polynomial function to backgroundpoints and adjusting fittings
• Background correction by subtracting the fitted spectrumfrom the spectrum
The overall algorithm is shown in Fig. 2, and the detailedflowchart for each section is provided in supporting informa-tion (Fig. S1–S7).
Data processing. This program allows the ability to cut theregions of interest in a particular spectrum, thereby (1) increasingthe visual resolution, (2) increasing the accuracy due to thedecrease in the amount of calculation, and (3) minimizingunexpected errors due to the abrupt change of background orimpurities from other erroneous peaks. This function was incor-porated as often only a section of the spectrum is required forspectral analysis.
Estimating SNR. The SNR equation used in this program is in dBmode and is given by:
SNRdb ¼ 20: log10RMS Signalð ÞRMS Noiseð Þ
� �(10)
where RMS represents the root mean square.For estimating the noise profile, initially, the spectrum is
divided into 30 scanning windows with equal lengths (X-axis),and each of these 30 windows is used independently to estimatethe local standard deviation (LSTD) in each of the scannedsections. Minimum LSTD is used as a region for estimating noiseprofile and calculating the RMS of the noise of the whole spectrum.For calculating RMS of the signal, the spectrum is smoothed usingSavitzky-Golay filter at different levels from 0.1 to 0.9, followed bysubtracting them individually from the main spectrum. Thereafter,SNRs of all the temporary background corrected signals are calcu-lated using Eqn (10). The average of these SNRs is then used tochoose the Best-Scale by input of peak width by user.
Calculating second derivative of the spectrum and correcting itsendpoint effects and residual noise. It is well known that due tothe discreet nature of a spectrum, artificial peaks are typicallygenerated at both the ends of the transformed signals duringtransformation. To address this issue, we added points to thestart and the end of the original spectrum and subsequentlyshifted and restricted this erroneous effect. After transformation,these erroneous areas could easily be removed from the signaland the second derivative. It should be noted that the pointswere added in a way that there would be minimal discontinuitiesor changes in the slope of the spectrum as this would haveconsiderably generated some unwarranted artificial peaks inthe derivative spectrum. Additionally, as noise does not followthe same frequency as in the experimental data, there were stillsome remanent traces of noise in the Best-Scale. These reminis-cent traces always have a smaller intensity in comparison withthe peaks related to the signal. For completely eliminating theeffect of residual noise in the second derivative, we squared thisspectrum to enhance the signals (SSDS).
Peak removal and finding background points. The method forbackground correction in this algorithm is based on SRM. In thistechnique, the first step involves the isolation of peaks from thesignal (the residual corresponds to background). During isolation,peak (signal) starting and finishing points were identified usingthe second derivative obtained from step 2.3.3.3, which allowedus to calculate the zero crossing that corresponded to the start-ing and finishing points. Based on the zero crossing, thespectrum could then be divided into different sections with clearstarting and finishing point pairs. Thereafter, the areas of eachsection within the second derivative spectrum of a particularsignal sandwiched between zero crossing pairs were calculated,followed by the selection of the minimum (i.e. largest negative)local area corresponding to the largest and the sharpest peak.For selecting areas related to background, a simple threshold isconsidered as:
Thershold ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiabs min local area ÞÞðð
p(11)
Any local areas smaller than this threshold were considered asbackground. These background points were saved into fittingarrays of FIT_X (Wavenumber) and FIT_Y (intensity). Furthermore,the next derivative of SSDS (endpoints corrected second
derivative of the spectrum) was calculated through CWT using‘Gauss1’ as mother wavelet, which produced third derivative ofthe spectrum. In the case of a positive area surrounded withtwo negative areas in SSDS, the former was scanned for theminimum extremum points, which correspond to zero crossingpoints of the first derivative of SSDS with negative slopes. Thesepoints were then added to the fitting arrays FIT_X (wavenumber)and FIT_Y (intensity) to bring the fitted background closer to thereal peak minima.
Adjusting endpoint effects. Estimation of the background usingsignal-deprived spectrum is based on the fitting of residualpoints with a ninth-order polynomial function. In the case wherewe have no background points at the start and finish of thespectrum, the fitting could select any arbitrary conditions inthese areas, thereby failing to have a correct background correc-tion towards these endpoints. The most commonly employedapproach to address this problem is to continue the minimumof the nearest background point as a horizontal line.[26] However,this approach produces an artificial offset at the ends of the spec-trum. In order to resolve this issue, we have fitted 100 points eachfrom the starting and the finishing parts of the spectrum with acubic polynomial. This process decreases the effect of noise inthe selected sections of the spectrum. Based on the locationof the starting and finishing background points and the slopesof the spectrum at endpoints, seven different conditions mayoccur, which are detailed in supporting information S2.
Fitting and adjustments. Following the endpoint correction, thepoints relating to the background are fitted with the ninth-orderpolynomial. For controlling the fitting behaviour, a correctioncondition has been considered. After finding the SNR, it isrelatively simple to estimate the peak-to-peak (PTP) value of thenoise. One important aspect that needs to be considered toobtain a precise background correction is that following thisprocess, the resultant background corrected spectrum shouldnot have any data lower than PTP
2 þ e� �
where e is related to thebackground correction error, which should not be more thanthe value of PTP itself. In the current algorithm, a simple loopvalidates this threshold for all the points of the spectrum. If thiscondition fails, the coordinates of the minimum of the spectrumat the failed ranges are added to the fitting arrays, and fittingprocess restarts until the PTP
2 þ e� �
condition becomes valid atall the points of the spectrum. This process eliminates thepossible fluctuation in the background estimation, which mayotherwise arise due to the lack of background points in somesections of the spectrum.
Testing the accuracy of current algorithm
For testing the accuracy of the designed algorithm, statisticalanalyses have been carried out.
The processes include:
• Producing 900 spectra, each containing ten Gaussian peaks ofrandom characteristics (i.e. m, s and intensity) with constantSNR=60 dB
• Changing the number of Gaussian peaks from 2 to 30 withrandom characteristics (i.e. m, s and intensity) and constantSNR=60 dB, with each condition applied for 200 times.
• Changing SNR of the spectrum from 10 to 130 dB with tenGaussian peaks of random characteristics (m, s and intensity),with each condition applied 200 times.
Wiley & Sons, Ltd. J. Raman Spectrosc. 2013, 44, 608–621
New paradigm for signal processing of Raman spectra
The spectrum after the background correction is compared withthe original spectrum and RMS error (RMSE) values is calculated.
Experimental data
Chemicals. Rhodamine B, crystal violet and methyl red werepurchased from Merck Chemicals and L-serine amino acid waspurchased from Sigma-Aldrich. All chemicals were used withoutfurther modifications.
Preparation of e-beam evaporated substrates. The metal layerswere deposited by a BalzersTM electron beam evaporator. Thelayer composed of 1000 Å Au with an underlying 100 Å Ti layer.The films were deposited sequentially by electron evaporationprocess onto the bare AT-cut quartz substrates. The purpose ofthe Ti layer is to assist with the adhesion of the Au layer to thesubstrate surface.
Raman scattering measurements. To obtain good Raman signals,gold substrates were immersed in 1 mM solutions of rhodamine B,crystal violet or methyl red for 1 h, followed by washing withdeionized water (MilliQ) and air drying. In case of L-serine aminoacid, the powder was directly placed on a flat gold substrate beforeRaman measurements. It is known that Au and Ag thin films andnanostructured substrates assist in increasing the Raman scatteringcross section of molecules by a surface enhanced Raman scattering(SERS) process.[1,2,27] The above samples containing differentRaman activemolecules were analysed using a Perkin Elmer RamanStation 200F (785nm laser, spot size of 100 mm) with an exposuretime of 1 s and 20 acquisitions, with disabled backgroundcorrection feature.
Results and discussion
The performance of the current algorithm is summarized bystating the results of the calibration curve calculations and furtherexplaining the results of each section outlined in the methods.
Figure 3. A Gaussian peak with different SNR values.
Figure 3 shows a single artificially synthesized Gaussian peakwith different SNR values. The width of the Gaussian peakequals 40 units in this analysis. Figure 4 shows the variation ofcorrelation coefficient (r) with SNR and CWT scales for the artifi-cially synthesized spectra outlined in Fig. 3. To generate thecalibration curve, the scale related to the highest correlationcoefficient (r) obtained for Gaussian peaks exhibiting differentSNR is plotted against the SNR values. A function can then befitted through these points, where the Best-Scale for obtainingthe second-derivative of any spectra after finding its SNR canbe estimated. The function which best fits this calibration curveis exponential in nature. The best fit function generates realnumbers, while scales for CWT should contain only integernumbers. Hence, rounding towards positive infinity of this fittedfunction is considered as the calibration curve. Calibration curvefor current example is shown in Fig. 5.
There are three important issues inworkingwith calibration curves:
• Changes in the number of signal data points can change thecalibration curve[19]
• The numbers of scales in CWT should be same for all investi-gated spectra
• The calibration curve is dependent on the width of the peaks(in Gaussian peaks this is known as variance)
To address the first condition, the number of data points foreach spectrum was fixed at 5000 points. This adjustment wascarried out using cubic spline data interpolation. The scales forall transforms have been chosen in a constant array from 1 to200. The variation of calibration curve for different variance ofsimulated Gaussian peak is shown in Fig. 6. In the case of con-stant SNR, an increase in peaks width results in an increase inBest-scale number, a phenomenon that has been widelyreported.[19] In the previous report, different situations influenc-ing the derivative estimation of a signal using CWT was consid-ered. Hence, in the current situation, by fixing data points andscale array lengths, the only user input will be the width
Figure 4. Variation of correlation coefficient with SNR and CWT scales. This figure is available in colour online at wileyonlinelibrary.com/journal/jrs
Figure 5. Calibration curve for signals in Figure 4. This figure is available in colour online at wileyonlinelibrary.com/journal/jrs
A. E. Kandjani et al.
614
estimation. In the case where a lower variance is selected, some-times, part of the peak base is selected as background as well. Inthis case, background estimation through fitting of ninth-orderpolynomial, which could be corrected with the algorithm, isexplained in section 2.2.3.6. In the case where the variance esti-mations are high, the areas that relate to the background wouldbe more confined due to the increase in Best-Scale values. Thismay result in false background estimation producing artificialhumps in the corrected spectrum. This can be addressed bydecreasing variance estimation number with user input. Animportant aspect of this is that if the average peak width inthe spectrum lies within the same range of the estimatedvariance number, then this fixed number would address allsimilar situations.In the investigated experimental results, keeping the variance
number a constant value of 20 resulted in appropriatebackground estimations in most of cases, which is explained inthe experimental results section.
SNR is an important factor to determine the Best-Scale values forestimating second derivative of a spectrum. Due to the depen-dency of Best-Scale to SNR, it is of utmost importance to estimateSNR of a spectrum before estimating second derivative. As previ-ously mentioned, the first step for this calculation is de-convolutingor estimating the noise profile from the signal. One method toaddress this issue is to smooth a noisy signal and subtracting thede-noised signal from the spectrum that results in the noise profile.Although this approach is extensively used, there are issues associ-ated with this approach. Primarily, in the case where there existshigh level denoising, if the signal has sharp peaks, the de-noisedspectrum could reduce the intensity of these peaks. Subsequently,the noise profile derived from simple subtraction of the de-noisedspectrum from the noisy spectrum would result in artificial peakswhere sharp peaks occur in the spectrum. This error induces higherintensities in the noise profile within the ranges where sharp peaks
Wiley & Sons, Ltd. J. Raman Spectrosc. 2013, 44, 608–621
Figure 6. The effect of peak width on Best-Scale in different SNR values. This figure is available in colour online at wileyonlinelibrary.com/journal/jrs
New paradigm for signal processing of Raman spectra
are smoothed in the spectrum. On the other hand, in the case oflow SNR values, the peaks with lower intensities are suppressedduring the denoising step, which introduces errors in estimatingthe noise profile.[14]
If we consider noise as a high-frequency signal distributedevenly over the whole spectrum, a section of its profile couldbe used as a representative for the noise profile where this rangeis comparably larger than the average noise wavelength. In otherwords, two different sections of a noise profile should havesimilar RMS values with negligible variance if they are distributedevenly and have the same intensity in the overall range. Animportant aspect that needs to be addressed is selecting thethreshold for dividing the spectrum into measurable sections.
Figure 7. SNR estimation steps: (a) Simulated Raman spectrum with ten peakshaded section represents window size for calculating LSTD; (b) STD in differred line shows linear fitting of the spectrum in selected region to find backgspectrum and (e) Different smoothing levels of the spectrum. This figure is a
This division window should be large enough to provide a signif-icant sample of the noise profile for calculations and also smallenough to make it possible to select a region that does notinclude peaks. After selecting the proper window size, the STDfor each window is calculated and the lowest value should corre-spond to a part of the signal which consists of noise and back-ground without peaks. In the case where the selected windowsize is small enough, the background can be estimated using asimple linear fit. The results of the noise profile selection areshown in Fig. 7 for a spectrum with sigmoid shaped backgroundwith ten peaks and initial SNR equal to 20. SNR estimation resultsfor a spectrum with other types of background are shown in thesupporting information S3.
s randomly distributed on the signal with sigmoidal background, whereinent divided ranges of spectrum; (c) Spectrum in minimum LSTD, whereinround, (d) Estimated noise profile by subtracting linear background andvailable in colour online at wileyonlinelibrary.com/journal/jrs
For estimating the accuracy of the current algorithm, testinghas been carried out using similar signal features (ten peaksrandomly distributed on the signal with sigmoidal background)while changing two parameters: SNR value of the signal andthe background intensity. The estimation of the accuracy of thecurrent algorithm is detailed in supporting information S4.
Second derivative and end effect
Most spectra are discrete in nature, i.e. they do not always tendto be of zero intensity at the starting and finishing points (endeffect). These points are considered as break points in the spec-trum and during wavelet transformation, an artificial peakappears in these areas.[28] The second derivative of the syntheticspectrum without applying end effect correction results in arti-ficial peaks (Fig. 8b, outlined in magenta colour). As previouslystated, negative peaks in the second derivative correspond tothe place of the peaks in the spectrum. Due to the end effect,there is an addition of two artificial peaks resulting in an errorduring the peak removal process. The second derivative of anyspectra is calculated using WT with ‘Mexican Hat’ as mother wave-let, as described previously. The active regions of ‘Mexican Hat’wavelet is equal to [�5 � a, 5 � a] where a represents the scaleof transform. Thus, if we extend the spectrum from both sidesin a way that the added points could have length widerthan 5 � a, the end effects would be confined to these regions.As the second derivative of a signal is highly sensitive to anybreaking points and sharp changes in slopes in the signal, theextending points should be added in a way that it follows itsadjacent slope of the signal.
Figure 8. (a) Synthetic spectrum with SNR=20 with 10 peaks; (b) Second derion slopes of the spectrum at ending points and (d) Second derivative throughis available in colour online at wileyonlinelibrary.com/journal/jrs
Following the estimation of Best-Scale (where the value for a canbe determined), 10*a points are added to the start and end of thesignal based on the signals local slope at these junctions (Fig. 8c).The active regions were doubled in this case to ensure that no traceof end effects remains in the second derivative. The correctedsecond derivative of the spectrum is shown in Fig. 8d.
Due to the existence of noise in the spectrum, the secondderivative of a spectrum, through numerical calculation, wouldresult in a noisy spectrum (Fig. 9b). This profile does not provideappropriate information that is required to determine the peakpositions in the second derivative of the spectrum. Estimatingthe second derivative of the spectrum after end point correctionusing CWT in Best-Scale would still exhibit traces of noise in thespectrum that can be suppressed by simply squaring of the esti-mated second derivative, as the intensity of the reminiscent noiseis less than 1 (Fig. 9d).
Reasons for adding helping points based on the degree ofseparation
When distance between peaks becomes less than their width, thepeaks overlap with each other. In the case when this phenome-non occurs, packing of peaks follows in one place. Under theseconditions, estimating background points using second deriva-tive is a challenging process. If two peaks exist in a signal, basedon their position and their Full Width at Half Maximum (FWHM),the degree of separation (R) could be defined as a variable toshow the overlap and peak conditions with respect to each other.If these peaks follow a Gaussian function, FWHM of each peak canbe calculated through:
vative through wavelet transform at Best-Scale=17; (c) Added points basedwavelet transform at Best-Scale=17 with end point correction. This figure
Wiley & Sons, Ltd. J. Raman Spectrosc. 2013, 44, 608–621
New paradigm for signal processing of Raman spectra
The smaller the value of R, the higher is the overlap of signals.This phenomenon is shown in Fig. 10 where the variation of Rwith position in two similar Gaussian peaks is explained. Byincrease in the distance between signals, R is increased. If weconsider the spectrum as not overlapping parts of Gaussianpeaks, the second derivative of the signal would show a positivepeak by increasing peaks distance from each other. When twopeaks are located within their FWHM range, this positive peak(Fig. 10b) reaches its maximum value. By moving from thesepoints, a minima extremum point is generated in the third deriva-tive. If we consider the areas of second derivative of the spectrumbetween zero crossing points, we can observe the location of thisminimum point that lies in a positive area sandwiched betweenthe two negative areas. The location of this minimum point canbe established by considering the zero crossing third derivativeof the spectrum. If the intensity of this point passes half of theintensity of max adjacent point, it could be considered roughlyas a part of the background.
This approximation, results in adding points to the backgroundarrays where second derivative fails to estimate the backgroundin the highly peak packed areas. Thus, instead of following anarbitrary shape, the fitting process progresses through thesepoints. The location of these points could be a little higher than
Figure 9. (a) Synthetic spectrum with SNR=20 with 10 peaks; (b) Numericaltransform and (d) Squared second derivative to suppress noises.
the base of the peaks, but, the iteration algorithm, described insection 2.2.3.6 will correct any overlapping due to added pointsfrom this section to the background arrays.
Background correction
After finding squared second derivative of the synthesized spec-trum, the peaks are removed from the spectrum by applying thealgorithm described in previous sections. The areas betweenstarting and ending points (represented as arrows in Fig. 11a)are related to the background. These areas are selected for fittingand estimating the background of the signal. Following thisprocess, the algorithm then finds the subclass of the startingand ending points. In this case, a subclass value of 0 for startingand 4 for ending points are detected (Fig. 11b and 11c). Thegreen dashed line is the first fitting estimation for background.As it can be clearly seen from Fig. 11d, the background estimatedcurve crosses the spectrum at the end. For correcting this issue,algorithm explained in section 2.2.3.6 is applied. By adding pointsin this area after 314 loops, the final background is determined(represented as magenta colour). This process eliminates creationof artificial peaks due to fluctuation of estimated background in areawhere there are not enough points to fit in ninth-order polynomial.A simple subtraction of this curve from the original spectrum wouldresult in the background corrected spectrum (Fig. 11e).
Testing the accuracy of proposed algorithm
Following 900 iterations testing of the algorithm outlined insection 2.2.4, a comparison between the background-removedspectrum and original signal before adding background (initialsignals) has been carried out. RMSE for the comparison between
second derivative; (c) Second derivative in Best-Scale=17 through wavelet
Figure 10. Overlapping Gaussian peaks and their second and third derivatives: (a) R=0.37; (b) R=1.11; (c) R=2.6; and (d) R=3.34.
A. E. Kandjani et al.
618
background corrected spectra with their initial signals is outlinedin Fig. 12a. Based on a previous study by C. Rowlands et al.,[26]
most of best background corrections have reported an RMSEvalue of more than 0.1. The median of RMSE calculated in the
Figure 11. (a) Spectrum with starting and finishing points of background; (bthe spectrum (Subclass=4); (d) Background points and their fittings, whereinrected background after 314 loops, respectively and (e) Original spectrum witwileyonlinelibrary.com/journal/jrs
current algorithm is about 0.075 which is less than 0.1 indicatingthat this algorithm could provide a good approximation for back-ground correction. The frequency changes of RMSE are charted inFig. 12b, which typically show that more than 94% of the points
) Starting condition of the spectrum (Subclass=0); (c) Ending condition ofgreen dashed and magenta curves relate to starting fitting curve and cor-h the background corrected one. This figure is available in colour online at
Wiley & Sons, Ltd. J. Raman Spectrosc. 2013, 44, 608–621
Figure 12. (a) Root mean squared error (RMSE) variation during testing 900 times random spectra with 10 peaks; and (b) Distribution of RMSE withnumber of tests. This figure is available in colour online at wileyonlinelibrary.com/journal/jrs
New paradigm for signal processing of Raman spectra
have an RMSE lower than 0.2, out of which more than 77%impressively lie below 0.1 RMSE, suggesting that this algorithmwould have less than 6% error in all conditions of signal features.Hence, the algorithm used in the current study could be anexcellent candidate for automation. Testing of the system withvarying peaks number also shows that by increase in thenumber of peaks in a spectrum, the median point of RMSEchanges slowly at the start but changes rapidly after 20 peaks(Fig. 13). This behaviour is a direct consequence of the decreasingnumber of background points in the spectrum.
As shown in section 3.1, the values of Best-Scale exponentiallyincrease with a decrease in SNR while an increase in the scaleresults in widening of the second derivative peaks. Wider peaksconfine the background points that can be selected betweenpeaks. Thus, in lower SNR,we have higher RMSE, as shown in Fig. 14.By an increase in SNR, values of RMSE initially show a drasticdecrease; however, after a while, a slight increase is observed that
Figure 13. Variation of RMSE with peak number in the spectrum. Dot pointsof points with higher and lower RMSE, respectively.
becomes constant at higher values. The explanation for thisbehaviour is related to the exponential nature of the calibrationcurve and inaccuracies introduced by performing CWT on an essen-tially noiseless signal. Rounding the value towards positive infinity(Ceil) in calculating Best-Scale values is inevitable due to integernature of the CWT scales. However, it makes the Best-Scale constantat higher level of SNR. The slight increase after the initial decreasein RMSE could be related to this feature where for all SNR valueslarger than 80, the Best-Scale varies from 6 to 1.
To further check the conditions, where currently proposedalgorithm might fail, we also chose extreme conditions byemploying either a very low SNR value of 5 (Fig. S16), or largenumber of peaks corresponding to 30 (Fig. S17), or a combina-tion of both (Fig. S18), as shown in the supporting informationS5. The slight errors result from the conditions when SNR issignificantly reduced or the number of peaks in a spectrum issignificantly large; the current algorithm might introduce
show median of RMSE, and upper and lower error bars are related to STD
Figure 15. Examples of application of the current algorithm for experimentally obtained real Raman spectra: (a) Serine amino acid; (b) rhodamine;(c) methyl red and (d) crystal violet. This figure is available in colour online at wileyonlinelibrary.com/journal/jrs
Figure 14. Variation of RMSE with SNR in the spectrum. Dot points show median of RMSE, and upper and lower error bars are related to STD of pointswith higher and lower RMSE, respectively.
A. E. Kandjani et al.
620
certain anomalies in the spectra. Therefore, care must be takenwhile dealing with such situations.
Experimental results
Application of the current algorithm for background correc-tion of four different noisy experimental systems (L-serine,rhodamine, methyl red and crystal violet) is outlined in Fig. 15.Analysis of the performance of the algorithm in real data
could not be done due to the inherent inability to obtainthe experimental data without a background to compare withresults. However, the proposed algorithm shows goodperformance in most cases, with the exception of a few minorerrors resulting from the condensation of peaks (Fig. 15b).Interestingly, the end effect errors are considerably less than thecommonly reported studies due to the ability of algorithm reportedin this study to follow background feature in the end points.
Wiley & Sons, Ltd. J. Raman Spectrosc. 2013, 44, 608–621
New paradigm for signal processing of Raman spectra
Conclusion
In conclusion, we have provided a new algorithm based on contin-uous wavelet transformation for baseline correction of Ramanspectra with the ability to work with noisy signals without de-noising. The algorithm is benefitted from CWT thereby enablingto work directly with noisy signal, and SRM for enabling peakremoval from the signal and finding the background shape. TheCWT method eliminates the needs for smoothing the signal andalso gives a good approximation to estimate peaks starting andfinishing points due to its ability to calculate second derivative ofthe noisy spectrum. On the other hand, using SRM, the peaksremain untouched, and background estimation can be achievedusing fitting of the remaining data points in the spectrum.
This algorithm has been tested for accuracy for each section ofprogramming and has acceptable errors that make it applicableto most of the data analysis essential for Raman spectroscopy.The accuracy tests as well as experimental results showed thatthis algorithm could be implemented in the cases whereautomatic baseline detection is necessary. This approach couldaddress the problems of background corrections on real datawhere the quality of spectra is low (e.g. biological low powerRaman spectroscopy). Also, based on the accuracy tests, thisapproach has a minimal variance in the relative peak intensitiesduring analyses.
Acknowledgements
AEK thanks RMIT University for a PhD scholarship. RR thanks theCommonwealth of Australia for an Australian Postgraduate Awardtowards his PhD at RMIT University and thanks RMIT Universityfor a higher degree by research publication grant. VB acknowl-edges the Australian Research Council (ARC) for the award of anAPD Fellowship and research support through the ARC Discovery(DP0988099, DP110105125), Linkage (LP100200859) and LIEF(LE0989615, LE110100097) grant schemes. VB also acknowledgesthe support of the Bill and Melinda Gates Foundation for providingfunding to develop a SERS-based malaria biosensor, and this studyresulted as a part of that project.
Supporting information may be found in the online version ofthis article.
References[1] A. F. Chrimes, K. Khoshmanesh, P. R. Stoddart, A. A. Kayani, A. Mitchell,
H. Daima, V. Bansal, K. Kalantar-zadeh, Anal. Chem. 2012, 84, 4029.[2] B. Plowman, S. J. Ippolito, V. Bansal, Y. M. Sabri, A. P. O’Mullane, S. K.
Bhargava, Chem. Commun. 2009, 5039.[3] M. Šćepanović, M. Grujić-Broj�cin, K. Vojisavljević, S. Bernik, T. Srećković,
J. Raman Spectrosc. 2010, 41, 914.[4] A. Pearson, A. P. O’Mullane, V. Bansal, S. K. Bhargava, Inorg. Chem.
2011, 50, 1705.[5] A. Pearson, A. P. O’Mullane, S. K. Bhargava, V. Bansal, Inorg. Chem.
2012, 51, 8791.[6] H. Fujimori, H. Komatsu, K. Ioku, S. Goto, M. Yoshimura, Phys. Rev. B
2002, 66, 064306.[7] H. Wang, Y. Fu, P. Zickmund, R. Shi, J.-X. Cheng, Biophys. J. 2005, 89, 581.[8] G. Schulze, A. Jirasek, M. M. L. Yu, A. Lim, R. F. B. Turner, M. W. Blades,
Appl. Spectrosc. 2005, 59, 545.[9] M. S. Friedrichs, J. Biomol. NMR 1995, 5, 147.
[10] M. A. Kneen, H. J. Annegarn, Nucl. Instrum. Methods Phys. Res., Sect. B1996, 109–110, 209.
[11] W. Dietrich, C. H. Rüdel, M. Neumann, J. Magn. Reson. 1991, 91, 1.[12] D. Marion, A. Bax, J. Magn. Reson. 1989, 83, 205.[13] B.-F. Liu, Y. Sera, N. Matsubara, K. Otsuka, S. Terabe, Electrophoresis
2003, 24, 3260.[14] M. �Clupek, P. Mat�ejka, K. Volka, J. Raman Spectrosc. 2007, 38, 1174.[15] C. Cai, P. D. B. Harrington, J. Chem. Inf. Comput. Sci. 1998, 38, 1161.[16] J. R. Beattie, J. Raman Spectrosc. 2011, 42, 1419.[17] A. K. Leung, F. T. Chau, J. B. Gao, Anal. Chem. 1998, 70, 5222.[18] X. Shao, C. Ma, Chemom. Intell. Lab. Syst. 2003, 69, 157.[19] L. Nie, S.Wu, X. Lin, L. Zheng, L. Rui, J. Chem. Inf. Comput. Sci. 2002, 42, 274.[20] X.-G. Shao, A. K.-M. Leung, F.-T. Chau, Acc. Chem. Res. 2003, 36, 276.[21] C. Charles, G. Leclerc, J.-J. Pireaux, J.-P. Rasson, Surf. Interface Anal.
2004, 36, 49.[22] Y. Sheng, D. Roberge, H. H. Szu, Opt. Eng. 1992, 31, 1840.[23] F. S. Rojas, C. B. Ojeda, J. M. C. Pavon, Talanta 1988, 35, 753.[24] J. R. Morrey, Anal. Chem. 1968, 40, 905.[25] M. P. Horvath, R. A. Copeland,M.W. Makinen, Biophys. J. 1999, 77, 1694.[26] C. Rowlands, S. Elliott, J. Raman Spectrosc. 2011, 42, 363.[27] A. Campion, P. Kambhampati, Chem. Soc. Rev. 1998, 27, 241.[28] J. M. Gregoire, D. Dale, R. B. V. Dover, Rev. Sci. Instrum. 2011, 82, 015105.[29] S. Wu, L. Nie, J. Wang, X. Lin, L. Zheng, L. Rui, J. Electroanal. Chem.
