A machine learning approach to Cepheid variable star classificationusing data alignment and maximum likelihoodRicardo Vilalta a,∗, Kinjal Dhar Gupta a, Lucas Macri ba Department of Computer Science, University of Houston, 501 Philip G. Hoffman Hall, Houston, TX 77204-3010, USAb Department of Physics and Astronomy, Texas A&M University, 4242 TAMU, College Station, TX 77843-4242501, USA
a r t i c l e i n f o
Article history:Received 26 March 2013Accepted 23 July 2013
Our study centers on the classification of two subtypes of Cepheid variable stars. Such a classificationis relatively easy to obtain for nearby galaxies, but as we incorporate new galaxies, the cost of labelingstars calls for some form of model adaptation. Adapting a predictive model to differentiate Cepheidsacross galaxies is difficult because of the sample bias problem in star distribution (due to the limitationof telescopes in observing faint stars as we try to reach distant galaxies). In addition, estimating theluminosity of a star as we reach distant galaxies carries some inevitable shift in the data distribution.We propose an approach to predict the class of Cepheid stars on a target domain, by first building amodelon an ‘‘anchor’’ source domain. Our methodology then shifts the target data until it is well aligned withthe source data by maximizing two different likelihood functions. Experimental results with two galaxydatasets (Large Magellanic Cloud as the source domain, and M33 as the target domain), show the efficacyof the proposed method.
A common assumption pervadingmost traditionalwork in clas-sification assigns the same joint probability distribution to trainingand testing sets. While such an assumption has found a plethora ofsuccessful real-world applications, recent work in machine learn-ing has revealed an equally rich source of applications when dis-pensing with such assumptions. An interesting example lies inlight curve classification from star samples obtained from differ-ent galaxies. A classification task set to differentiate different typesof stars observed in nearby galaxies, will experience a sample biasas our study encompasses galaxies lying farther away. This is dueto our limitation to observe faint stars; as the distance to a galaxyincreases, our star sample will inevitably concentrate on higher-luminosity stars. This is an instance of the dataset shift prob-lem (Quinonero-Candela et al., 2009; Shimodaira, 2000; Sugiyamaet al., 2009), and precludes the direct utilization of one singlemodel across galaxies; it calls for a form of model adaptation tocompensate for the change in the data distribution.
In this study we show one approach to deal with the datasetshift problem in the context of star classification. The task is toclassify Cepheid variable stars into two subtypes according to theirpulsation mode: fundamental or first-overtone. From an astro-nomical view, a robust discrimination between fundamental and
first-overtoneCepheidswill enable amoreprecise and accurate de-termination of distances to nearby galaxies, which in turn impactsthe uncertainty in the Hubble constant (current expansion rate ofthe Universe). A classification model with fairly high accuracy is infact attainable for nearby galaxies, but the cost of labeling variablestars, together with the large number of galaxy datasets, quicklyturn the goal of acquiring predictive models for many galaxies intoa daunting task. Additionally, switching to a new galaxy dataset in-volves a shift in the data distribution, as the proportion of higher-luminosity Cepheids inevitably rises.
Our study suggests a methodology to attack the dataset shiftproblem in Cepheid star classification by re-using a predictivemodel previously obtained on a source domain. The model canbe invoked again after transforming the testing or target data toaccount for (1) a shift in apparent mean magnitude, and (2) asample bias over luminous stars. The alignment is effected usingmaximum likelihood. Our experiments show results when usingthe Large Magellanic Cloud as the source domain, and M33 as thetarget domain; following our proposedmethodology, accuracy val-ues are similar to those obtained if class labels were available onthe target domain (M33). Our specific contribution lies on a con-crete methodology within machine learning and data mining spe-cialized to star classification, where predictive models are re-usedacross galaxy datasets.
This paper is organized as follows. Section 2 provides basic con-cepts in classification and a brief introduction to Cepheid stars.Section 3 explains our proposed methodology. Section 4 describes
R. Vilalta et al. / Astronomy and Computing 2 (2013) 46–53 47
Fig. 1. A linear relation exists between the logarithm of the period and themagnitude of a Cepheid star. The graph shows both ‘‘fundamental’’ and ‘‘first-overtone’’ Cepheids from the Large Magellanic Cloud that overlap more thanexpected due to interstellar dust and poor image resolution.
related work. Section 5 shows our empirical results. Lastly, Sec-tion 6 gives a summary and conclusions.
2. Preliminary concepts
2.1. Cepheid variables
Classical Cepheid variables (also commonly referred to asPopulation I Cepheids) are stars both massive 4–11 MSun (solarmasses) and luminous 1 × 103–5 × 104 LSun (solar luminosities),that undergo regular pulsationswith periods ranging from2 to 100days. The pulsations are driven by cyclical changes in the opacity ofhydrogen and helium in the outer atmosphere of these stars (Cox,1980). A tight correlation exists between period (P) and luminosity(L) for Cepheid variables:
log10 L = a + b ∗ log10 P. (1)
where a and b are fitting parameters. Fig. 1 shows an exampleof such relation for a sample of Cepheids populating the LargeMagellanic Cloud galaxy. The flux of each star is represented by itsapparent magnitude m, defined as m = −2.5 × log10
Ld2, where d
is the distance from Earth to the star measured in parsecs. Hence,smaller numbers correspond to brighter magnitudes (higherfluxes).
Our confidence on the modern version of the ‘‘Cepheid P–L re-lation’’ is supported by the small intrinsic data dispersion (only 4%of the mean luminosity at fixed period for near- and mid-infraredwavelengths Persson et al., 2004). Briefly, the physical reason be-hind the P–L relation can be understood as follows. Stars with dif-ferent masses have different central temperatures and densities,which greatly influence the rate of energy generation via fusion.Stars with different masses will achieve different equilibrium con-ditions (surface luminosity, ‘‘L’’ in Eq. (1), as well as temperatureand radius) at various stages of their evolution. For stars in a nar-row range of surface temperature, the internal conditions are pro-pitious for regular, stable pulsations. The period of these pulsations(‘‘P ’’ in Eq. (1)) is inversely related to the density of the star. Moremassive (i.e. more luminous) stars have lower average densities,leading to longer pulsation periods.
Cepheid variables can be classified according to their pulsationmode(s).1 Our study centers on the two most abundant classes,
1 Information about data sources and plotted data can be found in Section 5.
which pulsate in the fundamental and first-overtone modes(Fig. 1). Their Period–Luminosity relations are shifted in luminosityby 70% and thus should be easily separable. In practice, individualCepheids within a galaxy will encounter varying amounts ofinterstellar dust, which attenuates their brightness to the pointthat the first-overtone and fundamental P–L relations are blurredinto a single relation. Additionally, the superposition of multiplestars within a single resolution element of an image could biasthe measured flux of a fundamental-mode Cepheid, artificiallyincreasing its brightness to the point where it lies within thefirst-overtone relation. If one is unable to discriminate betweenfundamental and first-overtone Cepheids, a hard period cut mustbe imposed to ensure no overtones are present in the sample (thecut would be made at P = 8 days, or 0.9 in the logarithm ofthe period in Fig. 1). Otherwise, first-overtones will contaminatethe sample and bias the distance estimate, because first-overtonepulsators are more luminous than fundamental-mode pulsators atfixed periods. Overall this is not a satisfactory solution becausesuch a period cut will eliminate >50% of the fundamentalmode sample and greatly reduce the precision of the distancedetermination.
The observational limitations described abovewill blur the fun-damental and first-overtone relations of short-period (P < 8 days)Cepheids into a single relation. Since overtone Cepheids are nat-urally more luminous at fixed periods, their inclusion in a sam-ple of fundamental mode Cepheids will bias the slope of the P–Lrelation towards shallower (i.e., smaller) values. The astronomicalcommunity needs an efficient way to classify the pulsation modeof a Cepheid into fundamental and first-overtone to better under-stand the intrinsic slope of the P–L relation (‘‘b’’ in Eq. (1)). Specif-ically, a robust discrimination between fundamental-mode andfirst-overtone pulsators would enable a comparison of the slopeof the period–luminosity relation of each type, which can be com-pared with the predictions of stellar evolution and pulsation mod-els. These models compute changes in the internal structure andchemical compositions of stars from their initial conditions (in theso-called ‘‘main sequence’’) until the end of the Cepheid phase andbeyond. Incorrect assumptions in the initial conditions, or flaws inthe modeling, become readily apparent when comparing the pre-dicted and observed properties of the Cepheid period–luminosityrelation. For such a study, covering as much of the entire range ofCepheid periods (4 days . P . 100 days) as possible is criticalto reliably determine the value of the slope. Nearby galaxies (suchas the Large Magellanic Cloud, Messier 33, Messier 81 and Messier101) are good laboratories for such a study because their Cepheidsare bright enough to be studied in detail, even at the shortest peri-ods (lowest luminosities).
2.2. Basic concepts in classification and dataset shift
We now introduce basic notation in classification. We assumea classifier receives as input a set of training examples Ttr = {(xk,wk)}
pk=1, where x is a vector in the input space X, and w is a value
in the (discrete) output spaceW . We assume the training or sourcesample Ttr consists of independently and identically distributed(i.i.d.) examples obtained according to a fixed but unknown jointprobability distribution, Ptr(x, w), in the input–output space X ×
W . The outcome of the classifier is a hypothesis or function f (x|θ)(parameterized by θ ) mapping the input space to the output space,f : X → W . In our study, Ttr is a dataset corresponding to a sourcegalaxy (e.g., LargeMagellanic Cloud),where vector x contains light-curve properties; we focus exclusively on period (x1) and apparentmean magnitude (x2), such that x = (x1, x2). Variable w can takeon any of the two classes we wish to predict (fundamental or first-overtone).
48 R. Vilalta et al. / Astronomy and Computing 2 (2013) 46–53
Fig. 2. Left. The distribution of Cepheids along the Large Magellanic Cloud LMC (top sample), deviates significantly from M33 (bottom sample). Right. M33 is aligned withLMC by shifting along mean magnitude.
We will also assume a testing or target sample with no classlabels, corresponding to a different galaxy Tte = {xk}qk=1, thatconsists of i.i.d. examples obtained from the marginal distribu-tion Pte(x) according to a different joint distribution, Pte(x, w), overX × W . In the simplest scenario, we assume that the only differ-ence between the training and test distributions is in the marginaldistributions Ptr(x) = Pte(x), while the class posteriors (i.e., con-ditional distributions) remain identical Ptr(w|x) = Pte(w|x). Thisis known as the covariate shift problem. The output will still be afunction f : X → W , but with the goal of minimizing the general-ization error over the ‘‘test’’ distribution (as estimated from Tte).
3. Addressing the data alignment and sample bias problems
Although an accurate classification of Cepheids into fundamen-tal and first-overtone can be attained for nearby galaxies (e.g.,Large Magellanic Cloud), the high cost of manually labeling vari-able stars suggests a mode of operation where a predictive modelobtained on a data set from a source galaxy Ttr, is later used on atest set from a target galaxy Tte. Such a scenario is not immediatelyattainable because of two main problems: (1) a shift in the datadistribution along apparent magnitude m as we reach galaxies ly-ing farther away, and (2) a significant degree of sample bias wherePtr(x) = Pte(x), that increases the probability density over higher-luminosity stars. An example of the effect of these two problemsis shown in Fig. 2 (left), where the distribution of Cepheids in theLarge Magellanic Cloud (top sample), deviates significantly fromthat of M33 (bottom sample). Both the offset along apparent mag-nitude and the significant degree of sample bias are mostly due tothe fact that M33 is ∼16× farther than the LMC. At these greaterdistances, the shorter-period (i.e., less luminous) Cepheids fallbelow the detection threshold and longer-period (i.e., more lumi-nous) stars are preferentially detected. A less likely, but still pos-sible scenario could occur when deep observations of a farawaygalaxy are compared to shallow observations of a nearby galaxy. Inthis case, a sample bias would still take place, but it would occur inthe Cepheids of the closer galaxy.
Our general solution is to shift the data on Tte to the pointwherethe model obtained over Ttr is directly applicable. We have namedthis the data-alignment problem in star classification. This differsfrompreviousmethodologieswhere the goal is to generate amodelafresh by re-weighting examples on the training set (Quinonero-Candela et al., 2009); instead we leave the model intact, andtransform the testing set until the model becomes applicable. Anexample of a successful alignment is shown in Fig. 2 (right), whereshifting the M33 sample along m by the right amount, enables usto do classification with the same model obtained on Ttr.
3.1. Problem formulation
We formalize our problem as follows. We assume a training setTtr and a predictive model f (x|θ) obtained by training on datasetTtr. We assume also an unlabeled testing set Tte = {x}, where eachfeature vector x = (x1, x2), is made of two features2 correspondingto period (x1) and apparent mean magnitude (x2). We assumefeature x2 has suffered a shift (because the galaxy correspondingto Tte lies farther away than the galaxy corresponding to Ttr). Wewish to generate a new dataset T ′
te = {x′}, where x′
= (x1, x2 + δ),to achieve the goal of aligning Tte with Ttr. Our real focus is onthe value of δ; we aim at an optimal value that best achieves suchalignment. Note that even if the alignment is done appropriately,the joint distributions between training and testing could differ,because Ptr(x) = Pte(x) due to a form of sample bias. In our case,the sample bias favors highly luminous Cepheids.
3.2. Step 1: maximum likelihood based on shift
Our methodology to attack the problem above consists of firstshifting the data Tte using maximum likelihood to generate a firstestimate of δ at low computational cost. A second stepwill providea more refined estimate by considering the expected proportion ofclass labels in Tte as predicted by the model f (x|θ) (Section 3.3),albeit at a higher computational cost.Density estimation. We start by modeling the training set Ttr asa mixture of two Gaussians, based on the bimodal distributionobserved along the two existing classes: fundamental and first-overtone (Fig. 1), and assume two features (x1, x2), correspondingto period and apparent mean magnitude respectively3:
Ptr(x1, x2) =
2i=1
φi1
2πσi1σi2
1 − ρ2
i
e−vi
2(1−ρ2i ) (2)
where {φi} are the mixing components (0 ≤ φi ≤ 1, φ1 + φ2 = 1),µij and σij are the mean and standard deviation of the ith compo-nent along the jth feature, ρi is the correlation between x1 and x2
2 Besides period and magnitude, many other features have been proposed in theliterature to characterize light curves (Richards et al., 2011; Debosscher et al., 2007).3 Our limited feature space obviates a vectorial representation for the joint
density (mixture of two Gaussians).
R. Vilalta et al. / Astronomy and Computing 2 (2013) 46–53 49
in the ith component, and
vi =(x1 − µi1)
2
σ 2i1
−2ρi(x1 − µi1)(x2 − µi2)
σi1σi2+
(x2 − µi2)2
σ 2i2
. (3)
We estimate parameters {µi} and {φi} and {σij} directly fromour sample Ttr, since we know all class labels (i.e., we know whichpoints belong to each component or Gaussian distribution). Thisenables us to have a complete characterization of the training setdistribution Ptr(x).Alignment. Our next step is to define a new testing set T ′
te = {x′},
where x′= (x1, x2 + δ), since we know a shift has occurred along
the apparent mean magnitude. In this case we redefine vi as zi:
zi =(x1 − µi1)
2
σ 2i1
−2ρi(x1 − µi1)(x2 + δ − µi2)
σi1σi2
+(x2 + δ − µ12)
2
σ 2i2
. (4)
Our approach is to find the δ that maximizes the log likelihoodof T ′
te with respect to distribution Ptr(x):
L(δ|T ′
te) = logΠqk=1Ptr(x
k) =
qk=1
log Ptr(xk1, xk2). (5)
To proceed, we express our joint distribution alternatively asfollows:
Ptr(xk1, xk2) =
2i=1
φiαie−βizki ,
where αi =1
2πσi1σi2
1 − ρ2
i
and βi =1
2(1 − ρ2i )
. (6)
After some algebraic manipulation, it can be shown that:
e−βizki = ecki1+δcki2+δ2cki3 (7)
where
cki1 = −βi
(xk1 − µi1)
2
σ 2i1
−2ρi(xk1 − µi1)(xk2 − µi2)
σi1σi2
+(xk2 − µi2)
2
σ 2i2
cki2 = −βi
2(xk2 − µi2)
σ 2i2
−2ρi(xk1 − µi1)
σi1σi2
and cki3 =
−βi
σ 2i2
.
To find the δ that maximizes L(δ|T ′te), we use a derivative
approach (see Appendix for details):
ddδ
q
k=1
log Ptr(xk1, xk2)
=
qk=1
γ k11e
θk1 (δ)+ γ k
21eθk2 (δ)
Γ k1 e
θk1 (δ)+ Γ k
2 eθk2 (δ)
+ δγ k12e
θk1 (δ)+ γ k
22eθk2 (δ)
Γ k1 e
θk1 (δ)+ Γ k
2 eθk2 (δ)
= 0 (8)
where
θ ki (δ) = δcki2 + δ2cki3, γi1 = φiαicki2e
cki1 ,
γ ki2 = 2φiαicki3e
cki1 , and Γ ki = φiαiec
ki1 .
The algebraic manipulations above convert an optimizationproblem into a simple equation-solving problem. Specifically,solving for δ (Eq. (8)) enables us to know the exact shift necessaryto generate the new testing set T ′
te.
3.3. Step 2: maximum likelihood based on the induced classdistribution
The previous section explains how to obtain an initial approx-imation to the alignment problem using maximum likelihood af-ter estimating the probability density of Ptr(x). We now describe amore refined search that takes into account class distributions andthe predictions of model f (x|θ). The rationale is as follows.While atesting set Tte lacks any class label information, it is possible to de-termine its likelihood according to the class distribution inducedby model f (x|θ). Our aim is to find the data shift δ that maximizesthe likelihood of the induced class distributions. We explain thismethodology next.Class prior modeling. Our second search focuses on the class dis-tribution of the predictions made by f (x|θ), but not in the origi-nal class priors, since we assume an unlabeled testing set Tte. Wemodel this distribution by generating multiple bootstrap samplesfrom Ttr, and invoking f (x|θ) on each sample to predict the classof each observation. The result is an estimation of the distributionon the proportion of examples assigned to each class as dictatedby f (x|θ). Since we deal with a binary classification problem, wecan concentrate on the distribution for one class only (e.g., funda-mental), as clearly the two distributions have the same variance.The distribution is characterized by a mean µ and variance σ 2 onthe proportion of examples classified as class w. We assume a nor-mal distribution Qtr(y) ∼ N (µ, σ 2), where y is the proportion oftraining examples predicted as classw by f (x|θ). Fig. 3 shows den-sity functions on the proportion of examples assigned to each classusing LMC data and a normal distribution.Alignment. We then proceed to find the shift δ that maximizesthe log likelihood of T ′
te (starting with the value of δ found inthe previous section). We use an iterative approach, where theprevious value of δ is updated using fixed increments δ = δold ± τ .At iteration k, the log likelihood of T ′
te is computed as follows:
L(δ|T ′
te) = logQtr(yk) (9)
where yk is the proportion of testing examples predicted as classwby f (x|θ) at step k. The need to run model f (x|θ) to compute thelog likelihood precludes finding an analytical solution for δ. Insteadwe resort to a greedy-best search technique where we iterate onincrements τ until we reach a maximum number of iterations, orthe difference in L(δ|T ′
te) between the current and previous stepsis ≤ 0. The sign of the increments is decided on the first iteration,after which they always remain either positive or negative.
3.4. The sample bias problem
As mentioned before, even if the alignment is done appropri-ately, the joint distributions between training and testing coulddiffer due to a form of sample bias. As shown in Fig. 2 (right), M33appears aligned with LMC, but the concentration of stars in M33lies at higher periods; this is considered a case of sample bias. Sam-ple bias is one reason for the presence of covariate shift, which ischaracterized by a difference in the marginal distributions of thetraining and testing sets (i.e., occurs whenever Ptr(x) = Pte(x)). Inour case, the reason for such bias is due to our limitation to observefaint stars as we try to reach distant galaxies.
To address the sample bias problem, we narrow the training setTtr (Large Magellanic Cloud) to those examples where the periodP is within the range of values found in Tte, (M33). Fig. 4 showsthe range of values under consideration; the idea is to focus theanalysis to regions in the feature space with highest density basedon Tte. The strategy followed here looks at the range of values alongperiod P covered by the test set Tte. Specifically, we model thesource domain as a mixture of Gaussians only along the range of
50 R. Vilalta et al. / Astronomy and Computing 2 (2013) 46–53
Fig. 3. An estimation of the distribution on the proportion of examples assigned toeach class by f (x|θ) on LMC data.We assume a normal distribution for both classes.
Fig. 4. We address the sample bias problem by focusing on those examples on LMCthat fall within the same range of values along period for M33 (right side of thevertical line).
values embeddedby the target domain. Steps 1 and2 as specified inSections 3.2 and 3.3 are then applied directly, without any furthermodification. The advantage gained with this process is to narrowour attention to local regions over the input–output space wherewe wish to specialize our predictive model.
4. Related work
The covariate shift problem has been studied extensively in thepast years. A common solution is to generate a new model afreshwith the source training set Ttr ∼ Ptr(x), through a re-weightingscheme that gives more importance to regions of high probabilitydensity relative to Pte(x) as estimated from the testing set Tte(Shimodaira, 2000; Kanamori and Shimodaira, 2009; Sugiyamaet al., 2009; Bickel et al., 2009). Different from previous work, ourproblemdoes not initially fall into the covariate shift paradigmdueto the shift in apparent mean magnitude between Ttr and Tte; thiscauses Ptr(w|x) = Pte(w|x). We attack this problem using a data-alignment solution based on maximum likelihood, after which weare able to exploit the originalmodel f (x|θ) obtained from Ttr, sincethe goal of the alignment step is to make Ptr(w|x) = Pte(w|x).
The re-weighting scheme mentioned above follows two mainassumptions: (1) we can generate accurate estimations of Ptr(x)
and Pte(x) (Sugiyama and Muller, 2005; Lin et al., 2002; Zadrozny,2004), and (2) the support of Pte is contained in the support ofPtr (i.e., the testing set is encompassed or covered by the trainingset). Regarding the first assumption, nonparametric approacheshave been proposed too, where the weights Pte(x)
Ptr(x)can be obtained
indirectly, without explicit estimation of the training and testingset densities (Gretton et al., 2009). The second assumption placesa severe restriction on the applicability of the weighting scheme;when it does not hold, different techniques have been proposedto deal with concepts that change over time, most notably underthe umbrella of concept drift (Bach and Maloof, 2010), or conceptevolution (Masud et al., 2010). Our work avoids the difficultylinked to a change in the class posterior probabilities through thealignment step, by reinstating the original model built on Ttr.
5. Experiments
Our analysis uses data obtained by two surveys: (1) thethird phase of the Optical Gravitational Lensing Experiment(OGLE-III)4 and (2) the M33 Synoptic Stellar Survey.5 OGLE(Udalski, 2009) is a collaboration of mostly Polish astronomersoriginally designed to search for dark matter using gravitationalmicrolensing. The project has been observing Magellanic Cloudsand the Galactic Bulge using a dedicated 1.3-m telescope at LasCampanas Observatory in Chile. As a side product of the searchfor microlensing events, OGLE has discovered tens of thousandsof variable stars, including Cepheids. We analyze the third catalogof OGLE Large Magellanic Cloud Cepheids (Soszynski et al., 2008),on both the visible (V ) and infrared (I) bands; the training set forthe visible band contains 3059 points (i.e., light curves), while thetraining set for the infrared band contains 3056 points. Data isof excellent quality, with statistical uncertainties of 0.001% and0.00035% on each band respectively. The approximate detectionthresholds of these observations are V = 21.5 and I = 21mag, well below the faintest apparent magnitude of any Cepheidin the Large Magellanic Cloud. The M33 Synoptic Stellar Survey(Pellerin and Macri, 2011) is a follow-up study of Cepheids andother variables initially discovered by the DIRECT project (Staneket al., 1998; Macri et al., 2001). These testing sets contain 323points in the visible band, and 322 points in the infrared band(statistical uncertainties of 0.0125% and 0.0056% respectively). Theapproximate detection thresholds of these observations are V =
23 and I = 21.5mag,which are adequate to detect all fundamentalmode Cepheids with P > 3 days and first-overtone Cepheids withP > 2 days.
We refer to data from the Large Magellanic Cloud galaxy asthe source domain, and to data from the M33 galaxy as the targetdomain. As a pre-processing step, parameters for the mixture ofGaussians on the source domain are estimated directly throughthe data. Since class labels are available, it is straightforwardto estimate means, covariance matrices and mixing proportions.When narrowing the training set Ttr (source domain) to thoseexamples where the period is within the range of values found inthe target domain (Section 3.4), the size of the set decreases to 2313points. Implementation of the learning algorithms can be foundin WEKA (Hall et al., 2009) using default parameters. Specifically,neural networks are invoked with one hidden layer, two internalnodes, a learning rate of 0.3, momentum of 0.2, and 500 epochs.Support Vector Machines are invoked with polynomial kernels ofdegrees 1, 2, and 3 respectively. Decision trees are invoked witha confidence factor of 0.25. Random Forests combine 10 decision
4 Data repository available at http://ogledb.astrouw.edu.pl/~ogle/CVS/5 Data repository available at http://faculty.physics.tamu.edu/lmacri/M33SSS/
R. Vilalta et al. / Astronomy and Computing 2 (2013) 46–53 51
Table 1Classification performance under visible band. Numbers in parentheses represent standard deviations. The last two columns correspond to the proposed methodology.
Learning algorithm Accuracy on LMC with class labels Accuracy on M33 (target domain)With class labels With no class labels using data alignment
Table 2Classification performance under infrared band. Numbers in parentheses represent standard deviations. The last two columns correspond to the proposed methodology.
Learning algorithm Accuracy on LMC with class labels Accuracy on M33 (target domain)With class labels With no class labels using data alignment
trees on each run. Regarding the optimization of Eq. (8), we solvefor δ by invoking the ‘‘solve function’’ in the ‘‘symbolic mathtoolbox’’ in Matlab.
Our first set of experiments compute predictive accuracy whenthe model obtained on LMC is applied to the transformed targetdomain (when a shift δ is applied to apparent magnitude). Ac-curacy is obtained by creating bootstrap samples of the target ortest set (M33); each sample is generated by sampling with re-placement. We generated 10 samples, and averaged our results af-ter applying model f (x|θ) on each transformed sample. Table 1shows our results when the data refers to the visible band. Thefirst column corresponds to the learning algorithms. The secondcolumn corresponds to accuracy on the source domain (as refer-ence only). The third column shows accuracy on the target domainwhen class labels are available. Thenext columns showaccuracy onM33when class labels are unavailable. The fourth column (labeled‘‘Standard’’) shows results using the values of δ listed in Table 2 ofPellerin and Macri (2011). We take into consideration that the ap-parentmagnitudes of the OGLE LMC Cepheids are not corrected forthe average amount of interstellar dust towards this galaxy, while(Pellerin and Macri, 2011) applies a correction for dust.6 The fifthcolumn (labeled ‘‘Step 1’’) shows results when δ is determined us-ingmaximum likelihood based on density estimation (Section 3.2),while the sixth column (labeled ‘‘Step 2’’) shows results when weexploit the induced class distribution (Section 3.3, τ = 0.01).
Compared to previous work (Pellerin and Macri, 2011), we ob-serve a significant increase in accuracy when δ is computed usingeither step 1 or 2; this is true for all learning algorithms under con-sideration, which validates our proposed methodology. Further-more, results for both step 1 and 2 show accuracies similar to thoseobtained onM33when class labels are present, evidencing how anappropriate shift in magnitude renders the model trained on thesource domain valid under the target domain. The difference be-tween steps 1 and 2 is mixed; in two cases step 2 outperforms step1,while in two cases the opposite is true, and in two other cases thedifference is not significant (p = 0.05 level using a t-student distri-bution). Further investigation, however, shows some clear benefitswith step 2: the range of values along δ for which a maximum in
6 For reference, the adjustment values derived in Pellerin and Macri (2011) areδ = 6.45 ± 0.02 for the visible band, and 6.31 ± 0.02 for the infrared band.
the likelihood function is obtained is consistently narrower for step2, which helps us pinpoint more precisely our final estimation forδ (δvisible-final = −6.24).
Table 2 displays results similar to the ones describe above, butusing the infrared band. Compared to previous work, we observe ageneral improvement in accuracy when δ is computed using max-imum likelihood, but the improvement is less clear. The differencebetween steps 1 and 2 is also similar. Our final estimation for δ(δinfrared-final = −6.08) is different from the one obtained usingthe visible band, because the effects of reddening due to interstel-lar dust are significantly reduced at infrared waves (Madore andFreedman, 1991). Overall, empirical results support our proposedmethodology. In addition,weprovide an alternativemeans to com-pute the correction shift δ under a new approach based on maxi-mum likelihood and sample bias adjustment. Our values are lowerthan those derived by Pellerin and Macri (2011), but statisticallyconsistent, given the scatter in their Period–Luminosity relations(0.4 mag in V and 0.27 mag in I).
6. Summary and conclusions
Our study shows an approach to deal with the automatedprediction of Cepheid stars obtained from a target galaxy, after apredictive model has already been built on a source galaxy. Ourmethodology is based on manipulating the target data by shiftingalong magnitude to correct for uncertainty in luminosity andsample bias. Our first form of alignment uses maximum likelihoodbased on density estimation on the source domain. The secondform of alignment refines the previous one by exploiting the classdistribution over the source domain as predicted by the learningmodel. Our experiments show accuracy values similar to thoseobtained if class labels were available on the target galaxy.
Our approach is not limited to the star classification problem.The alignment problem can be found on other domains wherethere is a shift in data distribution due to a systematic changeduring data collection, caused mainly by a change in the physicalconditions surrounding an experiment. In particle physics, for ex-ample, a model built to identify a particle may by re-used on a newsample obtained through amore sophisticated accelerator, by sim-ply shifting the data along the set of parameterswhere reaching outto higher energies brings with it a displacement in the data distri-bution. Our study suggests, however, that the success of dataset
52 R. Vilalta et al. / Astronomy and Computing 2 (2013) 46–53
shift strategies depends on the global properties of the model builtover the source distribution. Specifically, if bothmarginal distribu-tions differ, Ptr(x) = Pte(x), but the class posteriors remain iden-tical, Ptr(w|x) = Pte(w|x), then in general the bias produced bytraining on the source domain is counterbalanced by a distributionexhibiting global patterns (i.e., exhibiting no local irregularities).In our case, this corresponds to the P–L linear relationship (Sec-tion 2.1), which is invariant across galaxies. When that is the case,model f (x|θ) can be re-used on different regions of the input space.
As future work, we plan to extend our ideas to the context oftransfer learning (Pan and Yang, 2010). Dataset shift resemblestransfer learning, but while the former deals with changes on thesame task, the latter comprises a broader setting where targettasks can differ substantially from the source task (Storkey, 2009).We plan to explore when a model can be re-used, based on thepresence of invariant global data properties across tasks.
Acknowledgments
We are grateful to the anonymous reviewers for their valuablesuggestions. Lucas Macri acknowledges support for this project bythe following sources: NASA, through a Hubble Fellowship grantHST-HF-01153 from the Space Telescope Science Institute; theNational Science Foundation, through a Goldberg Fellowship fromthe National Optical Astronomy Observatory and through grantnumber 1211603; and by Texas A&M University, through a facultystart-up fund.
Appendix
We provide a detailed derivation of the maximum likelihoodapproach presented in Section 3.2. The joint distribution of the datais expressed as follows:
Ptr(xk1, xk2) =
2i=1
φiαie−βizki ,
where αi =1
2πσi1σi2
1 − ρ2
i
and βi =1
2(1 − ρ2i )
where
e−βizki = ecki1+δcki2+δ2cki3
cki1 = −βi
(xk1 − µi1)
2
σ 2i1
−2ρi(xk1 − µi1)(xk2 − µi2)
σi1σi2
+(xk2 − µi2)
2
σ 2i2
cki2 = −βi
2(xk2 − µi2)
σ 2i2
−2ρi(xk1 − µi1)
σi1σi2
and cki3 =
−βi
σ 2i2
.
The derivative of the log likelihood function can be written as:
ddδ
q
k=1
log Ptr(xk1, xk2)
=
ddδ
q
k=1
log2
i=1
φiαie−βizki
=ddδ
q
k=1
log(φ1α1e−β1zk1 + φ2α2e−β2zk2 )
=
qk=1
ddδ
log(φ1α1e−β1zk1 + φ2α2e−β2zk2 )
=
qk=1
1
φ1α1e−β1zk1 + φ2α2e−β2zk2
×
ddδ
(φ1α1e−β1zk1 + φ2α2e−β2zk2 )
.
Now,
ddδ
φ1α1e−β1zk1 + φ2α2e−β2zk2
=
ddδ
φ1α1(ec
k11+δck12+δ2ck13) + φ2α2(ec
k21+δck22+δ2ck23)
= (ck12 + 2δck13)φ1α1(ec
k11+δck12+δ2ck13)
+ (ck22 + 2δck23)φ2α2(eck21+δck22+δ2ck23)
= φ1α1ck12eck11(eδck12+δ2ck13) + 2δck13φ1α1ec
k11(eδck12+δ2ck13)
+ φ2α2ck22eck21(eδck22+δ2ck23) + 2δck23φ2α2ec
k21(eδck22+δ2ck23)
R. Vilalta et al. / Astronomy and Computing 2 (2013) 46–53 53
Bach, S.H., Maloof, M.A., 2010. A Bayesian approach to concept drift. In: Proceedingsof the International Conference on Neural Information Processing Systems, pp.127–135.
Bickel, S., Bruckner, M., Scheffer, T., 2009. Discriminative learning under covariateshift. Journal of Machine Learning Research 10, 2137–2155.
Cox, J.P., 1980. Theory of Stellar Pulsation. Princeton University Press.Debosscher, J., Sarro, L.M., Aerts, C., Cuypers, J., Vandenbussche, B., Garrido,
R., Solano, E., 2007. Automated supervised classification of variable stars.Astronomy & Astrophysics 475, 1159–1183.
Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., Scholkopf, B.,2009. Covariate shift by kernel mean matching. In: Quinonero-Candela, J.,Sugiyama, M., Schwaighofer, A., Lawrence, N.D. (Eds.), Dataset Shift in MachineLearning. MIT Press, pp. 131–160.
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H., 2009. TheWEKA data mining software: an update. SIGKDD Explorations 11 (1).
Kanamori, T., Shimodaira, H., 2009. Geometry of covariate shift with applicationsto active learning. In: Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A.,Lawrence, N.D. (Eds.), Dataset Shift inMachine Learning. MIT Press, pp. 87–105.
Lin, Y., Lee, Y., Wahba, G., 2002. Support vector machines for classification innonstandard situations. Machine Learning 46, 191–202.
Macri, L.M., Stanek, K.Z., Sasselov, D.D., Krockenberger, M., Kaluzny, J., 2001. Directdistances to nearby galaxies using detached eclipsing binaries and cepheids. VI.variables in the central part of M33. The Astronomical Journal 121, 870–890.
Madore, B., Freedman, W.L., 1991. The cepheid distance scale. Publications of theAstronomical Society of the Pacific 103, 933–957.
Masud, M., Chen, Q., Khan, L., Aggarwal, C., Gao, J., Han, J., Thuraisingham, B., 2010.Addressing concept-evolution in concept-drifting data streams. In: Proceedingsof the IEEE International Conference on Data Mining, pp. 929–934.
Pan, S.J., Yang, Q., 2010. A survey on transfer learning. The IEEE Transactions onKnowledge and Data Engineering 22 (10), 1345–1359.
Pellerin, A., Macri, L.M., 2011. The M33 synoptic stellar survey. I. Cepheid variables.Astrophysical Journal Supplement Series 193, 26.
Persson, S.E., Madore, B.F., Krzemiński, W., Freedman, W.L., Roth, M., Murphy, D.C.,2004. New cepheid period–luminosity relations for the large Magellanic cloud:92 near-infrared light curves. The Astronomical Journal 128 (5), 2239–2264.
Richards, J.W., Starr, D.L., Butler, N.R., Bloom, J.S., Brewer, J.M., Crellin-Quick, A.,Higgins, J., Kennedy, R., Rischard,M., Shimodaira, H., 2011. Onmachine-learnedclassification of variable stars with sparse and noisy time-series data. TheAstrophysical Journal 733, 10.
Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D., 2009.Dataset Shift in Machine Learning. MIT Press.
Shimodaira, H., 2000. Improving predictive inference under covariate shift byweighting the log-likelihood function. Journal of Statistical Planning andInference 90, 227–244.
Soszynski, I., Poleski, R., Udalski, A., Szymanski, M.K., Kubiak, M., Pietrzynski, G.,Wyrzykowski, L., Szewczyk, O., Ulaczyk, K., 2008. The optical gravitationallensing experiment. The OGLE-III catalog of variable stars. I. classical cepheidsin the large Magellanic cloud. Acta Astronomica 58, 163–185.
Stanek, K.Z., Kaluzny, J., Krockenberger, M., Sasselov, D.D., Tonry, J.L., Mateo, M.,1998. Direct distances to nearby galaxies using detached eclipsing binariesand cepheids. II. variables in the field M31A. The Astronomical Journal 115,1894–1915.
Storkey, A., 2009. When training and test sets are different. In: Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D. (Eds.), Dataset Shiftin Machine Learning. MIT Press, pp. 3–28.
Sugiyama, M., Muller, K.R., 2005. Model selection under covariate shift. In:Proceedings of the International Conference on Artificial Neural Networks, pp.235–240.
Sugiyama, M., Rubens, N., Muller, K.R., 2009. A conditional expectation approachto model selection and active learning under covariate shift. In: Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D. (Eds.), Dataset Shiftin Machine Learning. MIT Press, pp. 107–130.
Udalski, A., 2009. The optical gravitational lensing experiment (OGLE): Bohdan’sand our great adventure. In: Stanek, K.Z. (Ed.), The Variable Universe: ACelebration of Bohdan Paczynski. In: Astronomical Society of the PacificConference Series, vol. 403. p. 110.
Zadrozny, B., 2004. Learning and evaluating classifiers under sample selectionbias. In: Proceedings of the International Conference on Machine Learning, pp.114–122.
