The Cleft Palate-Craniofacial JournaW)(5) pp. 609-617 September 2012 Copyright 2012 American Cleft Palate-Craniofacial Association

ORIGINAL ARTICLE

Validation of the Dutch Registry of Common Oral Clefts: Quality ofRecording Specific Oral Cleft Features

Anna M. Rozendaal, M.D., Antonius J.M. Luijsterburg, M.D., Ashna D. Mohangoo, M.Sc, M.P.H., Ph.D.,Edwin M. Ongkosuwito, D.D.S., Esther de Vries, M.Sc, Ph.D., Christi Vermeij-Keers, M.D., Ph.D.

Objective: Since 1997, common oral clefts in the Netherlands have been recorded in thenational oral cleft registry using a unique descriptive recording system. This study validatesdata on the topographic-anatomical structure, morphology, and side of individual anomalies ofthe primary palate and secondary palate that form the oral cleft.

Design: Validation study.Setting: All 15 Dutch cleft palate teams reporting presurgery oral cleft patients to the national

registry.Patients: A random sample of 250 cases registered in the national database with oral clefts

from 1997 through 2003; of these, 13 cases were excluded.Main Outcome Measures: By linking registry data with clinical data, we identified differential

recording rates by comparing the prevalence, and we measured the degree of agreement bycomputing validity and reliability statistics.

Results: The topographic-anatomical structures (lip, alveolus, and hard and soft palates) ofthe anomalies had near-perfect interdatabase agreement with a sensitivity of 88% to 99%.However, when analyzing the individual anomalies in detail (morphology and side), validitydecreased and depended on morphological severity. This association was most evident foranomalies of the secondary palate. For example, sensitivity was higher for "complete cleft hardpalate" (92%) than for "submucous cleft hard/soft palate" (69%).

Conclusions: Overall, the validity of Dutch registry data on oral clefts is good, supporting thefeasibility of this unique recording system. However, when analyzing oral cleft data in detail, thequality appears to be related to anatomical location and morphological severity. This might haveimplications for etiologic research based on registry data and for guidelines on neonatalexamination.

KEY WORDS: cleft lip, cleft palate, registry, validation

Dr. Rozendaal is Ph.D. student, Research Unit Department of Plasticand Reconstructive Surgery, Erasmus Medical Center, University MedicalCenter Rotterdam, and Department of Orthodontics, Erasmus MedicalCenter-Sophia Children's Hospital, University Medical Center Rotter-dam, Rotterdam, the Netherlands. Dr. Luijsterburg is Plastic andReconstructive Surgeon and Ph.D. student. Research Unit Departmentof Plastic and Reconstructive Surgery, Erasmus Medical Center,University Medical Center Rotterdam, Rotterdam, the Netherlands. Dr.Mohangoo is Epidemiologist, Department of Prevention and Care,Section of Maternal and Child Health, Netherlands Organisation forApplied Scientific Research (TNO), Quality of Life, Leiden, the Nether-lands. Dr. Ongkosuwito is Orthodontist and Head, Department ofOrthodontics, and Member, Cleft Palate Team and Craniofacial Team,Erasmus Medical Center-Sophia Children's Hospital, University MedicalCenter Rotterdam, Rotterdam, the Netherlands. Dr. de Vries isEpidemiologist, Department of Public Health, Erasmus Medical Center,University Medical Center Rotterdam, Rotterdam, the Netherlands. Dr.Vermeij-Keers is Embryologist and Dutch Association for Cleft Palateand Craniofacial Anomalies (NVSCA) Registry Leader, Research UnitDepartment of Plastic and Reconstructive Surgery, Erasmus MedicalCenter, University Medical Center Rotterdam, Rotterdam, the Nether-lands.

Submitted May 2010; Revised August 2010; Accepted September 2010.

Common oral clefts (OCs) are very complex andheterogeneous birth defects affecting the lip, alveolus, hardpalate, soft palate, and uvula. In the embryonic develop-ment of the primary palate (the presumptive lip andalveolus) and secondary palate (the presumptive hardpalate, soft palate, and uvula), many different cell-biological mechanisms and genes are involved, related todifferent time frames. During the formation of the primaryand secondary palates, complex embryological processesincluding outgrowth, fusion, and differentiation (into boneand musculature) of the facial swellings and of the palatineprocessestake place (Vermeij-Keers, 1990; Luijsterburgand Vermeij-Keers, 2001; Krapels et al., 2006). Disturbanceof these developmental processes can result in many

Address correspondence to: Dr. Anna M. Rozendaal, Research UnitDepartment of Plastic and Reconstructive Surgery, Room 1591, ErasmusMC, University Medical Center Rotterdam, P.O. Box 2040, 3000 CARotterdam, the Netherlands. E-mail am.rozendaal@gmail.com.

DOI: 10.1597/10-109

609

610 Cleft Palate-Craniofacial Journal, September 2012, Vol. 49 No. 5

different cleft types with variable degrees of severity onchnical presentation (Van der Meulen et al., 1990;Luijsterburg and Vermeij-Keers, 2001; Luijsterburg andVermeij-Keers, 2010).

Although the etiopathogenesis of OCs has been widelystudied, it is still poorly understood (Mossey et al., 2009). Tofacilitate further genetic and etiopathological studies and toimprove prevention, diagnostics, and treatment, it is ofparamount importance that details of all OC types aredescribed and recorded. Worldwide, many registrationsystems have been developed in order to record congenitalanomalies, including OCs (World Health Organization,2003; Boyd et al., 2005; International Clearinghouse forBirth Defects Surveillance and Research [ICBDSR], 2007;Kubon et al., 2007; Amini et al., 2009; EUROCAT NorthernNetherlands, 2010). These registries classify OCs accordingto the International Classification of Diseases (ICD) or itsextensions, thereby providing some information abouttopography but not always complete information aboutmorphological severity (e.g., completeness or incompletenessof the cleft) (World Health Organization, 2003, 2010;EUROCAT Northern Netherlands, 2010). Because differentcleft types, which have specific topographic and morpho-logic features, originate from different time frames and arerelated to specific genes and cell-biological mechanisms,detailed information on the topography and morphology isessential for fundamental research on OCs. Therefore, aunique detailed recording system for OCs and othercraniofacial anomalies has been developed on behalf of theDutch Association for Cleft Palate and CraniofacialAnomalies (NVSCA). This unique NVSCA system is basedon the embryology of the head and neck area and records allthe individual anomalies of the primary and/or secondarypalate that form the OC. Besides the topographic-anatom-ical structure and side, the morphology of each anomaly canbe described to anticipate all conceivable anomalies. Since itsestablishment in 1997, virtually all new live-born presurgerypatients with OCs in the Netherlandsan average of 351patients per yearhave been reported to the nationalNVSCA registry (Luijsterburg and Vermeij-Keers, 2010).

The main purpose of the NVSCA registry is to providea solid basis for epidemiological, clinical, and fundamen-tal research. To serve this purpose, it is crucial to ensurethat the data provided by the registry are of high quality.Sound description and complete reporting of OCs andtheir specific features are necessary to maintain highstandards of data quality. Previously, it was shown thatthe case-ascertainment of OCs in the NVSCA registry ishigh (Anthony et al., 2005). In addition, we found recentlythat the NVSCA registry has high-quality data on thethree OC categories: cleft lip/alveolus; cleft lip/alveolusand palate; and cleft palate (Rozendaal et al., 2010). Asdescribed by Luijsterburg and Vermeij-Keers (2010), thesethree categories manifest very heterogeneous cleft types,composed of individual anomalies of the primary and/or secondary palate having specific features regarding

topographic-anatomical structure, morphology, and side.However, it is unknown whether the individual anomaliesin OCs have been recorded completely and accurately inthe NVSCA registry.

The aim of this study was to investigate the quality of theNVSCA data on the individual anomalies of the primarypalate and secondary palate in OCs by validating theregistry data on the specific features of the anomalies:topographic-anatomical structure, morphology, and side.By linking the NVSCA database with a new independentreregister database derived from medical data review, wewere able to identify differential recording rates bycomparing the prevalence and to measure the degree ofagreement by computing validity and reliability statistics.

METHODS

NVSCA Registry

The methodology of the NVSCA registry is described indetail elsewhere (Luijsterburg and Vermeij-Keers, 2010;Rozendaal et al., 2010) and is summarized here. TheNVSCA registry is an anonymous registry that wasformally established in accordance with Dutch privacylaw. All Dutch cleft palate teams report their new live-born patients with OCsbefore these patients have anoral cleft operationusing the NVSCA recording form.This form is composed of three parts: a general section(infant/parent characteristics), a section for craniofacialanomalies including OCs, and a section for congenitalanomalies of other organ systems; a manual is available(Luijsterburg and Vermeij-Keers, 2010; Rozendaal et al.,2010). The section for OCs consists of a two-dimensionaltable, in which the specific features of the individualanomalies that form the OC can be described. As shown inFigure 1, the x-axis shows the topographic-anatomicalstructures: lip, alveolus (embryologically developed fromthe premaxillae and maxillae), hard palate (palatumdurum), soft palate (palatum molle), and uvula. The y-axis depicts the morphology (complete, incomplete, andsubmucous) and the check boxes represent the side (left,right, and median).

The recording form is completed by the consultingphysician during the first visit of the patient to the cleftpalate team, and subsequently the form is sent to theNVSCA registry. The Registration working group checksthe recorded data before these are transferred to theNVSCA database. In addition, the cleft palate teamsperform case-ascertainment activities annually. Note thatthe NVSCA does not have active follow-up of patients andthat no data from other sources are included.

Subjects

This validation study was initiated and carried out in the15 Dutch cleft palate teams; all gave written permission for

Rozendaal et al., VALIDATION REGISTRY DATA ON SPECIFIC ORAL CLEFT FEATURES 611

L = left

R = right

M = median

Cleft

Complete

Incomplete

Submucous

i/louth

Lip Pre.l Pre.* Pal. Pal. Uvula Ton.'max.* dur.* mol.*

L RM

L RM

L RM

L R

L R

L R

M

M

M

LIRM

L RM

L RM

M

M

M

M

M

M

M

M

M

FIGURE 1 Section of the NVSCA recording form for common oral cleftsin which the specific features of the individual anomalies that form the oralcleft can be described. The x-axis shows the topographic-anatomicalstructures: lip, alveolus (embryologically developed from the premaxillaeand maxillae), hard palate (palatum durum), soft palate (palatum molle),and uvula; the y-axis depicts the morphology: complete, incomplete, andsubmucous; and the check boxes represent the side: left, right, and median.ABBREVIATIONS: pre./max. = premaxilla - maxilla; Pre. = premax-illa; Pal.dur. = palatum durum; Pal.mol. = palatum molle.

review of patients' medical data. Principles outlined in theDeclaration of Helsinki were followed. During a 7-year-period (1997 to 2003), a total of 2553 patients wereregistered in the national NVSCA database with an OC.Patients with median cleft lip/alveolus or atypical facialclefts were excluded due to their different pathogenesis(Van der Meulen et al., 1990; Vermeij-Keers, 1990). Fromthis database, a study population of 250 cases was selectedusing a standard random-sampling technique in StatisticalPackage for the Social Sciences (SPSS) version 17.0 (SPSSInc., Chicago, IL).

Data Collection and Verification

We used medical data to validate the NVSCA data onthe specific features of the individual anomalies in OCs.The methods of medical data collection and verificationwere described in a previous paper by Rozendaal et al.(2010) and are summarized here.

The relevant medical information, including medicalrecords, color photographs, panoramic radiographs, anddental casts, was supplied by the cleft palate teams. For 241of the 250 cases (96.4%), the minimum criterion forinclusionthe availability of at least one medical re-cordwas met. Apart from the nine untraceable cases,we excluded one case that had insufficient medicalinformation to record the cleft and three patients whohad undergone oral cleft surgery before registration. Thisresulted in a total of 237 cases that remained in the study.

Using the medical information that was created before aswell as after completion of the original NVSCA recordingforms, a single investigator (A.M.R.) recorded each case

blindly on the standard NVSCA form (Rozendaal et al.,2010). The criteria used to define the type of OC wereestablished in accordance with existing literature (Vermeij-Keers, 1990). If the medical information was insufficient torecord a specific feature, for example, the morphology orside of the hard palate, the investigator noted this on theform. This was done to allow exclusion of the case at a laterstage from the specific feature's analysis. All the recordeddata were then transferred to an independent reregisterdatabase, and finally, this database was checked fornonexistent, inappropriate, and invalid data.

Statistical Analysis

To get complete insight into the quality of the detailedregistry data on the individual anomalies of the primarypalate and secondary palate, their specific features werevalidated step by step. First, we analyzed the topographic-anatomic structures (lip, alveolus, hard palate, and softpalate including the uvula), then the morphology of thetopographic-anatomic structures (e.g., complete cleft lip),then the side of the topographic-anatomical structures (e.g.,left cleft lip), and finally the morphology and side of thetopographic anatomical structures (e.g., left complete cleftlip), that is, the complete reflection of the individualanomaly. Note that the side of the soft palate including theuvula was not analyzed because clefts of the soft palate anduvula always develop in the median.

The prevalence of the specific features in the individualOC anomalies was calculated for both the NVSCA and thereregister. In addition, the NVSCA database was comparedwith the reregister database for concordance of individualpatient data. Note that one case may contribute to morethan one difference between the databases. In case ofdisagreement between the databases on a specific feature (n= 99), a second investigator (C.V.K.) reviewed the medicaldata blindly and recorded the OC independently on a newNVSCA recording form. If the two investigators disagreed,there was discussion until consensus was reached (n = 21).

It is known that the "disease prevalence" can affectreliability and validity statistics (Feinstein and Cicchetti,1990; Loong, 2003; Whiting et al., 2004) and that theconfidence intervals in reliability and validity statisticsrefiect the precision of the outcome measures. Wevalidated, therefore, only those anomalies individually thathad (1) a prevalence of n > 10 in the NVSCA database; and(2) a sufficiently small 95% confidence interval (CI) for allreliability and validity measures (distance between theupper and lower limits of 95% CI

612 Cleft Palate-Craniofacial Journal, September 2012, Vol. 49 No. 5

TABLE 1 Classifcation of the Individual Cleft Anomalies of the Primary Palate and Secondary Palate According to Fusion and DifferentiationDefects; Any Combination of Anomalies of the Lip, Alveolus, Hard Palate, and Soft Palate Is Allowed

Fusion defects

Differentiation defects

Primary palate

Secondary palate

Primary palate

Secondary palate

Complete cleft lipComplete cleft alveolusIncomplete cleft alveolus (only if the lip is normal or has a complete cleft)Complete cleft hard palateIncomplete cleft hard palateComplete cleft soft palate including uvulaIncomplete cleft soft palate including uvulaIncomplete cleft lipSubmucous cleft lipIncomplete cleft alveolus (only if the lip has an incomplete or submucous cleft)Submucous cleft alveolusHypoplastic lip/alveolusSubmucous cleft hard palateSubmucous cleft soft palate including uvula

explained here. This classificafion is based on the normaland abnormal development of the primary and secondarypalates. During the formation of these structures, fusionand differenfiafion processes are regulated in time andplace. Disturbances of these complex processes can give riseto fusion and/or differentiation defects of the lip, alveolus,hard palate, and soft palate including the uvula. Theoret-ically, all individual anomalies of the primary palate andsecondary palate that form the OC can be classified as afusion or differentiation defect. The template for decidingwhich anomaly is a fusion or differentiation defect is listedin Table 1 (Luijsterburg and Vermeij-Keers, 2001).

When analyzing the morphology and/or side of thetopographic-anatomical structures, the following anomalieswere grouped together. We grouped "submucous cleft lip"together with "incomplete cleft lip" because both aredifferentiation defects of the lip. The differentiation defect"submucous cleft alveolus" was grouped together with"incomplete cleft alveolus," which isin combination withan "incomplete/submucous cleft lip"also a differentiationdefect of the alveolus. We grouped "submucous cleft hardpalate" together with "submucous cleft soft palate" becauseboth anomalies are late differentiation defects of thesecondary palate. The new group "submucous cleft hard/soft palate," which still had a 95% CI that was too wide, wasnot grouped further because other differentiation defects ofthe secondary palate do not exist. "Incomplete cleft softpalate" and "complete cleft soft palate" were groupedtogether because both are fusion defects of the soft palate.The anomaly "right cleft hard palate" was grouped togetherwith "left cleft hard palate" because both are unilateralfusion defects of the hard palate. Because the anomaly "rightsubmucous cleft alveolus" had not been recorded in theNVSCA database, it was not validated. Finally, becausepractically all incomplete and submucous clefts of the hardpalate present in the NVSCA database were median clefts,the side was noi validated for these anomalies.

The prevalence data were presented as numbers andpercentages. Prevalence comparisons between the databas-es were performed using the chi-square test. All p values of

Rozendaal et al., VALIDATION REGISTRY DATA ON SPECIFIC ORAL CLEFT FEATURES 613

TABLE 2= 237)

Prevalence of Specific Features of Individual Anomalies of the Primary Palate and Secondary Palate in Common Oral Clefts (N

NVSCA Reregister

Specific Feature of Individual Anomaly

Cases With Information'

p Value]

Topographic-anatomical structure

Primary palate

Cleft lip 164Cleft alveolus 126

Secondary palate

Cleft hard palate 117

Cleft soft palate 160

Morphology of topographic-anatomical structure

Primary palate

Complete cleft lipIncomplete cleft lipSubmucous cleft lipComplete cleft alveolusIncomplete cleft alveolusSubmucous cleft alveolus

Secondary palateComplete cleft hard palateIncomplete cleft hard palateSubmucous cleft hard palateComplete cleft soft palateIncomplete cleft soft palateSubmucous cleft soft palate

Side of topographic-anatomical structure

Primary palate

Left cleft lipRight cleft lipLeft cleft alveolusRight cleft alveolus

Secondary palatej:Left cleft hard palateRight cleft hard palateMedian cleft hard palate

Morphology and side of topographic-anatomical structure

Primary palate

Left complete cleft lip 57Right complete cleft lip 45Left incomplete cleft lip 60Right incomplete cleft lip 32Left submucous cleft lip 3Right submucous cleft lip 2Left complete cleft alveolus 60Right complete cleft alveolus 43Left incomplete cleft alveolus 34Right incomplete cleft alveolus 14Left submucous cleft alveolus 2Right submucous cleft alveolus 0

Secondary palatej

Left complete cleft hard palate 25Right complete cleft hard palate 15Median complete cleft hard palate 54Left incomplete cleft hard palate 0Right incomplete cleft hard palate IMedian incomplete cleft hard palate 15Left submucous cleft hard palate 0Right submucous cleft hard palate 0Median suhmucous cleft hard palate 6

69.253.4

50.067.5

24.219.125.413.6

1.30.8

25.618.414.56.00.90.0

11.06.6

23.70.00.46.60.00.02.6

164139

128166

4737703885

5646421900

II9

61I0

33006

69.258.9

54.770.0

19.915.729.716.13.42.1

23.919.717.98.10.00.0

4.83.9

26.80.40.0

14.50.00.02.6

237236

234237

236236236236236236234234234234234234

228228228228228228228228228

1.000.228

.309

.552

8385580462

94166

15177

120799658

241573

35.236.02.134.219.70.9

40.97.02.664.83.03.0

50.633.340.924.7

10.66.632.2

671021383570

83346

154612

1227710270

137

100

28.443.25.535.524.40.0

36.114.82.666.12.65.2

51.532.543.429.8

5.73.144.1

236236236234234234

230230230233233233

237237235235

227227227

.114

.110

.055

.771

.220

.156

.292

.0071.000.770.778.242

.854

.845

.575

.214

.059

.080

.009

.267

.331

.303

.437

.127

.253

.668

.724

.316

.367

.1561.000

.015

.134

.387

.317

.317

.0061.0001.0001.000

Number of cases ihaC had sulTicient information to record the topographic-anatomical structure, morphology, and/or side of the individual anomalies.t p value presents stalislical significance level between the Dutch Association for Cleft Palate and Craniofacial Anomalies (NVSCA) and reregister databases in prevalence of feature/

anomaly; /; < .05 is used to determine statistical significance and is presented in bold formal. Side of the soft palate was not analyzed because clefts of the soft palate always develop in the median.

614 Cleft Palate-Craniofacial Journal, September 2012, Vol. 49 No. 5

TABLE 3 Agreement Between the NVSCA Datahase and Reregister Database (Criterion Standard) on Specific Features of IndividualAnomalies of the Primary Palate and Secondary Palate in Common Oral Clefts (N = 237)*

Specific Feature of Individual Anomaly

Topographic-anatomical structure

Primary palate

Cleft lipCleft alveolus

Secondary palate

Cleft hard palateCleft soft palate

K

.98

.82

.84

.92

Morphology of topographic-anatomical structure

Primary palate

Complete cleft lipIncomplete/submticous cleft lipComplete cleft alveolusIncomplete/submucous cleft alveolus

Secondary palate

Complete cleft hard palateIncomplete cleft hard palateSubmucous cleft hard/soft palatelComplete/incomplete cleft soft palate

Side of topographic-anatomical structure

Primary palate

Uft cleft lipRight cleft lipLeft cleft alveolusRight cleft alveolus

Secondary palateS

Left/right cleft hard palateMedian cleft hard palate

.81

.82

.84

.67

.77

.43

.77

.91

.95

.94

.84

.85

0.42.62

Value

(95% CI)

(.95-1.00)(.75-0.89)

(.77-.9I)(.87-.98)

(.73-.89)(.74-.89)(.77-.91)(.56-.78)

(.69-.86)(.25-.60)(.58-.97)(.86-.97)

(.91-.99)(.90-.99)(.73-.91)(.78-.93)

(.25-.58)(.52-.72)

Morphology and side of topographic-anatomical structure

Primary palate

Left complete clefl lipRight complete cleft lipLeft incomplete/submucous cleft lipRight incomplete/submucous cleft lipLeft complete cleft alveolusRight complete cleft alveolusLeft incomplete cleft alveolusRight incomplete cleft alveolus

Secondary palateS

Left/right complete cleft hard palateMedian complete cleft hard palate

.80

.79

.81

.79

.82

.88

.72

.64

.45

.49

(.71-.90)(.69-.90)(.73-.90)(.68-.90)(.73-.90)(.80-.96)(.60-.84)(.45-.84)

(.28-.61)(.36-.62)

Sensitivity

%

99.487.8

88.395.8

97.081.988.068.4

91.635.369.295.5

96.797.488.281.4

73.766.0

93.691.979.776.989.387.069.057.9

78.957.4

(95% CI)

(96.6-100.0)(81.1-92.7)

(81.4-93.3)(91.5-98.3)

(89.6-99.6)(73.2-88.7)(79.0-94.1)(54.8-80.1)

(83.4-96.5)(19.7-53.5)(38.6-90.9)(91.0-98.2)

(91.8-99.1)(90.9-99.7)(80.4-93.8)(70.3-89.7)

(48.8-90.9)(55.8-75.2)

(82.5-98.7)(78.1-98.3)(68.8-88.2)(60.7-88.9)(78.1-96.0)(73.7-95.1)(52.9-82.4)(33.5-79.7)

(54.4-93.9)(44.1-70.0)

Specificity

%

98.695.9

96.298.6

89.398.595.494.9

87.898.099.597.4

98.397.595.599.4

88.094.5

93.194.598.198.094.498.497.498.6

88.089.2

(95% CI)

(92.6-100.0)(89.8-98.9)

(90.6-99.0)(92.4-100.0)

(83.7-93.6)(94.6-99.8)(90.7-98.1)(90.6-97.6)

(81.3-92.6)(94.9-99.4)(97.5-100.0)(90.8-99.7)

(93.9-99.8)(93.7-99.3)(90.4-98.3)(96.7-100.0)

(82.8-92.1)(89.0-97.8)

(88.5-96.3)(90.3-97.2)(94.7-99.6)(94.9-99.4)(89.9-97.3)(95.4-99.7)(94.0-99.1)(96.0-99.7)

(82.8-92.1)(83.4-93.4)

(+) Predictive Value

%

99.496.8

96.699.4

78.397.791.381.3

80.975.090.098.7

98.394.993.898.3

35.990.4

n.275.695.288.283.393.085.378.6

37.566.0

(95% CI)

(96.6-100.0)(92.1-99.1)

(91.5-99.1)(96.6-100.0)

(67.9-86.6)(92.0-99.7)(82.8-96.4)(67.4-91.1)

(71.4-88.2)(47.6-92.7)(55.5-99.7)(95.3-99.8)

(94.1-99.8)(87.5-98.6)(86.9-97.7)(90.8-100.0)

(21.2-52.8)(81.2-96.1)

(64.2-87.3)(60.5-87.1)(86.5-99.0)(72.5-96.7)(71.5-91.7)(80.9-98.5)(68.9-95.0)(49.2-95.3)

(22.7-54.2)(51.7-78.5)

(-J

%

98.684.5

87.290.9

98.787.293.590.3

94.989.798.291.4

96.698.791.492.7

97.377.9

98.398.491.495.596.696.993.596.4

97.985.1

PredictiveValue

(95% CI)

(92.6-100.0)(76.4-90.7)

(79.7-92.6)(82.2-96.3)

(95.4-99.8)(80.7-92.1)(88.4-96.8)(85.1-94.2)

(89.7-97.9)(84.8-93.4)(95.4-99.5)(83.0-96.5)

(91.5-99.1)(95.5-99.8)(85.4-95.5)(87.8-96.0)

(93.9-99.1)(70.5-84.2)

(95.2-99.7)(95.5-99.7)(86.2-95.1)(91.7-97.9)(92.6-98.7)(93.3-98.8)(89.1-96.5)(93.0-98.4)

(94.6-99.4)(78.9-90.0)

Co.se.stn

237236

234237

236236234234

230230230233

237237235235

227227

236236236236234234234234

228228

NVSCA = Dutch Association for Cleft Palate and Craniofacial Anomalies; 95% CI = 95% confidence interval.t Number of cases that had sufficient information to record the topographic-anatomical structure, morphology and/or side of the individual anomalies. This group had a distance of >50% between the upper and lower limit of the 95% CI for the sensitivity but was not grouped further because other embryologically related anomalies do not

exist. The side of the soft palate was not analyzed because clefts of the soft palate always develop in the median.

palates by database. The prevalence of the four topographic-anatomical structures (lip, alveolus, hard palate, and softpalate including the uvula) in the NVSCA database wassimilar to that in the reregister database. For the twostructures of the primary palate (lip and alveolus), thedistribution of the morphology, of the side, and of themorphology and side in the NVSCA was similar to that in thereregister. For one structure of the secondary palate (hardpalate), however, three anomalies were underreported signif-icantly in the NVSCA (incomplete cleft hard palate: p = .007;

median cleft hard palate: p = .009; and median incompletecleft hard palate: p = .006). Only one anomaly (left completecleft hard palate) was significantly less frequent in thereregister than in the NVSCA (4.8% versus 11.0%, p = .015).

Agreement on Specific Features of Individual Anomalies inCommon Oral Clefts

Table 3 shows the degree of agreement between thedatabases for the specific features of the individual

Rozendaal et al., VALIDATION REGISTRY DATA ON SPECIFIC ORAL CLEFT FEATURES 615

anomalies of the primary and secondary palates. Whenanalyzing the morphology and/or side of the topographic-anatomical structures, several anomalies did not meet thecriteria for validation (i.e., they had a prevalence of n < 10in the NVSCA database and/or 95% CIs for reliability andvalidity measures that were too wide). These anomalieswere therefore grouped together with their embryologicallyrelated anomalies as described in the "Methods" section.

Topographlc-A natomical Structure

All four topographic-anatomical structures had near-perfect interdatabase agreement (K value: .82 to .98) with asensitivity of 87.8% or more, a specificity and positivepredictive value of more than 95%, and a negativepredictive of 84.5% or more.

Morphology of Topographic-Anatomical Structure

After regrouping the anomalies, four anomalies of theprimary palate remained. Table 3 shows that the kappavalues ranged from .67 to .84; one anomaly (incomplete/submucous cleft alveolus) was at the level of substantialagreement, and three were at near-perfect agreement.Sensitivity ranged from 68.4% for incomplete/submucouscleft alveolus to 97.0% for complete cleft lip. Positivepredictive values ranged from 78.3% for complete cleft lipto 97.7% for incomplete/submucous cleft lip. The specificityand negative predictive values were more than 87% for allfour anomalies.

For the remaining four anomalies of the secondarypalate, the kappa values ranged from .43 to .91; oneanomaly (incomplete cleft hard palate) was at the level ofmoderate agreement, two were at substantial agreement,and one (complete/incomplete cleft soft palate) was at near-perfect agreement. Sensitivity was 35.3% for incompletecleft hard palate, 69.2% for submucous cleft hard/softpalate, and more than 91% for the other two anomalies.Positive predictive values ranged from 75.0% for incom-plete cleft hard palate to 98.7% for complete/incompletecleft soft palate. The specificity and negative predictivevalues were more than 87% for all four anomalies.

Side of Topographic-Anatomical Structure

Table 3 shows that all four anomalies of the primarypalate had near-perfect interdatabase agreement (K value:.84 to .95), with a sensitivity of 81.4% or more and aspecificity, positive predictive value, and negative predictivevalue of more than 91%.

For the secondary palate, there were two remaininganomalies after regrouping. One anomaly (left/right clefthard palate) had a kappa value of .42 (moderateagreement), sensitivity of 73.7%, positive predictive valueof 35.9%, and specificity and negative predictive value of88.0% and over. The other anomaly (median cleft hard

palate) had a kappa value of .62 (substantial agreement),sensitivity of 66.0%, specificity and positive predictive valueof more than 90%, and negative predictive value of 77.9%.

Morphology and Side of Topographic-Anatomical Structure

For the eight anomalies of the primary palate thatremained after regrouping, the kappa values ranged from.64 for right incomplete cleft alveolus to .88 for rightcomplete cleft alveolus; five anomalies were at the level ofsubstantial agreement and three at near-perfect agreement.Sensitivity ranged from 57.9% for right incomplete cleftalveolus to 93.6% for left complete cleft lip. Positivepredictive values ranged from 75.6% for right complete cleftlip to 95.2% for left incomplete/submucous cleft lip.Meanwhile, specificity and negative predictive values werehigh for all eight anomalies (more than 91%).

For the secondary palate, two anomalies remained forvalidation. Left/right complete cleft hard palate had akappa value of .45 (moderate agreement), sensitivity of78.9%, positive predictive value of 37.5%, and specificityand negative predictive value of 88.0% and over. Theanomaly median complete cleft hard palate had a kappavalue of .49 (moderate agreement), sensitivity of 57.4%,positive predictive value of 66.0%, and specificity andnegative predictive value of more than 85%.

DISCUSSION

This continuation of the NVSCA validation study showsthat the quality of the NVSCA data on the specific featuresof the individual anomalies in OCs varies by type ofanomaly. By linking the NVSCA database with a newindependent reregister database derived from medical datareview, we found that validity of the registry data is relatedto anatomical location and morphological severity of theindividual anomalies.

The following results illustrate the pattern of recording inthe NVSCA. The topographic-anatomical structures of theindividual anomalies of the primary palate (lip andalveolus) and of the secondary palate (hard and softpalates) were identified perfectly in the NVSCA and hadhigh validity measures (85% to 99%) with near-perfectinterdatabase agreement. However, when analyzing theanomahes more in detail (i.e., analyzing the morphologyand/or side), the validity decreased and appeared to berelated to the type of anomaly. First, anomalies of theprimary palate were recorded better than anomalies of thesecondary palate; the interdatabase agreement was nearperfect for most primary palate anomalies; whereas, it wasmoderate to substantial for most secondary palate anom-alies. This suggests better registration of externally visibleanomalies (such as cleft lip/alveolus) than anomalies thatrequire a diagnostic procedure (such as opening the mouthfor inspection and palpating the palate). In addition,validity was related to morphological severity, given that

616 Cleft Palate-Craniofacial Journal, September 2012, Vol. 49 No. 5

severe anomalies were generally recorded better than mildanomalies. This association applied to both the primaryand secondary palates but was most evident for thesecondary palate. For example, 35% of the incompletecleft hard palates and 69% of the submucous cleft hard/softpalates present in the reregister were also present in theNVSCA, compared with more than 91% of the completecleft hard palates and complete/incomplete cleft softpalates.

Although many registries record OCs, studies on thevahdity of OC data are scarce. There are some studies,however, that describe the case-ascertainment of OCs inmedical registries (Cousley and Roberts-Harry, 2000; Boydet al., 2005; Kubon et al., 2007; Amini et al., 2009). In onestudy, that of Kubon et al. (2007), this was done in relationto the various cleft types within the three main OCcategories. Similar to our study, they found that registra-tion in the Norwegian medical birth registry was morecomplete for clefts of the primary palate than for clefts ofthe secondary palate. They suggested that this could beexplained by the delayed diagnoses of clefts of the hard/softpalate and thus incomplete routine examination ofnewborns, which was also reported in other studies(Cousley and Roberts-Harry, 2000; Armstrong and Simp-son, 2002; Habel et al., 2006). Different from registries thatreceive information from birth admissions or hospitaldischarge records, the NVSCA receives the OC datadirectly from the cleft palate teams, which are expected tobe focused on OCs and to examine patients carefully(Rozendaal et al., 2010). Still, part of our findings may beexplained by incomplete examination because the numberof patientsand probably the experience and routine ofdiagnosticsvaries strongly among the 15 Dutch cleftpalate teams.

Delayed diagnosis of cleft palate might have severalclinical implications. For example, the presence of a cleftpalate is often associated with additional congenitalanomalies and syndromes (Mossey et al., 2009), and thediagnosis of a cleft palate should therefore generate an evenmore extensive examination of the newborn.

Additionally, our findings that the quality of recordingincreased with the morphological severity of the anomaliesand that this association was most evident for thesecondary palate are also consistent with the findings ofKubon et al. (2007). Perhaps more unexpectedly, bothstudies showed that besides morphologically mild clefts ofthe secondary palate, those of the primary palate, which areclearly visible and require surgery, also tended to beunderreported. A possible explanation for these findings isthat greater morphological severity of an anomaly might bea factor that encourages doctors to report better.

The underrepresentation of morphologically mild anom-alies may have consequences for research on registry data.These anomalies develop during other stages in embryo-logical development and can be related to cell-biologicalmechanisms and genes other than morphologically severe

anomalies (Vermeij-Keers, 1990; Luijsterburg and Vermeij-Keers, 2001; Krapels et al., 2006). Consequently, studiesbased on registry data examining environmental factors orgenes that are associated with morphologically mild cleftsmight underestimate the importance of such factors andgenes.

The strength of this study is that all cleft palate teamsgave permission to collect the medical data. The samplingframe thus had a national distribution, including cleftpalate teams of large urban teaching and specialisthospitals as well as of small regional ones. Most of thesetreatment centers have carried out the high-qualitydocumentation needed for modern multidisciplinary treat-ment, which favors our retrospective detailed evaluation.However, the use of medical data to validate registry dataalso has its limitation. It can never be equal to thepresentation of the patient in the outpatient clinical setting,and therefore it is never 100% accurate (Quan et al., 2004;Whiting et al., 2004). As we showed previously (Rozendaalet al., 2010), the amount and quality of the medical datavaried by cleft palate team. For some cases, the collectedmedical information was insufficient to evaluate certainspecific features of the individual anomalies, and thereforethese cases had to be excluded from the features' analysis inthis study.

Another limitation is that, although we grouped anom-alies having a sample prevalence of n < 10 together withtheir embryologically related anomalies, there were stillconsiderable differences in the prevalence rates of theevaluated anomalies; morphologically mild anomalies were,for example, less prevalent in the study sample thanmorphologically severe anomalies. Because it is knownthat disease prevalence can affect the reliability (kappa) orvalidity statistics (sensitivity, specificity, and positive andnegative predictive values) (Feinstein and Cicchetti, 1990;Loong, 2003; Whiting et al., 2004) we used to measure thedegree of agreement between the NVSCA and reregister,the differences in validity of registry data on morpholog-ically severe and mild anomalies might partially beexplained by the differences in prevalence.

Finally, the study sample was not large enough toexamine all anomalies of the primary and secondary palatesindividually. Nevertheless, we were able to analyze most ofthe individual anomalies in OCs recorded over a 7-year-period, thereby evaluating the feasibility of the uniquedescriptive NVSCA recording system for OCs.

CONCLUSIONS

Our study is the first that validates descriptive registrydata on OCs. The unique NVSCA system records theindividual anomalies of the primary palate and secondarypalate that form the OC by describing the specific features(topographic-anatomical structure, morphology, and side)of each anomaly. This study shows that the quality of theNVSCA data on the specific features of the individual

Rozendaal et al., VALIDATION REGISTRY DATA ON SPECIFIC ORAL CLEFT FEATURES 617

anomalies in OCs varies by type of anomaly and is relatedto anatomical locafion and morphological severity. Greatermorphological severity of an anomaly might be a factorthat encourages doctors to report better, but underreport-ing might also partly be explained by incomplete examina-fion of the oral cleft. These factors might have implicationsfor genetic and etiologic research based on registry data, forexample, and for guidelines on neonatal examinafion by thecleft palate teams.

Despite the limitations and challenges described, this studyshows together with other quality studies (Anthony et al.,2005; Rozendaal et al., 2010) that, overall, the data quality ofthe NVSCA registry on OCs is high, supporting the feasibilityof the unique NVSCA recording system. However, data onmorphologically severe clefts can be interpreted with higherconfidence than those on morphologically mild clefts. Incontrast to ICD-based registries, the NVSCA registry hasvalid, detailed OC data that are collapsible to more generaldiagnoses or codes, which allows classifying OCs in manydifferent ways. This makes the NVSCA registry a veryvaluable tool for epidemiological, clinical, and fundamentalresearch and for the improvement of OC care.

Aeknowledgments. This study took place under the auspices of the DutchAssociation for Cleft Palate and Craniofaciai Anomalies (NVSCA). Theauthors wish to thank all the cleft palate teams in the Netherlands for theirassistance and support. The cleft palate teams are located in AcademicMedical Center in Amsterdam, Erasmus Medical Center-Sophia Chil-dren's Hospital in Rotterdam, IJsselland Hospital in Capelle a/d IJssel,Leiden University Medical Center in Leiden/Juliana Hospital in TheHague, Medical Center Alkmaar in Alkmaar, Medical Center Leeuwar-den in Leeuwarden, Rijnstate Hospital in Arnhem, Sophia Hospital inZwolle, St. Elisabeth Hospital in Tilburg, University Medical CenterGroningen in Groningen, University Medical Center Maastricht inMaastricht, University Medical Center St. Radboud in Nijmegen,University Medical Center Utrecht in Utrecht, Victor Veau Foundationin Almelo, and VU University Medical Center in Amsterdam.

REFERENCES

Amini H, Axelsson O, Ollars B, Anneren G. The Swedish Birth DefectsRegistry: ascertainment and incidence of spina bifida and cleft lip/palate. Aeta Obstet Gyneeol Seand. 2009;88:654-659.

Anthony S, Jacobusse GW, Vermeij-Keers C, Weijerman ME, VanWouwe JP, Van der Pal-de Bruin KM. Vergelijking van prevalenties uitde LVR/LNR registratie met afwijking specifieke registraties. In:Anthony S, Dorrepaal SA, Kateman K, Van der Pal-de Bruin KM,eds. Aangeboren afwijkingen in Nederland 1996-2003: gebaseerd op delandelijke verloskunde en neonatologie registraties. Leiden: TNO-Kwaliteit van Leven; 2005:41-59.

Armstrong H, Simpson RM. Examination of the neonatal palate. ArehDis Child Fetal Neonatal Ed. 2002;86:F210.

Boyd PA, Armstrong B, Dolk H, Bouing B, Pattenden S, Abramsky L,Rankin J, Vrijheid M, Wellesley D. Congenital anomaly surveillance inEnglandascertainment dericiencies in the national system. BMJ.2005;330:27.

Cousley RR, Roberts-Harry D. An audit of the Yorkshire Regional CleftDatabase. J Orthod. 2000;27:319-322.

EUROCAT Northern Netherlands. Prevalence of congenital malforma-tions in the Northern Netherlands 1981-2007. Available at http://www.rug.nl/umcg/faculteit/disciplinegroepen/medischegenetica/eurocat/professionals/tabellen. Accessed February 24, 2010.

Feinstein AR, Cicchetti DV. High agreement but low kappa: I. Theproblems of two paradoxes. J Clin Epidemiol. 1990;43:543-549.

Habel A, Elhadi N, Sommerlad B, Powell J. Delayed detection of cleftpalate: an audit of newborn examination. Areh Dis Child. 2006;91:238-240.

Humphries KH, Rankin JM, Carere RG, Buller CE, Kiely FM, SpinelliJJ. Co-morbidity data in outcomes research: are clinical data derivedfrom administrative databases a reliable alternative to chart review?J Clin Epidemiol. 2000;53:343-349.

International Clearinghouse for Birth Defects Surveillance and Research(ICBDSR). Annual report 2007, with data for 2005. Rome: ICBDSRCentre; 2007.

Krapels IP, Vermeij-Keers C, Mller M, De Klein A, Steegers-TheunissenRP. Nutrition and genes in the development of orofacial clefting. NutrRev. 2006;64:280-288.

Kubon C, Sivertsen A, Vindenes HA, Abyholm F, Wilcox A, Lie RT.Completeness of registration of oral clefts in a medical birth registry: apopulation-based study. Aeta Obstet Gyneeol Seand. 2007;86:1453-1457.

Landis JR, Koch GG. The measurement of observer agreement forcategorical data. Biometries. 1977;33:I59-I74.

Loong TW. Understanding sensitivity and specificity with the right side ofthe brain. BMJ. 2003;327:7l6-719.

Luijsterburg AJM, Vermeij-Keers C. Recording and classificationcommon oral clefts. In: Lilja J, ed. Transaetions, The 9th InternationalCongress on Cleft Palate and Related Craniofaeial Anomalies.Gtenborg, 2001:243-250.

Luijsterburg AJM, Vermeij-Keers C. Recording common oral clefts: anew descriptive system. Cleft Palate Craniofae J. 2010: in press.Published online, doi 10.1597/08-150.

Mossey PA, Little J, Munger RG, Dixon MJ, Shaw WC. Cleft lip andpalate. Lancet. 2009;374:1773-1785.

Quan H, Parsons GA, Ghali WA. Validity of procedure codes inInternational Classification of Diseases, 9th revision, clinical modifi-cation administrative data. Med Care. 2004;42:801-809.

Rozendaal AM, Luijsterburg AJM, Mohangoo AD, Anthony S,Ongkosuwito EM, Vermeij-Keers C. Validation of the NVSCA registrycommon oral clefts: study design and first results. Cleft PalateCraniofae J. 2010;47(5):534-543.

Van der Meulen J, Mazzola R, Strickler M, Raphael B. Classification ofcraniofaciai malformations. In: Strieker M, Van der Meulen JC,Raphael B, Mazzola R, Tolhurst DE, Murray JE, eds. CraniofaeialMalformations. Edinburgh: Churchill Livingstone; 1990:149-309.

Vermeij-Keers C. Craniofaciai embryology and morphogenesis: normaland abnormal. In: Strieker M, Van der Meulen JC, Raphael B,Mazzola R, Tolhurst DE, Murray JE, eds. Craniofaeial Malformations.Edinburgh: Churchill Livingstone; 1990:27-60.

Whiting P, Rutjes AW, Reitsma JB, Glas AS, Bossuyt PM, Kleijnen J.Sources of variation and bias in studies of diagnostic accuracy: asystematic review. Ann Intern Med. 2004; 140:189-202.

World Health Organization. The development of registries to supportbirth defect research. In: Mossey P, Castilla E, eds. Global registry anddatabase on eraniofaeial anomalies. Report of a WHO Registry Meetingon Craniofaciai Anomalies. Geneva: World Health Organization;2003:1-14.

World Health Organization. International statistical classification ofdiseases and related health problems, lOth revision. Version for 2007.Available at http://www.who.int/classifications/apps/icd/icd 1 Oonline/.Accessed February 24, 2010.

Copyright of Cleft Palate-Craniofacial Journal is the property of Allen Press Publishing Services Inc. and its

content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's

express written permission. However, users may print, download, or email articles for individual use.