1
The search strategy was based on a the Montori’s filter (1) for identifying SRs in PubMed with emphasis in specificity, and the Ecuador Cochrane Center strategy). (see Table 1) The search strategy covered the period between January 2008 and March 2011. Study selection was performed through EROS, a web-based software to serve in the initial phases of the SR process. All retrieved studies were independently assessed by pairs of researchers, both by title/abstract and full text. Identification of systematic reviews in LILACS Ciapponi A 1 , Glujovsky D 2 , Aleman A 3 , Caccavo F 4 , Claro García-Atance JC 5 , Comandé D 6 , Corbalán Pössel J 7 , Daray FM 8 , Eslava Schmalbach JH 9 , Fernández Toricez A 10 , García Dieguez M 11 , García Martí S 12 , Gianneo O 13 , Gutiérrez Castrellón P 14 , Jimenez Gutierrez C 15 , Klajn DS 16 , López L 17 , Loza Munarriz CA 18 , Malaga G 19 , Patrón Sentena C 20 , Pinto R 21 , Rada G 22 , Rey Ares L 4 , Rivera Mercado S 23 , Rodriguez Funes V 24 , Rodríguez R 25 , Sguassero Y 26 , Tisi Baña M 27 , Abdala V 28 1 IECS - Institute for Clinical Effectiveness and Health Policy. Argentina; 2 IECS - Institute for Clinical Effectiveness and Health Policy, Argentina; 3 Departamento de Medicina Preventiva y Social, Facultad de Medicina. Universidad de la Republica, Uruguay; 4 Argentine Cochrane Centre IECS, Institute for Clinical Effectiveness and Health Policy. Buenos Aires, Argentina.; 5 EBM Unit, Faculty of Medicine, Pontificia Universidad Católica de Chile, Chile; 6 Argentine Cochrane Centre IECS, Institute for Clinical Effectiveness and Health Policy. Buenos Aires, Argentina; 7 Health Policy and Systems Research Unit, Pontificia Universidad Católica de Chile, Chile.; 8 3° Cátedra de Farmacología, Facultad de Medicina, UBA. Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Argentina.; 9 Clinical Research Institute, School of Medicine, Universidad Nacional de Colombia, Colombia.; 10 Centro Documentación Conjunto OPS/OMS y CLAP/SMR-OPS/OMS. PAHO/WHO and CLAP/WR-PAHO/WHO Joint Documentation Center, Uruguay.; 11 Departamento de Ciencias de la Salud- Universidad Nacional del Sur, Argentina.; 12 Institute for Clinical Effectiveness and Health Policy. Buenos Aires, Argentina.; 13 Centro Colaborador Uruguay – Fondo Nacional de Recursos, Uruguay.; 14 Dirección de Investigación y Centro Cochrane. Instituto Nacional de Perinatologia. Ministerio de Salud, México; 15 Subdirección de Evaluación Clínica de Tecnologías en Salud. Centro Nacional de Excelencia Tecnológica en Salud (CENETEC- SALUD), México; 16 Comité de Docencia e Investigación, Hospital Tornú, Buenos Aires, Argentina.; 17 Centro de la Colaboración Cochrane Guatemala. Centro de Investigaciones de las Ciencias de la Salud. Facultad de Ciencias Médicas. Universidad de San Carlos de Guatemala, Guatemala.; 18 Unidad de Epidemiologia Clinica. Universidad Peruana Cayetano Heredia, Perú.; 19 CRONICAS-Universidad Peruana Cayetano Heredia, Perú.; 20 Facultad de Odontología, Universidad de la República, Uruguay; 21 Servicios Cooperativos de Información - SCI, BIREME/OPS/OMS, Brasil; 22 Department of Internal Medicine. Faculty of Medicine, Pontificia Universidad Católica de Chile, Chile.; 23 Department of Family Medicine. Faculty of Medicine, Pontificia Universidad Católica de Chile, Chile.; 24 El Salvador Cochrane Division from Central American Branch of the Cochrane Iberoamerican Center, El Salvador.; 25 Health Policiy and System Research Unit, Pontificia Universidad Católica de Chile, Chile.; 26 Centro Rosarino de Estudios Perinatales, Argentina.; 27 Clinica Medica. Hospital Universitario Austral Pilar, Argentina.; 28 Servicios Cooperativos de Información - SCI, BIREME/OPS/OMS, Brasil. Introduction To identify SR in LILACS by DARE (Database of Abstracts of Reviews of Effects) and by other organisations criteria To identify the accuracy of Title/Abstract criteria to identify SR by DARE Our search, applying the Montori’s filter, identified 2241 references as potentially eligible SRs 1786 with full-text available have been included in the current analysis 15% were classified as a SR, according to the used DARE criteria, and about 1% when using more strict criteria (see Table 3) The accuracy of Title/Abstract criteria to identify Systematic Reviews (DARE) are presented in Table 4 o It is desirable that users have easy access to systematic reviews (SR) instead of potentially biased narrative reviews (NR). o Databases methodological filters are efficient tools retrieving SRs but do not discriminate them of NRs also retrieved. Some users’ abilities are required for this discrimination, through a process which is frustrating and prone to mistakes. o Considering the databases limitations to differentiate them, we organized a collaborative web-based model to identify SRs in LILACS. Results Aim Methods References Conclusion This collaborative model will allow LILAC’s users to easily access to confirmed SRs by incorporating labels into LILACS. The proportion of confirmed SRs out of references retrieved by Montori’s filter was still low. Identifying SRs by title/abstract criteria it´s difficult. Mentioning meta-analysis had 100% specificity but the positive predictive value it’s not so high (88%) Identifying itself as a SR or MA criteria had the highest negative predictive value (97%) An improved filter could be developed with these inputs. 1. Montori VM, Wilczynski NL, Morgan D, Haynes B, for the Hedges Team. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ 2005; 330: 68-72. 2. DARE. (Accessed 04/08/2011, 2011, at http://www.crd.york.ac.uk/cms2web/AboutDare.asp.) 3. Sander L, Kitcher H. Systematic and Other Reviews: Terms and Definitions Used by UK Organizations and Selected Databases. Systematic Review and Del-phi Survey. In: National Institute for Health and Clinical Excellence. London; 2006 ((TW:"revision sistematica" OR TW:"revisao sistematica" OR TW:"systematic review") OR ((MH:"review literature as topic" OR MH:"meta-analysis as topic" OR PT:revision OR PT:"meta-analysis") AND (TW:metaanal$ OR TW:"meta-analysis" OR TW:"meta- analise" OR TW:"meta-analisis" OR TI:overview$ or TW:"estudio sistematico" OR TW:"systematic study" OR TW:"estudo sistematico" OR TI:review OR TI:revisao OR TI:revision)) T able 2. Definitions of systematic review by criterion Table 1. Search strategy to identify relevant systematic reviews in LILACS (1) Table 4. Tests properties to detect Systematic Reviews (DARE) by the presence of marker concepts in Title/Abstract Identified itself as a Systematic Review Search strategy description Mentioned a Meta-analysis Quality assessing description Systematic Review or Search strategy Sensitivity 0.81 (0.77-0.86) 0.80 (0.76-0.85) 0.14 (0.10-0.18) 0.20 (0.15-0.25) 0.96 (0.93-0.98) Specificity 0.92 (0.90-0.92) 0.79 (0.77-0.81) 1.00 (0.99-1.00) 0.99 (0.99-1.00) 0.76 (0.74-0.79) +PV 0.61 (0.56-0.66) 0.40 (0.35-0.44) 0.88 (0.78-0.98) 0.78 (0.68-0.88) 0.41 (0.37-0.45) -PV 0.97 (0.96-0.98) 0.96 (0.95-0.97) 0.87 (0.85-0.89) 0.88 (0.86-0.89) 0.99 (0.99-1.00) +LR 9.05 (7.63-10.72) 3.77 (7.63-10.72) 42.66 (16.92-107.56) 20.37 (11.66-35.60) 4.06 (3.70-4,46) -LR 0.20 (0.16-0.26) 0.25 (0.20-0.32) 0.88 (0.76-0.86) 0.81 (0.82-0.91) 0.05 (0.03-0.10) +PV / -PV: Positive / Negative predictive Value, predictive Value. +LR / -LR: Positive / Negative likelihood ratio Table 3. Proportion of 1786 studies meeting each Systematic Review`s criterion When titles/abstracts were analyzed, the following criteria were used: • Identifies itself as a systematic review • Describes a search strategy • Describes assessing study quality • Mentions a Meta-analysis When full texts were assessed, DARE criteria (2) an other criteria (3) for quality assessment were used (see Table 2). When DARE criteria were met, the study was classified as SR. Otherwise, it was classified as non-SR. Discrepancies were solved by consensus.
