Coursebook

EPISODE RESEARCH TRAINING

CKO-9 course

and

Research Training Internship

February – 2015 Plu-nr: xx-xxx

CONTENTS 3 PREFACE 4 1 CHOOSING YOUR RESEARCH PROJECT 5 Procedure 5 Selecting your project 6 Supervision during the Research Training Internship (RTI) 7 2 ORGANISATION EPISODE RESEARCH TRAINING 9 Coordination and learning objectives 9 Central Clinical Education (CKO-9) 9 Research Training Internship 10 Personal learning objectives 10 Assessment

Central Clinical Education (CKO-9) Research Training Internship Attendance and absence (CKO-9) Evaluation

10 10 10 11 12

3 TWO-WEEK CKO-9 COURSE 13 Structure and learning components 14 L

Module 1: Rationale of the researchecture 1: Profile of CKO-9 15

17 Module 2: First appraisal of the literature 19   Module 3: Specifying the research question 22 S

Module 4: Pathways to efficient study designeminar 1: A first outline of your research 24

25 Module 5: Measurement of key variables 27 Module 6: Considering the study subjects 30 Seminar 2: Designing your data collection 33 L

Module 7: Societal and ethical justificationecture 2: Philosophy of science influencing your research 34

36 Module 8: Designing the data analysis 39 Module 9: Organisation and management summary 46 S

Computer lab: SPSS practicalitieseminar 3: Practical organisation and numerical thoughts 49

50 Module 10: Reporting your Plan of Work and management summary 54 Seminar 4: Sales and communication 57 CKO-9 Evaluation form 58 4 TWELVE-WEEK RESEARCH TRAINING INTERNSHIP 60 General information 60 Supervision 61 Writing your research report 63 APPENDICES 66 Registration form 67 Template Project Outline and tips 69 Two examples

Assessment protocols 72 76

3

PREFACE Scientific research is driven by scientific curiosity, by professional and social ambition, and – certainly in today’s medicine – by evidence-based motives as well as by serendipity (i.e. making a fortunate discovery while looking for something else). So, there are many factors that influence the health care people receive following or during a visit to the doctor. It often starts with discoveries made by scientists working in laboratories. Once scientists or medical-specialists spot the potential of the scientists’ work, they begin developing it further, with clear goals in mind; to help to cure or prevent disease. In some cases medical breakthroughs arise following a researcher’s study of the spread of a disease between people or from the question whether it is desirable to implement a new treatment. Medical-specialists have to deal with this continuous development in the field of medical knowledge. They have to keep up and anticipate on new insights and advanced technical possibilities. One of the important skills required in this process of advancement of knowledge is a critical attitude towards research practice. Furthermore, medical specialist themselves conduct experiments to aid and support these developments in scientific knowledge. Medical-students have to familiarize themselves with this always changing and exciting world of scientific research. This acquaintance starts in the first-year Medical Professional Training course (MPV-1) and continuous in other MPV, 5BOSA8 and O courses in the bachelor’s phase. Their theoretical scientific training ends in the master’s phase in Episode Research Training / Episode Onderzoeksstage in which they choose an Research Training Internship and prepare this internship in CKO-9. This episode can be located in the final stages of their master’s phase, when they have already gained experience at various clinical departments or at the beginning, prior to their clinical internships, in case of an waiting period. We hope that the CKO-9 course and the Research Training Internship that forms part of this Episode increases your enthusiasm for scientific work and form the kick off for continuing a research career in this exciting and important medical field following your graduation.

4

1 CHOOSING YOUR RESEARCH PROJECT There is a wide range of research projects open to students when preparing for their Research Training Internship (RTI). These can be found in any department of the Radboud University Medical Centre (Radboudumc) or in many institutions elsewhere in the Netherlands and abroad. These may be projects in clinical specializations within or outside the hospital, in basic science subjects, in meta-medical domains, or in the behavioural sciences. In each and every case, the RTI must meet the following quality requirements: - the internship plan must be approved by the Research Training Committee (RTC); - the internship will be pre-trained in the CKO-9 course; - the host organization for the internship must guarantee host supervision quality; - the internship plan must make clear that the proposed time frame can be met and can be completed

with a report; - a senior member of staff, holding a doctorate degree and in the employment of the Radboudumc,

must declare him/herself to be prepared to take on final responsibility for supervision and assessment (the Radboudumc final supervisor).

Research Training Committee (RTC) The chair of the Research Training Committee: Professor J.P.H. Drenth - phone 024-3616999 - email Joost.Drenth@radboudumc.umcn.nl The managing director is drs. C.C.M. Janssen The Committee consists of 4 members, all of whom are qualified principle investigators and represent scientific disciplines ranging from applied medicine (causal and predictive research) to (molecular) life sciences.

Students are strongly encouraged to apply for an RTI abroad as this is in line with the University’s pursuit of internationalization goals. A great many of medical faculty staff have contacts with leading research institutes elsewhere in the world. They are often prepared to assist motivated students in the organisation of an internship project abroad. These internships, however, are subject to the same quality requirements as internal RTIs or other external RTI’s. Students must be aware that RTIs abroad require more preparation and additional effort; often this is a preparatory period of at least one year. The internationalization officer of the Radboudumc Education Institute has been specially appointed to support students in preparing for these training programmes abroad. Procedure Before students embark on their RTI, they have submitted a project plan for approval by the RTC. The plan has to be handed in at the Student Information Point (StIP) at OIO/OSZ. Forms for internship project plans can be obtained from the OIO/OSZ desk or can be downloaded from www.umcn.nl/student. Students have to include information about supervision arrangements, and they need to submit a concise project outline of preferable max 2 A4 (background, research question, study design, impact, plan of work, feasability, literature). A template for the concise project outline has been included in the Appendices. Students hand in their completed project plan and the (StIP) “Registration form” (see Appendices) for the CKO-9 course and the Research Training. On this registration form, which has to be signed by both the student and the UMC final supervisor, both declare that they are acquainted with the Episode-9 Course Book and that there is sufficient opportunity for achieving the general learning objectives of the internship in the 12-week period available. This then forms the first quality control step in internship projects. Registration form and the project plan must be submitted before the first day of the period preceding the block period in which students wish to enrol in the CKO-9 training period.

5

Subsequently, the RTC reviews the project plan and provides commentary as she finds the project outline lacks information or is of insufficient quality. Also, before approving an internship, the RTC verifies whether the conditions necessary for supervision have been met. Without this approval, students cannot embark upon the CKO-9 course nor on their internship. Please note: In case accreditation by Ethics Committee is necessary and NOT already approved, the RTI can’t be approved by the RTC. Students can monitor the progress of the approval procedure via their Student Web Dossier. After formal approval and sign off of the plan by the RTC and the students have been registered for the CKO-9 course, the quality control process has been completed. Please note: students may only embark on their RTI after they have completed the CKO-9 course. If two students together embark on an internship, both students must submit their own (and hence not identical) project proposal and registration form. In the case of an internships not in Radboud University Medical Centre (abroad or external), written confirmation from the host institution is required, specifying who will supervise the student at the host institution Selecting your project First of all, students need to decide on the country, city, institution, and discipline in which they mean to take the RTI. Subsequently, they need to investigate the practicalities and verify whether their ideas are feasible. When selecting an RTI, there are two routes for students to take: either they make a choice from the list of projects available on the web or they take the initiative to arrange an internship themselves. These two routes are described below. Project proposals initiated by scientific departments On the Radboudimc website students can find descriptions or suggestions for RTI projects. Members of staff in scientific departments often frame project proposals that link to the research interests of the department concerned. These proposals sometimes are ready made and allow students to make an immediate start. More commonly, however, project proposals have an open character and describe the research theme or research methodology in broad terms. In this case, it is up to students and their supervisors to formulate a concrete internship plan that is feasible in a 12-week period.

SELECTING A RESEARCH PROJECT FROM THE WEB-LIST Student selects a project from the web-list and contacts the organization that offers an internship (often Radboudumc)   Both host (or daily) supervisor and Radboudumc final supervisor confirm or refute students’ placement.   Student hands in his/her project plan and registration form (see Appendices) to StIP.   Student checks whether the internship proposal has been approved by the RTC and when he/she is enrolled in the CKO-9 course.

Project proposals initiated by students Students are at liberty to look for or formulate their own RTI position and find an appropriate host for internship and supervision. A common procedure is for students who are interested in a particular field to contact a department or a researcher to discuss the option for an RTI. These then jointly formulate an internship plan, which may match the framework of an ongoing research interest. Then the student needs to arrange host-based supervision which meets the regulations for supervision and assessment. Students then contact the Radboudumc supervisor, presenting an outline of the activities proposed at the host organization and supervision arrangements made at the host institution. The Radboudumc supervisor then decides whether he/she is prepared to take on final responsibility for the internship. On acceptance, the student informs the host supervisor.

6

DEVELOPING YOUR OWN INTERNSHIP Student contacts the host organization to discuss options (can also be Radboudumc).   Student presents a draft version of activities and arranges host-based supervision at the host organization of their choice.   Student contacts a Radboudumc final supervisor.   The Radboudumc final supervisor decides whether he/she is prepared to take final responsibility for the internship and makes arrangements with the host tutor.   Student contacts the host organization. Host tutor confirms whether students can or cannot be accepted for an internship.   Student hands in his/her project plan and registration form (see Appendices) to StIP   Student checks whether the internship proposal has been approved or not by the RTC and when he/she is enrolled in the CKO-9 course.

Supervision during the Research Training Internship (RTI) A senior member of staff, holding a PhD and being an employee of Radboudumc, has final responsibility for the RTI. This also applies to internships taken outside Radboudumc. This Radboudumc final supervisor is also responsible for approving the assessments on the first assessment form. This form, therefore, must also be co-signed by the Radboudumc final supervisor before being submitted to the RTC. Internal internship RTI performed at Radboudumc must be supervised by a Radboudumc member of staff holding a doctorate. In everyday practice, non-PhD staff can also be involved in supervisory activities. In these cases a senior member of staff holding a doctorate (the Radboudumc final supervisor) will formally be in charge of project’s supervision and is responsible for signing the project proposal in advance, and/or signed approval of the daily supervisor’s assessments on the first assessment protocol. The (daily) supervisor holds regular supervisory meetings with students and provides day-to-day project supervision. The internship student takes the initiative for scheduling supervisory meetings. Should any problems occur, the Radboudumc final supervisor is the person for students to turn to and discuss issues regarding the discharge of their research tasks at the department. If this does not help to improve matters or if the daily and final supervisor are one and the same person, students can contact the RTC. External internship In the case of external internships (either in the Netherlands or abroad), supervision at the host institution needs to be guaranteed. The Radbboudumc final supervisor who was responsible for initial project approval also has the final responsibility for supervision and makes sure that supervision at the host institution is practised in the right way. He/she is also responsible for signed approval of the host supervisor’s assessments on the first assessment protocol. Prior to the start of an external internship the Radboudumc supervisor needs to be convinced of the likelihood of the internship being completed successfully. In preliminary contacts, attention needs to be paid to the suitability of the internship topic, its feasibility, host supervision arrangements, and to assessment procedures. The host institution agrees to provide the required supervision. In the case of an internship abroad, travel, accommodation, and organizational issues involved need to be taken care of. The host supervisor at the internship institution is responsible for supervision throughout the internship and holds regular supervisory meetings with the student. The daily supervision need not necessarily be provided by the host supervisor; they can delegate qualified replacements. The internship student takes the initiative for scheduling supervisory meetings. In addition, the host supervisor is responsible for assessing the internship, both its final product and that the process leading to it. Should any problems occur, the host supervisor is also the person for students to turn to and discuss issues regarding the discharge of their

7

research tasks at the department. If this does not help to improve matters, students should first contact their Radboudumc final supervisor. If no solution is found, students can contact the RTC.

8

2 ORGANISATION EPISODE RESEARCH TRAINING Episode Research Training consists of two parts: - The Central Clinical Education CKO-9 course ‘Professionalising your approach to Medical Research’ and, - A twelve week Research Training Programme.

Coordination The responsibility for the entire episode programme and its execution lies with Episode Research Training / Episode Onderzoeksstage Coordinator professor A.L.M. Verbeek

- phone 024 -3613102 - email Andre.Verbeek@radboudumc.nl

The final responsibility for the Central Clinical Education (CKO-9) lies with Executive Coordinator dr. J.D.M. Otten - phone 024-3616979 - email Hans.Otten@radboudumc.nl The final responsibility for the Research Training lies with the chair of the Research Training Committee professor J.P.H. Drenth - phone 024-3616999 - email Joost.Drenth@radboudumc.nl

Learning Objectives Central Clinical Education (CKO-9) An inventory of the scientific topics of the Nijmegen research internships shows that the majority of these projects can be classified as applied medical research. These studies provide quantitative evidence on the effect of medical interventions, and on the diagnosis, aetiology and prognosis of disease. The remainder are mechanism studies (how illnesses develop) and health care evaluations. The CKO-9 course deals with the aims, methodology and procedures of conducting scientific research. Up to this point in the curriculum, students have engaged with other people’s scientific work, but, in their 12-week RTI, based on the Plan of Work created in this course, they conduct their own first scientific study. Central learning objectives CKO-9 is a two-week basic training course preceding the internship on medical research. After completing the training course students are able to conduct their internship following the proposed Plan of Work. Key Features The course teaches you to: - relate current medical research (literature) to your research project - motivate the rationale of your research and address the main research question - follow pathways resulting in an efficient study design - understand common study designs, meta-analysis and qualitative studies - define key variables and how to measure them - consider the number of subjects / materials / tissues needed - organize the data collection - analyse data (computer lab with SPSS) - organise, manage and report a Plan of Work.

9

Research Training Internship The main objective of the RTI is that students will enhance the scientific understanding they gained in the Bachelor’s cycle and in CKO parts of the Master’s cycle by deepening, widening, and applying their understanding to more specific and more complex situations. The internship should be considered an educational or learning activity, which is also true for preparing the research report: the internship is an exercise in performing research, not a test of researcher competence. This does not preclude certain students from actually achieving a more or less independent researcher level. General learning objectives The RTI comprises the following general learning objectives. Students can: 1. specify the motive for the study, the interests it serves, and the medical scientific context in which it

fits.(= CKO-9) 2. draft a definitive research proposal on the basis of the following skills:

a. They can retrieve, critically read, and systematically review relevant literature. b. They can give greater precision or depth to the research question on the basis of the literature. c. They can formulate a study design that fits the definitive research question.

3. organize and execute the proposed study. 4. systematically analyse the data they have collected. 5. provide clear descriptions of their results and neat summaries in tables and figures. 6. point out errors of measurement and other limitations in the data they collected. 7. critically reflect on the study design and the results. 8. produce a written report that conforms to requirements:

a. In terms of content (justification of the study design, clear description, analysis, critical reflection). b. In terms of form (carefully edited, trim layout, clear language, references).

9. deliver a concise oral presentation of research results to the department where the study was performed and engage in productive discussion of its results.

Personal learning objectives of the Research Training Internship In addition to the general learning objectives, there may be specific and personal learning objectives relating to a particular type of internship: in one type of internship students will need to compose a questionnaire for data collection purposes, and in another type, they may need to learn how to use a pipette; in one, they will need to recruit test subjects, in another, they will be working with experimental animals or with laboratory samples that have already been collected. Sometimes, they need to write an application to the Central Committee on Research Involving Human Subjects, and sometimes, this will not be necessary. This shows how a range of personal learning objectives may be required on a case-by-case basis. Students have to include three specific learning objectives in their Plan of Work. It is important that students and their supervisors, in advance, agree both the general learning objectives as well as the personal learning objectives: which specific skills must have been developed by students upon completing their internship? Assessment Central Clinical Education (CKO-9) course During the CKO-9 course which lasts two weeks, students prepare a Plan of Work for their own RTI. This written plan is the final product of the CKO-9 course, to be assessed by the CKO-9 course lecturer on a pass-fail basis. In case of a fail, the course coordinator will ask students to revise their plan; only when students have produced the desired result can they officially embark on their RTI. As additional course requirement, students must have discussed their Plan of Work with their supervisors, and supervisors must have approved the plan as feasible and as suitable for achieving the internship objectives. Also the student’s presence and dedication during the scheduled instruction activities, including the running of the quick tests have to be satisfactorily fulfilled. Research Training Internship The RTI will be assessed by means of two assessment protocols. This has been done to validate the reliability of the assessment procedure. If the first assessment is biased owing to process aspects – for

10

example the project-based collaboration between student and supervisor – the independent assessment of the written report by a second assessor may redress the balance. First assessment protocol The first assessment protocol (see Appendices) is used by the internship daily or host supervisor. If this supervisor is not the same person as the Radboudumc final supervisor, the latter must express his/her agreement with the assessments of the former. Especially for foreign internships, adaptation of grades is often required, because foreign grading systems can deviate substantially from the Dutch system. The assessment on the first assessment protocol of the Radboudumc final supervisor is decisive. In all cases, the Radboudumc final supervisor has to sign the protocol. The first assessment protocol distinguishes between assessment of students’ overall performance (process aspects) and their achievements (learning outcomes), i.e. the degree to which they have accomplished the general internship learning objectives. These general learning objectives bear on aspects like the research rationale, study design, organization and execution, data analysis, discussion of results, and the written research report. The protocol also records specific internship learning objectives, which, however, do not contribute towards the final grade. Second assessment protocol An independent second assessor will be asked to assess the RTI report (product aspects) by means of a second assessment protocol. If the first assessment should be biased owing to process aspects – for example the project-based collaboration between students and supervisors – the independent assessment of the written report by a second assessor may redress the balance. As the written report (product) may be considered an account of the students’ learning, it may indirectly and implicitly reveal something about the general learning objectives of the internship. The report can be assessed objectively by this independent second assessor as he/she is unaware of students’ and supervisors’ day-to-day conduct throughout the internship. The protocol used in assessing the report deals with aspects of both content and form (see Appendices). Final grade of the episode As soon as the assessments have been recorded in the protocols concerned, the final grade of the RTI is determined by the chairperson or a member of the RTC who examines the protocols. Only after a member of the committee has co-signed the final assessment form, will the final grade be valid. The final grade of the internship is calculated as follows: the overall performance grade + the general learning objectives grade + 2 x the written report grade / 4. Each of these separate grades must be a pass for students to pass their internship. The RTC chairperson acts as the examiner on behalf of the Examining Board. If the assessments in the first protocol are not validated by the Radboudumc final supervisor or if the grades on both protocols are too far apart (with a gap of three or more), it is up to the chairperson to take a decision and decide on measures to be taken, if any. The final result is registered in OSIRIS the student information system. In conformity with examination regulations, this episode is given a single grade. The final grade is the same as the final grade awarded to the RTI. The grade will only be valid if students have passed their CKO. The above is a paraphrase of the assessment regulations for Episode Research Training / Episode Onderzoeksstage. The full text of these episode assessment regulations can be found on the Episode Research Training BlackBoard page. Attendance and absence (CKO-9) As participation in all scheduled instruction activities at CKO-9 (lectures, seminars and computer lab) in this course is compulsory, absence will be registered. If students should be unable to attend compulsory course components, they must inform the course coordinator (via BlackBoard) who will then decide whether their absence is permitted or not. If a student’s absence is legitimate, the coordinator will assess whether lost ground needs to be caught up by means of an assignment. If students are absent without leave or show lack of commitment, they will fail the CKO-9 course.

11

Evaluation The “Instituut Wetenschappelijk Onderwijs en Opleidingen” (IWOO, the Scientific Education Institute) has, in collaboration with the IOWO, developed a questionnaire to evaluate all components of the clinical master’s phase. After having finished the RTI and submitting it to StIP, the student receives an automatically generated email after one week to evaluate both the CKO-9 course and the RTI using our web-based questionnaire. Students can open this questionnaire in BlackBoard via “Evaluatie / Esurvey” or directly through http://www.evaluatieservice.com/projects/evaluatie_episode_onderzoeksstage. It will be greatly appreciated if this evaluation is completed immediately after receiving the email.

12

3 TWO-WEEK CKO-9 COURSE

Professionalising your Approach to Medical Research

Over the last four years, almost 800 medical students have participated in and successfully completed the CKO-9 course. Their feedback:

‘I’m glad I took this intensive 2 week crash course in doing research. I wanted to learn the basics quickly before I started my research internship.’

‘Looking back at the CKO‐9 course, I notice that I learned a lot. In the beginning, I honestly had no clue how to start a research project. Now I have a clear plan of approach’  ‘I have more ideas about the possibilities and limitations of my research project. Now it feels like a personal piece of work.’ ‘We learned to make a Plan of Work for a scientific research project. This will be very useful in the near future when I decide to do more scientific research elsewhere.’

You are now nearing your choice of clinical specialism. The CKO-9 course has been designed to demonstrate how medical research will guide and support you as a practicing specialist. Research forms the basis for creating new knowledge, which in turn leads to new or improved medical practice. By experiencing a professional approach to research, we hope that, in the future, a number of you will continue to be active researchers alongside your daily work in your chosen specialism.

13

Structure and Learning Components Main Objective CKO-9 After completing this course you are ready to conduct your medical research internship following the proposed Plan of Work. To get the most out of your internship we have developed ten modules, which together with lectures, four work groups and one computer lab on SPSS biostatistics, form the learning components of the research methodology course. Each of the learning components is described in detail below.

Team of teachers Course coordinator Central Clinical Education (CKO-9) dr. J.D.M. Otten - phone 024-3616979 - email Hans.Otten@radboudumc.umcn.nl Teachers: Dr. J.A.A.M. van Dijck (department for Health Evidence)

Dr. J. Fransen (dep. of Rheumatology) Dr. M.M..H.J. van Gelder (department for Health Evidence)

Drs. H. Groenewoud (department for Health Evidence) Dr. R.J. Melis (dep. of Geriatrics)

Prof. dr. A.L.M. Verbeek (department for Health Evidence)

Blended learning: The CKO-9 way of working In CKO-9 we stimulate the use of peer review, which you will become familiar with in the first lecture. This formula is used in all workgroups and is strongly recommended, although not compulsory, for all modules by working in couples. The strength of peer review will become clear as you interact with your immediate fellow students and when preparing the workgroups by means of the modules. Peer review through interaction with the rest of the class and the lecturer in the workgroup will re-enforce the process of critical reflection. This will help you when deciding on the various aspects of scientific research and will eventually improve the products you have to make in the assignments, leading to a reasoned plan of approach for your RTI. The assignments start with reading chapters in the book of Zielhuis et al., which provide the theoretical background and are a lead-in to more practical skills necessary for fulfilling the assignments. The course comprises an interactive mix of teaching and learning: two interactive lectures, ten study assignments, four work groups including peer and intervision elements, one computer lab session, and individual coaching & support throughout. These have been designed to help you independently understand and apply the principles and methods of predictive and causal research. At the end of the second week, students have to defend their written Plan of Work. The course text is in English, whereas the course itself will be delivered in Dutch.

COMPULSORY TEXTBOOKS Zielhuis GA, Heydendael PHJM, Maltha JC, van Riel PLCM. Handleiding Medisch-wetenschappelijk Onderzoek. Zesde herziene druk. Amsterdam: Elsevier Gezondheidszorg, 2010, pp. 220.

Petrie A, Sabin C. Medical Statistics at a Glance. Oxford: Blackwell Publishing Ltd, 2009, 3rd edt.

Fletcher RW, Fletcher SW. Clinical Epidemiology: The Essentials. Philadelphia: Lippincott Williams & Wilkins, 2005, 4th edt.

14

ELECTIVE TEXTBOOKS ON SPSS: de Vocht A. Basishandboek SPSS 20.Eerste druk. Utrecht: Bijleveld Press, 2012, pp. 255. QUALITATIVE STUDIES: Lucassen PLBJ, Olde Hartman TC. Kwalitatief onderzoek. Praktische methoden voor de medische praktijk. Bohn Satfleu van Loghem. Houten 2007, pp. META-ANALYSIS: Egger m, Smith GD, Altman DG. Systematic reviews in health care. Meta-analysis in context. London: BMJ Publishing Group, 2001, 2nd edt.

Lecture 1: PROFILE OF CKO-9 What’s it all about? In this kick-off lecture we get acquainted with each other and consider the aim of the RTI: namely to acquire knowledge. Before starting the internship, you have to develop a reasoned Plan of Work. How? In this lecture, we present a structured approach that will convert your enthusiasm and curiosity for your research project into an informative summary accompanied by an appealing title. In the next two weeks, you will refresh your general research skills in pairs and peer-review your work in seminar groups. Finally you will write and defend your own Plan of Work.

INTRODUCING YOUR RESEARCH SUBJECT The main part of the exercise lasts about twenty minutes (for groups of two to three students) and gives you the chance to find out a little about yourself and your partner through some 'what-will-the-project-be-about' questions. You work together to develop a précis of your research project as well as creating an appealing and appropriate title. You can present both in class, and you can give each other (positive) feedback on your proposals. Simple questions to help 'break the ice' are, for example, those about your research project, its title, the medical topic, where it’s conducted, the institute, or what your interests are. After twenty minutes, one or two of you will be requested to present your project: title, type of research, and most importantly, the précis. It is also important to think about the kind of research type (see next page) Tips! - Demonstrate the value of your research to others. - Make your title active. - Show that you are committed to your research project. Learning objective Students can give a lively and interesting précis of their internship project. Précis: (Longman Dictionary) Fr, a shortened form of a speech or piece of writing, giving only the main points.

15

Type of research Medical research is defined by its object and its methodology. It can be divided into four basic and distinct categories, each with a different kind of study object, design, data collection, analysis and reporting. - The first type is diagnosis (e.g. developing new tests). This kind of study can be presented, for instance,

with outcome measures like Sensitivity & Specificity and analysed by using a ROC-analysis. - The second is treatment (comprising medical intervention in general and health care research). A

survival analysis is useful – for example presented as the median of 5-years survival etc. - The third type is aetiology (pathogenesis, how does disease develop). A specific tool in this case is for

instance the use of a case-control design, presented with odds ratios. - The fourth addresses prognosis of disease (how does disease progress, what’s the mechanism). Making

a Clinical prediction rule is for instance the possible solution here, with estimation of the risk for a certain outcome.

Thanks to developments in techniques in biomedicine, new categories are continuously being added to this list. For instance, the internist Alvan Feinstein added ‘clinimetrics’, but in essence this is a tool, technique or measurement instrument and not a special medical research category. Biomarkers, computer-aided cytological testing, measurement research (correlation), imaging, and genetic measurements have also been added. Some methodologists even have added EBM, meta-analysis, biostatistics, bioinformatics, library science, web-based questionnaires and qualitative studies to the list of research methods. In essence, all of these new methods can be brought back to one of the four research topics described above. Question Does this taxonomy of types of medical research objects and research methods or techniques make any sense to you? What kind of research would you like to get involved in? We’d love to hear where your research fits in. Collocation of medical domains with methodology Which methodology in B is commonly applied in the medical domains of A? Match a methodology in B to a type of research methodology in A. There may be more than one possibility. For instance i with 1.3 etc. If you don’t recognize some of the items, use Wikipedia as a quick and dirty method and take this for granted. (See also in Fletcher, Clinical Epidemiology, The Essentials Table 12.1 Matching the Strongest Research designs to Clinical Questions). B

i. qualitative study A ii. evidence-based medicine 1 diagnosis iii. clinimetrics 2 medical intervention: treatment, prevention iv. biochemistry 3 etiology, pathogenesis v. clinical psychology 4 prognosis vi. pilot study.

vii. polygenic approach

viii. survey

16

Module 1: RATIONALE OF THE RESEARCH Is my research worth doing? There must be a valid reason for launching a scientific study: a question that has been raised by clinical practice, (applied) scientific researchers, or society and that needs to be addressed. Both for yourself and for all other players in practice, it is vital that the motivation of why the research should take place is stated clearly and concisely (remember the word précis). If you can do this, you will find that you will be able to communicate the content and importance of your research to peers. Learning Objective Students are able to demonstrate that their research is worthwhile and relevant. The information should include: what their project is about, why it should be conducted (the motivation, either from a scientific or societal viewpoint), why and for whom this medical knowledge is important, and what the expected impact of the research outcome will be. Assignment

1. You are requested to study the textbook by Zielhuis et al, Chapters 1 and 2 until step 4 on page 33.

2. In order to prepare for the assignment, take Quick Test 1, which can be found on Blackboard. The test contains thirteen multiple choice questions. It highlights, at an introductory level, concepts like rationale, relevance and reflections on doing research, as well as type of medical research. All 10 Modules of the CKO-9 course will succeed with a Quick Tests after reading chapters from Zielhuis et al.

3. After reading the text from Zielhuis et al, you should be able to answer questions a – d.

a. Which function does medical research have? b. Research methodology consists of four types of criteria. What do ‘transparency’, ‘validity’,

‘generalizability’ and ‘precision’ refer to? c. Read the twelve steps in the tracking of medical research and think about which step best

fits the 10 modular learning components of CKO-9 (see Table of Contents, Chapter 3, and all learning objectives of the ten Modules).

d. What’s the difference between your ‘rationale of research’ and ‘rational research’? Why is your rationale rational?

4. Look at the title (4 words) and the précis (86 words) about chocolate (in the box below) and answer

the questions e- g. Précis: The temptations of chocolate Based on scientific research, consumption of flavenoid containing products like green tea,

red wine, etc. seem to be associated with a substantial reduction in the risk of cardio-metabolic disorders. There are studies suggesting that consumption of dark chocolate, containing flavonole, has a positive influence on human health in particular by antihypertensive, anti-inflammatory, activation of nitric oxide and other beneficial effects. We want to carry out a study to evaluate whether an intervention of daily consumption of dark chocolate will lower the blood pressure in normotensive persons.

e. Is the title appropriate? Why, why not? f. What will this study be about?

17

g. Is there a rationale to do this research? To get going we will show you the so-called BMJ boxes. So far we know, the British Medical Journal, BMJ was the first journal where authors are required to accompany their article with some statements about the study. Please see the examples below.

WHAT THIS STUDY ADDS (chocolate and blood pressure) Consumption of dark chocolate, containing cocoa and a richness of flavonoles, will have a

beneficial effect on blood pressure by lowering the systolic and diastolic pressure with 5-10 mm Hg.

The diminishing effect for consumption of milk chocolate (more cocoa butter and less cocoa) and

white chocolate (no cocoa) indicate a dose-response relation of cocoa in the reduction of blood pressure.

5. Now it’s time for your research. Write an appropriate title for your research in 15 words. Complete your précis in 75 words. Remember, a précis should cover relevance and content of your research altogether. Then, answer the question “What’s the rationale of your research?” To start up with the rationale, make a BMJ-box for your study, providing 3 statements about “What your study will (probably) add”. Write your rationale in 220-260 words. These three elements will form part of your Plan of Work (to be completed in Module 10). Tips! Tip 1 - Before you start to write, you should spend at least 15 minutes planning your writing. Your answers should be well organised with clear linking between sentences and paragraphs. Remember from your Academic Writing Course: your written work should have a clear beginning, middle and an end, an appropriate style and layout, and be written legibly as you will be assessed by the target reader(s) of your Plan of Work. The rationale has to be convincing in order to write a powerful Plan of Work. Information should be relevant and your rationale should be concise: “Less is more”, or “Kort is Prachtig”, or “In der Beschränkung zeigt sich der Meister” or “La limitación se muestra por el maestro” etc. Tip 2 - Start writing your Plan of Work now. This take-home message was launched by Professor Jan Vandenbroucke in 1989. Read his text in Vandenbroucke JP . Een recept voor een klinisch-wetenschappelijk leeronderzoek. Ned Tijdschr Geneeskd 1989; 133: 34-8 (especially pages 37-38 “Vroeg opschrijven”. Tip 3 - Another Tip and Take-home message! If you have problems writing good English sentences, please take a look at the Academic Phrase bank (http://www.phrasebank.manchester.ac.uk/index.htm).

18

Module 2: FIRST APPRAISAL OF THE LITERATURE What have the ‘others’ been doing? In the previous module, you discussed the motive or rationale of your research project and internship. A literature search is a logical consequence of developing your research motive: you need to understand what others are doing and what the state-of-the-art is in this field. Learning objective By doing a first literature search, students are able to select the three most important papers which are related to their project. They can present the ‘state-of-the-art’ of the research problem and can criticise these papers. Assignment

1. First, read Chapter 4 on “Literatuur en literatuuronderzoek” from the textbook by Zielhuis et al. which focuses on the words and phrases in the box below.

Critical appraisal – MeSH – NTvG uitgebreid zoeken – State-of-the-art medical knowledge

2. After reading Chapter 4 from Zielhuis et al, you should complete the Quick Test No. 2

Here another five questions a – e about doing a literature search. a. If you want to speed up your literature search, is Google of benefit? Why? Why not? b. PICO is especially connected with one of the four types of medical activities. Which? c. As in any other science, researchers like to structure their publications. In evidence-based

medicine these are registered in primary, secondary and tertiary publications. Give examples of each kind of these papers.

d. Will the Science Citation Index be of value to you? e. To become familiar with your research object (and if Dutch is your (near) native language), try: “Google zoeken: ntvg.nl - Nederlands Tijdschrift voor Geneeskunde| - Uitgebreid zoeken”.

3. We would now like to learn something about your ability to search for literature, to raid libraries

and delve into electronic archives when doing research. So, select the three most relevant articles in relation to the rationale of your study and download the papers. Describe your search strategy, and make a reference list in the style of the references in the box below. For the three references you have selected, extract the relevant information for your research and make a well-ordered table. Tabulate information like study units or population, design, sample size, measurement instruments, determinant, outcome and, importantly, all notes you made on strong and weak aspects of the studies.

Example: Temptations of Chocolate on next page

19

...

Search for evidence – Between May and August 2011 we carried out a search for the most recently published studies to identify the main elements for our research question on the influence of dark chocolate on blood pressure. We used two databases for the literature search: PubMed and Web of Knowledge. The terms chocolate, chocolate* [Title/Abstract], cocoa [MesH], cocoa* [Title/Abstract] were applied in combination with blood pressure [MeSH], blood pressure* [Title/Abstract].The bibliographies of selected references were also searched. More than 40 articles were found from the start of 2003 to mid 2011, of which 5 met the specified inclusion criteria: [add your specifications here] normotensive healthy persons. Eventually, we selected three articles for the first critical appraisal in which we extracted technical information on populations, study design, numbers of subjects, measure instruments, etc. The selection was based on [add your selection criteria here] two systematic reviews and one study on working of polyphenoles which was excluded in the meta analysis. In Table 1, we made an overview of the main findings and notes on my literature appraisal. Table 1. Title [part of it] No. Author, yr Design Main findings Criticisms Recommendations own research 1. Ried, 2010 Meta-analysis …………………… …………… …………………… …………………… …………… …………………… …………………… …………… …………………… 2. etc.

References 1. Ried K, Sullivan T, Fakler P, Frank OR, Stocks NP. Does chocolate reduce blood pressure? A meta-

analysis. BMC Med. 2010;8:39.

2. Desch S, Schmidt J, Kobler D, Sonnabend M, Eitel I, Sareban M, Rahimi K, Schuler G, Thiele H. Effect of cocoa products on blood pressure: systematic review and meta-analysis. Am J Hypertens. 2010;23: 97-103.

3. Cienfuegos-Jovellanos E, Quiñones Mdel M, Muguerza B, Moulay L, Miguel M, Aleixandre A. Antihypertensive effect of a polyphenol-rich cocoa powder industrially processed to preserve the original flavonoids of the cocoa beans. J Agric Food Chem. 2009;57:6156-62.

4. To get going we will first show you the BMJ box “What is already known on this topic” on our chocolate-blood pressure item. Then you are requested to make a box like this for your own study.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Studies on Kuna Indians reveal that high cacao intake is correlated with low prevalence in cardio metabolic disorders. Migration and changes in diet suggest cacao as an explanation for this phenomenon and not a genetic predetermination.

High intake of cacao polyphenols lowers the risk of cardio-metabolic disorders. Some fruit (berries grapes

etc.), red wine, green tea and cacao products are Ployphenol rich. In vitro research suggests that cacao in dark chocolate has an influence on vascular endothelial function

and activation of nitric oxide.

20

5. Finally, guided by the table you made under instruction 3 (which in this stage is based on the three

articles you have selected), write your critical appraisal (i.e. strong and weak points inclusive) in 220-260 words in an appropriate scientific style. Start with state-of-the-art evidence (step 4, Chapter 4, textbook by Zielhuis et al). Include all information which is worthwhile and relevant for your research! It will help you when formulating your research objective and research question (Module 3). Tip! For search strategies, see BlackBoard Hulp bij literatuuronderzoek with MeSH-terms. In case of a systematic review, meta-analysis, qualitative study or any problems with your search, you may contact our Medical Library as described on BlackBoard, or ask your teachers.

21

Module 3: SPECIFYING THE RESEARCH QUESTION What exactly will I be doing? In this module, we will deal with the link between the state-of-the-art for the literature and the rationale of your study which will result in your Research Question. An idea for scientific research does not just ‘jump out of the blue’; it is often generated by an observation or by the results of previous research. This leads to a preliminary formulation of the Research Question, which is sometimes converted into a hypothesis and tested by the researcher with observations in practice. This is called ‘empirical’ research. Based on the formulation of your Research Question, you determine what the best form of research is: will it be a pilot study, a qualitative study, or a full-blown quantitative research programme? Learning objective Students can translate the scientific objective into one (yes, 1!) well-defined Research Question and/or testable hypothesis. Furthermore, they can relate the Research Question to its outcome, its determinant, the timing of determinant and outcome and the relevant domain. Assignment

1. Today’s theory revisits some texts you have read before.But firstly, please consider page 35, section “De beperking van de vraagstelling” in JP Vandenbroucke’s Een recept voor een klinisch-wetenschappelijk leeronderzoek. Ned Tijdschr Geneeskd 1989; 133: 34-8. Reread Chapter 2, step 1-3 in Zielhuis et al.

2. Complete Quick Test 3 that consists of thirteen questions on the theme of specifying a Research Question.

Now, focus on the 11-word Research Question “Does consumption of chocolate have influence on IMT and blood pressure?” Apply the seven assessment criteria (Table 2.2 in Zielhuis et al.) to the Research Question in order to demonstrate the (in)-appropriateness of the scientific question by answering questions a - c. a. What does the RQ demonstrate about the criteria of unambiguity, singularity, simplicity

and formulation? b. Is the RQ scientifically relevant? How do you know this research is worthwhile doing

(what did you say in module 1)? c. Are determinant and outcome specified?

3. Read the extract below which is about particularistic (or factual knowledge) and abstract

knowledge. It has been paraphrased from Bouter – van Dongen – Zielhuis’ textbook on Theoretical Epidemiology (2009).

d. Does the RQ on chocolate and blood pressure address particularistic or abstract knowledge?

e. What do you think about (determinant-outcome) timing and domain?

PARTICULARISTIC VERSUS ABSTRACT KNOWLEDGE Research questions describe issues that either refer to particularistic or abstract knowledge. Particularistic research intends to affirm propositions about a specified relation between determinants and outcomes under given circumstances of time and place. Most of these studies are used to map health conditions (disease frequency in relation to, e.g., age, gender, SES), use of medicines in health surveys, and evaluations or bench marking of medical interventions. (See next page)

22

This kind of description is of value to support policy makers, doctors in their daily medical decision making, and to direct future research. An example of particularistic research is an investigation of a Salmonella outbreak in a nursing home.

In contrast, abstract knowledge is devoid of content that is time-place dependent. Abstract knowledge

is expected to be generalizable to all human-kind (even to other species). With this kind of research we aim to study general, abstract, scientific knowledge on (causal) determinant-outcome associations or disease mechanisms, which are seldom descriptive ‘by nature’. Think about the question whether alcohol (abuse) is a causal factor for liver cirrhosis. The causal link between alcohol abuse and liver cirrhosis holds true throughout the world; it is not scientifically restricted to De Achterhoek, the Netherlands in 1977, or the Castro-Castro Prison in Peru in 2011.

Once a link like this has been scientifically established, next steps in medical research will be aimed at gaining further insights into types of alcohol abuse, types of users, and perhaps type of cirrhosis as well.

NB: For particularistic research problems, the specific features (species, time and place) of the study must be reflected in the research question. For abstract research problems, these features are usually not part of the Research Question, but described in the design part of the study.

Adapted from: Bouter LM, van Dongen MCJM, Zielhuis GA. Epidemiologisch onderzoek. Opzet en interpretatie. Houten: Bohn Stafleu van Loghum; 2010: pp 81-2.

4. Application task - Look at the Précis you made in Module 1. What is your domain? Does it concern a specific target population or an abstract general situation? Conceptualize and write your Research Question in an appropriate style and number of words.

5. You will need all the products of the tasks in Modules 1-3 in Seminar 1. Are you happy with them? Did you succeed in writing the outline? Seminar 1 is a small class where you and your fellow medical students together with a specialised lecturer / researcher will be meeting to peer-review your rationale-based Research Question and to improve your approach to developing your Plan of Work. Read the main text for Seminar 1 carefully to get a general idea of the programme.

23

Seminar 1: A FIRST OUTLINE OF YOUR RESEARCH Is it relevant? To stress the focus of your research topic, you are going to sum up rationale of your study and state-of-the-art of the literature in so-called British Medical Journal (BMJ)-boxes: What is already known? And ‘what will my study add? In addition, the most important issue in designing your research is to come up with a crystal clear Research Question (RQ). Your RQ must meet all the requirements of scientifically feasibility completeness, uniformity, singularity, scientific relevance, simplicity, and correct formulation. In the peer review session of this seminar meeting you will be asked if your RQ meets these requirements. Learning objective Students can outline their proposed study conceptually and defend its relevance to fellow students, leading to an appropriate research question (RQ). Based on their RQ, the students are able to look ahead to designing the data collection, the analysis and its provisional embedding in a societal and ethical context. These will be discussed in Modules 4-8. Workgroup process

1. First we start with a moment for reflection: What have you learnt most from Modules 1-3 on Rationale, Literature and the Research Question? Gather all your learning experiences, pieces of information, and evidence from the assignments so far. Remember Vandenbroucke’s adage “Vroeg opschrijven”. In how far have you succeeded in writing the outline (background or rationale, aim of the study etc.) of your Plan of Work (major issue of Module 10)?

2. Now we will evaluate your experience with the quick tests and the modules you have taken so far.

Do you understand the theoretical components? This evaluation will be supported by a presentation on the effects of chocolate intake on blood pressure in adults.

3. Then, in small groups of 3 students, review the outline of your study using the précis you made in Module 1. Highlight the state-of-the-art of your research topic (which is a product of Module 2) answering the question “What is already known on this topic” in three concise statements. Then, present the rationale of your research topic (also a product of Module 1) by means of three statements in a separate box entitled “What does my study add”. Keep both preliminary BMJ-boxes in stock for Module 10 when, amongst all the other issues, your Plan of Work approaches its final stage.

4. Then still within the peer review format, scrutinize your research question based on the criteria in Table 2.2 in Zielhuis’ textbook. You now have to submit your RQ to the lecturer. You will receive feedback in the following seminar.

5. During the last part of the Seminar 1 the group will prepare for the following modules on

designing your data collection.

24

Module 4: PATHWAYS TO EFFICIENT STUDY DESIGN Designing the yellow brick road The next three modules are on data collection. Data collection has two forms, namely collecting in the conceptual dimension and in the operational dimension. The conceptual dimension is about how to design your data collection efficiently and is addressed in Module 4. The operational dimension of data collection concerns the study subjects to be measured; this is addressed in Modules 5 and 6. In Seminar 1 you finalized your Research Question. During this process, you have unwittingly thought through the research typology and methodology. Types of methodology are pilot studies, descriptive studies, and explorative and hypothesis testing. To harvest scientific information for your Research Question you have to shape or form the data to be collected. A template can help you to gather the empirical facts. There are two standard templates: a survey-like study design and an experimental investigation. Both forms are applied to help you ‘mould’ your Research Question to arrive at a representative, valid, reproducible and reliable answer for your target reader. Learning objective Students demonstrate that their study makes sense; that it will produce a valid answer to the Research Question. They are able to do so by selecting the optimal study design which will allow them to effectively obtain empirical data from the study domain. Assignment

1. In what way will your study conceptually be set up? The title of Module 4 “Pathways to Efficient Study Design” suggests two ways: the practical and the nominal way. In a practical pathway you ask the question, “How can I efficiently harvest the information?” In the nominal pathway you choose a study design from a list of types such as experimental studies, observational studies, in vitro studies, etc. Now read Chapter 2 (steps 4 - 8) and Chapter 3 (Vormen van onderzoek) of Zielhuis’ textbook.

2. To become further acquainted with the topic, match the concepts in the box below to the thirteen

questions in Quick Test 4.

Design (noun) - Design (verb) – Experiment - Pilot – ‘Trial and Error’

3. Please, answer the extra questions a – f. a. Describing the overall design consists of five elements. Which? b. How do you sketch or model a study design? See for example the sketch for the chocolate study on the next page. What are the essential elements? c. What’s the difference between an experiment and a descriptive / survey-like study? d. To what extent is tissue engineering a type of research? e. Do you happen to know what, in medicine, is meant by a translational study? f. Assess the appropriateness of the designs chosen for the question whether chocolate intake does lower blood pressure? See the textbox below with three researchers and their proposals.

25

4. Based on the Précis and Research Question of your project, select a suitable design (use Zielhuis, Step 4, p. 33 and Chapter 3) and sketch the study model (Fig. 2.1 and 2.2 in Zielhuis, p. 37). Write and motivate the design of your study project in 75 words, as shown in Zielhuis’ Examples 2.14 – 2.20. Tip! Still puzzled and fascinated about your study design? Then read: Beurskens AJHM, de Vet HCW, Kant IJ. Dwalingen in de methodologie. Pilotonderzoeken: zin en onzin. Ned Tijdschr Geneeskd 1998; 142: 2142-5. For matching the strongest research designs to Clinical Questions the box below may help.

Temptations of chocolate Researcher1 (Born in Belgium and now working at University Hospital Maastricht) Study design: Survey, comparison systolic and diastolic blood pressure in medical students in Brussels and Maastricht. Researcher 2 (Researcher appreciating the article of Grassi et al. Am J Clin Nutr 2005;81:611– 4 and decided on a “me too” study) Study design: Cross-over design, comparison of blood pressure (systolic and diastolic) before and after consumption of dark chocolate. Researcher 3 (PhD student, just followed a course on randomised clinical trials) Study design: Randomised trial, random allocation of different volumes of cacao to a group of medical students.

Question Design Diagnosis Survey study Prevalence Survey study Incidence Cohort study Risk Cohort study Case control study Prognosis Cohort study Treatment Clinical trial Prevention Clinical trial Cause Cohort study

Case control study

26

Module 5: MEASUREMENT OF KEY VARIABLES What am I measuring? Medical research is founded on observations: measurements. To perform these measurements, one needs instruments. Measuring instruments include not only things like rulers or devices but also questionnaires, visual inspections, chemical or laboratory analyses, and registrations. For both dependent, independent variables and for all other variables, measuring instruments are necessary. But can you really rely on your instruments? What happens if the reliability is not guaranteed? Learning objective Students can describe how they measure the key variables of the Research Question, and which instruments they will use. By doing so, the students show that they understand the (potential) magnitude of measurement errors and their effect on the research result. Assignment

1. Now you’re on the verge of taking the fifth Quick Self-test. Quick Test 5 addresses seemingly unrelated topics, but they are all about with variation and measurement. They are about the constructs: ‘variable’, ‘random vs. systematic measurement error’ and ‘inter-/intra-rater variability’.

2. Module 5 theory focuses on the qualities of measurement instruments. Read Chapter 7 in Zielhuis’ book. Then answer questions a – e. a. Is ‘taking medical history’ a type of measurement instrument? What about the accuracy? And,

how can you learn to quantify its accuracy? b. Have you ever worked with depression scales? Or the APGAR test? How many items were

involved? Do you agree or disagree with the number of items needed to accurately measure somebody’s health status: The more, the better?

c. Describe two types of measurement research. d. What’s the purpose of the kappa statistic? e. Is measurement research an issue in your project? If so, see the textbox below.

TO AGREE TO DISAGREE ON MEASUREMENTS

- De Vet HCW, Beurskens AJHM. Reproduceerbaarheid van metingen. Ned Tijdschr Geneeskd 1998; 142: 2040-3.

- Pouwer F, van der Ploeg HM, Bramsen I. Bias door vragenlijsten. Ned Tijdschr Geneeskd 1998; 142: 1556-8.

- Van Breukelen GJP, Berger MPF. Hoe te komen tot een gewogen totaalscore van afzonderlijke scores op een vragenlijst. Ned Tijdschr Geneeskd 2000; 144: 1580-4.

- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of

clinical measurement. Lancet 1986; i: 307-10.

- Petrie A, Sabin C. Medical Statistics at a Glance. Oxford: Blackwell Publishing Ltd, 2005, 2nd edt. Chapter 39: Assessing agreement.

- Fletcher RW, Fletcher SW. Clinical Epidemiology: The Essentials. Philadelphia: Lippincott Williams & Wilkins, 2005, 4th edt. Chapter 2: Abnormality.

27

3. At least three different kinds of variables can be distinguished in every study (see Figure 2.1 in Zielhuis et al, Chapter 2, page 37). Look how these have been registered in Table 1, with four aspects A – D for each kind of variable. Table 1. Example - Register and order the main variables of your research. Three examples are presented.

CENTRAL / KEY VARIABLE

DETERMINANT OUTCOME FURTHER CHARACTERISTICS (confounding factor, intermediate, or domain variable)

VARIABLE*

e.g. AGE e.g. BLOOD PRESSURE e.g. PHYSICAL FITNESS

A - Conceptual scale B - Measurement scale C - Measurement instrument

D - Sensitivity to measurement error

A - young, middle aged, old

B - year of age

C – questionnaire D - not at issue.

A - hypertension, yes/no

B - systolic blood pressure, mmHg

C - automated Sphygmo type® D – reliability being tested yet.

A - poor, borderline, sufficient, good, excellent

B - predicted VO2max,

ml/min/kg

C - Åstrand-Rhyming sub maximal cycle exercise test D - reproducibility study published by JT Groothuis et al. (in press)

*) Something which represents something (such as temperature) which can vary in quantity or size

For your study, list the key variables when measuring determinant, outcome and further relevant characteristics in Table 2. Complete the list with useful information regarding question marks I-V.

I. Indicate the likelihood (low, medium, high) of the occurrence of random measurement errors.

II. And what about the occurrence of systematic errors? III. What makes you think that the likelihood of making measurement errors will be great or

small? IV. What can you do to reduce the potential of making serious errors of measurement? V. How do you judge the impact of the errors on the study result?

28

Table 2. My project - Main variables of my research project.

CENTRAL / KEY CONCEPT DETERMINANT OUTCOME FURTHER CHARACTERISTICS (confounding factor, intermediate, domain)

MAIN VARIABLE

A - Conceptual scale B - Measurement scale C - Measurement instrument D - Sensitivity to measurement error

I - random measurement error

II - systematic errors

III - likelihood of occurrences of measurement errors

IV - how to restrict errors

V - impact on study

29

Module 6: CONSIDERING THE STUDY SUBJECTS Who and what do I need? In Module 4 you decided what your study population would be: humans, fruit flies, or (bio)materials. Now you will focus on different aspects of this population. For instance, how many subjects do you need for your study? This question relates to the accuracy with which you want to estimate the outcome measures of your research object. Consider study populations or research subjects in a broad perspective, for instance, “the people to be interviewed” in case of a qualitative research, or a specific group of articles about a subject in case of a systematic review or meta-analysis. Learning objective Students can qualify their choice of research subjects (representative vs. selective) for the study. They show how they determine the number of subjects to be used and how they will be recruited. Additionally, they can demonstrate the effect of sample size on the confidence interval of an effect parameter. Assignment

1. The theory in this module is preceded by Quick Test 6, which consists of 13 two/three-option multiple-choice questions on the concepts below. Which conceptual qualities are you confident of?

1 study unit 2 numbers needed to study 3 random sample 4 representativeness 5 selective study population 6 type of sampling

2. Developing your skills – As the CKO-9 course develops, you are becoming familiar with the fact that you always have to motivate your decisions at all stages of your research. Today’s stage is about study subjects and sample size. How to be precise with minimal effort? Before reading about formulas for sample size calculation in Zielhuis, let’s take a look at the reasoning behind these in the next boxes.

A population represents the entire group of individuals in whom we are interested for the value of a certain parameter. We usually cannot study the entire population, so we have to take a sample instead. After measuring a number (size = n) of the subjects, we can calculate the mean. To have an idea about the variability of our measurements, we can take many repeated samples of size n from the population and look at the distribution of all the means (see Chapter 10 in Petrie and Sabin, Medical Statistics at a Glance). This variability of the distribution of the mean is known as the Standard Error of the Mean (SEM) and can be calculated by SEM = SD/√n in which SD is the standard deviation of the sample and n the sample size. With this SEM we can calculate the 95% Confidence Interval (CI) of the mean. In a formula: 95% CI: [sample mean - 1.96 * SEM, sample mean + 1.96 * SEM]. This factor of 1.96 (Zα/2) is because, in the theoretical normal distribution, numbers more than 1.96 standard deviations away from their mean occur with a frequency of less than 2.5 % (= α/2). See Example 1 below for some practice with SEMs and calculating 95% CIs.

30

The temptations of Chocolate Example 1 Study design: ecological study, comparison of systolic and diastolic blood pressure amongst medical students in the cities of Brussels and Maastricht. Findings: The mean systolic blood pressure was 130 mm Hg (SD 10) for 16 medical students in Brussels. In Maastricht, for the same number of students, the mean systolic blood pressure was 135 (SD 10). Now we can calculate the SEM: 10/√16 = 2.5 mmHg for both groups. The 95% CI for the Brussels group is about [130 – 5 , 130 + 5] = [125 - 135]. For Maastricht the outcome is 135 mm Hg, 95% CI [130 - 140]. Further questions: - Notice, there is an overlap between the 95% CIs of the mean blood pressure in Brussels and in Maastricht. What is your conclusion? Are both means significantly different? - Can you discern the difference between the means in the two cities by calculating the 95% CIs in case 100 medical students were measured for each group? What is your conclusion now? Of course the number of subjects also influences the precision of a proportion (e.g. the sensitivity of a test (%). The calculation of the standard error (here of the Proportion, SEP) is somewhat different. The formula for this SEP = √ (p*(100%-p)/n, where p is the proportion, 1-p is the complement of p and n the sample size. For calculating the 95% CI of the proportion the same formula can be used: [proportion - 1.96 * SEP, proportion + 1.96 * SEP].See example 2 for calculating SEPs and 95% CIs. Example 2 Study design: before/after design, comparison of blood pressure (systolic and diastolic) before and after consumption of dark chocolate in 15 subjects. Findings: See figure for the systolic blood pressure before and after two week’s consumption of dark chocolate. In 13 of the 15 medical students (87%) the systolic blood pressure showed a decrease of more than 3 mm Hg. Questions: - How ‘precise’ is our result of 87% of students with reduced

blood pressure? Calculating the SEP: (√ (p*(1-p)/n = √ (87 %*( 100 - 87% / n= 8.7 Now the 95% CI: CI= (87% - 1.96*8.7%, 87% + 1.96*8.7%) = (70% - 100%). Is this a reasonable outcome for you? Why not fill out 104% as the upper boundary? - We studied another group of students in which white chocolate was prescribed.

Suppose we found a reduced blood pressure in 53% of the students i.e. 8 out of 15 or 53 out of 100 students. Is this outcome (53%) for white chocolate different from dark chocolate in a study group of 15 students? And in case in both groups 100 students were studied?

3. For further theoretical considerations, read Chapter 6 ‘steekproefomvang en powerberekening’ in Zielhuis’ textbook. This time there are no questions to answer, just mark all sections which are directly beneficial for your research and can easily be applied. If you face difficulties with the theory, discuss your difficulties in Seminar 2.

31

4. In this task you are going to write around 250 words on the number of units needed in your study, and how they will be recruited. The task focus is on the reasoning behind the number, rather than the fact that your supervisor told you that you needed ‘so many subjects’. Do you really need to work with the whole group of study units, or should you give priority to a sample - random or selective? To help you, look back at the articles you selected in Module 2 on ‘literature’. What numbers did they use and how were these motivated? Requirement! In your Plan of Work you will need a numerical motivation for your study size. What are the three main qualitative reasons behind reaching this point? (Think about time, effort, and your role in the research process). Tips! In your Bachelor of Medicine period you studied biostatistics using the book by Petrie & Sabin “Medical Statistics at a Glance”. In the 2nd edition (Oxford: Blackwell Publishing Ltd, 2005), Chapters 10 and 11 focus on quantitative population parameters like proportions and means. As you may have noticed, the precision of these estimates are of value for your numerical considerations on the number of subjects needed. Refresh your knowledge on 95% CI, (confidence intervals). Some like to cast their Research Question in terms of hypothesis testing: Is the outcome (mean or proportion) in group A different from that in group B? T-tests are commonly applied when comparing means; Chi-square tests [pronounce ‘chi’ as ‘kai’] are commonly applied when comparing proportions. For details of hypothesis testing and these statistical tests, consider Petri & Sabin.

5. Finally, prepare well for Seminar 2. Skim the main text of Seminar 2 for a general understanding of the theory and as preparation for peer review.

32

Seminar 2: DESIGNING YOUR DATA COLLECTION Learning objective Students can outline their study design to fellow students and justify the choices for the study units, key variables and measurement instruments. Furthermore, they can underpin the number of subjects needed and can present their expected research outcome using a graph and/or table. Seminar way of working As in Seminar 1, Seminar 2 includes three parts: review of the theory from Modules 4-6, peer review on ‘collecting the right data’, and reflection on the next modules.

1. First things first: are there any questions left after completing the Quick Tests and Modules 4-6? And having received all kinds of feedback on your RQ, is everything crystal clear to you and your peers? How many study subjects or units form part of your project?

2. During the peer review, which forms the main part of this seminar, you will discuss your choices of measurement instruments. Working in small groups, present your précis and discuss all items you have entered in Table 1 on the measurement of the main variables. Consider, choose and decide on the instruments which you think fit your RQ best. Note: Your decisions have bearing on the cost, time, effort and organisation of your further research preparations. In addition, your data analysis is at stake! Then, having decided on the most important determinant and outcome, take the plunge and sketch the association (recall Module 4 assignment 3 of the study model)! What numbers do you expect in your research, based on previous Modules? Following this, design an n x m table, representing the association between outcome and determinant. Sketch a Figure that illustrates your (expected) outcome. Don’t forget to give your Table and Figure a title and the appropriate legend.

3. During the last part of Seminar 2, we will ask you to think about where you are heading in this course. This is more than – ‘I’m just here to follow the course’: we would like you to consider aspects like: - points you like - points that you might do differently - the way we cooperate with peers - the learning and teaching materials. It is essential for all of you to look for ways that you can change / improve / vary your own learning and benefit from experiences.

4. After this moment of reflection on what you have done so far in this pre-internship preparation

course, we will look forward to what will happen in the second week. Read the textbox below and your CKO-9 course book.

Lecture 2: Philosophy of science; it’s influence on your research Module 7: Societal and ethical justification

Module 8: Designing the data analysis Module 9: Organisation and management summary Seminar 3: Practical organisation and numerical thoughts Computer lab: SPSS practicalities Module 10: Reporting your Plan of Work and management summary Seminar 4: Sales and communication

33

Lecture 2: PHILOSOPHY OF SCIENCE: INFLUENCING YOUR RESEARCH Thinking about things In this second lecture we will attempt to place your research project in the perspective of epistemology [Epistemol, what? Read the following textbox]. This is the field of philosophy that deals with aspects of how to learn about the object of research, thus the nature, scope and the limitations of scientific knowledge. Do you think that your research will result in the right answer (and... what is right?) and, is the right answer that what we want to know? And what powers influence and manipulate the choices to be made in your research? Learning objective Students can position their study in the field of epistemology and can evaluate the various influences on the dynamics of science on the outcomes of their own research.

Epistemology

Epistemology originates from the Greek words epistèmè and logos, meaning knowledge,

science

This interactive lecture on the theme of acquiring medical knowledge consists of three parts. Firstly, students (co-assistants) refresh their knowledge and understanding of philosophical perspectives on scientific medical knowledge. Secondly, basic understanding is acquired of the actual influences on the dynamics of science (conceptual, academic, economic, political) Thirdly, students will reflect on the role of (the latest) medical knowledge for future medical practice as a medical professional.

, and study of, respectively. It is the branch of philosophy concerned with the nature and

scope (limitations) of knowledge. It addresses the questions:

• What is knowledge?

• How is knowledge acquired?

• How do we know what we know?

• And is it all true?

Much of the debate in this field has focused on analyzing the nature of knowledge and how it

relates to connected notions such as truth, belief, and justification. It also deals with the means

of production of knowledge, as well as skepticism about different knowledge claims.

Many dictionary definitions may give the impression that epistemology is closely related to

critical thinking or, why not, critical appraisal: "the study or a theory of the nature and grounds

of knowledge especially with reference to its limits and validity" (Merriam-Webster's Online

Dictionary, 11th Edition).

34

35

To refresh your memory, review the textbox on schools of philosophy relevant to medical sciences:

Logical positivism, 1910 - 1950 Logical positivism (also known as logical empiricism and neo-positivism) is a stream of philosophy that combines empiricism—the idea that observational evidence is indispensable for knowledge—with a version of rationalism incorporating mathematical and logico-linguistic constructs and deductions of epistemology.

Critical rationalism, 1950 - 1975 Critical rationalists hold that scientific theories, and any other claims to knowledge, can and should be rationally criticized, and (if they have empirical content) can and should be subjected to tests that may falsify them. Thus claims to knowledge may be contrastively, normatively evaluated. They are either falsifiable and thus empirical (in a very broad sense), or not falsifiable and thus non-empirical.

Paradigm theory, 1975 - 1990 A paradigm shift (or revolutionary science) is, according to Thomas Kuhn in his influential book The Structure of Scientific Revolutions (1962), a change in the basic assumptions, or paradigms, within the ruling theory of science. It is in contrast to his idea of normal science. 

Social constructivism, 1990 - 2005 Social constructionism and social constructivism are sociological theories of knowledge that consider how social phenomena or objects of consciousness develop in social contexts. A social construction (social construct) is a concept or practice that is the construct (or artefact) of a particular group.

New pragmatism, 2005 – 2015 The central claim here is that organization of scientific studies needs to be fundamentally reshaped. This change is needed to provide room for ethics and to increase the relevance of research in society and clinical practice. It is argued that the new pragmatism provides critical resources for this change. Pragmatism is a particularly helpful approach to define the use of scientific knowledge in the social reality while it avoids entrenched epistemological distinctions that marginalize ethics and make research less useful.

To give your own study more flavour:

- Are there issues of integrity at stake in your project or the way in which it will be carried out? - Are the rivalling groups that deny the relevance of your research goals? - Does your research aim at the discovery of truth or knowledge, or merely the likelihood that things

are true? - Does your approach endeavour to be objective? - Nota bene: Scientific knowledge is often deducted from subjects, which means that it is inherently

subjective. - How is your project organized? Who is ultimately responsible? - Which interests from economics, society and politics are involved? - What is your ambition with your project? - Should society invest in your project? - Are the possible clinical results for helping patient theoretical, practical or ideological?

Module 7: SOCIETAL AND ETHICAL JUSTIFICATION Am I doing the right thing? The choice whether to finance a specific research project depends on the preferences of the funding agency. Media and interest groups can exert considerable social pressure to prioritize particular topics. Once the topic and the Research Question of a study have been decided on, interested parties have to refrain from interfering in the following steps of the research process until the study has been completed (no conflict of interests). Furthermore, conducting research involving humans is subject to increasingly complex legal requirements. This means that you, as a clinical researcher, are not only expected to be an expert in the field of research but are also expected to be fully informed about legislation and arrangements with supporting departments in order to carry out the research. All research has to be conducted in accordance with the Law for the Protection of Personal Data (WBP) and the Law for Agreement on Medical Treatment (WGBO). The manner in which these regulations are incorporated can be found in the Code of Conduct. In case of clinical research with medicinal products, the Good Clinical Practice guideline was developed to protect the (human) rights of a subject in a clinical trial. It also provides assurance regarding the safety and efficacy of the newly developed drugs. Similar requirements have been made for non-human research. Learning objective Students can argue to what extent their research proposal requires societal and moral justification (in view of legislation, Good Clinical Practice, and accreditation by the Ethics Committee). Furthermore, they can demonstrate that they have accounted for privacy matters applicable to the study subjects. Assignment

1. Whether or not you are dealing with humans, animals or biomaterials, it’s always good to have a clear idea about the human rights regarding the data and information you need for your research. Start reading Chapter 10 Bescherming van Onderzoeksdeelnemers in Zielhuis’ research manual. Don’t forget to visit the websites on Codes Goed Gedrag, Goed Gebruik (see www.federa.org or Zielhuis pp 210-211for some other sites).

2. Quick Test 7 introduces the changes in science, society, philosophy and ethics over the last decades. It looks at the concepts of: - conflict of interest - informed consent - proportionality - subsidiarity - and abbreviations like CMO, METC, and WMO.

3. Please, address the following issues a and b. a. We would like you to reflect on your own or to talk to your partner about the email message in the textbox. It’s about efficient information exchange. Discuss whether it is a correct way of working and why. Otherwise, what might be an alternative solution?

36

Email: subject information Forward: list of records Hello Hans, I’m sorry, I have to cancel tomorrow’s delivery. No – I haven’t run off on holiday – but today our management team announced unexpectedly that we will be starting tomorrow instead of next week with the replacement of our lab department. That means I have scanned and digitized all test outcomes and linked these with information from the medical records. I’ve filed them electronically for you and ordered the records by patient name – so you will be able to screen for the right patient for your research. My apologies for canceling at such short notice! Kind regards, Julia PS: Our ICT department can generate the most up-to-date copy of the list, if convenient.

b. Do research proposals in general, but also research internships, need CMO accordance or

approval by relevant authorities? Read the next internship proposal (for a ‘me-too’ study) and decide whether it requires approval from a relevant review committee.

RESEARCH INTERNSHIP PROPOSAL

Based on “Gait speed and survival in older adults” by Stephanie Studenski,

MD, MPH, et al. JAMA 2011; 305: 50-58. Proposal: Development of an innovative WalkingTest Gordon Cooper developed his well-known fitness test for military personnel half a century ago: the Cooper-test, running for 12 minutes. In older adults a walking test is more appropriate. The elderly, especially, vary in life expectancy and physical fitness. Both aspects should be considered when setting goals for healthy ageing. In the last few decades, a number of papers were published about walking or gait speed tests, although most of the time, gold-standard tests were lacking. The purpose of this research internship is to develop a new test covering a 400m long-distance walk, to be conducted outdoors. The 6-minute Åstrand-rhyming sub maximal cycle exercise test provides the gold-standard physical condition. The setting, environmental conditions, number of subjects (age, gender, and multi-morbidity), recruitment, physical exams and lab are all up to you, as you are the clinimetric data-analyst. So, will your innovative walking test prove to be a useful predictor of physical fitness for all of our 55+ adults?

3. During each module, we will be asking you to think about what the theory means for your research project:

- So, is your research ‘CMO-compulsory’? - Does it concern biomaterials and/or humans; and, in what way? - Will it use medical records, questionnaires, Internet information?

37

- Who’s the owner of the collected data? How are these data protected? - Do you anticipate any conflict of interest? Why? And why not?

Make notes on these issues. These notes are not only for you, the trainee, but also for your peers, team members, and supervisor. You will have to compile them in your Plan of Work (Module 10).

38

Module 8: DESIGNING YOUR DATA ANALYSIS Getting the right result! Every scientific question can be cast in a general research model. These models consist of many kinds of variables. The determinant and outcome variables are the most important when answering the research question. Many research questions in medical science are about an association between these variables. All variables must be (made) measurable, before appropriate measuring instruments are selected to carry out the measurements (Module 5 on measurements of key variables). In order to collect the data, you first need to design the data file with an accompanying codebook. Designing and performing data analysis are tasks preferably done together with your supervisor (and sometimes with experts in statistics) to preclude the possibility of you getting completely lost in your data file. Wrong interpretation of the output, might lead to contradictory interpretation of the results of the analyses. If you have designed your research well, the data analysis will provide the answer to your research question. But please note: it is a great shame whenever the following quote by PWA Willems (Utrecht University) becomes applicable to your study: “Unfortunately, only after having analysed the data, has it become clear how the research should have been designed.” Learning objective In their Research Question, students can distinguish the key variables of the association to be studied and convert the variables to a codebook. Based on this, they can design the crude data analysis aiming to answer the Research Question. Students can indicate how to use a statistical package like SPSS for data entry, data management, data analysis, and result presentation and interpretation. Important: This module is a large one, comprising six parts! Instruction

1. Read Chapter 8 in Zielhuis’ textbook and choose the topics which best fit your Research Question: - measurement scale of your outcome and determinant variable - codebook terminology - statistical analysis, especially estimating parameters and their precision and, - is hypothesis testing part of your research project? For further consideration: have a look at the book by Petrie & Sabin “Medical Statistics at a Glance”, 3rd edition (Oxford: Blackwell Publishing Ltd, 2009). Are Chapters 20, 24, 26 and 44 relevant to your study?

2. Quick Test 8 looks at the basics of statistical analysis describing the data: the ‘average’ and the ‘spread’, displaying the data graphically, but most and foremost, the interpretation of the population on parameters and their standard errors

3. Collocation of medical research domains, methodology and data-analysis: In extension to a former task (Lecture 1), please match a domain of research (A), and a study method (B) to a type of data analysis (C). There may be more than one possibility. For instance 1 with c, d etc. If you don’t recognize some of the items, please use Wikipedia as a quick and dirty method to gain an understanding of the contents.

39

Return to Module 2 and the review of the three papers on your subject: what kind of data analysis did the authors use?

Medical research domains (A), study methods (B) and data-analyses (C) the last 500 CKO-9 students put forward in their Plan of Work in the period 2009-2011

A 1. diagnosis

2. medical intervention: C 3. treatment, prevention a. observation series

4. etiology, pathogenesis, polymorphisms b. survival analysis c. t-test 5. prognosis d. ROC-analysis e. proportional hazard modelling f. logistic regression analysis g. chi-square test

B h. patient interview i. qualitative study i. ANCOVA ii. evidence-based medicine j. focus group interview iii. clinimetrics k. longitudinal data-analysis iv. biochemistry v. clinical psychology

vi. pilot study. vii. polygenic approach

viii. survey ix. medical ethics x. social medicine

4. To give you an even more “A-ha” feeling on data analysis, you are going to read about the topic of

graft loss following kidney transplantation and the influence of the ‘heart-beating donor’.

5.

Précis - Kidney transplantation is the current treatment of choice in patients with end-stage renal disease. Substantial changes have occurred (new immunosuppression agents, new selection criteria for donors and recipients in the intervention procedures). We want to study to what extent outcomes like graft loss and patient survival still are dependent on donor source, i.e. living-donor grafts or cadaveric grafts / (non-) heart beating donor organ? This research question is relevant for the current and future availability of donors.

Study the codebook on the dataset NIERTRANSPLANT.SAV concerning 1228 patients. Using SPSS as statistical software, we analysed the data that have been made anonymous and which were made available to us by the courtesy of Professor Andries Hoitsma, transplantation internist at the University Medical Centre in Nijmegen. Look at the outcomes carefully and answer the questions.

40

Codebook on kidney transplantation and graft loss

VARIABLE NAME

VARIABLE LABEL

VALUE LABELS

ID identification number

unique number ranging from 1863 to 110465

GEND gender

1 = male 2 = female

AGE age at transplantation 18 – 76 yr

S_BD systolic blood pressure before transplantation

80 – 240 mmHg “ . “ is missing value

DONOR heart beating donor organ (HBD)

1 = heart beating donor 2 = non-heart beating

GL graft loss 1 = yes 0 = no

TIME_TO_GL time to graft loss or censored 1 – 21 yr

Applying SPSS statements such as: Analyze – Descriptive statistics – Frequencies / Descriptives / Explore / Crosstabs, resulted in several outcome tables and graphs (below) on domain variables. Study these descriptives and then answer questions a – c. a. What is the prevalence rate of hypertension, defined as S_BD > 180 mmHg? b. What are the mean age and systolic blood pressure of the recipients? How many digits (and what

unit) are appropriate? c. What kind of information on the correlation between age and systolic blood pressure can you defer

from the output? (Table II, Figure I)

Table 1. Systolic blood pressure before transplantation of 1228 recipients

Systolic blood pressure before transplantation (mmHg) Frequency Percent Valid Percent

Cumulative Percent

Valid 80-110 115 120 125 130 135 140 145 150 154 155 156 158 160 165 170 173 175 180 185

39 5

40 11 60 11 79 13

100 1

13 1 1

85 5

73 1 4

40 1

3.2 .4

3.3 .9

4.9 .9

6.4 1.1 8.1

.1 1.1

.1

.1 6.9

.4 5.9

.1

.3 3.3

.1

6.0 .8

6.2 1.7 9.3 1.7

12.2 2.0

15.5 .2

2.0 .2 .2

13.2 .8

11.3 .2 .6

6.2 .2

6.0 6.8

13.0 14.7 24.0 25.7 37.9 39.9 55.4 55.6 57.6 57.7 57.9 71.1 71.8 83.1 83.3 83.9 90.1 90.2

41

190 192-240

Total

Missing System Total

29 34

646

582 1228

2.4 5.3

52.6

47.4 100.0

4.5 5.4

100.0

94.7 100.0

Table II. Correlation and regression between systolic blood pressure and age among 1228 recipients.

** Correlation is significant at the 0.01 level (2-tailed)

Mean Std, Deviation

N Correlations

Age at

transplantation

Systolic blood pressure before transplantation

Age at transplantation

45.99

12.993

1228

Pearson Correlation Sig. (2-tailed) N

1

1228

.160** .000 646

Systolic blood pressure before transplantation

151.96 25.016 646 Sig. (2-tailed) N

.160** .000 646

1

646

Regression Coefficientsa (linear regression) Model

Unstandardized Coefficients

B Std. Error t Sig.

1 (Constant) Age at transplantation

137.476 .313

3.656 .076

37.602 4.111

.000

.000

a Dependent Variable: Systolic blood pressure before transplantation

In Figure I the results are shown of linear regression of S_BP as a function of AGE. Answer question d.

d. Can you set up the regression line: S_BP = B0 + B1 x AGE? What increase in systolic blood

(mmHg) pressure do you expect in case a person becomes 1 year older?

Figure I. Correlation and regression between systolic blood pressure and age

42

6. Now, regarding our key Research Question: What is your prior belief of graft loss rate in type of transplantation, which is operationalised in HBD (heart beating donor transplantation) vs. non-HBD patients. How many recipients sustain graft loss, and what %, of all the recipients is this? Is this different for type of donor (HBD vs. non-HBD patients)? Has your hypothesis about the effect of HBD-donor been confirmed or refuted by Table III and Figure II, respectively?

e. What confounding factors would you like to address? Would you consider systolic blood

pressure as a confounder in the relationship of HBD-transplantation and graft loss? What does Table IV tell you in this respect?

For the statistical analysis, we quoted SPSS statements like: Analyze – Descriptive statistics – Crosstabs, mark in ‘Cells’ option ‘Percentages’: Row, Column and Total and in ‘Statistics’: Chi-square.

Analyze – Compare means, followed by Independent - Samples – T Test Analyze – Survival – Kaplan-Meier, with Options ... , as Plots, Survival and thick under Compare factor for log-rank.)

Table III. Heart beating donor intervention and the effect on graft loss.

Heart beating donor organ (HBD) Graft loss (GL)

No Yes Total No Count Expected count % within HBD % within GL % of Total

403 439.4

39.7% 76.6% 33.2%

612 575.6

60.3% 88.8% 50.4%

1015 1015.0

100.0% 83.5% 83.5%

Yes Count Expected count % within HBD % within GL % of Total

123 86.6

61.5% 23.4% 10.1%

77 113.4

38.5% 11.2% 6.3%

200 200.0

100.0% 16.5% 16.5%

Total Count Expected count % within HBD % within GL % of Total

526 526.0

43.3% 100.0%

43.3%

689 689.0

56.7% 100.0%

56.7%

1215 1215.0

100.0% 100.0% 100.0%

43

non-HBD n= 1015 HBD-donor n= 200 Figure II. Graft loss according to kind of kidney transplantation from (non-)heart beating donor.

Table IV. T-test statistics on the association of systolic blood pressure before transplantation and type of kidney donor.

HBD N Mean Std. Deviation Std. Error Mean

Systolic Blood Pressure non_HBDonor 537 153.08 25.417 1.097

HBDonor 108 146.53 22.334 2.149

t-test for Equality of Means

95% Confidence

Interval of the Difference

F

Sig. t df

Sig. (2-

tailed)

Mean

Difference

Std. Error

Difference Lower Upper

Systolic

Blood

Pressure

Equal variances

assumed

2.591 .108 2.491 643 .013 6.550 2.629 1.388 11.713

Equal variances

not assumed

2.715 167.729 .007 6.550 2.413 1.787 11.314

44

7. Back to your own research proposal: What is your basic analysis of the Research Question? Sketch the key Table and the key Figure. Concerning the Table, is the outcome presented in rows or columns? What about the determinant? Do the same for the Figure: which variable is on the X-axis and which on the Y-axis? Complete your draft Table and Figure with appropriate title and legends and 95% CIs or statistical test outcomes, whatever is most convenient. NB: you will need this work on draft tables and figures for peer review in Seminar 3 and the SPSS computer lab afterwards. Tip 1! For statistical problems during your internship, first of all mobilize co-workers in your host department. In case of major problems about your data analysis, you may use the email address for statistical consultation by RUNMC experts; this is posted on BlackBoard. Tip 2! To display tables and figures, see section Resultaten (pp167-9) in Chapter 9 Publiceren of Zielhuis’ textbook and Chapter 4 in Petrie & Sabin’s Medical statistics at a glance.

Tip 3! Which language is used in your text, tables and figures? Use a dot as delimiter if you use English (2½ -> 2.5) and a comma in case of Dutch (2½ -> 2,5).

45

Module 9: ORGANISATION AND MANAGEMENT SUMMARY Justifying your case! We can imagine that you would like to perform your study alone. Usually, however, you will be working in a team. In a team, all members have their own tasks and roles and their own planning, timetables, and priorities. In addition, other important facilities like the availability of a workplace, a personal computer, and in case of research abroad, accommodation can be very important. More directly connected to your research is the question, ‘is your study feasible?’ This means, is your research affordable in time and money, is there enough technical expertise, and is it possible to include the estimated number of subjects. Is it possible to use measuring instruments and are there others users? All these and more are part of planning your research organization. Learning objective Students can discern the main players related to their research project: scientific support, information collection, collaboration and use of measurement instruments. They can also demonstrate the feasibility of achieving their proposed deadlines in the given time span of three months. Assignment N.B.: today’s assignment consists of six parts (Quick Test excluded). 1. Nothing to read this time, but Quick Test 9 requires you to take a deeper look at organising your

progress. You will think about communication – logistics – piloting – time-planning.

2. You now have less than a few days before you start your internship. Are you prepared? Is your supervisor prepared? Search the department you are going to on the Internet. Who is who? And what have they published recently? What about your supervisor, his/her (major) articles (impact factors) and H-index / citation report? For the latter, go to the “ISI Web of Knowledge” (via University Library), fill out the name of your supervisor, select the right articles and choose “Analyze Results” or “Create Citation Report”. Plan an email to send to your supervisor saying (once again) who you are, anticipating potential starting points and that you are looking forward to the internship. Don’t forget to mention the CKO-9 pre-training track: that you will forward your Plan of Work to them within one week.

3. So far, you have worked on setting goals and objectives. Now it is time to organise your actions into a

plan. Actions set into a time-frame make up your plan. Usually, we see plans like the one in the table below:

46

Time schedule research internship

Week 1 Complete preparations. Contact department MDs

Weeks 2-6 Enrolment. Input case record forms. Follow-up

Weeks 7-9 Measurements. Input findings

Week 10 Data analysis

Weeks 11-12 Writing paper. Presentation. Completion Internship Can you be a bit more specific for your own plan? Put also the writing of some of the parts of your report / draft article on the time table. Make sure for yourself and for us, CKO-9 teachers that your plan is workable. Who are your team members? Make sure you have organised your actions in a logical and practical order. Can you accomplish the actions you have set for yourself in the time frame allotted?

4. Time-management is one thing. Self-management is another thing. How will you approach your research internship further? In textbooks on self-management, so-called SWOT analyses are advised. In SWOT you have to pinpoint your own Strengths (S) and Weaknesses (W) and confront these with the Opportunities (O) and Threats (T) emerging from your environment (trainee post). Read the Table and fill out your SWOT items. What issues of your approach deserve your special attention during your research internship?

No. MAIN POINTS

MY SWOTs

1 Manage your time

S: W: O: T:

...

... ... ...

2 Team work, cooperation with subjects

S: W: O: T:

...

... ... ...

3 Approach to end result and products of your internship: - What do you have to do? - What standard of work is

expected?

S: W: O: T:

...

... ... ...

4 Presentations / written texts S: W: O: T:

...

... ... ...

5 Fulfilling your own learning objectives (see below, task 5)

S: W: O: T:

...

... ... ...

5. The main objective of your research internship is ‘learning by doing’ - how to conduct research.

Nine general objectives have been formulated in that regard (see this course book Episode Research Training, Chapter 2, Learning Objectives).

47

But you also have to master three specific (personal) research objectives (See p. 10 “Personal learning objectives of the research training internship”). But first in the textbox below, you are going to read about four different medical students and their personal learning objectives.

6. Which students erroneously mention general learning objectives, instead of personal objectives? Master Student in 2011 Personal Learning Objectives

A. H. • Critically read literature

• Analyse the collected data • Produce an article

B. I. i. Speaking Spanish

ii. Improving interview techniques iii. Team building iv. Back-packing through the country-side

C. J. - Gain experience in setting up and conducting

qualitative research - Develop interview techniques further - Practice writing a good report

D. K. 1. Competences in Pubmed

2. Skills in statistical testing and SPSS 3. Developmental experiences in writing scientific

papers 4. Visiting my first scientific congress

Write your three personal learning objectives. These personal learning objectives are more specific goals for your internship. You have to put time and effort into this - beyond your plan scheduled above (assignment 3). Make clear how, which activities and courses you will take, to achieve these goals. So make sure your host supervisor agrees with you and supports you.

7. Finally, have a quick look at Seminar 3, tomorrow. It’s all about Practical organization, and … some numerical thoughts. You will work on the latter in a computer lab on SPSS, which is scheduled following Seminar 3. But first, review Modules 7 – 9.

48

Seminar 3: PRACTICAL ORGANISATION AND NUMERICAL THOUGHTS

Get organized! In this seminar, you will reflect on the approach, planning, and organisation of your own internship. Independent of your internship supervisor or the actual research supervisor, you have to, in advance, formulate all the points that deserve special attention regarding the management of your internship. Management largely comes down to self-management. What are your well- and less-developed self-management competences? The other aspect for reflection is the design of your data analysis. How will you conduct your basic statistical analysis and what is the expected outcome, and how will it be visualized into a figure / graph?

Learning objective Students can demonstrate their firm grasp of the anticipated organisational aspects of their study and can specify time constraints and the relevant team players. Additionally, they can make a blueprint of the descriptive and inferential data analyses as presented in the result section of a scientific article.

Method of working (instructions will follow in the workgroup seminar) In the peer review part of the seminar, you will review the design of your data analysis. In order to do so, please bring along the final productions of the Module 8 home assignment; these include drafts and printouts dealing with practical issues involved in data analysis, and sketches of the final Table and Figure. This seminar concludes with the important subject of your own organizational skills for managing your internship. You will focus on the products of Module 9 and the tables completed.

49

Computer lab: SPSS PRACTICALITIES (translation under construction)

Are my results significant? Now things really get going! You enter your data into the computer, check the quality of the data files for all separate variables, make descriptive statistics, and, finally, develop statistical data analyses of the association that is to be studied. Until quite recently, this was done by noting down measurements followed by some simple arithmetic, but now statistical software packages are commonly used everywhere. In the CKO-9 course, we use SPSS. Do you remember your first SPSS data analysis in the Bachelor phase? Before using the package, you may benefit from a (re-)introduction to this software at the level of codebook, data entry, elementary procedures, recall for the help function, and simple output. Learning objective Students are able to process and analyse their own virtual data file or make use of the presented examples, resulting in (simple) tables and graphs. Interactive statistical lab In the1st hour, a demonstration is given of how to build a data base with records and different kind of variables, how to do simple descriptive analyses, and how to compare means and proportions. We will then briefly touch on how to transform variables into other variables and to use different windows in SPSS. During the 2nd hour you will complete the ‘muesli’ exercise: how does muesli intake affect mental concentration tests in some volunteers? (Creating a data base, data entry, descriptive analysis of all variables, comparing pre- and post- measurements, confounding and or subgroup analyse).

CKO-9 COO-SPSS: Data entry and simple statistical analyses “Muesli study” Basic skills in SPSS

Précis Some resident physician underestimates the importance of well-balanced diets. The internships are long and aggravating, and there is simply not enough time to eat. Even though they are aware that diet has an important effect on their ability to concentrate and capacity to handle swiftly, their diet suffers from both ‘too much’ and ‘too little’ – resulting in a negative influence on physical and mental strain. We hypothesize that breakfast with muesli will facilitate slower carbohydrate uptake and have a positive influence on the physician’s reaction time. We will study this by giving a muesli diet to 20 resident physicians for a period of 4 weeks. Their reaction time will be assessed before and after this period. The next table shows the data of 20 participants.

50

Table 1: Data from the muesli study

ID Gender Total reaction time Total reaction time Birth date (before) (after)

1 male 78 75 12.01.85 2 female 90 81 06.05.88 3 female 68 65 05.11.83 4 female 77 70 07.11.88 5 female 78 77 21.12.83 6 male 81 76 25.05.89 7 female 73 72 01.03.85 8 female 57 54 04.08.89 9 male 55 52 19.03.83

10 female 74 69 22.05.84 11 male 63 58 10.04.85 12 male 61 58 23.04.81 13 male 80 75 31.01.89 14 female 82 77 13.09.83 15 male 72 65 17.07.86 16 male 65 55 15.06.88 17 male 90 79 16.12.83 18 female 68 65 05.11.87 19 female 77 68 07.11.88 20 male 81 70 25.02.89

Code book Variable label Variable name Value range Identification number ID 1-20 Gender Gender 1=male, 2=female Reaction time (RT) before muesli diet (sec). RTbefore 55-90 Reaction time after 4 weeks muesli (sec.) RTafter 52-81 Birth date Birth 23.4.1981 – 25.5.1989 Variables, yet to make (see under 5): Difference in reaction time (RTbefore – RTafter) in seconds

Diff

Age at 1.1.2010 (years) Age Data entry:

1. Enter the Table 1 data directly in the data editor (Data view-window). Tip: Start to define the variables in the Variable view-window. For help about the different types of variables see “help>>topics”.

2. Save the file via <file> <save as> with file name “Muesli_ZSO9.SAV”. It is now possible to conduct some simple analyses.

Understanding the data: 3. Briefly describe the variables in the data file.

Step 1: Click on <analyze>.Search for the analysis you want to perform (to analyze you can use <help> <statistics coach>): Use <Descriptive Statistics> and next <Frequencies> and <Descriptives>.

51

Step 2: Click on <paste> to save the syntax. The commands for the analysis will automatically be pasted into the syntax window. (Clicking on <OK> will run the analysis immediately without saving the commands in the Syntax window. Saving your analyses in the Syntax window with <paste> provides a journal of all the steps you have taken. Step 3: Next, go to the Syntax window and select the command just pasted. Click on <run> or on ►Now the data will be analyzed and the results will be written to the Output window. Step 4: Click on the Output window and study the results. Step 5: Start a new analysis: Go to step 1 and choose another analysis from the drop-down menu.

- How many male and female participants are available from the file? - What is the mean reaction time before the start of the muesli? And after 4 weeks eating muesli for breakfast? Are these results different?

4. Try also <Analyze>, <Descriptive Statistics>, <Explore>. This procedure gives you different

outcome measurements and produces graphs simultaneously.

Making a new variable:

5. Make a new variable in which the difference between reaction time before and after the muesli diet is calculated (variable name ‘Diff’) and the age at the start of the study (variable name ‘Age’), that is the 1st of January 2010 (see box) Step 1: Click on <Transform>. Use <Compute variable>. Add the new name of the variable in ‘Target variable’. Choose the desired calculation by entering the numeric expression in the box (see below). Step 2: Save the calculation in the Syntax (<Paste>, see assignment 3) or use OK directly. Step 3: Behold the results in the Data view-window.

Variable Numeric expressionDifference in reaction time:

RTbefore – RTafter

Age: DATEDIFF(DATE.DMY(01,01,2010),birth,"years") (see function group-> date arithmetic, datediff and date creation)

Consider the age. What is the mean age? Is this okay? What is the mean difference in reaction time before and after the 4 weeks muesli diet?

Data-analysis:

6. In assessment 3 step 5 you compared the mean reaction times at the start of the study and after the diet. Is this a significant difference? The null hypothesis is no difference (i.e. mean Diff = 0). Use <Analyze>, <Compare Means>. Which procedure do you prefer? The unpaired T-test ( <Independent-Samples T Test> ) or the paired T-test ( < Paired-Samples T Test> )? What is the mean change in reaction time? What is the 95% Confidence interval? Is null (‘0’) included? What does the outcome of the P-value suggest?

52

7. Now we want to know whether the reaction time will be different between male and female

participants (effect modification). Step 1: Click on <Analyze>, <Compare Means>, <Independent-Samples T Test>. Step 2: Use ‘Diff’ for Test variable and ‘Gender’ for Grouping variable. Define Groups and click on OK. What is the difference in reaction time between males and females? Is the difference null? What is the 95% Confidence interval? Is ‘0’ included? What can you conclude from the P-value?

8. Visualise the reaction time at the start of the study for males and females in one graph. Use the <Graphs> menu. Which type of graphs are useful (try BAR-chart, BOX-plot, ERRORBAR and HISTOGRAM)? Which is the most suitable? Visualise the coherence between the reaction time before and after the muesli diet. Use a SCATTER-plot. Can you draw a regression line through the scatter plot (click twice on the plot and use options)?

Here ends the computer lab SPSS practicalities. If you have any questions about the analysis of your own study, don’t hesitate and ask the SPSS lecturer

53

Module 10: REPORTING YOUR PLAN OF WORK AND MANAGEMENT SUMMARY Convincing your readers! After a fortnight of preparation and anticipation, you now have reached the final and most comprehensive assignment. Why are you actually doing this internship and why have you chosen to work on this specific research topic? How will you organise the whole project? You will complete a Plan of Work, parts of which you have already worked on during your previous assignments. With this final Plan of Work, you will not just do yourself and your supervisor a favour, but you have also taken the first steps towards producing your final internship report and a possible scientific publication. The CKO-9 lecturing team really hope that you achieve this last target: publication! Learning objective Students can write the final Plan of Work for their internship. Furthermore, they can summarize the relevance of the anticipated results, and can deliberate on the strengths and anticipated weaknesses of their proposed study at a scientific and societal level. Assignment

1. Make the Quick Test. In addition, to warm you up on the art of good communication resulting in your Plan of Work, here are some additional questions: - What is the maximum acceptable number of slides for your 10-minutes PowerPoint presentation? - What is, in general, the number of words allotted for an abstract (e.g. for NTvG) and how many for a scientific article ( e.g. the New England Journal of Medicine). How many words are you allowed for your internship report? - What are “Authorship-responsibility / Copyright transfer forms”? - Does your type of research require a special form of reporting (CONSORT, MOOSE STARD, STROBE etc)? (See Zielhuis, Chapter 9)

2. In this Module you complete your Plan of Work. As in any other piece of written work, it should be preceded by a summary. This summary, however, will differ from the summary of a scientific paper. Firstly, think about whom are you writing the Plan of Work for? Secondly, start with background information. But thirdly, and most importantly, address logistics and practicalities in the plan of investigation. Write a management summary in 500 words: - title and author … your name - background …. rationale, what’s already known - purpose … what will this study add, and research question - setting - most important part of the management summary: the plan of investigation (for example enrolment of subjects or materials, numbers, measurement tools, data collection, human subjects protection, data reporting, data analysis) - expected results and relevance, impact.

54

Template for writing your Plan of work

• Front page

• Management Summary

• Introduction with rationale, literature (search, overview (table), critical appraisal and reference list) and a paragraph specifying the Research question

• Proposed plan of investigation with study design, study model including key variables, study population and number of subjects and measurement instruments

• Societal and ethical justification

• Proposed data analysis with main table and figure

• Organization with time table and SWOT analysis

• Personal learning objectives

For some examples of well-done Plans of Work see BlackBoard

3. Completing your Plan of Work What have you learned from all the modules? Have you made informative and additional notes in the seminars? The aim of the final module is to complete your Plan of Work. You can use the template presented in the box as guide for writing your Plan of Work. But, please note, this is not virtual ‘run through’ of all the previous modules, it’s about getting a clear view of the end result of CKO-9: a report of your Plan of Work. Your report will let your research internship supervisor see how you are trying to interest fellow researchers and ‘sell’ the project. Make sure you cover all the points required, and write enough detail to fulfil the task. Take care of an appropriate style and layout. But, most importantly, show a well-ordered structure in what you present the logical development of ideas, facts and expectations. For example, the chapter on ‘research question’ should not be found ‘just anywhere’ in your plan, nor should it contain just one statement like “Does chocolate lower blood pressure in hypertensive patients?” Even a research question on medical chocolate intervention should be introduced by a strong opening, motivating the decisions regarding the domain, determinant and outcome. For this task, you have to produce a single piece of writing, of 10 – 15 (maximum) sheets of A4 (in professional English or Dutch).

4. This part is not an assignment, but describes how we will assess your Plan of Work. It may, therefore, facilitate completion of your final report. Assessment - Your performance at CKO-9 is assessed at two levels which score your ability to write a Plan of Work: A = sufficient / pass and B = insufficient / fail. The score is based on the criteria for style (see left hand box below) and content (see right hand box) and the requirements presented in Chapter 2 of this course book. For the assessment of the content every item of your Plan of Work is scored on a scale ranging from -, ±, +, which is short for absent or weak, borderline, and good. The scores are assigned to your text, graphs, formulas and, more importantly, the structure within and between the chapters.

55

Plan of Work: Plan of Work: Your Writing Checklist Your Content Checklist

• Have you included information about all the previous module assignments?

• Title and front page information • Management summary • Rationale • Have you used your own words and

avoided copying from the specific instruction text?

• Literature • Research Question • Study design • Is your chapter structure sufficiently clear

to show a good control of content and form (opening , middle and closing section)

• Study population • Measurements • Data analysis • Are the front page requirements met? (title,

extension, ‘Plan of Work’, author, student nr., creation date, emails student/supervisor, department, calendar period)

• Organization • Personal objectives

• Have you included the management summary following the front page?

• Did you avoid redundant information, grammar, spelling and other errors that might cause confusion?

5. Prepare Seminar 4 well. Skim its text for a general understanding of the peer review process and

the task. NB: The 500-word management summary of your Plan of Work is pivotal for Seminar 4. Print three hard copies and bring them to Seminar 4 for your peers.

56

Seminar 4: SALES AND COMMUNICATION Selling your story Using the management summary of their Plan of Work, students can reflect on the strong and weak points of their own study and communicate these to fellow students. They should ask critical questions about the relevance of the research and the quality of the proposed work. Learning objective Students can summarize their study and present the aspects that make their research project worthwhile. Method of working In this seminar, we start by handing out the management summaries (Module 10) for peer review in small groups. Then, the seminar group are given an imaginary budget of € 1,000,000 to be spent on a maximum of 2 research projects that appear to its members to be most worth funding. In the final part of the seminar, students reflect on the research process in its entirety. What makes a research project:

- Useful? (for scientific knowledge - for clinical and health policy - for future research directions) - Good, according to what you have learned about research questions - Good, according to what you have learned about study design / data acquisition - Feasible - Clear and unambiguous.

Evaluation This course has been developed especially for you, the student. We need to know what you think .. in order to improve and develop the course further. Some of the verbal feedback given during the course has already been included in parts of the seminars, however we would like to ask you to spend a few minutes filling in the form below (distributed during the seminar) and handing it in. It is anonymous, .. unless we discuss your feedback during the last five minutes of this CKO-9 course, before you leave to start your internship next Monday.

57

CKO-9 Evaluation Form Calendar year: ... Month: ... 1. Was the content and nature of the course sufficiently clear after the first lecture (HC1)? 1 2 3 4 5 No absolutely not O O O O O Yes completely Comment..................................................................................................................................................... 2. Did the course meet your expectations? No absolutely not O O O O O Yes completely Comment..................................................................................................................................................... 3. Was the time you had to invest in the course in accordance with your expectations? No absolutely not O O O O O Yes completely Comment..................................................................................................................................................... 4. What is your overall judgement of the teacher’s performance? Very bad O O O O O Very good Comment..................................................................................................................................................... 5. What is your overall judgement of the teaching materials used (module guide, books, powerpoint presentations etc?) Very bad O O O O O Very good Comment..................................................................................................................................................... 6. How useful was the course to you? Not at all O O O O O Very useful Comment..................................................................................................................................................... 7. If CKO-9 was a voluntary course, would you recommend the course to others? Not at all O O O O O Very much Comment.....................................................................................................................................................

58

8. Were the Quick Tests useful studying Textbook and Modules? Absolutely not O O O O O Very useful Comment..................................................................................................................................................... 9. Did you appreciate working in peer review groups? Not at all O O O O O Very well Comment..................................................................................................................................................... 10. How do you rate this course? Very bad 1 2 2 4 5 6 7 8 9 10 Very good 11. Please give three positive aspects of CKO-9 and three aspects that will have to improve

................................................................................................................................................................

................................................................................................................................................................

................................................................................................................................................................

................................................................................................................................................................

................................................................................................................................................................

................................................................................................................................................................

59

4 TWELVE-WEEK RESEARCH TRAINING

General information Besides practical professional training, scientific education is an important component of a medical study programme, involving gaining knowledge of both scientific methods and a series of skills, like an investigative and inquisitive mindset, the power of precise formulation, and a wide-ranging orientation towards disciplines other than your own. These elements jointly foster the development of a scientific attitude. Having received scientific training in the Bachelor’s cycle (MPV, optional courses, and 5BOSA8) and in the Master’s cycle (CKO and CKO-9), students round off their scientific training in the Research Training Internship (RTI). This internship can be located in the final stages of their master’s phase - generally in the final six months - when they have already gained experience at various clinical departments. However, when students are on a residency waiting list, they may opt to take their RTI prior to CKO-1. In all cases, students must contact the host organization well before the start of the internship to regarding the actual date of commencement and other important matters. It is important that students submit major changes of the approved project plan to the Research Training Committee (RTC) confirmed by the Radboudumc final supervisor. In case of doubt (whether changes are minor or major), submission to the RTC is advisable to prevent problems with the assessment in the end.

Coordination The final responsibility for the Research Training lies with the chair of the Research Training Committee professor J.P.H. Drenth - phone 024-3616999 - email Joost.Drenth@radboudumc.nl

The Research Training takes 12 consecutive full-time weeks, a period the medical faculty considers a minimum requirement for achieving the set objectives. The internship must be completed within this period, including the completion of the report and assessment by the supervisor. If students should wish to pursue their research projects on a voluntary basis after this period, they are at liberty to do so if their research interests do not interfere with their residency or their exams. The number of credits awarded to the RTI will be the same in all cases. Extended RTI There is an option for students to take an extended RTI, in which the internship is extended by a maximum of three months and replaces their optional residency. If students indicate in advance they like to take an extended RTI that does not replace their optional residency but that is additional to it, this can be recorded on their degree certificate. If students wish to take an extended RTI (replacing their optional residency and with a maximum duration of three months), they must supply a motivation indicating how the extended length of the internship will be expressed in the final product. This motivation must be signed by the Radboudumc final supervisor. Obviously, if students want to take an extended internship that is additional to the regular curriculum, they must also provide a motivation in advance. Exemption in special cases. The scientific internship is a compulsory exam component for all students. However, for those who have performed scientific work at a level that clearly meets the requirements made during the RTI, there is an exemption option. The main requirement for exemption is that students should have produced an approved dissertation leading to a doctoraal(master’s) examination on a topic or subject that is relevant to medicine.

60

Granting exemptions is the exclusive privilege of the Examining Board in Medicine, on the advice of the Research Training Committee (RTC). An application for exemption addressed to the Examining Board in Medicine must be made in writing, stating reasons and submitting proper evidence (report and assessment). Supervision Supervision Protocol It is self-evident that, in supervision, matters of content need to be dealt with. These might relate to topics such as inevitable constraints to the study with a view to the time available, making content choices, mastering the necessary research skills, the availability of materials, etc. Time devoted to supervision by supervisors may vary, but a minimum time investment of 30 to 40 hours must be expected. In the initial and concluding stages, a great deal of supervision time will be required, with less in the middle stages. In supervision of final projects, there are several moments when students meet up with their host supervisor. It is up to students to take the initiative for arranging supervisory meetings. In addition, students must also make sure that their supervisor receives the relevant information in writing several days prior to the meeting, in order for him/her to be able to prepare for the meeting properly. After the meeting has taken place, students write a concise report of the major observations, conclusions, and appointments. This report is a memory aid for both students and supervisors. In the case of an internship outside the Radboudumc, moreover, students should also send a copy of the report to the Radboudumc final supervisor to keep him/her informed of the students’ activities at the host institution. The following delineation of six supervisory meetings, needless to say, is not meant to be rigidly prescribed. This description is to convince students that they not only may expect to receive regular supervision but also that students themselves are expected to carefully plan and prepare supervisory meetings. Introductory meeting During their CKO-9 course, students have contacted their internship supervisor to make sure he/she agrees on the process of research topic development during the CKO-9 course. This contact may lead to an introductory meeting. If this is impossible, an introductory meeting at a later point in time is inevitable. During this introductory meeting, the following points need to be raised: - organizational matters, such as contracts, the Radboudumc supervisor, day-to-day supervision at the

host institution, facilities at the host institution, and time frame. These have obviously been arranged in advance, but need to be checked during this meeting;

- the final products that are to be handed in (report, paper, oral presentation) and any possible additional details, such as the language of the report;

- supervision: what arrangements have been made?; - personal learning objectives: How they van be achieved; - important information on the day-to-day state of affairs at the host institution; - information about the research topic that is important for students to have in advance. In summary:

preparing the topic. First meeting: the research plan (general learning objectives 1 and 2) In the very early stages of the internship, the provisional research plan that was the final product of the CKO-9 course needs to be finalized into a finished design. First of all, students need to reconsider the motive of their study. What is its relevance? In which medical scientific context does the study fit? If necessary, students need to trace, critically read, and systematically review additional literature. They re-examine and improve their research question and work towards a fitting study design. This stage is all about writing a sound and detailed research plan, which includes a time frame. Students hand in this plan to their supervisor several days before their supervisory meeting is scheduled. Topics for discussion in this meeting may include: - the plan’s strong and weak points; - unfeasible points and suggestions for improving these; - appointments. Students write a concise report of this meeting.

61

Second meeting: making the study design more concrete (general learning objectives 2 and 3) The study design must now obtain its final shape. Choices made now need to be justified. New choices must be made on the operationalization of the major variables. Several days before the meeting is scheduled, students hand in the required information to their supervisor in writing. Topics for discussion at this meeting may include: - the fit between the study design and the research question; is the design complete?; - the suitability and quality of the measuring instruments that are to be used; - a data collection strategy; - organizational aspects of data collection; - appointments. Students write a concise report of this meeting. Third meeting: data analysis and description of results (general learning objectives 4, 5, and 6) Several days before the supervisory meeting is scheduled, students hand in a partial report including data and results. Topics for discussion at this meeting may include: - data analysis; - results description; - appointments. Students write a concise report of this meeting. Fourth meeting: first draft of the research report (general learning objectives 7 and 8) Several days before the meeting is scheduled, students hand in a first draft of their report. Topics for discussion at this meeting may include: - the structure of the report; - the contents of the report; - the results and statistics; - the discussion, conclusions drawn, critical reflection on design and results; - language: clarity, unambiguity, reasoning, and style. Students write a concise report of this meeting. Fifth meeting: second draft of the research report (general learning objectives 7, 8, and 9) Several days before the meeting is scheduled, students hand in the second draft of their report, having processed any remarks made in the previous meeting. Topics for discussion at this meeting may include: - the processing of remarks made in the previous meeting; - extra attention now to be paid to the discussion, the conclusion, and putting conclusions in a wider

perspective; - agreements about finishing the report; - agreements about the oral presentation to be given at the host institution.

Students write a concise report of this meeting.

Sixth meeting: final meeting (general learning objectives 7, 8, and 9) Before this final meeting, the supervisor has examined the final draft of the research report. The oral presentation of the study at the host institution has been given. This final meeting deals with: - discussion of the oral presentation; - assessment of the internship, report, and oral presentation; - evaluation of the internship; - any further positive criticisms relating to the internship, supervision, etc; - the future: paper or follow-up study. When the final meeting has taken place, students themselves make sure that the Radboudumc final supervisor – if this is not the same person as the (daily or host) supervisor – is presented with a copy of the report and the completed assessment protocol and expresses his/her (non-) agreement with the assessments of the (daily or host) supervisor. In case of non-agreement the Radboudumc final supervisor adjusts the grade and motivates the grade change. Especially for foreign internships, adaptation of grades is often required, because foreign grading systems can deviate substantially from the Dutch system. Finally, students must hand in the completed assessment protocol and two binded copies of the research report to the StIP at the Radboudumc. The RTC does not play a direct role in the actual supervision of RTI; it has a conditional and monitoring role. By making sure that expedient procedures are in place and by spot-checking these procedures, the

62

committee aims to make sure that proper student supervision during the RTI is guaranteed. If there are any problems during the internship, the committee is available for mediation and, if necessary, correction.

Writing your research report All students finalize their Research Training with a written report, preferable in English. This report may be one of two kinds: it may either be a research report or a draft paper. In the case of a twin internship, both students must each produce their own individual written report, as they also submitted two individual project proposals.

ADDITIONAL TEXTBOOK A very useful guide to performing medical research is:

Overbeke AJPM, Gijn J van, Walvoort HC. Publiceren in biomedische tijdschriften. Een praktische handleiding. Houten: Bohn, Stafleu & Van Loghum, 1999, pp. 220.

The results of scientific research are commonly reported in journals. Writing and publishing a paper requires proficiency in precise and concise formulation. Reporting the results of the RTI is therefore an excellent opportunity to practise this skill. This is why students are expected to produce a concise report, whether they choose to hand in a draft paper or a research report. Research report Students need to bear in mind that an exhaustive or wordy report causes reader exasperation and generally does not contribute to a favourable assessment. The maximum length of the text is 5,000 words (appendices and reference list included). The number of words must be stated on the title page. The RTI report may be written in Dutch or English. The RTI report must include the following elements: - a full title page (title, author, supervisor(s), department, internship period, duration of internship,

date of the report); - structured summary (aim, method, results, discussion) not exceeding one page; - contents including page numbers; - orderly structure (see below) (example: background, research question, method, statistical analysis,

results, discussion, putting your own results in a wider perspective); - tables and figures, if any (always provided with a number and title); - footnotes and references. There are several ways to structure a report. An example is given below. - Summary

This is a brief but complete description of all important elements (one page at the most). It includes the aim, place where the study was performed, research design, methods, results, and conclusion.

- Contents - Preface - Chapter 1: Introduction

The introduction outlines the motive for the study. It also details the wider framework by briefly summarising the literature relevant to the issue. The introduction ends with the formulation of the research question.

- Chapter 2: Research question This chapter expands the research question, paying clear attention to bottlenecks, preconditions, and any additional questions. This is also where the concepts are defined.

- Chapter 3: Method This chapter describes exactly how the study was designed. It deals with the following issues:

63

study design; population or materials investigated; techniques and instruments; interventions or more general determinants; method of analysis.

- Chapter 4: Results The results are presented in this chapter.

- Chapter 5: Discussion This chapter deals with the following issues:

linking the results to the research question; any methodological limitations in the design or the execution; discussion of the results in the perspective of the literature; any recommendations.

- References References must be represented consistently in line with journal regulations (See Zielhuis et al. p. 170). Example 1: Zielhuis G.A., Heydendael P.H.J.M., Maltha J.C., Riel P.L.C.M. van (2010). Handleiding medisch-wetenschappelijk onderzoek. Utrecht: Wetenschappelijk uitgeverij Bunge. Example 2: Taylor S.E. Hospital patient behavior: reactance, helplessness or control? Journal of Social Issue (1979); 35: 156_184.

- Appendices For example: research instruments, letters, additional tables and graphics, glossary, etc.

A draft paper If students choose to write a draft paper as their final report, their contribution is likely to be part of a team product, as there often are multiple authors. To make sure that the students’ contribution is acceptable, students are required to be either the 1st or the 2nd author of the draft paper. The RTC prefers a (first) draft article over a final one. Not only because a final one usually requires more time than there is available (see deadline), but also because a draft version provides more insight into what students themselves are able to c.q. what they have learned. Also, in addition to the draft article, the RTC calls for a separate process description (max 2 A4) where students describe which research activities they have conducted themselves and how this went. The draft paper must be structured in accordance with a structure that is common for research papers. Maximum length is about 3,000 words (author guidelines for the aimed journal is decisive). The number of words must be stated on the title page. Handing in the report After completion of the internship, the final report (either the research report or the draft paper plus the process description) must be handed in to the following people as soon as possible (at the very latest after 3 months): 1. the (daily or host supervisor. This person assesses the students’ performance on the basis of the first

assessment protocol and, therefore, includes the general learning objectives in his or her assessment and, hence, also the quality of the report as well (general learning objective 8);

2. the Radboudumc final supervisor. This may be – but need not be – the same person as the one above. In any case, the Radboudumc final supervisor must always be presented with a copy of the report. This Radboudumc final supervisor must also indicate whether he/she accepts the assessments of the (daily or host) supervisor. In case of non-agreement the Radboudumc final supervisor adjusts the grade and motivates the grade change. Especially for foreign internships, adaptation of grades is often required, because foreign grading systems can deviate substantially from the Dutch system. The assessment on the first assessment protocol of the Radboudumc final supervisor is decisive. In all cases, the Radboudumc final supervisor has to sign the protocol.

3. StIP (two binded copies). Two copies of the internship report AND the completed first assessment protocol must immediately be handed in to StIP. These two copies should be binded! The StIP will then forward one copy of the report to a second assessor appointed by the RTC. The second copy will be kept on file in the library. With a view to their possible graduation date, students are advised

64

to bear in mind that assessment by this second assessor may take a number of weeks. Students are therefore advised to hand in their internship report at least six weeks before the graduation date of their choice. Students with a final grade of 8 or above may qualify for Faculty or University awards (see next section).

Faculty and university awards In order to promote students’ motivation, Radboud University and Radboudumc award a number of prizes for the best internship reports or master theses in an academic year. a. The Radboud University Student Prize (€ 1000) is awarded by the University to one student in each

faculty. This prize is awarded on the recommendation of the Radboudumc’s Board of Directors. b. Faculty student prizes of € 350 each (one for medicine, one for dentistry, and one for biomedical

health sciences) may be awarded by the Radboudumc’s Board of Directors on the recommendation of the Commission Master Awards.

c. Faculty incentive prizes of € 125 each (one for each study programme) may be awarded by the Board of Directors on the recommendation of the Commission Master Awards.

Award candidates can be registered with the Commission Master Awards by the internship supervisor and/or the Radboudumc final supervisor. For this purpose, they should submit a written motivation of their registration, including two copies of the reports concerned. The Commission Master Awards assesses all registrations and advises the University Council on the awards. The Commission consists of eight members (with doctoral degrees) of the scientific staff of the Faculty of Medical Sciences, selected from departments that are involved in the three study programmes.

65

APPENDICES

Registration form: page 67

Template Project Outline and tips page 69

Two examples page 72

1ste Assessment protocol page 76

2nd Assessment protocol page 80

66

Persoonsgegevens

Algemene stagegegevens

To complete: Fill in more information and read explanation

Registration CKO9 & Research Training GNK

For extended internship, I wish to receive extra credits

Please, provide all extra information requested and provide a project plan (max. 2 A4) according to the desired model (title, background, research question/hypothesis ,design/population/ n subjects/variables, rationale, detailled plan of work, timetable , feasibility and literature). The undersigned hereby declare that they have read the content of the coursebook Episode Research Training and also declare that this internship offers sufficient opportunity to achieve the general learning objectives of the research internship.

in year of study 02 0 2

I will use these to compensate optional internships in epi 10:

Registration CKO9 & Research Training GNK

For extended internship, I wish to receive extra credits

Please, provide all extra information requested and provide a project plan (max. 2 A4) according to the desired model (title, background, research question/hypothesis ,design/population/ n subjects/variables, rationale, detailled plan of work, timetable , feasibility and literature). The undersigned hereby declare that they have read the content of the coursebook Episode Research Training and also declare that this internship offers sufficient opportunity to achieve the general learning objectives of the research internship.

in year of study 02 0 2

I will use these to compensate optional internships in epi 10:

Name and initials

Host institution

Department

Host supervisor (as well in case of Radboudumc as non-Radboudumc)

Radboudumc final supervisor (first assessor)

Discipline

Title of research internship

Background/research question/design/rationale/concrte activities and timetable/ feasibility and literature

Patient file study

Nature of the study

Laboratory studySystematic review and/or meta-analysis

Data collection in study subjects Other i.e.:

Check by Studie Progress Monitoring dept.

pass:

Admission requirements Research Internship at start before CKO1v

fail:datedate

date

Signature student

date

Student number

4 weeks

12 week starting date

Signatue of Radboudumc final supervisor

CKO9 in week 3 en 4 in periode:

starting date Extended internship

finishing date

finishing date

8 weeks 12 weeks

nee

yes, 8 weeks

yes, 4 weeks

yes,12 week

Co groupnumber

If not Radboudumc also city & Country

Internship before/ after coschappen

after before

67

COMPLETING THIS EXTRA INFORMATION IS OBLIGATORY (more choices possible). Internship is based on: □ (Only) existing material □ Gathering new material Accreditation by Ethics Committee is: □ Necessary □ Not necessary

□ Already approved Activities to be performed by student: □ Acquisition of study subjects □ Taking measurements on study subjects (patients, animals etc.) □ Taking measurements on (bio)material □ Taking interviews □ Encoding interviews □ Learning specific (lab) skills

□ Attending policlinic / operations with regard to the study □ Preparing an accreditation by the Ethics Committee □ Analysing already prepaired data □ Building own data files □ Simple statistic analyses (descriptive, T-test etc.) □ Advanced statistic analyses supported by a statistician (e.g. multivariate analysis etc.)

EXPLANATION AND INSTRUCTIONS CKO9 • In CKO9, you will be preparing your internship by expanding your research question, methodology,

statistics, etc. The internship always follows on to CKO9 but need not follow immediately. • The CKO9 course will structurally be offered in weeks 3 and 4 of each period. • Enrolment in CKO9 is conditional upon:

1) your having completed your residency or your waiting for CKO1v to start while already having been assigned to a residency group; and

2) your having arranged an internship position; and 3) your internship proposal having been approved.

• Enrolment in CKO9 is also conditional upon your signing up well ahead of time; this means your registration must be in on the 1st day of the period preceding the CKO9 period. For example: if you wish to enrol in CKO9 in weeks 3 and 4 of period 2, your registration form must be in on day 1 of period 1 at the very latest.

• There is the possibility of doing an extended research internship (4, 8, or 12 week extension). If you wish to receive additional credits for the extended internship, please indicate this on your registration form. If you wish use these credits to compensate one or several optional residencies in Episode 10, you should also indicate this separately. Once launched, internships cannot be changed into extended internships, and any additional internship time will not earn you extra credits.

GENERAL LEARNING OBJECTIVES of the research internship

1. Students can specify the motive for their study, the interest it serves, and the medical scientific context it fits into.

2. Students can propose a definitive study design on the basis of the following skills: 2A - Students can retrieve, critically read, and systematically review relevant literature;

2B - Students can fine-tune/elaborate the research question on the basis of the literature; 2C - Students can frame a study design that fits the definitive research question.

3. Students can organize and carry out the proposed study. 4. Students can systematically analyse the collected data. 5. Students can lucidly describe the results and neatly summarize them in tables and figures. 6. Students can indicate errors of measurement and other limitations in the collected data. 7. Students can reflect critically on their study design and results. 8. Students can produce a written research report that is in line with the requirements made upon

such reports. 9. Students can orally present the outcomes of their study to the department where the study was

performed and are able to engage in fruitful discussion of these outcomes. PERSONAL LEARNING OBJECTIVES of the research internship In addition to general learning objectives, there may be specific personal learning objectives relating to a particular type of internship: in one type of internship, students will need to compose a questionnaire for data collection purposes, and, in another type, they will need to recruit test subjects. A range of specific learning objectives may be called for on a case-by-case basis. It is important for students and their supervisors to agree in advance about the specific learning objectives: which specific skills must have been developed by students upon completing their internship?

68

Tips at page 26 - 27 Examples at page 28 -31 Template Project Outline - Research Training

nternship I (preferable 2A4 max, incl. timetable) Title Tip A Background of the research (based on 1-2 key publications Tip B1-B2 Research question / Hypothesis Tip C Study design / study population / number of subjects / variables (including primary outcome)

Tip D1-D8

Rationale (Why is the research important? / How the knowledge can be of use? / What is the scientific/clinical/societal impact)?

Tip E

Detailed plan of work: specification of students’ activities and completed timetable (* see at bottom)

Tip F

Feasibility? Any problems anticipated? Which? Tip G Reference list of literature (1-2 key publications) Tip H

69

* TIME TABLE Research Training Internship (Do not include CKO9!) Please mark with a cross (X) the applicable activities and, if necessery, expend the table with more relevant activities (and weeks). Week 1 2 3 4 5 6 7 8 9 10 11 12 Study of literature Technical instruction / practise technique / learning specific lab skills

Preparations in advance, logistics (A)*

Taking measurements (B)*

Data analyses / statistics (C)*

Write introduction Write method Write results Write discussion Prepare presentation Presentation to host department

Miscellaneous : Miscellaneous : * A Please specify A (for instance: Draw sample, invite/recruite subjects etc) * B Please specify B (for instance: Distract data from statuses, taking interviews, other

measurements on study subjects, measurements on (bio)material etc)

70

TIPS ON WRITING THE PROJECT OUTLINE

Title Tip A: Give succinct, short catchy title

Background Tip B1: Set project in 3-4 sentences, describe the field and provide the information that led up to the research question

Tip B2: Funnel background information further, focus on the unmet needs and describe which elements are worth to study

Research question Tip C: Fix project and frame the most important research question, if applicable, in the format of an hypothesis that can be tested

Study design Tip D1: Label the type of study

Tip D2: Describe accurately main characteristics of study population Tip D3: Describe major inclusion and major exclusion criteria Tip D4: Give accurate description of the control population Tip D5: Describe precisely the most important outcome measure you

want to assess Tip D6: Then define the secondary outcome measures Tip D7: Describe the rationale for the size of the study Tip D8: If you already know which statistical analyses you are going

to do, please describe

Rationale Tip E: Why in the first place should you start this study? Is there some overarching importance to do so? What is the significance of this study proposal?

Plan of work Tip F: Put into words the specific activities you are going to carry out and also complete the time table

Feasibility Tip G: Show you really have thought about the project and whether it is feasible to do. Explain possible difficulties that you expect to meet and how you think to overcome these

Literature Tip H: 1-2 key publications

71

Example 1 - Project Outline Research Training Internship Department: Vascular Surgery, Catharina Hospital Eindhoven Title Gait analysis in patients with peripheral arterial disease. Background Patients with peripheral arterial disease (PAD) often present with intermittent claudication. This typical pain in the leg musculature during ambulation decreases in rest. Patients with PAD show reduced strength in the lower extremities, lower physical activity levels, limited walking performance and reduced health-related quality of life.[1,2] Previous research described reduced step length and decreased walking speed during over ground walking measures after ischemic pain was induced on a tread-mill.[2-4] Sufficient gait analysis during the development of ischemic pain on a tread-mill is lacking. Since tread-mill walking consumes significant more energy compared to over ground walking,[5] continuous measurement on a tread-mill could monitor specific changes during walking in similar circumstances as supervised exercise therapy. The Optogait is an advanced instrument for quantifying spatiotemporal gait parameters and allows for continuous measurements during treadmill walking. It has already been used in healthy adults and patients with a total knee replacement.[6] Proper gait analysis could contribute to the development of improved treatment regimes. Research question/hypothesis Do patients with PAD have a shorter step length compared to healthy controls?. Design We will perform a prospective case control study including 200 patients with PAD and 200 healthy controls. Patients diagnosed PAD (Fontaine-score of II and ankle-brachial index of <0,9) will be recruited from our outpatient department. Patients will be excluded if they have comorbidities that are expected to interfere with their walking performance, including amputation, osteoarthritis, pulmonary disease, heart failure or use of walking aids. A total of 200 healthy controls (ankle-brachial index >0.9) without any significant comorbidities interfering with their walking performance (aforementioned) will be recruited. Our primary outcome is step length as assessed with Optogait. Secondary outcome measures are walk cycle time, stance time, swing time (in which the foot is suspended and proceeds in the air), stride length, cadence (steps/min), walking speed, single and double support phase. We expect that patients with PAD will have a 8% shorter step length and a sample size of 200 patients per arm is needed to detect a significant difference between the both groups (power: 80%; α: 0.05). Rationale By providing data on gait pattern characteristics we will be able to develop an optimized treatment protocol for patients with PAD to improve overall walking performance. Ultimately, this will lead to an improvement of health-related quality of life. Plan of work 1. Patients will be recruited from the vascular surgery outpatient clinic of Catharina Hospital in Eindhoven. 2. After their visit at the outpatient clinic, patients will undergo gait analysis on a tread-mill in a test laboratory in the same hospital. Patients will be transported by wheelchair or asked to rest at least twenty minutes after their walk through the hospital before they are tested. 3. Gait analysis is performed and the patients will be asked to walk as far as possible on their preferable walking speed. Patients are instructed to inform the researcher about the onset of pain. The Optogait monitor will be used to measure the data mentioned above. 4. The same gait analysis will be performed on 30 healthy subjects, after PAD is excluded by a normal ankle-brachial index. 5. Outcome measurements will be analyzed using SPSS. See also timetable Feasibility This research internship is part of a larger study (PhD thesis). The study has been approved by the Ethical Board of the Radboud University Medical Center. I will be responsible for the measurement of 30 patients and 30 controls. That makes it possible for me to finish this project within 12 weeks time. Identifying controls could be difficult as we need to randomly select healthy individuals from the population, matched

72

for possible confounders. Currently we are discussing the use of random volunteers from our personal inner circle versus partners of patients. Reference list 1. Altered gait profile in subjects with peripheral arterial disease. Gardner AW, Forrester L, Smith

GV. s.l. : Vascular Medicine, 2001, Vols. 6:31-34. 2. Relationship between temporal-spatial gait parameters, gait kinematics, walking performance,

exercise capacity, and physical activity level in peripheral arterial disease. Crowther RG, Spinks WL, Leicht AS, Quigley F, Golledge J. s.l. : Journal of Vascular Surgery, 2007, Vols. 45: 1172-1178.

3. Peripheral arterial disease affects kinematics during walking. Celis R, Pipinos II, Scott-Pandorf MM, Myers SA, Stergiou N, Johanning JM. s.l. : Journal of Vascular Surgery, 2009, Vols. 49:127-132.

4. Peripheral arterial disease affects the frequency response of ground reaction forces during walking. McGrath D, Judkins TN, Pipinos II, Johanning JM, Myers SA. s.l. : Clinical Biomechanics, 2012, Vols. 27: 1058-1063.

5. Kinematic, kinetic and metabolic parameters of treadmill versus overground walking in healthy older adults. Parvataneni K, Ploeg L, Olney SJ, Brouwer B. s.l. : Clinical Biomechanics, 2009, Vols. 24:96-100.

6. Validity of the Optogait photoelectric system for the assessment of spatiotemporal gait parameters. Lienhard K, Schneider D, Maffiuletti NA. s.l. : Medical engineering & physics, 2013, Vols. vol:35 iss:4 pg 500-504.

7. Exercise Training for Claudication. Stewart KJ, Hiatt WR, Regensteiner JG, Hirsch AT. s.l. : New England Journal of Medicine, 2002, Vols. 347(24):1941-51.

TIMETABLE Weeks 1 2 3 4 5 6 7 8 9 10 11 12 Project Finishing literature study X X Technical instruction/practice technique/learning lab skills

X X

Preparations, logistics, testing of equipment (specify under A.)

X X

Taking measurements (specify under B.) X X X X X X X Statistical analysis X X X X Interpreting results X X Writing study report Writing introduction of study report X X X Writing methods X X X X Writing results X X X Writing discussion X X X Presenting results Prepare oral presentation X X Oral presentation at (host) department X Miscellaneous / other …… ……

A. Specification of study preparations: Installation Optogait and tread-mill, writing research protocol en patient information for approvement METC, start preparations for including patients at outpatient department, test analyses on tread-mill B. Specification of study measurements: Patient recruitment, examination of gait pattern of patients and controls following a standardized research protocol, data extraction to SPSS/Excel.

73

Example 2 - Project Outline Research Training Internship Department: Fysiology, Radboudumc, Nijmegen Title Influence of an (in)active lifestyle on vascular endothelial function in men with and without a history of myocardial infarction. Background of the research Worldwide cardiovascular diseases (CVD) are still a growing health problem and the leading cause of death. In 2012, CVD accounted for 17.3 million deaths. This number is expected to grow to 23.6 million deaths in 2030. One third to half of the CVD consists of coronary heart diseases1. A recent paper in The Lancet revealed that physical inactivity has overtaken smoking as the principal cause of cardiovascular death worldwide2. Although exercise seems to be a powerful tool to prevent CVD, not all cardio-protective effects of exercise are understood. Analyses of the literature support several potential mechanisms for the cardio-protective effects of exercise by an improved endothelial function3. However, a previous study revealed that healthy active individuals are not immune to CVD4. If differences are present in endothelial function between athletes with and without a myocardial infarction, it might provide new insight for a non-response to the cardio-protective effects of exercise. Therefore, the purpose of this study is to examine the differences in endothelial function in athletes with a history of myocardial infarction, healthy athletes, sedentary individuals with a history of myocardial infarction, and healthy sedentary individuals. I hypothesize that the healthy athletes will have a better endothelial function compared to the athletes with a myocardial infarction. In addition, I expect that athletes with a myocardial infarction and inactive subjects without a myocardial infarction will have a comparable endothelial function. Research question (or hypothesis) What are the differences in endothelial function between athletes with a history of myocardial infarction, healthy athletes, sedentary individuals with a history of myocardial infarction, and healthy sedentary individuals? Study design Study design: Cross-sectional observational study. Study population: 40 male subjects, aged > 40 years. Based on physical lifestyle and history of myocardial infarction they will be divided into four groups (Table 1).

Athletes Sedentary individuals History of myocardial infarction Group A Group C Healthy Group B Group D Table 1: Schematic overview of the groups in the proposed study

Power: Based on anticipated differences in %FMD between study groups (active vs. inactive) of 3.5% with a SD of 2.4,5,6 a power of 90% and alpha 5% significance level, we calculated that 2*((1.65+1.28)2*2.42)/3.52)=8 subjects per group should be included. We will include 10 subjects per study group to prevent problems in case subjects quit the experiment. Variables: Endothelial function of the brachial artery as measured by the flow-mediated dilation (FMD). Study methods: The endothelial function of the brachial artery will be examined by inflating a blood pressure cuff around the forearm, distal from the imaged artery. Using echo-Doppler, the brachial artery baseline diameter and blood flow will be assessed. Subsequently, the blood pressure cuff will be inflated to 220 mmHg to block arterial inflow. After 5 minutes, the blood pressure cuff is released and hyperemic blood flow and changes in arterial diameter and flow will be assessed. These responses are commonly referred to as flow-mediated dilatation and represent a nitric oxide-mediated endothelium-dependent vasodilatation in the brachial and artery. Primary outcome: To determine the brachial endothelial function measured by FMD of athletes with a history of myocardial infarction (A), healthy athletes (B), sedentary individuals with a history of myocardial infarction (C), and healthy sedentary individuals (D). Analysis: Statistical analysis will be performed by two tailed ANOVA analysis and subsequent contrast analysis via Bonferroni.

74

Rationale The findings of this study will enhance our understanding of potential underlying mechanisms of cardiovascular disease and myocardial infarction and the role of physical activity in prevention of myocardial infarction. Feasibility This is an ongoing study, in which three groups have been measured (A, B and D). During my internship, I will include physically inactive subjects with a history of a myocardial infarction (group C) and I will screen subjects to determine eligibility for the study. Although this is a common group of subjects, some difficulties and motivational problems to complete the study might occur. After the measurements of group B, I will analyze the data of all four groups and write a paper including all four groups. Since the Echo-Doppler measurements will be conducted by an experienced sonographer I will assist during measurements. I will also interpret the results. Time table Research Training Internship Week 1 2 3 4 5 6 7 8 9 10 11 12Study of literature X X Learn how to assist and interpret measurements X Preparations in advance (A) X X Taking measurements (B) X X X X Data analyses/statistics X X Write introduction X X Write method X X Write results X X Write discussion X X X Prepare presentation X X Presentation to host department X X Finish research report X *A: Invite/recruit subjects *B: The flow mediated dilatation (FMD) of the brachial artery. References 1. Laslett LJ, Alagona P Jr, and all. The worldwide environment of cardiovascular disease: prevalence,

diagnosis, therapy, and policy issues: a report from the American College of Cardiology, Journal of American College of Cardiology, 2012;60(25 Suppl):S1.

2. Lee, I.M., et al., Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet, 2012. 380(9838): p. 219-29.

3. Bowles DK, Laughlin MH. Mechanism of beneficial effects of physical activity on atherosclerosis and coronary heart disease. J Appl Physiol. 2011;111(1):308-310.

4. Green, L.H., S.I. Cohen, and G. Kurland, Fatal myocardial infarction in marathon racing. Ann Intern Med, 1976. 84(6): p. 704-6.

5. Black MA, Cable NT, Thijssen DH, Green DJ. Impact of age, sex, and exercise on brachial artery flow-mediated dilatation. Am J Physiol Heart Circ Physiol.2009;297(3):H1109-1116.

6. Dalli E, Segarra L, Ruvira J, Esteban E, Cabrera A, Lliso R, Lopez E, Llopis E, Sotillo JF. [Brachial artery flow-mediated dilation in healthy men, men with risk factors, and men with acute myocardial infarction. Importance of occlusion-cuff position]. Rev Esp Cardiol. 2002;55(9):928-935.

75

1st ASSESSMENT PROTOCOL RESEARCH INTERNSHIP IN MEDICINE • For host supervisor and to be completed by final Radboudumc supervisor • Students hands in this form to StIP with two binded copies of the internship report Student’s name and initials Student number

Title of the internship project

Internship starting date Internship finishing date Report handed in on (date)

What was the nature of the research internship (please tick as appropriate)

Mainly literature study Mainly study of patient file data Study of available patient-related data Study based on student’s performance of measurements in patients Mainly laboratory study performed by student Other: …………………………………………..

Supervisor’s institution, department, and name Name and phone no. of UMC supervisor in charge

Supervisor’s signature

ASSESSMENT BY SUPERVISOR A. Assessment of the student’s PERFORMANCE in the research internship

1. Inquisitive and interested in topic and research

2. Critical mindset in study design, implementation, and report

Fail

question-able

pass

fair

good Very

good fail question-able

pass

fair good very

good

3. Organization and efficiency in designing and implementing the study

4. Time planning

Fail question-

able pass

fair

good very

good fail question-

able

pass

fair good very

good 5. Autonomy in performance 6. Commitment and participation

Fail question-

able

pass

fair

good very

good fail question-

able

pass

fair good very

good

7. Openness to feedback/criticism and willingness to use it

8. Adequately seeking assistance (well in time and not indolently)

Fail

Question-

able

pass

fair

good very

good fail question-

able

pass

fair good very

good 9. Interaction with patients (if applicable) 10. Interaction with colleagues during internship

Fail question-

able

pass

fair

good

very-good

Not appl.

fail question-able

pass

fair good very

good

A. GRADE for the student’s PERFORMANCE in his/her internship on the basis of the above (half marks are allowed) (1 = unworthy of marking, 2 = very poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = sufficient, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding).

Please continue with your assessment of the GENERAL LEARNING OBJECTIVES

76

B. Assessment of the GENERAL LEARNING OBJECTIVES of the research internship To what degree has the student shown that he/she possesses the following skills? Your explanation of these items is greatly appreciated (see next page under E). If a learning objective has not been dealt with, we kindly ask you to indicate why, also under E.

1. Can indicate motive for the study, the interest it serves, and the medical scientific context into which it fits

2a Can retrieve, critically read, and systematically review relevant literature with a view to a definitive study design

fail

question-

able

pass

fair

good very

good fail question-able

pass

fair

good very

good

2b. Can fine-tune/elaborate the research question with the aid of the literature with a view to a definitive study design

2c. Can frame a study design that fits the definitive research question

fail

question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good

3. Can organize and implement the proposed study

4. Can systematically analyse the collected data

fail

question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good

5. Can describe the results clearly and neatly arrange them in tables and figures

6. Can point out errors of measurement and other limitations in the collected data

fail

question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good

7. Can critically reflect on the study’s design and results

8a. Can produce a written report in line with content requirements made upon such reports. The quality of the report’s content is:

Fail

question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good

8b. Can produce a written report in line with formal requirements made upon such reports (neat editing, layout, use of language, correct references, etc.). The formal quality of the report is:

9. Can orally present the study’s outcomes to the department where the study was performed and can engage in a fruitful debate on the subject

Fail

question-

able

pass

fair

good

very- good

fail question-able

pass

fair

good very

good

B. GRADE for ACHIEVING THE GENERAL LEARNING OBJECTIVES of the internship on the basis of the above (half marks are allowed) (1 = no performance, 2 = extremely poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = satisfactory, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding).

Please note: The final grade for this research internship will be determined by the Research Internship Committee, after the report has also been assessed by a 2nd assessor appointed by the Committee.

Please continue with the SPECIFIC LEARNING OBJECTIVES

77

C. Evaluation of SPECIFIC LEARNING OBJECTIVES of the research internship Please mention specific learning objectives and explain in your own words whether and why these have been achieved or not.

Learning objective 1: Learning objective 2: Learning objective 3:

D. Declaration of the Radboudumc FINAL SUPERVISOR

I Agree / Disagree with the assessments and grades given under A and B Agree with A and B (in case 9 or 10 please fill out “ad 1”) Disagree with both A and B (fill out “ad 2”) Partial agreement (fill out both ad 1 if applicable) and ad 2) Ad 1. Agree with grading 9 or 10: The research Training Committee strongly urges giving grades 9 or 10 as an exception, and only in case of outstanding quality. Please note: abroad the grading system often deviates from the Dutch one (higher grades abroad). If you agree with a 9 or 10, please state your reasons below. Motivation: Ad 2. Disagree with the grades of part A and/or B: Please state your reasons for disagreeing with the grading of part A or B and bear in mind that abroad the grading system often deviates from the Dutch one (higher grades abroad). The research Training Committee strongly urges giving grades 9 or 10 as an exception, and only in case of outstanding quality. Please give your grades. A B Your grades: Half marks are allowed Motivation: NB: 1 = no performance, 2 = extremely poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = satisfactory, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding

Please continue with Explanation / Other remarks and your signature

78

E. EXPLANATION of assessment of general learning objectives / Other remarks

Please indicate concisely and explicitly which comments, advice, or areas for further development you wish to point out to the student with a view to his/her prospective career. Please use the reverse side of this form if needed.

Signature of Radboudumc final supervisor Date Student’s signature

   

79

2nd ASSESSMENT PROTOCOL                                                                 RESEARCH INTERNSHIP IN MEDICINE • To be completed by the 2nd assessor (appointed by the Research Internship Committee) Student’s name and initials Student number

Title of the research report

Name of 2nd assessor Department Phone number

ASSESSMENT OF THE REPORT If one of the elements has not been dealt with in the report, we kindly ask you to indicate this in the Remarks section.

1. general report layout (title page, contents, numbering, notes, literature)

2. quality of figures, tables, and graphics

Fail

question-able

pass

fair

good very

good fail question-

able

pass

fair

good very

good 3. quality of the summary 4. quality of the introduction

Fail question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good 5. justification and formulation of the research question

6. justification of the study design

Fail

question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good 7. clear description of materials and methods 8. clear description of analysis

Fail question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good 9. clear description of results and conclusions 10. quality of discussion of literature

Fail question-

able

pass

fair

good very

good fail question-

able

pass

fair

good very

good 11. critical reflection on study and findings 12. quality of written use of language

Fail question-

able

pass

fair

good very

good fail question-able

pass

fair

good very

good Remarks (e.g., regarding work load, innovativeness, risks, etc.)

Grade for the research internship REPORTWith a view to the above, grade given by 2nd assessor (half marks are allowed). Give marks 9 or 10 as an exception, and only in case of outstanding quality. If do so, please motivate under “Remarks”. 1 = no performance, 2 = extremely poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = satisfactory, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding

Signature of 2nd assessor Date

80

81