Grad Edge Insights and Research on Graduate Education WWW.CGSNET.ORG | VOLUME 4, NUMBER 4 | MAY 2015 Making Assessment Part of Our Genetic “Makeup”: A View from the CGS/NSF Dean- in-Residence Inside Data Sources . . . . . . . . . . . . . . 3 Did You Know? . . . . . . . . . . . 4 What’s the colorful circular image to the right? (Figure 1) Even the biologically-trained graduate deans among us probably didn’t learn about Circos plots of the human genome in graduate school, but you can undoubtedly help the rest of us understand it. In a single Circos plot, genomic variations between individuals and populations can be identified (the gene sequences and chromosomes on the perimeter of the plot), the similarity of genomic regions (the red, blue, orange and green colored lines connecting sequences on the perimeter of the plot), and even genes responsible for biochemical pathways linked to cancer and other diseases (grey connecting lines) are represented. The Human Genome Project model has been extended to other disciplinary areas. The Art Genome Project 1 and the Materials Genome Initiative 2 are just two such examples. All such projects employ a top-down approach to organizing constituencies (genes, art, advanced materials) that simultaneously allows for enormous depth of research and investigation into both the parts and the whole, while discovering commonalities and links (biochemical pathways, artistic media, processing techniques) between like items. But what about the people doing the work in these projects? Have we maximized the return on our human capital investment portion in the Human Genome Project? Do the obvious beneficial ends justify the means of training multitudes of biomedical researchers who are highly technically skilled, but are now having difficulties finding suitable careers? 3 We’ll never know the answers to these questions because no one ever asked them. We missed the opportunity to do research while we were doing research. As graduate deans, we generally do not have the time or resources to apply the same level of academic rigor to what we do in our graduate programs as our colleagues (and some of us still) do in almost every other area of research, scholarship, and artistic endeavor. Even undergraduate learning has its own place in the research literature. Some of our colleagues have built their careers around better understanding how students learn; our practices in the classroom have changed as a result of that research. But why do we know more about the undergraduate experience than we know about the graduate experience? There are a plethora of peer-reviewed journals, a boundless number Figure 1: Circos plot of the human genome. http://circos.ca/intro/genomic_data/img/circos-conde-nast-large.png
Transcript
GradEdgeInsights and Research on Graduate Education
W W W . C G S N E T . O R G | V O L U M E 4 , N U M B E R 4 | M A Y 2 0 1 5
Making Assessment Part of Our Genetic“Makeup”: A View from the CGS/NSF Dean-in-Residence
InsideData Sources . . . . . . . . . . . . . . 3
Did You Know? . . . . . . . . . . . 4
What’s the colorful circular image to theright? (Figure 1) Even the biologically-trainedgraduate deans among us probably didn’tlearn about Circos plots of the humangenome in graduate school, but you canundoubtedly help the rest of us understandit. In a single Circos plot, genomic variationsbetween individuals and populations can beidentified (the gene sequences andchromosomes on the perimeter of the plot),the similarity of genomic regions (the red,blue, orange and green colored linesconnecting sequences on the perimeter ofthe plot), and even genes responsible forbiochemical pathways linked to cancer andother diseases (grey connecting lines) arerepresented. The Human Genome Projectmodel has been extended to otherdisciplinary areas. The Art Genome Project1and the Materials Genome Initiative2 are justtwo such examples. All such projects employa top-down approach to organizingconstituencies (genes, art, advancedmaterials) that simultaneously allows forenormous depth of research andinvestigation into both the parts and thewhole, while discovering commonalities andlinks (biochemical pathways, artistic media,processing techniques) between like items.But what about the people doing the work inthese projects? Have we maximized thereturn on our human capital investmentportion in the Human Genome Project? Dothe obvious beneficial ends justify the meansof training multitudes of biomedicalresearchers who are highly technicallyskilled, but are now having difficulties
finding suitable careers?3We’ll never knowthe answers to these questions because noone ever asked them. We missed theopportunity to do research while we weredoing research.As graduate deans, we generally do not
have the time or resources to apply the samelevel of academic rigor to what we do in ourgraduate programs as our colleagues (andsome of us still) do in almost every other areaof research, scholarship, and artisticendeavor. Even undergraduate learning hasits own place in the research literature. Someof our colleagues have built their careers
around better understanding how studentslearn; our practices in the classroom havechanged as a result of that research. But whydo we know more about the undergraduateexperience than we know about thegraduate experience? There are a plethora ofpeer-reviewed journals, a boundless number
Figure 1: Circos plot of the human genome.http://circos.ca/intro/genomic_data/img/circos-conde-nast-large.png
Page 2 GradEdge
of books, and a panoply of professionalsocieties devoted to elucidating the impactof undergraduate research, study abroad,service learning, peer mentoring, pre-professional preparation and numerousother experiential topics on undergraduatestudent development, but not a singlejournal (that I could find) related to even thebroad topic of graduate education. There areas many reasons for this lack of research asthere are disciplines, but in the end thedearth of research and scholarlyunderstanding of what occurs in ourgraduate programs is hurting us. It’s hurtingus in terms of public perception,4 state andfederal funding,5 and the concept ofdisciplinarity.6 But most importantly, it’shurting our graduate students. It’s hurtingtheir professional preparation—even forfaculty careers,7 their job opportunities,8their personal finances,9 and presumablytheir ability to participate in large,interdisciplinary projects that tackle some ofthe toughest problems facing our society.To be fair, there are some excellent
institutional- and discipline-based studies that can serve as the basis for future research on graduate education. As part of an ongoing CGS/National Science Foundation (NSF) Dean-in-Residence project, I have come across a body of literature related to evaluating international experiences for graduate students, for example, including work by Michigan State,10 the University of Washington,11 and the Office of International Science and Engineering at NSF.12 These studies are either limited in the number and type of participating institutions, the disciplines involved, or the nature of the international experience (credit-bearing vs. research, for example), but they lay the groundwork for future and longitudinal studies on this important aspect of graduate student professional development. Related opportunities to engage in meaningful research on graduate education are myriad in number and multi-faceted in scope.These research questions—and the
associated funding—await those who are willing to connect the skills at the perimeter of the graduate studies career genome and find the pathways that lead to success. In my new role as CGS/NSF Dean-in-Residence, it is my task to connect the needs of the graduate education community with existing funding mechanisms, and to look for ways that the funding agencies can better respond to the emerging trends in our graduate programs. NSF’s multi-pronged mission is accomplished across its seven directorates
through discipline-specific as well as cross-cutting programs. Two current cross-cuttingthemes are related exactly to understandingcomplex technological problems, analogousto the Human Genome Project or the BrainInitiative.13 One is in “Big Data” and the otheris “Innovations at the Nexus of Food, Energy,and Water Systems” (INFEWS). The INFEWSprogram goals are to: 1) support integratedexperimental research towards creating acomprehensive food-energy-water socio-technical systems model; 2) advanceknowledge/technologies that foster safer,more secure, and more efficient use ofresources within the food-energy-waternexus, and; 3) support an integratedapproach to build the next-generationINFEWS workforce. Part of that ability tosupport integrated research, advanceknowledge and build workforce necessarilymust involve research into how thoseprocesses best occur, that is, research on theprocess of doing research and buildingresearch capacity.Within NSF’s Directorate for Education and
Human Resources (EHR) is the Division ofGraduate Education (DGE). DGE has longsupported graduate students through itsGraduate Research Fellowship Program(GRFP), and has been forward-looking in itssupport of science, technology, engineering,and mathematics (STEM) graduate educationthrough its Innovative Graduate Educationand Research Training (IGERT) program. Ithopes to integrate research on graduateeducation directly into its programs. Themost recent program—National ResearchTraining (NRT)—has two foci, one of which isInnovations in Graduate Education (IGE). IGEpresents a unique opportunity to study thescience of graduate STEM education (seeRich Boone’s GradEdge article from March2015). In an effort to publicize these researchopportunities, eight of our graduateeducation colleagues from DGE spent partsof March and April traveling to the four CGS-affiliate regional meetings in order to engagein dialogue directly with the graduateeducation community. In addition to theprograms described above, these programofficers highlighted the opportunity for us toparticipate in fundamental research ongraduate learning through the EducationCore Research (ECR) program. Master’s levelgraduate education, for example, is ofprimary concern to many CGS institutions.Research on new training and educationmodels at the master’s degree level fitperfectly within the goals of the ECRprogram.
The interest in graduate educationresearch extends beyond DGE and into thedisciplinary directorates of NSF as well. Arecent “Dear Colleague Letter” (DCL in NSFparlance) from the Division of Chemistry14invited researchers in the chemical sciencesto submit proposals for conferences “thatassess the current status and develop plansfor alternative approaches to research andgraduate education in chemistry.”Professional societies such as the AmericanChemical Society have long been interestedin the research of education at all levels, asevidenced by their most recent report ongraduate education in the chemicalsciences.15 Finally, the Office of InternationalScience and Engineering (OISE) at NSF islaunching an evaluation of its long-standingPartnerships for International Research andEducation (PIRE) program to learn, in part,how support for international researchexperiences benefit the careers of graduatestudent researchers.I encourage you to think broadly about
how your institution, programs and theiraffiliated professional society and private-sector partners can find support fordocumenting the impact of your uniquegraduate education and professional trainingprograms. Together we must also start thestopwatch on much-needed longitudinalstudies on the pedagogical and knowledge-building components of graduate education.As the Carnegie Foundation for theAdvancement of Teaching puts it in one oftheir core principles: “we cannot improve atscale what we cannot measure.”16 We needto make assessment and evaluation part ofour graduate education genetic makeup.
By Brian Mitchell, CGS/NSF Dean-in-Residence
This program, including this article was madepossible by a grant from NSF (grant #1102970).Any opinions, findings, and conclusions orrecommendations expressed in this article donot necessarily reflect the views of NSF.
6Jessica K. Graybill, Sarah Dooling, VivekShandas, John Withey, Adrienne Greve, andGregory L. Simon, “A Rough Guide toInterdisciplinarity: Graduate StudentPerspectives,” BioScience (2006) 56 (9): 757-763 doi:10.1641/0006-3568(2006)56[757:ARGTIG]2.0.CO;2.7http://www.aacu.org/publications-research/periodicals/responsibility-doctoral-programs-career-preparation-future-faculty8Taylor, M, Reform the Ph.D. System or Closeit Down, 20 April 2011. Nature 472, 261(2011) | doi:10.1038/472261a.
sf15055.jsp?WT.mc_id=USNSF_25&WT.mc_ev=click.15“Advancing Graduate Education in theChemical Sciences” Full Report of an ACSPresidential Commission Submitted to ACSPresident Bassam Z. Shakhashiri onDecember 6, 2012.http://www.acs.org/content/dam/acsorg/about/governance/acs-presidential-graduate-education-commission-full-report.pdf.16http://www.carnegiefoundation.org/our-ideas/six-core-principles-improvement/.
Data Sources: Highlights from the DoctoralInitiative on Minority Attrition and CompletionThe Council of Graduate Schools (CGS) has
just published Doctoral Initiative on MinorityAttrition and Completion (Sowell, Allum, &Okahana, 2015), which discusses patterns ofdoctoral completion and attrition amongunderrepresented minority (URM) studentsin science, technology, engineering, andmathematics (STEM) fields, as well as theexperiences of students and graduateprograms related to program completion.The publication was made possible by agrant from the National Science Foundation(#1138814) as well as the participation of 21U.S.-based CGS member institutions. Theproject, which began in 2011, assembled thelargest dataset of its kind that includes: 7,500student-level records, 1,600 responses to adoctoral student survey, and insight frommore than 320 students and as many ormore faculty and administrators whoparticipated in focus group sessions at 16 ofthe 21 institutions. The report gives the mostcomprehensive account of completion andattrition rates of URM STEM doctoralstudents in the U.S. to date. The full report isavailable at the CGS website, and this articlehighlights some of findings.
URM Doctoral Completion RatesThe study found that 54% of URM doctoral
students who entered their STEM programsat the participating institutions between May1992 and April 2002 earned their doctorateswithin ten years. As shown in Figure 1, therewas a 12 percentage point increase in thedoctoral completion rate between yearsseven and ten for URM STEM students. Whilethe 10-year URM STEM doctoral completion
rate in this study was fairly close to the 10-year doctoral completion rate for all students(55%, excluding international students)reported in the CGS Ph.D. CompletionProject, the Doctoral Initiative on MinorityAttrition and Completion (DIMAC) resultsreveal some striking differences betweenURM STEM doctoral students. For example,the 10-year completion rate of physical &mathematics sciences URM doctoral studentswas 18 percentage points below that ofstudents in life sciences fields. The 10-yearcompletion rate for Black/African Americanstudents was eight percentage points belowthat of Hispanic/Latino doctoral students.While this difference between Black/AfricanAmerican and Hispanic/Latino students waspartly explained by field and gender effects,factors other than race/ethnicity did notseem to fully explain the difference. CGSresearchers will explore this differencefurther in the coming months and prepare aseparate research brief.The results also indicated that the seven-
year completion rate for URM STEM doctoralstudents increased by five percentage pointswhen comparing the earliest academic yearcohort group (AYs 1992/93 – 1994/95) in theproject population and the most recentcohort group (AYs 2003/04 and 2004/05).While this finding is not attributed to anyone policy, program, or intervention aimedto facilitate the success of URM STEMdoctoral students, it is understood that manyprograms, such as NSF’s Alliance for GraduateEducation and the Professoriate (AGEP), theU.S. Department of Education’s GraduateAssistance in Areas of National Need
(GAANN), and the U.S. Department ofEducation’s McNair Scholars Programemerged during this time period. In otherwords, the findings of this study suggestthat, collectively speaking, “something isworking” to facilitate the success of URMSTEM doctoral students. This finding alsohighlights a need for better assessment andevaluation mechanisms to help guidegraduate schools and policymakers indetermining optimal resource allocation thatfacilitate greater success of URM STEMdoctoral students.
Support for URM STEM Doctoral StudentsData gathered from the inventory of
policies, practices, and interventions, as wellas from the student survey and focus groupsessions provided a rich source ofinformation about the URM STEM doctoralexperience, as well as how graduate schoolshave been working to help facilitate studentsuccess. The results suggest that many of thepolicies, practices, and interventionsintended to support STEM doctoral studentsin general are long-standing; however, fewinterventions at the institutional level arededicated specifically to URM students inSTEM doctoral programs. The study alsofound that a number of interventionsfocused on transitional challenges thatincoming STEM doctoral students face.Specifically, there is a particularly strongemphasis on the recruitment, selection, andadmission of STEM doctoral students and theacclimation of new doctoral students to theculture of graduate school. For example,national programs such as the McNair
Scholars Program and NSF’s Louis StokesAlliance for Minority Participation and Bridgeto Doctorate programs are credited for beingeffective means of recruiting and preparingURM students for the academic rigor of STEMdoctoral programs. However, few formalprograms are available for URM STEMdoctoral students to seek support in thelatter stage of the doctoral process.This finding poses some concerns given
the fact that the study suggests thatstudents, although generally satisfied withsupport they receive from their facultymembers and graduate programs, seemed togrow more skeptical at the dissertationphase of the doctoral experience. Forexample, the survey results from this studyindicated that 30% of doctoral candidatesdisagreed or strongly disagreed with thestatement, “Faculty are aware of issues facingURM students,” and 20% disagreed orstrongly disagreed with the statement, “Thisprogram is doing a good job helping URMstudents succeed,” while 22% and 13% ofpre-candidates, respectively, did the same.Also, of the candidates who responded tothe survey, 65% indicated that theyfrequently or occasionally felt “Worried abouttheir mental and physical health,” while 59%of pre-candidates in the survey did the same.Nearly one-half (45%) of candidates in thesurvey responded that they frequently oroccasionally felt “Isolated from otherstudents,” while roughly one-third (32%) ofpre-candidates did the same.Programs such as NSF’s AGEP and the
Sloan Foundation’s Minority Ph.D. Program,as well as regional or institutional effortssuch as the McKnight Doctoral Fellowshipand the Meyerhoff Scholars Program offersupport that is specifically targeted toward
URM students in STEM doctoral programs.However, this study found that aside fromthese kinds of programs, many interventionsfor URM STEM doctoral students are ofteninformal and sometimes ambiguous innature. Doctoral students, especially thosewho are in the dissertation phase of theirprogram, rely on peer support, mentorships,and personal determination.
Next StepsResults of the DIMAC project offer valuable
national benchmarks for CGS memberinstitutions to assess the state of URM STEMdoctoral students at individual institutions. Inaddition, the report also suggests severalways to further facilitate their successfulcompletion. First, the fact that formalinterventions are largely front-loaded duringthe early stages of the doctoral processsuggest that additional support beyond thefirst year of the doctoral program could helpensure that such early investments are notwasted. Such latter-stage interventions donot necessarily need to be focused oncurricular aspects of the doctoral experience.For example, resources explaining each stageof the doctoral experience could be useful tofirst-generation students and could helpreinforce positive communication betweenstudents and their families. Latter-stageinterventions may also include professionaldevelopment opportunities for faculty inorder to help them be more effectiveadvisors.The report findings also suggest that
additional monitoring and evaluation ofprograms and interventions could provideevidence to insights into effective andineffective programs. This study alsosuggests that visible commitments to the
diversification of the student body in allaspects of the graduate school communitycan have positive effects on STEM doctoralcompletion. For example, theimplementation of diversity/culturalsensitivity training for administrators, faculty,and students as well as the provision ofincentives to mentors and champions couldhelp to promote a culture of diversity andinclusion. To paraphrase the words of oneuniversity staff member who participated ina DIMAC focus group session, diversity andinclusion are not just matters of increasingthe number of URM students; they arematters of changing the climate of diversityand inclusion on campus. Diversity andinclusion must be seen as a part of theexcellence agenda, rather than bejuxtaposed to it.
This article was prepared by Jeff Allum,Director, Research and Policy Analysis andHironao Okahana, Research Associate, Councilof Graduate Schools. Robert Sowell, former CGSVice President of Operations and Programs,also co-authored the CGS report, “DoctoralInitiative on Minority Attrition andCompletion.”
This project, including this article was madepossible by a grant from NSF (grant #1138814).Any opinions, findings, and conclusions orrecommendations expressed in this article donot necessarily reflect the views of NSF.
ReferencesSowell, R., Allum, J., Okahana, H. (2015).Doctoral Initiative on Minority Attrition andCompletion. Washington, DC: Council ofGraduate Schools.
Figure 1: 10-Year STEM Doctoral Completion Rates
Did you knowthat beginningJuly 1 of thisyear, all publiccolleges and
universities are required to offer in-state tuition rates, including forgraduate programs, to qualifiedveterans and to their dependents,regardless of state residency status?
