Jan Alexis NielsenDepartment of Science EducationUniversity of Copenhagen
Presentation at the ESERA Summer School, June 26, 2018, Jyväskylä, Finland
A challenge for practice and a topic for research.
Generic competences in science teaching.
Generic Learning Goals
Policy-makers (in many parts of the world) increasingly call for science teaching to foster student learning and competence development related to:
• Inquiry• Modelling• Argumentation • Technology • Engineering • Innovation• Entrepreneurship• Communication• Science Practices, • 21st century skills
27/06/2018 2
Learning goals that fall under these areas can be called generic: they are targets of teaching in multiple disciplines or subjects, and there is often no school discipline that they belong exclusively to.
Key definitions
A generic learning goal is a target of teaching in multiple disciplines or subjects, and does not exclusively belong to any one school discipline.
A competence is a readiness to act based on relevant knowledge and skills in a way that meets the challenges of a given situation (cf. Blomhøj & Jensen, 2003).
27/06/2018 3
Example 1: Socioscientific Argumentation Competence
27/06/2018 4
Disciplinary knowledge/skills
Other relevant knowledge/skills
Societal issues
Paraphrased learning goal:Students should be able to engage in argumentation and make informed decisions about societal issues by drawing on relevant knowledge and skills
27/06/2018 5
Physics
Biology
Chemistry
Geograhy
Uni-disciplinary: Generic goal is woven
into the disciplines
In Denmark (year 10-12), SSI argumentationcompetence is a goal of most (science) disciplines on par with core disciplinary skills/knowledge
Example 2: Inquiry Competence
27/06/2018 6
Formulate questions
Plan and carry out investigations
Analyse data
Communicate and discuss results
Disciplinary knowledge/skills
Paraphrased learning goal:Students should be able to formulate questions, plan and carry out investigations, analyse data, and communicate and discuss results by drawing on relevant knowledge and skills.
27/06/2018 7
Physics
Biology
Chemistry
Geography
Cross-disciplinary:Generic goal is explicitly
cross-disciplinary
In Denmark (year 7-9), inquiry competence is one of four overarching competence goals for all science disciplines – these goals complement disciplinary knowledge/skills
27/06/2018 8
Physics
Biology
Chemistry
Geography
Cross-disciplinary:Generic goal is explicitly
cross-disciplinary
Physics
Biology
Chemistry
Geograhy
Uni-disciplinary: Generic goal is woven
into the disciplines
Activity 1:From your perspective as researchers, what do you think could challenge the inclusion of the two learning goals in the science disciplines in a curriculum?
It may help to talk about how a teacher could teach for these learning goals in your own discipline.
One in each group types in your ideas on www.menti.comuse code 86 64 70
27/06/2018 9
SSI argumentation:Students should be able to engage in argumentation and make informed decisions about societal issues by drawing on relevant knowledge and skills.
Inquiry:Students should be able to formulate questions, plan and carry out investigations, analyse data, and communicate and discuss results by drawing on relevant knowledge and skills.
Main thematic challenges in the research
27/06/2018 10
Conceptual unclarity: It is unclear (even to researchers) what these goals actually mean
Primacy of content: A focus on disciplinary content often eclipses a focus on these goal
Issues about assessment: These goals may be difficult for teaches to assess (formatively and summatively)
Main thematic challenges in the research
27/06/2018 11
SSI argumentation Inquiry
Conceptual unclarity: It is unclear (even to researchers) what these goals actually mean
Primacy of content: A focus on disciplinary content often eclipses a focus on these goal
Issues about assessment: These goals may be difficult for teaches to assess (formatively and summatively)
Research on SSI argumentation: Conceptual unclarity
27/06/2018 12
• Many scholars have focused on whetherstudents use science evidence in SSI argumentation; but it is beginning to emerge that SSI argumentation should be treated as being much richer (Nielsen, 2013b, 2017)
• Students can use science as strategic rhetoric devices or to ‘win the argument’ (Nielsen, 2011; 2012).
• We lack an operational concept of SSI argumentation that outlines what it means to balance facts and values in legitimate ways.
Changes from somatic gene therapy are not heritable
Factual premise
We should allow somatic gene therapy!
Value-laden statement
It is OK to make changes to the body that are not heritable
Value premise
Research on SSI argumentation: Primacy of content
Discipline Discipline
Socioscientific issues
Socioscientific Issues
Disciplinary content
Competences
Scenario 1 (the curriculum)
Scenario 2 (our respondents)• SSIs are often implemented in a
superficial fashion (e.g. Lazarowitz & Bloch, 2005).
• Teachers’ narratives often reflect a content-centred understanding of SSI – where SSIs merely play an instrumental role in motivating the disciplinary content (Tidemand & Nielsen, 2017).
Research on Inquiry: Conceptual unclarity
27/06/2018 14
• While ‘inquiry’ has been one of the most central notions of science education for at least 20 years, inquiry is not close to being uniformly defined
• A recent review identified no less than 19 distinct skills/sub-competences that are regularly associated with inquiry competence in multifarious combinations (Bernholt et al., 2013).
www.assistme.ku.dk 15 October 2013 31
Table 9: Scheme for the evaluation of the literature
Column Content Literature author(s) General information about the investiga-tion/ analysis
year country design (Survey, Intervention, Evaluation, Case Study, Meta-analysis) domain (Science, Technology, Mathematics) sample(s) size (N) sample characteristics: grade (school type) sample characteristics: age
Content focus of the investigation/ analysis (either as focus of the intervention/learning environment/curricula or as focus of the assessment)
scientific inquiry/science process skills diagnosing problems/ identifying questions searching for information considering alternative or multiple solutions creating mental representations constructing and using models formulating hypotheses planning investigations constructing prototypes finding structures or patterns researching conjectures collecting and interpreting data evaluating results searching for alternatives/ modifying designs constructing and critiquing arguments or explanations/ argumentation/ reasoning/ using evidence debating with peers/ communication searching for generalizations dealing with uncertainty knowledge/ achievement/ understanding/ conceptual change problem solving other
(Bernholt et al, 2013)
Research on Inquiry: Primacy of content
• In inquiry-based teaching the teacher-role becomes complex and focus is on guiding students through processes (e.g. Crawford, 2000).
• The complexity can lead teachers to stick to a role of covering content when teaching inquiry (e.g. Nielsen, 2017).
MotivatorGuide
Diagnostician
Innovator
ExperimenterResearcher
Modeler
Mentor
Collaborator
Learner
Teacher roles in inquiry teaching, after Crawford (2000)
Main thematic challenges in the research
27/06/2018 16
Socioscientific argumentation competence
Inquiry competence
Conceptual unclarity Tendency to forget that SSI argumentation involves much more than the use of disciplinary knowledge
Multifarious definitions and descriptions of a wide variety of subordinate skills
Primacy of content SSI is often used as a means to an end (framing disciplinary content), rather than an end in itself
Uncertainty about the role of teacher-as-guide may lead teachers to focus on ‘delivering’ content
Issues about assessment
27/06/2018 17
Student activity
Data
Judgement of level
Report of level
Summative
Next step in learning
Formative
(after Harlen, 2013)
Assessment
Justification in terms of assessment criteria
Assessment: one of the most important factors in education
• Washback effect: Over time the goings-on in teaching will become more and more aligned with what is being measured at in summative assessment (Harlen, 2007).
27/06/2018 18
Activity 2:Choose one of the two learning goals. From your perspective as researchers, what are your ideas about how to summatively assess the chosen learning goal?
Discuss what you think the key assessment criteria are and then come up with ideas for which kind of student activity is best suited for assessing the learning goal.
One in each group types in your ideas on www.menti.comSSI: code 26 14 39Inquiry: code 76 88 51(you can write more than one idea)
27/06/2018 19
SSI argumentation:Students should be able to engage in argumentation and make informed decisions about societal issues by drawing on relevant knowledge and skills.
Inquiry:Students should be able to formulate questions, plan and carry out investigations, analyse data, and communicate and discuss results by drawing on relevant knowledge and skills.
Research on SSI argumentation: Issues about assessment
• Many science teachers feel unable to handle and give feedback on students’ SSI discussions (Bryce & Gray, 2004)
• There is a general lack of effective assessment criteria for assessing SSI argumentation (Levinson & Turner, 2001);
• Examinations rarely test SSI argumentation validly (if at all) (Tidemand & Nielsen, 2017).
• Most assessment frameworks for SSI argumentation are constructed for assessing written arguments – and would not lead to valid assessment of dialogic argumentation (Nielsen, 2013b)
27/06/2018 21
Research on SSI argumentation: Issues about assessment
Student activity
Data
Judgement of level
Report of level
Summative
Next step in learning
Formative
Justification in terms of assessment criteria
Lack of support for teachers to translate the competence to criteria
$
Typical summative assessment activities are not ideal for this competence
Lack of guides for teacher-judgement
Research on Inquiry: Issues about (summative) assessment
• Large body of research into summative assessment of inquiry competences using multiple-choice, constructed-response or open-ended items (Bernholt et al., 2013).
• But it can be argued that this type of activity offers only a reduced vision of students’ inquiry learning – primarily focusing on student understanding (e.g. Hume & Coll, 2010)
www.assistme.ku.dk 15 October 2013 63
Figure 2: Formative assessment item on dominance relationships (Hickey & Zuiker, 2012, p. 24)
To assess students’ understanding of key concepts, concept maps instead of items are often used for a summative assessment. For example, Brandstädter, Harms, and Großschedl (2012) investigate concept maps as an assessment tool for system think-ing in biology education. As the process of the concept map development is quite com-plex, some approaches use computer-assisted methods (e. g. Schaal, Bogner, & Gir-widz, 2010).
On the other hand, concept maps can be used for formative assessment. In this case, the focus lies on checking students’ progress in understanding key concepts at several times during a treatment (e. g. Furtak et al., 2008). The analysis of concept maps can be organised by rubrics as shown in Table 29 (e. g. Nantawanit, Panijpan, & Ruen-wongsa, 2012).
In general, it is important to train students in the procedure of making a concept map (Nantawanit et al., 2012). One possible way is the think-pair-share method: First, stu-dents make an individual map, then, they build a map in a small group, and finally, they construct a concept map as a class (e. g. Furtak et al., 2008). Another common method is to give the concepts and linking words to the students (see Figure 3). Both ap-proaches have a more formative than summative character.
(Hickey & Zuiker, 2012, p. 24)
Research on Inquiry: Issues about (summative) assessment
• It may be, that using performanceassessment is the best way to secure valid summative assessment of inquiry competence (e.g. Barron & Darling-Hammond, 2008).
• But performance assessment is more complex and time-consuming – and requires substantial support for teachers (Bernholt et al., 2013).
http://ictflwinterfest-2012.wikispaces.com/file/view/ATM_cartoon.jpg
Research on Inquiry: Issues about (formative) assessment
• We have knowledge about good practices for formative assessment for inquiry – e.g. using structured whole-class discussions as assessment conversations (Ruiz-Primo & Furtak, 2007).
• But even experienced teachers need support in translating inquiry competence into assessment criteria that suits their practice (Nielsen, Tidemand & Dolin, 2018).
• Knowledge of good practices are only slowly being implemented in national assessment systems (Nielsen, 2017).
thinking requires the teacher to judge differences between students’ responses, explanations, ormental models so the critical dimensions relevant for their learning can be made explicit (e.g.,teacher compares students’ responses according to the evidence provided or responds to studentsby askingwhich explanation is more scientifically acceptable based on the information provided).Using information from assessment conversations involves taking action on the basis of studentresponses to help students move toward learning goals (e.g., helping students to reach consensuson a particular explanation based on scientific evidence). The range of student conceptions atdifferent points of a unit should determine the nature of the conversation; therefore, more than oneiteration of the cycle of eliciting, recognizing, and using may be needed to reach a consensus thatreflects the most complete and appropriate understanding.
Assessment conversations can be thus be described in the context of classroom talk moves(Christie, 2002) as ESRU cycles—the teacher Elicits a question; the Student responds; the teacherRecognizes the student’s response; and then Uses the information collected to support studentlearning.2 This model of informal formative assessment is depicted in Figure 1.
We distinguish the ESRU model from the IRE/F sequence in three ways: First, the elicitingquestions used to initiate a sequence have the potential for providing information on the evolvingstatus of student conceptions and understanding about scientific inquiry skills and habits of mind.For example, teachers can ask questions about understanding key points (e.g., clarificationquestions about vague statements or lifting questions to improve the level of discussion). Second,recognizing students’ responses in diverse ways (e.g., revoicing, rephrasing) is consideredfundamental because it indicates to the student that his/her contribution has been heard andaccepted into the ongoing classroomnarrative (Scott, 1998). Recognizing students’ responses alsoprovides the opportunity for the teacher to pull out the essential aspects of student participationand to act on them, and also provides the student the opportunity to evaluate the correctness of theteacher’s interpretation of their contribution.
Third, our conception of using comes from the formative assessment and the scientific inquiryliterature (Black & Wiliam, 1998; Clarke, 2000; Duschl, 2000, 2003; Duschl & Gitomer, 1997;National Research Council, 2001; Ramaprasad, 1983; Sadler, 1989, 1998). Use, in our sequence,is more specific than the meaning assigned to feedback in the IRF sequence. Within ‘‘use,’’ ateacher can provide students with specific information on actions they may take to reach learninggoals: ask another question that challenges or redirects the students’ thinking; modelcommunication; promote the exploration and contrast of students’ ideas; make connectionsbetween new ideas and familiar ones; recognize a student’s contribution with respect to the topicunder discussion; or increase the difficulty of the task at hand. Clearly, evaluations by themselves(e.g., ‘‘good’’ or ‘‘excellent’’) cannot be part of our ESRU pattern unless they are embedded in
Figure 1. The ESRU model of informal formative assessment.
INFORMAL ASSESSMENT PRACTICES 61
Journal of Research in Science Teaching. DOI 10.1002/tea
Ruiz-Primo & Furtak’s (2007, figure 1, p. 61) ESRU model of structuring formative assessment in classroom conversations.
27/06/2018 25
Research on Inquiry: Issues about assessment
Student activity
Data
Judgement of level
Report of level
Summative
Next step in learning
Formative
Justification in terms of assessment criteria
Multifarious skills that need to be translated into criteria
$
Typical summative assessment activities are not ideal for this competence
Teachers need substantial support, may increase in performance assessment
Main thematic challenges in the research
27/06/2018 26
Socioscientific argumentation competence
Inquiry competence
Conceptual unclarity Tendency to forget that SSI argumentation involves much more than the use of disciplinary knowledge
Multifarious definitions and descriptions of a wide variety of subordinate skills
Primacy of content SSI is often used as a means to an end (framing disciplinary content), rather than an end in itself
Uncertainty about the role of teacher-as-guide may lead teachers to focus on ‘delivering’ content
Issues about assessment There is a lack of effective formative and summative assessment practices
We have knowledge of good formative assessment practice, but changing assessment practices requires support
Generic learning goals face challenges in practice
27/06/2018 27
Primacy of content Issues about assessment
Conceptual confusion
Future research needs to investigate the direction of the relations in order to provide strategies for professional development solutions.
How to welcome a generic learning goal to the curriculum!
27/06/2018 28
Combining ideas from two research projects on generic learning goals, may be a path to follow
“Upper secondary school rethought” (innovation competence) (Inquiry competence)
How to welcome a generic learning goal to the curriculum!
Conceptual clarity: Experienced teachers formulate overarching assessment criteria for “new” generic learning goals together with researchers and other stakeholders (Nielsen, 2015a).
Micro-level insight into how the learning goal can be operationalised into constructs that can inform teaching and assessment (Nielsen, Tidemand & Dolin, 2018)
27/06/2018 29
Assessment practice: Experiencedteachers systematically implement different assessment methods for assessing the generic learning goal (Dolin, Nielsen & Tidemand, 2017).
Enables deep teacher understanding of criteria and assessment practice for the learning goal (Nielsen, 2015b)
Solid basis for changing wider assessment culture so as to better accommodate the learning goal (Nielsen, Tidemand & Dolin, 2018).
ReferencesAbd El Khalick, F., Boujaoude, S., Duschl, R. A., Lederman, N. G., Mamlok-Naaman, R., Hofstein, A., . . . Taylor, P. C. (2004). Inquiry in science
education: International perspectives. Science Education, 88(3), 397–419.Barron, B. & Darling-Hammond, L. (2008). Teaching for meaningful learning: A review of research on inquiry-based and cooperative learning. In L.
Darling-Hammond, B. Barron, P. D. Pearson, A. H. Schoenfeld, E. K. Stage, T. D. Zimmermann, G. N. Cervetti, & J. Tilson (Eds.), Powerful Learning. What we know about teaching for understanding. San Francisco: Jossey-Bass.
Bernholt, S., Rönnebeck, S., Ropohl, M., Köller, O., & Parchmann, I. (2013). Report on current state of the art in formative and summative assessment in IBE in STM-Part 1. ASSIST-ME Report Series(1).
Blomhøj, M., & Jensen, T. H. (2003). Developing mathematical modelling competence: conceptual clarification and educational planning. Teaching Mathematics and its Applications, 22(3), 123-139. doi:10.1093/teamat/22.3.123
Christenson, N., & Chang Rundgren, S.-N. (2015). A Framework for Teachers’ Assessment of Socio-scientific Argumentation: An example using the GMO issue. Journal of Biological Education, 49(2), 204-212. doi:10.1080/00219266.2014.923486
Crawford, B. A. (2000). Embracing the essence of inquiry: New roles for science teachers. Journal of Research in Science Teaching, 37(9), 916–937.Day, S. P., & Bryce, T. G. K. (2010). Does the Discussion of Socio-Scientific Issues require a Paradigm Shift in Science Teachers' Thinking?
International Journal of Science Education, 33(12), 1675-1702. doi:10.1080/09500693.2010.519804Dolin, J., Black, P., Harlen, W., & Tiberghien, A. (2018). Exploring Relations Between Formative and Summative Assessment. In J. Dolin & R. Evans
(Eds.), Transforming Assessment: Through an Interplay Between Practice, Research and Policy (pp. 53-80). Cham: Springer International Publishing.
Dolin, J., Nielsen, J. A., & Tidemand, S. (2017). Evaluering af naturfaglige kompetencer. Acta Didactica Norge, 11(3), 1-28. Duschl, R. A. (2007). Quality Argumentation and Epistemic Criteria. In S. Erduran & M. P. Jiménez-Aleixandre (Eds.), Argumentation in Science
Education (pp. 159-175): Springer Netherlands. Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and Quasi- Experimental Studies of Inquiry-Based Science Teaching: A
Meta-Analysis. Review of Educational Research, 82(3), 300–329. Harlen, W. (2007). Assessment of Learning. London: Sage Publications Ltd.Harlen, W. (2013). Assessment and Inquiry-Based Science Education: Issues in Policy and Practice. Trieste: IAP.Hickey, D. T., & Zuiker, S. J. (2012). Multilevel Assessment for Discourse, Understand- ing, and Achievement. Journal of the Learning Sciences, 21(4),
522–582.Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and Achieve- ment in Problem-Based and Inquiry Learning: A Response to
Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107.Hume, A., & Coll, R. K. (2010). Authentic student inquiry: The mismatch between the intended curriculum and the student-experienced curriculum.
Research in Science & Technological Education, 28(1), 43–62.
27/06/2018 31
ReferencesIrwin, A., & Wynne, G. (1996). Misunderstanding science. Cambridge: Cambridge University Press.Kolstø, S. D. (2001). ‘To trust or not to trust, . . .’- pupils’ ways of judging information encountered in a socio-scientific issue. International Journal
of Science Education, 23(9), 877-901.Lazarowitz, R., & Bloch, I. (2005). Awareness of Societal Issues Among High School Biology Teachers Teaching Genetics. Journal of Science
Education and Technology, 14(5-6), 437-457. doi:10.1007/s10956-005-0220-4Lee, H., & Witz, K. G. (2009). Science Teachers' Inspiration for Teaching Socio-scientific Issues: Disconnection with reform efforts. International
Journal of Science Education, 31(7), 931-960. doi:10.1080/09500690801898903Levinson, R. (2006). Towards a Theoretical Framework for Teaching Controversial Socio-scientific Issues. International Journal of Science Education,
28(10), 1201-1224. doi:10.1080/09500690600560753Levinson, R., & Turner, S. (2001). The teaching of social and ethical issues in theschool curriculum, arising from developments in biomedical
research: a research study of teachers. London: The Welcome Trust.Lindahl, M. G., & Linder, C. (2013). Students’ Ontological Security and Agency in Science Education—An Example from Reasoning about the Use of
Gene Technology. International Journal of Science Education, 35(14), 2299-2330. doi:10.1080/09500693.2011.618516Millar, R., & Osborne, J. (1998). Beyond 2000: science education for the future. London: King's College London.Nielsen, J. A. (2011). Co-opting Science: A preliminary study of how students invoke science in value-laden discussions. International Journal of
Science Education, 34(2), 275-299. doi:10.1080/09500693.2011.572305Nielsen, J. A. (2013a). Dialectical Features of Students’ Argumentation: A Critical Review of Argumentation Studies in Science Education. Research in
Science Education, 43(1), 371-393. doi:10.1007/s11165-011-9266-xNielsen, J. A. (2013b). Delusions About Evidence: On Why Scientific Evidence Should Not Be the Main Concern in Socioscientific Decision-Making.
Canadian Journal for Science, Mathematics, and Technology Education, 13(4), 373-385. doi:10.1080/14926156.2013.845323Nielsen, J. A. (Ed.) (2017). Litteraturstudium til arbejdet med en national naturvidenskabsstrategi. København: Institut for Naturfagenes Didaktik.Nielsen, J. A., Tidemand, S., & Dolin, J. (2018). Transforming Assessment Research: Recommendations for Future Research. In J. Dolin & R. Evans
(Eds.), Transforming Assessment: Through an Interplay Between Practice, Research and Policy. (pp. 279-290). Cham: Springer.Nordenbo, S., Allerup, P., Andersen, H. L., Korp, H., & Dolin, J. (2009). Pædagogisk brug af test – en systematisk review. København: Danmarks
pædagogiske universitetsforlag og Dansk Clearinghouse for Uddanelsesforskning.Orlander Arvola, A., & Lundegård, I. (2011). ‘It’s Her Body’. When Students’ Argumentation Shows Displacement of Content in a Science Classroom.
Research in Science Education, (Advance online publication), 1-25. Retrieved from http://dx.doi.org/10.1007/s11165-011-9237-2 doi:10.1007/s11165-011-9237-2
Ratcliffe, M., & Grace, M. (2003). Science education for citizenship: teaching socio-scientific issues. Maidenhead: Open University Press.
27/06/2018 32
ReferencesRuiz-Primo, M. A., & Furtak, E. M. (2006). Informal Formative Assessment and Scien- tific Inquiry: Exploring Teachers' Practices and Student
Learning. Educational As- sessment, 11(3-4), 205–235.Ruiz-Primo, M. A., & Furtak, E. M. (2007). Exploring Teachers' Informal Formative As- sessment Practices and Students' Understanding in the
Context of Scientific Inquiry. Journal of Research in Science Teaching, 44(1), 57–84.Rönnebeck, S., Nielsen, J. A., Olley, C., Ropohl, M., & Stables, K. (2018). The Teaching and Assessment of Inquiry Competences. In J. Dolin & R. Evans
(Eds.), Transforming Assessment: Through an Interplay Between Practice, Research and Policy. (pp. 27-52). Cham: Springer.Sadler, T. D., & Zeidler, D. L. (2003). The morality of socioscientific issues: Construal and resolution of genetic engineering dilemmas. Science
Education, 88(1), 4-27. doi:10.1002/sce.10101Simonneaux, L. (2014). Questions Socialement Vives and Socio-scientific Issues: New Trends of Research to Meet the Training Needs of Postmodern
Society. In C. Bruguière, A. Tiberghien, & P. Clément (Eds.), Topics and Trends in Current Science Education: 9th ESERA Conference Selected Contributions (pp. 37-54). Dordrecht: Springer Netherlands.
Tidemand, S., & Nielsen, J. A. (2017). The role of socioscientific issues in biology teaching: from the perspective of teachers. International Journal of Science Education, 39(1), 44-61. doi:10.1080/09500693.2016.1264644
Zuiker, S., & Whitaker, J. R. (2014). Refining Inquiry with Multi-Form Assessment: Formative and summative assessment functions for flexible inquiry. International Journal of Science Education, 36(6), 1037-1059. doi:10.1080/09500693.2013.834489
27/06/2018 33