action researchIn this article we explore the development of some different traditions of action research and provide an introductory guide to the literature.contents: introduction origins the decline and rediscovery of action research conclusion further reading how to cite this article. see, also: research for practice.

In the literature, discussion of action research tends to fall into two distinctive camps. The British tradition - especially that linked to education - tends to view action research as research oriented toward the enhancement of direct practice. For example, Carr and Kemmis provide a classic definition: Action research is simply a form of self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justice of their own practices, their understanding of these practices, and the situations in which the practices are carried out (Carr and Kemmis 1986: 162). Many people are drawn to this understanding of action research because it is firmly located in the realm of the practitioner - it is tied to self-reflection. As a way of working it is very close to the notion of reflective practice coined by Donald Schn (1983). The second tradition, perhaps more widely approached within the social welfare field - and most certainly the broader understanding in the USA is of action research as 'the systematic collection of information that is designed to bring about social change' (Bogdan and Biklen 1992: 223). Bogdan and Biklen continue by saying that its practitioners marshal evidence or data to expose unjust practices or environmental dangers and recommend actions for change. In many respects, for them, it is linked into traditions of citizens action and community organizing. The practitioner is actively involved in the cause for which the research is conducted. For others, it is such commitment is a necessary part of being a practitioner or member of a community of practice. Thus, various projects designed to enhance practice

within youth work, for example, such as the detached work reported on by Goetschius and Tash (1967) could be talked of as action research. Origins Kurt Lewin is generally credited as the person who coined the term 'action research': The research needed for social practice can best be characterized as research for social management or social engineering. It is a type of action-research, a comparative research on the conditions and effects of various forms of social action, and research leading to social action. Research that produces nothing but books will not suffice (Lewin 1946, reproduced in Lewin 1948: 202-3) His approach involves a spiral of steps, each of which is composed of a circle of planning, action and fact-finding about the result of the action (ibid.: 206). The basic cycle involves the following:

This is how Lewin describes the initial cycle: The first step then is to examine the idea carefully in the light of the means available. Frequently more fact-finding about the situation is required. If this first period of planning is successful, two items emerge: namely, an overall plan of how to reach the objective and secondly, a decision in regard to the first step of action. Usually this planning has also somewhat modified the original idea. (ibid.: 205) The next step is composed of a circle of planning, executing, and reconnaissance or fact finding for the purpose of evaluating the results of the second step, and preparing the rational basis for planning the third step, and for perhaps modifying again the overall plan (ibid.:

206). What we can see here is an approach to research that is oriented to problem-solving in social and organizational settings, and that has a form that parallels Deweys conception of learning from experience. The approach, as presented, does take a fairly sequential form and it is open to literal interpretation. Following it can lead to practice that is correct rather than good as we will see. It can also be argued that model itself places insufficient emphasis on analysis at key points. Elliott (1991: 70), for example, believed that the basic model allows those who use it to assume that the general idea can be fixed in advance, that reconnaissance is merely fact-finding, and that implementation is a fairly straightforward process. As might be expected there was some questioning as to whether this was real research. There were questions around action researchs partisan nature the fact that it served particular causes. The decline and rediscovery of action research Action research did suffer a decline in favour during the 1960s because of its association with radical political activism (Stringer 2007: 9). There were, and are, questions concerning its rigour, and the training of those undertaking it. However, as Bogdan and Biklen (1992: 223) point out, research is a frame of mind a perspective that people take toward objects and activities. Once we have satisfied ourselves that the collection of information is systematic, and that any interpretations made have a proper regard for satisfying truth claims, then much of the critique aimed at action research disappears. In some of Lewins earlier work on action research (e.g. Lewin and Grabbe 1945) there was a tension between providing a rational basis for change through research, and the recognition that individuals are constrained in their ability to change by their cultural and social perceptions, and the systems of which they are a part. Having correct knowledge does not of itself lead to change, attention also needs to be paid to the matrix of cultural and psychic forces through which the subject is constituted (Winter 1987: 48). Subsequently, action research has gained a significant foothold both within the realm of community-based, and participatory action research; and as a form of practice oriented to the improvement of educative encounters (e.g. Carr and Kemmis 1986). Exhibit 1: Stringer on community-based action research

A fundamental premise of community-based action research is that it commences with an interest in the problems of a group, a community, or an organization. Its purpose is to assist people in extending their understanding of their situation and thus resolving problems that confront them. Community-based action research is always enacted through an explicit set of social values. In modern, democratic social contexts, it is seen as a process of inquiry that has the following characteristics: It is democratic, enabling the participation of all people. It is equitable, acknowledging peoples equality of worth. It is liberating, providing freedom from oppressive, debilitating conditions. It is life enhancing, enabling the expression of peoples full human potential. (Stringer 1999: 9-10)

The action research process works through three basic phases: Look - building a picture and gathering information. When evaluating we define and describe the problem to be investigated and the context in which it is set. We also describe what all the participants (educators, group members, managers etc.) have been doing. Think interpreting and explaining. When evaluating we analyse and interpret the situation. We reflect on what participants have been doing. We look at areas of success and any deficiencies, issues or problems. Act resolving issues and problems. In evaluation we judge the worth, effectiveness, appropriateness, and outcomes of those activities. We act to formulate solutions to any problems. (Stringer 1999: 18; 43-44;160)

The use of action research to deepen and develop classroom practice has grown into a strong tradition of practice (one of the first examples being the work of Stephen Corey in 1949). For some there is an insistence that action research must be collaborative and entail groupwork. Action research is a form of collective self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justice of their own social or educational practices, as well as their understanding of those practices and the situations in which the practices are carried out The approach is only action research when it is collaborative, though it is important to realise that action research of the group is achieved through the critically examined action of individual group members. (Kemmis and McTaggart 1988: 5-6) Just why it must be collective is open to some question and debate (Webb 1996), but there is an important point here concerning the commitments and orientations of those involved in action research. Conclusion One of the legacies Kurt Lewin left us is the action research spiral and with it there is the danger that action research becomes little more than a procedure. It is a mistake, according to McTaggart (1996: 248) to think that following the action research spiral constitutes doing action research. He continues, Action research is not a method or a procedure for research but a series of commitments to observe and problematize through practice a series of principles for conducting social enquiry. It is his argument that Lewin has been misunderstood or, rather, misused. When set in historical context, while Lewin does talk about action research as a method, he is stressing a contrast between this form of interpretative practice and more traditional empirical-analytic research. The notion of a spiral may be a useful teaching device but it is all too easily to slip into using it as the template for practice (McTaggart 1996: 249). Further reading This select, annotated, bibliography has been designed to give a flavour of the possibilities of action research and includes some useful guides to practice. As ever, if you have suggestions about areas or specific texts for inclusion, I'd like to hear from you.

Explorations of action research

Atweh, B., Kemmis, S. and Weeks, P. (eds.) (1998) Action Research in Practice: Partnership for Social Justice in Education, London: Routledge. Presents a collection of stories from action research projects in schools and a university. The book begins with theme chapters discussing action research, social justice and partnerships in research. The case study chapters cover topics such as: school environment - how to make a school a healthier place to be; parents - how to involve them more in decision-making; students as action researchers; gender - how to promote gender equity in schools; writing up action research projects. Carr, W. and Kemmis, S. (1986) Becoming Critical. Education, knowledge and action research, Lewes: Falmer. Influential book that provides a good account of 'action research' in education. Chapters on teachers, researchers and curriculum; the natural scientific view of educational theory and practice; the interpretative view of educational theory and practice; theory and practice - redefining the problem; a critical approach to theory and practice; towards a critical educational science; action research as critical education science; educational research, educational reform and the role of the profession. Carson, T. R. and Sumara, D. J. (ed.) (1997) Action Research as a Living Practice, New York: Peter Lang. 140 pages. Book draws on a wide range of sources to develop an understanding of action research. Explores action research as a lived practice, 'that asks the researcher to not only investigate the subject at hand but, as well, to provide some account of the way in which the investigation both shapes and is shaped by the investigator. Dadds, M. (1995) Passionate Enquiry and School Development. A story about action research, London: Falmer. 192 + ix pages. Examines three action research studies undertaken by a teacher and how they related to work in school - how she did the research, the problems she experienced, her feelings, the impact on her feelings and ideas, and some of the outcomes. In his introduction, John Elliot comments that the book is 'the most readable, thoughtful, and detailed study of the potential of action-research in professional education that I have read'. Ghaye, T. and Wakefield, P. (eds.) CARN Critical Conversations. Book one: the role of the self in action, Bournemouth: Hyde Publications. 146 + xiii pages. Collection of five pieces from the Classroom Action Research Network. Chapters on: dialectical forms; graduate medical education - research's outer limits; democratic education; managing action research; writing up. McNiff, J. (1993) Teaching as Learning: An Action Research Approach, London: Routledge. Argues that educational knowledge is created by individual teachers as they attempt to express their own values in their professional lives. Sets out familiar action research model: identifying a problem, devising, implementing and evaluating a solution and modifying practice. Includes advice on how working in this way can aid the professional development of action researcher and practitioner. Quigley, B. A. and Kuhne, G. W. (eds.) (1997)Creating Practical Knowledge Through Action Research, San Fransisco: Jossey Bass.Guide to action research that outlines the action research process, provides a project planner, and presents examples to show how action research can yield improvements in six different settings, including a hospital, a university and a literacy education program.

Plummer, G. and Edwards, G. (eds.) CARN Critical Conversations. Book two: dimensions of action research - people, practice and power, Bournemouth: Hyde Publications. 142 + xvii pages. Collection of five pieces from the Classroom Action Research Network. Chapters on: exchanging letters and collaborative research; diary writing; personal and professional learning - on teaching and self knowledge; anti-racist approaches; psychodynamic group theory in action research. Whyte, W. F. (ed.) (1991) Participatory Action Research, Newbury Park: Sage. 247 pages. Chapters explore the development of participatory action research and its relation with action science; and examines its usages in various agricultural and industrial settings Zuber-Skerritt, O. (ed.) (1996) New Directions in Action Research, London; Falmer Press. 266 + xii pages. Useful collection that explores principles and procedures for critical action research; problems and suggested solutions; and postmodernism and critical action research.Action research guides

Coghlan, D. and Brannick, D. (2000) Doing Action Research in your own Organization, London: Sage. 128 pages. Popular introduction. Part one covers the basics of action research including the action research cycle, the role of the 'insider' action researcher and the complexities of undertaking action research within your own organisation. Part two looks at the implementation of the action research project (including managing internal politics and the ethics and politics of action research). New edition due late 2004. Elliot, J. (1991) Action Research for Educational Change, Buckingham: Open University Press. 163 + x pages Collection of various articles written by Elliot in which he develops his own particular interpretation of action research as a form of teacher professional development. In some ways close to a form of 'reflective practice'. Chapter 6, 'A practical guide to action research' - builds a staged model on Lewin's work and on developments by writers such as Kemmis. Johnson, A. P. (2007) A short guide to action research 3e. Allyn and Bacon. Popular step by step guide for master's work. Macintyre, C. (2002) The Art of the Action Research in the Classroom, London: David Fulton. 138 pages. Includes sections on action research, the role of literature, formulating a research question, gathering data, analysing data and writing a dissertation. Useful and readable guide for students. McNiff, J., Whitehead, J., Lomax, P. (2003) You and Your Action Research Project, London: Routledge. Practical guidance on doing an action research project.Takes the practitionerresearcher through the various stages of a project. Each section of the book is supported by case studies Stringer, E. T. (2007) Action Research: A handbook for practitioners 3e, Newbury Park, ca.: Sage. 304 pages. Sets community-based action research in context and develops a model. Chapters on information gathering, interpretation, resolving issues; legitimacy etc. See, also Stringer's (2003) Action Research in Education, Prentice Hall.

Winter, R. (1989) Learning From Experience. Principles and practice in action research, Lewes: Falmer Press. 200 + 10 pages. Introduces the idea of action research; the basic process; theoretical issues; and provides six principles for the conduct of action research. Includes examples of action research. Further chapters on from principles to practice; the learner's experience; and research topics and personal interests.Action research in informal education

Usher, R., Bryant, I. and Johnston, R. (1997) Adult Education and the Postmodern Challenge. Learning beyond the limits, London: Routledge. 248 + xvi pages. Has some interesting chapters that relate to action research: on reflective practice; changing paradigms and traditions of research; new approaches to research; writing and learning about research. Other references Bogdan, R. and Biklen, S. K. (1992) Qualitative Research For Education, Boston: Allyn and Bacon. Goetschius, G. and Tash, J. (1967) Working with the Unattached, London: Routledge and Kegan Paul. McTaggart, R. (1996) Issues for participatory action researchers in O. Zuber-Skerritt (ed.) New Directions in Action Research, London: Falmer Press. Acknowledgements: The picture is by 'today is a good day' and is of a sculpture by Desiree Hope. It is reproduced under a Creative Commons licence: attribution, non-commercial, no derivs. 2.0. (flickr: http://www.flickr.com/photos/good_day/22895946/). How to cite this article: Smith, M. K. (1996; 2001, 2007) 'Action research', the encyclopedia of informal education, www.infed.org/research/b-actres.htm. Mark K. Smith 1996; 2001, 2007

infed is a not-for-profit site [about us] [disclaimer] provided by the YMCA George Williams College. Give us feedback; write for us. Check our copyright notice when copying. Join us on Facebook and Twitter. Hosting by Memset Dedicated Servers [CarbonNeutral].

The Design of Classroom Action Research in Teaching English ResearchPosted by Teaching English 4 All June 29, 2010 2 Comments Filed Under action research, CAR, classroom action research, descriptive, recount, report, teaching english Nowadays, in teaching English, a teacher usually makes and needs development and empowerment into his class. One way to improve the quality of the teaching English process

is to do Classroom Action Research (CAR). The term classroom action research is very familiar with English teachers, particularly professional or certified teachers. In doing it, the teacher also functions as a researcher because it involves teachers inside the classroom rather than being carried out solely by specialists from outside the classroom (Phillips & Carr, 2010). From this way, it is well-known for teacher-researcher. The Classroom Action Research is a part of various kinds of action research out of some other themes like participatory research, critical action research, and action learning (Phillips & Carr, 2010). Considering this, the following explanation starts from presenting definition of action research and then goes to the detailed explanation about the Classroom Action Research.According to Koshy (2005) & Bassey (1998) action research is an enquiry which is carried out in order to understand, evaluate and then modify educational program in order to improve educational practice. Koshy (2005), furthermore, states that an important characteristics of action research is that it proposes opportunities for collaborative work. In addition, Mills (2003) says that action research is any systematic inquiry conducted by teacher researchers, principals, school counselors, or other stakeholders in the teaching and learning environment to gather information about how their particular schools operate, how they teach, and how well their students learn. The explanation above implies that one of the proposed designs of action research is Classroom Action Research involving a teacher in the classroom to empower and develop the quality of the classroom process. The design of classroom action research based on the consideration that the researcher attempts to solve the problem of the particular classroom. It provides a way of thinking systematically about what happens in the school or classroom, implementing critically informed action where improvements are thought to be possible (Kemmis & McTaggart, 1992). Snell (1999) states that action study is concerned with trying to improve one specific point in a teachers technique in a particular classroom using empirical measurement. Additionally, Kasbollah & Sukarnyana (1988) states classroom action research is a reflective research conducted by giving a certain action to improve an unsatisfactory condition and increase the quality of teaching practices in the classroom in order that those practices become more professional. In short, a classroom action research focuses on a particular classroom problem or a group of students and aims at helping the teacher solve the problems in finding an appropriate technique in teaching. Classroom action research is also done by a teacher on the basis of the phenomena on his/her own classroom. McNiff (1992) asserts that a classroom action research is participatory and collaborative. That is why, the researcher participated as the teacher who is teaching English through a proposed strategy. The phenomenon is usually an issue in the classroom, e.g. for MTs or SMP English teacher, he/she faces a problem on a strategy to enhance or improve the students ability in specific type of genre such as: recount text, narrative text, descriptive text, report text, and procedure text. In brief, classroom action research functions as a way, which is designed to enhance the quality of both result and process of the teaching and learning due to satisfactory condition that takes place in the classroom. There are various kinds of classroom action research models. I would like to describe the model of Kemmis and McTaggart (1992). In implementing this model, the following visualization is presented:

The Design of Classroom Action Research The model consists of four steps: (1) planning of the action, (2) implementing of the action, (3) observing of the action, and (4) making analysis and reflection. This advantage of this model is that within the model, a new cycle covering those above stages can be reimplemented if satisfactory results of the teaching and learning process are not achieved. References Bassey, M. 1998. Action Research for Improving Practice. In R. Halsall. (Ed.). Teacher Research & School Improvement: Opening Doors from the Inside, (93). Buckingham: Open University Press. Kasbollah, K.E.S. & Sukarnyana, W.I. 1988. Penelitian Tindakan Kelas. Malang: Penerbit Universitas Negeri Malang. Kemmis, S. & McTaggart, R. 1992. The Action Research Planner. Victoria: Deaken University Press. Koshy, V. 2005. Action Research for Improving Practice: A Practical Guide. Great Britain: TJ International Ltd. McNiff, J. 1992. Action Research: Principle and Practice. New York: Routledge.

Mills, G.E. 2003. Action Research: A Guide for the Teacher Researcher. (2nd Edition). Upper Saddle River, NJ: Pearson Education. Phillips, D.A. & Carr, K. 2010. Becoming A Teacher through Action Research. New York: Taylor & Francis. Snell, J. 1999. Improving Teacher-Student Interaction in the EFL Classroom: An Action Research Report. The Internet TESL Journal, Vol. 5, No. 4. (Online) (http://iteslj.org)

Holistic Scoring Method Definition of Holistic Scoring Holistic scoring is a method by which trained readers evaluate a piece of writing for its overall quality. The holistic scoring used in Florida requires readers to evaluate the work as a whole, while considering four elements: focus, organization, support, and conventions. This method is sometimes called focused holistic scoring. In this type of scoring, readers are trained not to become overly concerned with any one aspect of writing but to look at a response as a whole. Focus Focus refers to how clearly the paper presents and maintains a main idea, theme, or unifying point. Papers representing the higher end of the point scale demonstrate a consistent awareness of the topic and do not contain extraneous information. Organization Organization refers to the structure or plan of development (beginning, middle, and end) and whether the points logically relate to one another. Organization refers to (1) the use of transitional devices to signal the relationship of the supporting ideas to the main idea, theme, or unifying point and (2) the evidence of a connection between sentences. Papers representing the higher end of the point scale use transitions to signal the plan or text structure and end with summary or concluding statements. Support Support refers to the quality of the details used to explain, clarify, or define. The quality of support depends on word choice, specificity, depth, credibility, and thoroughness. Papers representing the higher end of the point scale provide fully developed examples and illustrations in which the relationship between the supporting ideas and the topic is clear. Conventions Conventions refer to punctuation, capitalization, spelling, and variation in sentence used in the paper. These conventions are basic writing skills included in Florida's Minimum Student Performance Standards and the Uniform Student Performance Standards for Language Arts. Papers representing the higher end of the scale follow, with few exceptions, the conventions of punctuation, capitalization, and spelling and use a variety of sentence structures to present ideas. To Forward to Next FLORIDA WRITES! ItemCustom Search

Florida Writing Assessment Program (FLORIDA WRITES!)Assessment Home | Florida Writes Home

Score Points in Rubric The rubric further interprets the four major areas of consideration into levels of achievement. The rubric used to score papers in spring 1995 is shown below.6 Points The writing is focused, purposeful, and reflects insight into the writing situation. The paper conveys a sense of completeness and wholeness with adherence to the main idea, and its organizational pattern provides for a logical progression of ideas. The support is substantial, specific, relevant, concrete, and/or illustrative. The paper demonstrates a commitment to and an involvement with the subject, clarity in presentation of ideas, and may use creative writing strategies appropriate to the purpose of the paper. The writing demonstrates a mature command of language (word choice) with freshness of expression. Sentence structure is varied, and sentences are complete except when fragments are used purposefully. Few, if any, convention errors occur in mechanics, usage, and punctuation. 5 Points The writing focuses on the topic, and its organizational pattern provides for a progression of ideas, although some lapses may occur. The paper conveys a sense of completeness or wholeness. The support is ample. The writing demonstrates a mature command of language, including precision in word choice. There is variation in sentence structure, and, with rare exceptions, sentences are complete except when fragments are used purposefully. The paper generally follows the conventions of mechanics, usage, and spelling. 4 Points The writing is generally focused on the topic but may include extraneous or loosely related material. An organizational pattern is apparent, although some lapses may occur. The paper exhibits some sense of completeness or wholeness. The support, including word choice, is adequate, although development may be uneven. There is little variation in sentence structure, and most sentences are complete. The paper generally follows the conventions of mechanics, usage, and spelling. 3 Points The writing is generally focused on the topic but may include extraneous or loosely related material. An organizational pattern has been attempted, but the paper may lack a sense of completeness or wholeness. Some support is included, but development is erratic. Word choice is adequate but may be limited, predictable, or occasionally vague. There is little, if

any, variation in sentence structure. Knowledge of the conventions of mechanics and usage is usually demonstrated, and commonly used words are usually spelled correctly. 2 Points 2 Points The writing is related to the topic but include extraneous or loosely related material. Little evidence of an organizational pattern may be demonstrated, and the paper may lack a sense of completeness or wholeness. Development of support is inadequate or illogical. Word choice is limited, inappropriate or vague. There is little, if any, variation in sentence structure, and gross errors in sentence structure may occur. Errors in basic conventions of mechanics and usage may occur, and commonly used words may be misspelled. 1 Point The writing may only minimally address the topic. The paper is a fragmentary or incoherent listing of related ideas or sentences or both. Little, if any, development of support or an organizational pattern or both is apparent. Limited or inappropriate word choice may obscure meaning. Gross errors in sentence structure and usage may impede communication. Frequent and blatant errors may occur in the basic conventions of mechanics and usage, and commonly used words may be misspelled. Unscorable The paper is unscorable because

the response is not related to what the prompt requested the student to do. the response is simply a rewording of the prompt. the response is a copy of a published work. the student refused to write. the response is illegible. the response is incomprehensible (words are arranged in such a way that no meaning is conveyed). the response contains an insufficient amount of writing to determine if the student was attempting to address the prompt. the writing folder is blank.

Florida Writing Assessment Program (FLORIDA WRITES!)Assessment Home | Florida Writes Home

Description of Writing Scores

For the Florida Writing Assessment, students are given 45 minutes to read their assigned topic, plan what to write, and then write their responses. The descriptions of eleven possible scores from 6.0 - 1.0 are given below. Score 6.0:The writing focuses on the topic, is logically organized, and includes ample development of supporting ideas or examples. It demonstrates a mature command of language, including precision in word choice. Sentences vary in structure. Punctuation, capitalization, and spelling are generally correct.

Score 5.5:The writing was given a 5 by one reader and 6 by the other reader.

Score 5.0:The writing focuses on the topic with adequate development of supporting ideas or examples. It has an organizational pattern, though lapses may occur. Word choice is adequate. Sentences vary in structure. Punctuation, capitalization, and spelling are generally correct.

Score 4.5:The writing was given a 4 by one reader and a 5 by the other reader.

Score 4.0:The writing focuses on the topic, though it may contain extraneous information. An organizational pattern is evident, but lapses may occur. Some supporting ideas contain specifics and details, but others are not developed. Word choice is adequate. Sentences vary somewhat in structure, though many are simple. Punctuation and capitalization are sometimes incorrect, but most commonly used words are spelled correctly.

Score 3.5:The writing was given a 3 by one reader and a 4 by the other reader.

Score 3.0:The writing generally focuses on the topic, though it may contain extraneous information. An organizational pattern has been attempted, but lapses may occur. Some of the supporting ideas or examples may not be developed. Word choice is adequate. Sentences vary somewhat in structure, though many are simple. Punctuation and capitalization are sometimes incorrect, but most commonly used words are spelled correctly.

Score 2.5:

The writing was given a 2 by one reader and a 3 by the other reader.

Score 2.0:The writing may be slightly related to the topic or offer little relevant information and few supporting ideas or examples. There is little evidence of an organizational pattern. Word choice may be limited or immature. Sentences may be limited to simple constructions. Frequent errors may occur in punctuation, capitalization, and spelling.

Score 1.5:The writing was given a 1 by one reader and a 2 by the other reader.

Score 1.0:The writing may only minimally address the topic because there is little or no development of supporting ideas or examples. No organizational pattern is evident. Ideas are provided through lists, and word choice is limited or immature. Unrelated information may be included. Frequent errors in punctuation, capitalization, and spelling may impede communication.

Florida Writing Assessment Program (FLORIDA WRITES!)Assessment Home | Florida Writes Home

Use of Results Interpretations of the FLORIDA WRITES! achievement results are subject to several limitations. Writing achievement is limited in definition to the writing task completed during the assessment: during a 45-minute time period, students write in response to an assigned topic for the purposes of writing specified by the program without access to dictionaries or other reference materials. Because only one example of writing is collected from each student under these conditions, scores for individual students are not reliable measures of a student's general writing achievement. For each student, scores from the FLORIDA WRITES! must be considered in relation to how well the student writes when given a variety of classroom writing assignments. The FLORIDA WRITES! results will provide a basis for identifying trends in writing over a period of several years, but do not provide an exact index of changes in performance from one year to the next year. The writing assessment employs one topic for each type of writing at each grade level. Because a topic given in any one year, by its nature, may be somewhat easier or harder for students to respond to than the topic given the previous year, differences seen in results from one year to the next are generally due to both differences in the difficulty of the topics as well as actual changes in student achievement. Performance tests like the Florida Writing Assessment that rely on the use of one topic do not lend themselves to statistical methods that can control for differences in the difficulty of the tests.

Taking into account these limitations, the results can assist in the identification of strengths and weaknesses in programs of writing instruction. Teachers and administrators may find it useful to examine the number of students scoring at different levels and the differences in student scores for each type of writing. The scores for students in a classroom or school can be interpreted in reference to sample student papers provided in the Florida Writes! publications for each grade level assessed and other materials available from the Department. Student writing performance can be further evaluated through carefully designed classroom or schoolwide writing assignments. The FLORIDA WRITES! results do not represent a comprehensive evaluation of writing instruction programs. The FLORIDA WRITES! does not measure all important aspects of writing. For example, student achievement in writing an extended manuscript or a report involving detailed information is currently not measured by the FLORIDA WRITES!. A comprehensive evaluation of student writing would take into consideration writing for a number of purposes under different conditions and would utilize information from a number of sources, such as student portfolios, classroom teacher observations, and interviews with students.

Holistic Scoring MethodsTo provide consistency in scoring, use anchor sets and scorer training that require all scorers to come to a consensus about the qualities required for a score point. Scoring guides, or rubrics are also developed. The rubric matches the chosen type of holistic scoring. Scoring of student work follows a series of steps, whether the type of subjective scoring is focused holistic scoring, analytic holistic scoring, scoring by major and minor errors, or some variation of these methods.

The Steps To Holistic ScoringScoring of student work should be meaningful to students, efficient in terms of time, and provide consistent results. Holistic scoring can meet these requirements, but it does require some preparation prior to scoring. The following steps outline the general process used for many formal holistic scoring programs. The exact sequence of these steps varies to meet the needs of the program.

Define the main goals and supporting specific objectives of the course.

Very specific detailed objectives can usually be measured efficiently in machinescoreable formats. Tasks that require students to meet more global goals and therefore many objectives must usually be scored subjectively.

Design a task that cannot be completed without meeting the main goals and the supporting objectives of the course.

Since holistic scoring takes time, it is often prudent to utilize one task that covers many objectives or a main goal rather than developing many smaller tasks that address each objective specifically.

Define the essential factors of the work that students are expected to produce.

These factors might include accuracy, completeness, thoroughness, clarity, synthesis of concepts, supported inferences, or any other factor that is an essential part of the highest quality work.

Develop a rubric based on the essential factors of the work and the general value of each score point.

Each score point of the rubric should address each of the factors that should be included in the work. After the work is examined or the task is defined, the rubric may need to be modified.

Internalize what the rubric means in terms of student products.

For example, if you were scoring a diving competition you would have to thoroughly understand the rubric before you saw the dive because you only get to see the dive once. The same holds true for student products. Rare is the teacher who will have time to reread every paper many times, judging it one time for grammar and another for accuracy, etc. You have to be able to form an overall impression of the quality of the work by quickly determining the relevant strengths and weaknesses of the work.

Prepare the students and parents.

Share the rubric with your students. Explain how the rubric is used. Show some work from past students. Allow students to score this work using the rubric. The more accurate students become in evaluating others work, the more accurate their selfevaluations become.

Define an anchor set.

The number of groups in the anchor set depends on the number of score points used. For a four point scale the separation would occur as follows: High, medium, and low groups would be determined by comparing papers. Separate the medium group into high medium and low medium. This will provide a four point rangehigh, high medium, low medium, and low. Write brief annotations describing each set according to the rubric with some specific details. These papers with their annotations become your anchor sets. These sets help you maintain consistency from year to year. If you are working with another teacher, a teacher's aide, parent, or even a large group of scorers the set provides anchors. The best anchor sets for this example will have the lowest of the high papers (4-) so the scorer will say anything better than this must be a 4. The set will also contain the best (3+) and the worst (3-) of the 3's. This pattern is followed for the rest of the anchor sets. Students, parents, or any interested party can take any paper and say this

is most like these papers. The score that they arrive at should be the same or adjacent to the teacher's score. Any discrepancies should be discussed. Annotations serve as a reminder of how anchor set scores were derived and are usually representative of the types of strengths and weaknesses found at that score point. If there are many assignments that produce the same type of product, one anchor set will usually suffice for classroom assessment. District or state level assessments generally require an anchor set for every prompt.

Score student work.

Keep the rubric and anchor set close by. There may be an occasional student product that requires a second reading or viewing. You may want to annotate these "line calls" to help you explain to score to the student. When two scorers are scoring the same work, which is usually the case in large scale assessment systems, averages, and third readers are employed to resolve differences.

After long breaks from scoring it is best to carefully review the rubric and the anchor set. Train students to score papers using the anchor sets. Allow them to score some unmarked papers. Compare their scores to the scores you have previously assigned. Discuss the reasons for your scores.

Types of Holistic ScoringAll types of subjective scoring center on the type of product used to demonstrate various aspects of what the student has learned in performing particular tasks. The product, which may be complex and open-ended, is scored as a whole, in comparison to objective scoring, which scores right or wrong answers. Examples of complex, open-ended tasks include answering discussion questions on a test or performing a particular task, such as debate, reports, or presentations. Subjective scoring requires the scorer to make judgment regarding the quality of the student's work. Focused holistic scoring, analytic holistic scoring, scoring by major and minor errors, or some variation of these methods, are used in holistic scoring methods.

Focused Holistic ScoringFocused holistic scoring centers on the product as a whole. Numerous pieces of work are studied and a scale is designed by describing factors inherent in the product at the various scoring levels. Each score point of the scoring guide, or rubric, addresses each of the major factors that should be included in the work. In some science inquiry tasks these may include observations, conclusions with supporting evidence, and experimental design. After the work is examined or the task defined, the rubric may need to be modified as factors are added or subtracted and descriptions of score points are reworked. Focused holistic scoring produces only one final number that is assigned to represent the student's work as a whole. It is focused because it focuses on the total product, not on separate aspects of the student's work. This scoring method allows a relatively fast scoring of process skills. However, it fails to pinpoint strengths and weaknesses of students. Annotations can be used to detail strengths and weaknesses.

The following is an example of a focused holistic rubric in which the scorer uses only one number to represent the student's work as a whole: Example of a Focused Holistic Rubric

4 The student's work exceeds the required standard 3 The student's work meets the required standard 2 This student's work falls just short of the standard 1 The student's work does not meet the standard

Analytic Holistic ScoringAnalytic holistic scoring can be used as a means of informing both the scorer and students of general areas of high and low quality. Analytic holistic scoring follows the same procedures as focused holistic scoring, but. the rubrics are more specific. The information provided by the rubrics in analytic holistic scoring is generally more useful to students, especially for beginners. Analytic holistic scoring rubrics can be used when only part of an entire paper is to be scored. Several analytic scores can be given to one paper. With a little practice a teacher will be able to read a paper one time and assign several analytic scores. Analytic rubrics are often used with short answer essays. Discussion of the analytic rubrics before and after the task can provide a vehicle of instruction. This example shows how numeric scores can be assigned to the key performance areas of Observing and Drawing conclusions. Example of an Analytic Holistic Rubric Performance Area: Observing

4 Information is accurate and precise. 3 Some inaccuracies are present or the preciseness varies considerably. 2 Many inaccuracies exist. Precision is mostly lacking. 1 Little, if any, accurate information is present.

Performance Area: Drawing Conclusions

4 Inferences are reasonable and supported by factual information. 3 Inferences are generally reasonable and for the most part supported adequately. 2 Inferences are given, but lack support and are sometimes illogical. 1 Inferences, if present, are illogical and little, if any, support is offered.

Analytic holistic scoring produces several numeric scores, each associated with a different aspect of the student's work. Focused holistic scoring produces only one number that is assigned to represent the student's work as a whole. It is focused because it focuses on the total product, not on separate aspects of the student's work. Focused holistic scoring is appropriately used when a relatively quick and superficial, yet consistent, assessment technique is needed. This method can be used as a precursor to or as a follow-up to other assessment techniques aimed at identifying students' strengths and weaknesses. Analytic holistic scoring provides students and teachers with diagnostic information about students'

particular strengths and weaknesses and is desirable when students need feedback about their performance in key areas of their learning products.

Major and Minor ErrorsMajor and minor errors is a type of scoring that uses a general rubric defined by major and minor errors followed by a list of errors. The errors may tend to focus on the components of the task, as in the example below, or they may be centered on qualities of the work such as clarity, reasoning, etc. The type of error listed depends on the product to be reviewed. This type of scoring is tedious to set up, but is very helpful in identifying exactly what the scorer is trying to assess. Example of Scoring by Major and Minor Errors

4 No major errors, few if any minor errors 3 One or two major errors or too many minor errors to warrant a higher score 2 Several major errors or a few major errors with many minor errors 1 Numerous major errors and overwhelming numbers of minor errors

Major Errors Minor ErrorsIdentify a problem.

The problem is NOT clearly defined in text NOR is the problem inferred by the given experimental technique.Design an experiment to investigate the problem.

A problem statement is omitted but the student obviously has identified the problem.

Design does not permit investigation of a The design includes unnecessary procedures problem. Design omits appropriate control of but this does not have detrimental effects on variables, or has a small sample size or trial the results. number.Collect meaningful, organized data for ease of analysis.

Appropriate charts/tables are missing OR are totally unorganized. Missing labels make it impossible to identify the type of data collected. Appropriate graphs are missing. Graphs may lack enough information to be interpreted correctly, e.g., graph labels are missing.Correct analysis of data.

Charts may be slightly disorganized, but is understandable. Labels may be missing but it is apparent what data has been collected. Appropriate graphs are given but there are some style errors. Labels may not be complete, but they are understandable.

No attempt is made to analyze data or to identify trends or patterns from the data.

There may be a few inconsistencies in the analysis. Major patterns in the data are

correctly identified.Conclusion

Conclusion is NOT supported by the data or contradicts known facts. Conclusion is missing, incorrect, or is based upon extraneous information.

Conclusion is generally supported by the data, and factual information but there are a few inconsistencies.

Copyright 2012 | Charles A. Dana Center at The University of Texas at Austin Copyright Information | Privacy Policy | Email questions or comments to Science TEKS Toolkit.