Project SummaryThe purpose of this three-year research project is to build a successful software environment for realtime, applied programming for underrepresented students' early literacy (RAPUNSEL) in order to address the critical shortage of women in Computer Science (CS) careers and degree programs. Fewer girls than boys enroll in CS related programs, feel self-confident with computers, and use computers outside the classroom. Much research ties this shortage to problems at the middle school age, and both women and girls report a lack of confidence in their computer skills. The goal is to develop an engaging system with which to teach computer programming to middle school girls. We believe that the way to build confidence in computer science is to teach girls real, applicable skills gradually through an engaging game environment. The system is designed in units to become a scalable, approachable simulation game that appeals to girls' sense of communication, curiosity, and play while boosting confidence and developing programming expertise.

Specifically, this project aims to study three areas related to gender and CS. First, how do the design of the programming software environment and the role of the peer group within such an environment affects girls' motivation and engagement to learn programming? Second, what is the depth to which middle school girls are able to learn programming concepts such as procedural thinking? Third, can we increase girls’ overall confidence and self-efficacy levels in a software environment? RAPUNSEL addresses these issues through the design of a JAVA-based networked play environment in which girls can alter and create original characters, scenes, and scripts to build collaborative narratives online. Using four web-based modules, girls will learn programming progressively through manipulating the objects in the modules and receiving immediate feedback. RAPUNSEL allows girls to share their creations and gradually gain access to programming through guided hint systems in the form of character agents. No previous initiatives have focused specifically on the tiered use of a commonly used programming language for middle school girls within an online software environment, and no other project has empowered girls to create their own programmatic objects for others to use.

Working in research and development “minicycles,” we will research and build technical modules such as an intelligent script editor, content such as behaviors and objects, the graphical user interface (GUI), and monitoring components to create the client side of a software environment which encourages girls to learn JAVA. We will begin with existing research focusing on girls and technology and children learning programming, and will design three client side core instructional modules during the course of the research. In each phase of the research, we will build these prototypes to fit into a larger framework for what we call “unfoldable environments” which motivate girls to unwrap the characters and worlds presented to them, manipulate them, and create new objects and environments. We will work with small groups of Oregon and New York girls as users, testers, informants, and design partners. Large-scale assessment will occur in the Eugene Oregon 4J school district.

The research team includes experienced, award-winning scientists, artists, designers, and educators with a unique blend of academic and industry experience. PIs Ken Perlin and Mary Flanagan have years of experience designing influential software. PI Andrea Hollingshead, a social psychologist who studies the socio-cognitive and motivational processes involved in effective collaborations, will lead assessment with consultant Gerald Tindal, University of Oregon. Industry-related consultants on the project include Will Wright, the founder of Maxis Entertainment and creator of The Sims games, and Lindsay Gupton, Executive Studio Director of Know Wonder, maker of the Harry Potter, American Girl, and Magic Schoolbus games. Educators serving as consultants are middle school teachers in Eugene, Oregon as well as the 4J school district in Eugene.

The research results will be disseminated in the form of peer-reviewed publications and conference talks as well as a set of online resources. When completed, RAPUNSEL will be distributed online as a user-friendly environment and will also be available as open source for other researchers to use.

1

Project NarrativeThere is a critical shortage of women in Computer Science (CS) careers and degree programs. Margolis and Fisher, in their landmark study of gender and CS, note that the male dominance in information technology can be linked to the social, cultural, and educational influences and patterns formed in childhood [2, 38]. Research shows that although girls are as talented as boys in math and science, and although most girls are excited about science in childhood, these same girls begin to lose interest in math and science in middle school [12, 14]. By the eighth grade, twice as many boys as girls show an interest in science, engineering, and mathematics careers [15]. While opportunities for CS-related careers are broadening, and programming skills are required in many diverse fields, fewer and fewer girls are attracted to CS related activities. New approaches and resources are needed to engage girls in computer programming activities.

The purpose of this research project is to design a successful web-based software environment for realtime, applied programming for underrepresented students' early literacy (RAPUNSEL). The goal is to develop an engaging way to teach computer programming to middle school girls in a scalable, approachable manner that appeals to girls' sense of communication, curiosity, and play. Specifically, this project aims to study how the design of a programming software environment affects girls' motivations to program, their ability to learn programming concepts, and the extent to which girls' overall confidence and self-efficacy levels are affected by the RAPUNSEL environment.

We will construct RAPUNSEL to be a JAVA-based online learning tool which teaches procedural thinking and, ultimately, computer programming through an attractive simulation game. The game environment is designed to be a modular, tiered, creative tool to instigate middle school girls' interest in programming. RAPUNSEL will encourage girls to alter and create original characters, scenes, and scripts to build programming-based narratives. When complete, RAPUNSEL will provide an inventive and participatory motivational space for multiple users. The software program, created through instructional modules that fit into a larger framework, is the basis of what we call an “unfoldable environment.” By engaging middle school girls within the unfoldable environment, we will motivate them to unwrap the characters and worlds presented to them, manipulate them according to their own preferences, and create new objects and environments with the software. No previous initiatives have focused specifically on teaching a commonly used programming language to middle school girls from within an online software environment, and no other project has thus far empowered girls to create and save their own language-based creations for others to use and share.

RAPUNSEL will enable multiple simultaneous participants to: Access an intuitive framework configured for tiered learning of computer programming Interact with digital characters and represent themselves through the creation of new digital

artifacts Program new, sharable objects in Java, save material to the web, and ultimately add their work

to help build the environment Solve complex problems and engage in software design, algorithmic, and procedural thinking

through computer programming Express themselves using technology and create new game goals and frameworks for play

through the objects they create Communicate with other participants and enact collaborative activities Develop self-esteem and self-efficacy while working with technology

The goal of RAPUNSEL is to empower middle school girls as designers, programmers, and inventors, enabling them to learn about the world through their own experimentation and exploration. To do this, the design of RAPUNSEL relies on the development of new technologies. In particular, we must build a progressive educational environment (referred to as “scaffolding” design in Guzdial [26]) which

2

eases girls into increasingly complex Java programming. This environment will allow us to study the learning potential of computer science programming tools, the impact of learning procedural thinking on the group, and girls' interest and confidence levels in various aspects of the environment through implementation [67]. This research also will study the practicalities of implementing RAPUNSEL in a classroom or in online-only settings.

During the course of our research, we will build, test, rework, and combine technical modules such as an active script editor, content such as behaviors, objects, and narratives, the graphical user interface (GUI), and monitoring components to create the client side of the software environment. We will design, evaluate and combine the final prototypes representing several foundational aspects of the research into a cohesive storytelling environment. The team will begin with existing research focusing on girls, middle school learners, and programming (specifically the significant work of Kafai, Guzdial, Resnick, Bruckman, Papert, Laurel, and the AAUW studies). In each phase of the research, we will design tools to address girls’ needs in the unfoldable environment framework.

NEED FOR PROJECTAt the same time the demand for participants in CS is increasing, fewer girls than boys enroll in CS classes, feel self-confident with computers, and use computers outside the classroom-- less than 33% of participants in computer courses and related programs are girls [1,15]. Research shows that females experience a reduction in self-esteem during adolescence, negatively affecting their achievements and narrowing their aspirations [57, 66]. Middle school girls statistically drop out of math and science classes or do not perform well, and adolescence is often the final time girls consider the diverse array of career opportunities in technical areas, especially CS [62]. The most significant reasons cited in research for this lack of interest are that girls often underestimate their own abilities and are not engaged by the content of such programs [5, 6, 28]. Women and girls consistently report a lack of confidence in their computer skills [11, 39, 24, 29,44, 67]. Although women constitute roughly half of the US population, they are significantly underrepresented in CS degree programs and professions [48]. Nearly 75% of future jobs will require computer use, and yet fewer than 33% of participants in computer courses and related activities are girls; by 2010, the largest industries and fastest growing job opportunities will be computer related, specifically, in CS and engineering fields [64]. The current science, engineering and technology workforce is only 19% female [15].

These figures point to an emergency situation in computer literacy for girls. Girls need to be reached before adolescence to keep them interested in science and math and to foster their achievement in these curricular areas. Specifically, girls need to be encouraged not only to be computer users, but also become those at the forefront of creating new computer technologies. Bruckman et. al. found that gender does not affect programming ability or performance, but while girls spend significantly more time than boys communicating with others in computer supported learning environments, boys are more likely than girls to have prior programming experience, and spend more time programming on average [8]. This research recommends that in order to increase gender equity in technical computer skills, developers should focus on strategies for fostering interest among girls. Yet no commercial software exists to encourage middle school girls to learn software design and computer programming, and the types of software directed at girls has neither increased nor diversified since the seminal work of Laurel [37].

Computer games that currently attract girls are thus important to our research. Games for girls have been studied through commissions sponsored by groups such as the AAUW. A recent review of popular mathematics programs targeted at K – 6th grade showed that only 12 % of the characters were female, and those played passive roles such as “princess” [3]. Studies of gender and play have shown that girls are more likely than boys to engage in parallel and constructive play as well as peer conversations [41]. Game environments such as The SIMS and Neopets engage players in interesting

3

variations of constructive play – players create virtual households or create and care for virtual pets. However, players do not learn extensive technical skills playing in such commercial game environments. Criteria for truly equitable software must go beyond representation and game scenarios and allow models which empower students to be software designers and have technical and creative control over their own environments.

The goal is that by being able to access programming processes and compelling environments early, middle school girls will be able to make their own creations, construct significant objects and environments which reflect the way girls think, represent their values and opinions, and lead to the formation of new knowledge. In addition, such experience will enable them to participate in the currently male-dominated software industry as a generation of girls becomes technological creators [3]. If we can broaden the participation of all middle school girls with techniques and possibilities that they can relate to their actual, everyday lives and can expand upon in a virtual environment, this will rearrange the dynamics in the creative terrain of computational media. Indeed, the eventual aim of gender equity research should emphasize the diverse range of interests and preferences for all students, so that through good design, creative environments, and customization, we are able to create experiences which will ultimately support a broad spectrum of learners.

Innovative Features of the WorkSeveral computer programming initiatives for middle school children have been funded by NSF. The NSF-funded MUVEES, a collaboration between George Mason U., Harvard U., the Smithsonian, and teachers from Arlington, VA, produced a collaborative environment for middle school science education [42]. This research team's use of online VR worlds to motivate students to explore curricular areas within a space is important, but it does not involve computer programming or construction of new knowledge or artifacts. Mary Flanagan, a PI on this proposal, has an ongoing project, The Adventures of Josie True, which is a software program to encourage girls in 5th grade math and science. This project, funded in 1999 by NSF, focuses on intricate narrative, compelling characters, and historical role models to provide a motivational context for girls’ exploration of 5 th grade curricular areas (http://www.josietrue.com). Initial research has shown that girls are more likely to excel in math challenges in a game environment due to the instant feedback of the system and a compelling narrative scenario [23]. The not-for-profit team is currently producing Episode 2 set in ancient Egypt. Focus groups conducted in 2002-3 show that girls wish to create their own game scenarios, behaviors, and characters to build upon existing characters and frameworks provided in the software, and thus Flanagan has taken this feedback into the design of RAPUNSEL. Several other initiatives focusing on mentoring girls in math and science have also been funded by NSF, and very important new studies, such as the Ohio University study begun in 2001 by Bernt et. al., are generating large scale data pools about interests in math and science for middle school girls.

NSF has funded several research projects which developed programming projects for children, including the groundbreaking work of Papert and the Logo programming language initiative in the 1980s. The KidSim/Cocoa/ project in the 1990s was another attempt to teach programming to children [56]. NSF funded several research initiatives which developed Alan Kay et. al.’s Squeak programming language which is specifically designed for children. Squeak and Logo offer extremely open programming platforms, as does Python [65]. We see several challenges for middle school girls to use these languages and associated “flat” programming platforms which do not offer clear, leveled challenges, expectations, built-in narrative starting points, and cognitive reinforcement. The Squeak and Logo initiatives clearly prove that children can indeed learn to program, and that they program well. Without supervision, instruction, or in-person mentorship, however, girls are unlikely to use these tools. In addition, they are static environments, not responding to student level or need. In contrast, commercial games such as The SIMS or SIMS Online need no instruction in order for girls to learn how to play; these games offer players dynamic and automatic leveling through game design,

4

and allow creation of new items for the world through “reskinning.” In commercial games, though, girls cannot gain access to the programming level of the software, and The SIMS is not designed as an instructional environment. Because The SIMS is only a game and not a learning environment, an environment like RAPUNSEL is needed to bridge between the popularity of commercial, “self-teaching” games such as The SIMS and the programming learning projects directed at children. Commercial and academic initiatives tend to be profoundly different in their design, ease of use, and approach, and this gap between popular commercial game design and existing resources for learning programming must be addressed. Girls are attracted to and succeed in playing many commercial games (45 percent of the 20 million SIMS players are female, and two-thirds of those are girls [13]) and we must incorporate the gradual introduction of complexity and motivation in the research and design of RAPUNSEL. Content leveling is a significant design challenge we address through carefully staging girls' exposure to our dynamic Java environment.

Our system is a manifestation of design-based learning and constructivist theories developed by computer science educators [47, 54 58, 59]. We build on the research of innovators who explored the groundwork of computer science instruction in young age groups. In addition, we build on the expertise of the research team in creating believable characters that respond to users and to each other in real-time [49]. Our web-based learning environment offers a unique focus and features not explored in prior work, such as the gradual and level-dependent introduction to programming concepts, the focus on behavior and narrative, and the reliance on a universally-usable programming language.

The AAUW commission on gender and software recommended several design features specifically for girls a result of their far-ranging research [2, 3]. These characteristics included a rich narrative and intricate, multilevel/appropriately leveled challenges, engaging characters, roles which affect positive social change, social interaction, opportunities to design and create, and strategy/skill aspects. The RAPUNSEL research focuses on building modules as part of a larger whole to particularly address each of these recommendations. RAPUNSEL is innovative in several ways. Our plan offers:

a scaffolded environment to introduce CS principles gradually through gameplay an environment that is balanced between an open flexible programming environment and one

that is structured as narrative game empowerment and self management for middle school girls to help build confidence in their

own computer programming abilities Our design 1) Uses sequencing of material in the simulation, so girls do not start out facing the intricacies of full programming and rather begin to learn procedural and object-oriented thinking through everyday examples; 2) Encourages girls by offering situations and environments which appeal to them. Many examples in computer science instructional environments ask learners to model automobiles and organizations. Instead, girls will be offered challenges as easy as changing a character’s color to as difficult as having one character respond to another through behavior change; 3) Provides a motivational framework for creation and collaboration by encouraging girls to create and share their own images, objects, behaviors, and other products within the program. Early work on this system is located at http://mrl.nyu.edu/~perlin/polly/.

The RAPUNSEL environment will become a sequenced learning environment consisting of four main modules, with an additional introductory demonstration module. In the first scenario, girls will guide characters in simple narrative activities such as eating and sleeping. Girls are given a standard polygon character (or Polly) and are coached to change its color, size, and sequence of activities. Pedagogically, this section focuses on programming commands, coloring objects, and making decisions. In the second scenario, girls will get characters to interact with another character and react

5

to the environment. Girls will control the behavior of their character to communicate with a pre-programmed intelligent agent. This section pedagogically involves programming behaviors, conditional statements, and dialogue. The third scenario will be a simple multiuser, asynchronous collaboration environment in which girls will have their characters negotiate with other girls’ characters. Pedagogically, this section involves multiple ways of responding to other user’s character behaviors and allows users to program dialogue between them. The web-based, asynchronous nature of the third module will foster multiple users and collaboration to promote the cooperation of dispersed members of the learning community [8]. Miller notes that collaborating, rather than competing, and providing rich, believable environments are successful in getting girls interested in technology [40]. The fourth scenario will be an open play environment for girls to construct and save their own characters. The design of the archive for such material must foster individual recognition in order to draw participants to create and share. We will design each scenario through the computational model of storytelling proposed by Elliot, representing each story through emotion content and behavior [21] Girls will have unlimited flexibility in how they are able to alter the story through manipulating the reasoning system that understands the relationships between the characters and their environment.

None of the previous initiatives focused specifically on teaching programming to middle school girls from an online software environment and community. None have used a scaled environment that taught children a commonly used programming language children could gradually and effectively teach themselves. Finally, no other project has thus far empowered girls to work collaboratively and create and save their own language-based creations for others to use and share. RAPUNSEL meets these requirements. We believe that the way to build confidence in computer science is to teach girls real, applicable skills gradually and thoroughly through an engaging game environment. We will focus on a multimedia approach to learning computer science to satisfy a new generation’s interest in creating media [22].

Project DesignWe plan to research, build, research, and rebuild the client side of the overall RAPUNSEL environment during this three year project specifically intended to teach programming to middle school girls who have no previous programming experience and perhaps little familiarity with computers. Our aim for this research is to provide tools to support girls through all aspects of learning programming so they will also be able to utilize this knowledge in our environment as well as other venues. We realistically imagine girls entering University being able to bypass basic programming courses due to their expertise with our software—in other words, girls will be learning JAVA programming skills that will directly be able to transfer to other real-world applications and programming environments. We are following the constructionist goal proposed by Papert to design a learning environment in such a way as to produce the most learning for the least teaching [46]. Therefore, while we will build a support network around the software in the form of software-based intelligent tutors and a demonstration scenario, the software must be able to be used by girls by themselves and be useful as a stand-alone learning tool. It is imperative that girls be able to use RAPUNSEL as intuitively as they are able to learn to play popular commercial games such as The SIMS, Neopets, and Harry Potter.

We will construct related modules with the ultimate goal of linking them together to create a cohesive educational and research tool in which girls will be able to use programming to support other activities such as storytelling and character manipulation. Each of the research modules will be instinctively easy to navigate upon first approach. In the final integration of the evolved game and the unfoldable environments, we will design attractive, animated, and entertaining tutorials and “trailers” for the

6

project to help new users. These will be triggered by entry into various aspects of the program so that girls are gradually cued into new possibilities.

Research components we plan to construct to research the cognitive and pedagogical level, ability, efficacy, and interest aspects of the target age group include:

Module 0, Demo The RAPUNSEL environment will be introduced through a demo module. This demo shows a preset scenario and will act as a means of establishing the functionality of the software environment.

Module 1, Level 1Single Character Behaviors

The first level involves one character controlled by the player. For example, the character, a Polygon (Polly), is hungry. It searches for food high and low. It decides not to eat bananas. Then it finds and eats some eucalyptus leaves, curls up in a ball and goes to sleep.

Module 2, Level 2Intra-character Behaviors/Dialogue

The second level involves an exchange between two characters. For example, Polly1 wants to play ball with Polly 2, but Polly 2 will not play with Polly 1. When Polly 1 finally gives up and walks away, Polly 2 chases after it and gives it the ball. Then they throw the ball back and forth.

Module 3, Level 3 Three character Behaviors/Exchange/ Dialogue

The third level incorporates multiuser, asynchronous collaboration between three characters. For example, Polly 1 wants to dance, but only with Polly 2. Polly 3 wants Polly 1 to sing. It instead plays the sitar while Polly behaves like a lava lamp.

Module 4, Level 4 Open Creation Environment

The fourth level allows girls to take the skills they have learned and develop their own characters, situations/stories and behaviors. Here, girls are the active builders of their own conceptual structures. The creation process is one that is iterative and time based, to incorporate new ideas and play.

Each module will have 10 overall possible tasks associated with it, and upon the completion of three tasks per module, girls will be moved on to the next module. The modules represent the stages in the incremental progression of learning programming skills and are self-paced. Activity within each module will be followed with online evaluation, described in the evaluation section of this proposal.

When a child learns to write English, she is effectively working in a system that never breaks. The rich semantic structure of English provides a forgiving context in which the child's work can be “parsed” even in the presence of errors. This allows the child to learn even in the presence of grammatical mistakes; these mistakes are in fact part of the learning process.

In order to provide children with such a “system that never breaks” we take a top-down approach to the learning of programming. Consistent with [20] and [9] respectively, we first introduce the concepts of objects and multi-threading. These two concepts suffice to allow the student to manipulate a simulated world in which multiple virtual characters interact with each other. Object oriented interaction allows the student to create and to program the various properties that influence a character's appearance and behavior, without ever requiring the simulation world to stop running. Multi-threading allows the student to begin immediately to create and to program multiple interacting characters and objects in the virtual world. These two concepts, when used together, allow a forgiving environment to be provided, in which the student's initial programming errors will not have catastrophic effects.

7

After these first concepts have been mastered, only then do we introduce the more fragile concepts of sequential execution: conditional blocks, iterative loops, parameterized procedures, global versus local variable scoping. Even then, we let the student play with these concepts within a protected block of the multi-threaded framework. This ensures that while a scripted animated character may behave oddly, the simulation world itself cannot “crash”.

We also use a semantically aware intelligent code editor to help the student quickly progress even in the early learning stages of knowledge of syntax. Initially, the code editor is constrained to act essentially as a fully constrained GUI, albeit one with a text-like look and feel. As the student advances, she can choose to progressively loosen these constraints.

The student can set object and character attributes by direct drag-and-drop operations. Every time the student effects such an operation, a code equivalent is generated by the system. The student can always modify this code via the code editor window. For example, let us say that Sally is customizing the character Polly. Sally decides to make the character Polly GREEN. She can do this by mouse-dragging a GREEN color swatch over to Polly's 3D representation. This creates the text in the code editor window:

Polly.set(COLOR, GREEN);

If Sally then moves the mouse over to the word GREEN in the code editor window, a menu of alternate colors automatically appears from which to choose (see Appendix).

In order to make a system like this work as a shared on-line world, the software is structured within a server/client component framework. This framework is used even when the system is used in an off-line stand-alone mode (in which case the server component can be running on the same computer as the client component). While our focus for this proposal is on the creation of the client side of RAPUNSEL, we plan to simulate how the on-line server component will eventually function in the modules 3 and 4. The development of the infrastructure and support for shared on-line server/client component framework will be the focus of a future grant proposal. The on-line server component will implement all notions of persistence in the world, so that all participants experience the same consistent world. Whenever a character walks through a doorway, or jumps on a table, or falls asleep, this information is maintained on the server. We will design the client to fit into this eventual server component. Each client is responsible for interaction with its individual user, for providing feedback both for direct manipulation and for code interactions, as well as for effecting the real-time animation and rendering of characters and objects in the game world.

Software components:

8

The burden is on the research team to design hypothetical scenarios in which girls are motivated to go online to create and participate in communities. In developing RAPUNSEL, girls themselves must be involved as active partners in the development process. We plan to involve cooperative inquiry, an approach to creating new technologies for children, with children [19]. Two groups of approximately 8 girls located in New York and in Oregon will act our design partners, regularly informing our design and research practices.

Pedagogical ApproachWe are utilizing a mixed-initiative framework in our pedagogical design to offer several ways to approach the domain tasks designed in the research [34]. The staged modules in RAPUNSEL rely on scaffolding girls’ learning through appropriate tasks, encouragement, and quality tools which offer girls formative feedback. Scenario design, character interaction, and environment design are our primary approaches to support learner growth and self-instruction. We are designing learning strategies which can be acquired in a real context and learned in such a way as to encourage transfer [43]. The learning environment monitors progress of students and agents present curious, motivating clues to players to encourage coding and spark trial and error experiments. Such interactions are particularly important for our constructivist learning approach, and encourage learners to participate in active problem solving.

The overall environment is designed to be intuitive and engaging. We will begin the interaction with the project through Module 0, a demonstration; this demonstration also essential to our evaluation process, in which we will ask participants to describe the events of the demo in a pre-test. The following four modules of the program offer menus which allow students to combine and compose behaviors together – a multi-select palette. A Java window is always available to users, although many participants will probably not use it during the first stages of gameplay. Thus as girls play, the characters’ hints provide motivation while the environment itself is engaging to explore. We believe our design will influence play interaction to encourage it to be an iterative process rather than a product-oriented process, and girls will feel comfortable in active roles [3]. To help the learner evolve the environment, code hints and code segments will be introduced through our intelligent, software-based tutors which offer us the chance to reinforce the social aspects inherent in human-agent interaction [33]. Characters and agents will be available as menu items offering feedback to prompt girls to discover new features during gameplay and boosting their confidence levels. Elliot et. al. recommend that agents should be designed to engage the user, foster enthusiasm for the subject by adding emotional aspects to the environment, and strongly recommend offering two or more tutors working in concert instead of lone character agents [22]. For example, the EVA agent is a poetry agent who offers code snippets for girls to try early on in the code and poses “poetic code clues” that participants can use to affect character looks and behaviors. These code snippets may change character colors, for example. The Z-bot, a second agent, offers conversation codes which unlock control of the environment through puns. These are possible agents, the form of which will develop in collaboration with our design partners.

The essential element of RAPUNSEL's design is the ability for our program to grow, change, and expose more over time through the modules. In Modules 1 and 2, girls will be able to use drag and drop menu commands and icon interaction on the palettes to initiate action and response using a limited range of presets. They will soon learn, however, that the entire world, from objects to the environment itself, is active and programmable. Most girls will utilize their growing programming knowledge to customize and extend RAPUNSEL, writing code to create new objects, create new characters, and “skin” existing ones in Module 4. Motivation to delve into the code is provided through the project design. By sharing characters in a database in Module 4, players differentiate themselves by creating outlandish characters, new scripts, and actions. In other words, an urge to show

9

their talent telling stories will draw girls into the online aspect of the community in a variety of ways. This approach is supported by Kafai's work with children designing their own activities in after school LOGO programs: girls often created their own worlds when making their own games, substituting other activities for violence and other aspects of games which may not appeal to them [25, 31]. Girls were also found to enjoy shifting rules and feedback mechanisms of the games, and in a study by Honey et. al., girls enjoyed making new technological devices if they were to be used as people connectors and collaboration activities [29]. Our RAPUNSEL software offers three top-down designed worlds for the learners; they are able to participate in these and other worlds from the bottom up, or through the bricolage approach proposed by Turkle and Papert [63] in which they build objects and subversions into the game. Or, girls may choose to create their own game worlds with rules, goals, and objects, in which they could work from the top down or from bottom up. We strongly believe that this system can address both ways of working and enable girls with many learning and creation styles to fully participate. Feedback is especially important in the software design and in the scaffolding design, as is the ability to always see one's affects on the world being created. The system offers temporary support for student learning, available until the student can perform independent of the menu support. The system will also be “break proof,” so that one mistyped element will not cause the whole world to stop functioning and discourage the girl player from participating further. This will be done through the construction of an intelligent active script editor.

Good educational design makes central the contextualization of thinking [50]. We will rely most on Schank and Cleary’s “teaching architectures” [55] which combine several models that can be used to attract middle school minority girls to math and science [62]. These architectures overlap several other learning models and are useful to the design of an interdisciplinary computer project [36, 45]. While the Shank and Cleary model consists of 5 principle areas, two relate to RAPUNSEL strongly:

Simulation-based Learning by Doing –offering students the opportunity to learn by “doing.” Based on “active learning,” the activities and simulations require active participation by the learner, who enters the simulated world as an actual participant. Environments that encourage active learning are based on learners making decisions about task, content, navigation, presentation, and assessment. Active learning environments make use of a number of cognitive strategies that enable the learner to elaborate on their own existing knowledge structures (schema), to construct new knowledge and understanding. Learning by doing activities in teaching architectures and simulators provide multiple representations of reality and represent the natural complexity of the real world; present authentic tasks that conceptualize rather than abstract information; provide real-world, case-based contexts; foster reflective practice, enable context and content dependent knowledge construction, and support collaborative construction of knowledge through social negotiation, as opposed to competition among learners for recognition. Middle school teachers cite that some of the best reasons to use multimedia tools in science education are the motivation to fully and actively participate and the students' increased self-esteem when they show and see the results on the screen [58]. We incorporate an active learning approach by designing RAPUNSEL to incorporate real world activities that could actively be solved. Simulations can be very effective teaching tools across the curriculum [16]. Our simulation environment will at the introductory level allow for play and drag and drop programming, and at the advanced level encourage advanced work; the system fosters imaginative play, math, and physics to freely mingle—a merging of fantasy, storytelling, and science [47].

Case-based Teaching—in RAPUNSEL, learners draw upon their own knowledge and their own experiences of procedural thinking to progress. This is one of the most challenging aspects of the RAPUNSEL design, for case based learning is difficult to use in today's classroom due to the necessity for teachers to be on hand, one on one, to provide background cases related to the curriculum [55]. Girls discover new elements of the program through specific challenges in the

10

scaffolded modules and by trial and error based on their own cultural knowledge, especially knowledge of other games, popular culture, and the everyday. However, while Lewis, Turkle, and Pappert [35, 63] note that girls engage better in trial and error exploration than do boys of the same age, girls do not easily allow themselves such freedom to explore if they are not confident with the material or learning context. Therefore, we hypothesize that if girls' sense of experimentation were fostered from within the program, encouraging confidence from their prior knowledge of other games and from common sense, they will excel.

There are significant challenges that must be investigated and addressed in our research. Traditional computer-based learning environments still rely on teacher-centered instruction, single path progression, isolated work, passive learning, factual, knowledge-based learning, reactive responses, and isolated contexts. Our learning environment focuses on student-centered learning, multiple path progression, collaborative work, active, exploratory, and constructionist learning, critical thinking and informed decision making, proactive and planned action, and real-scenario contexts [30].

Research TeamOur team has a demonstrated track record for educational software development, outreach, and technology transfer. Mary Flanagan, University of Oregon (UO), and Ken Perlin, New York University (NYU), will co-direct the project. Flanagan has over a decade of experience in educational software design, and has garnered over 20 international awards for the innovative products she has produced. She is also the producer of The Adventures of Josie True (http://www.josietrue.com), the award winning science and mathematics web environment for middle school girls (NSF 99). Her co-edited book about women and girls in cyberculture was recently published by MIT (2002). Ken Perlin is a Professor of CS, and Director of NYU’s Media Research Laboratory and Center for Advanced Technology. Perlin's research interests include graphics, animation, and multimedia, and he has received several awards for teaching excellence in science and technology. In 1997 he won an Academy Award for Technical Achievement from the Academy of Motion Picture Arts and Sciences for his noise and turbulence procedural texturing techniques, which are widely used in feature films and television. He is well informed on issues in the software industry through serving on the Board of Directors of the NY chapter of ACM/SIGGRAPH and the NY Software Industry Association. Together the PIs have designed the structure for the project, assembled the team of researchers, and set up the framework for the research. PI Andrea Hollingshead, a social psychologist who studies the socio-cognitive and motivational processes involved in effective collaborations, will lead evaluation. Her NSF-funded research on the development and maintenance of knowledge networks in IT-enabled collaborations is particularly relevant to this project (NSF KDI IIS-9980109). The PIs will also be lead the interface design, character design, create the foundation for the software architecture, and participate in user testing in schools.

Educators serving as consultants are middle school teachers in Eugene, Oregon as well as the entire 4J school district in Eugene. Computer teacher Nancy Newman, Roosevelt Middle School, will be involved in shaping the work as a subject matter expert and research affiliate. Finally, our team will work with two groups of 10-12 year old children who will serve as design partners in Oregon and New York. The assessment team is critical to the project. Dr. Gerald Tindal, Director of Behavioral Research and the Teaching Research Center in the College of Education, UO (http://brt.uoregon.edu/), will lead the educational assessment for the project with PI Hollingshead. Consultants, advisors, and staff for the project include internationally recognized leaders in education and software design. Integral to the design of the project will be interface design by Rebecca Ross. Ross, professor in the Gallatin School of NYU, has expertise in designing interfaces for social engagement in online software environments. Research scientist Chris Poultney, a skilled programmer extensive experience implementing user interfaces, will construct the program's flexible and extendible architecture. Industry-related consultants on the project are known leaders in the field. Will Wright, creator of the

11

The Sims, the best selling PC game of all time, will consult on distribution and overall design issues. Lindsay Gupton, Executive Studio Director of Know Wonder (maker of the Harry Potter, American Girl, and Magic Schoolbus software) will consult on scenario designs.

Evaluation and Research MethodologyThe underlying theoretical model for our project is that RAPUNSEL will serve as an effective intervention for reducing the digital divide by empowering girls to learn and to pursue further training in computer programming. RAPUNSEL will accomplish this by providing an engaging learning environment that not only will lead to the acquisition of basic programming skills, which can transfer to other programming contexts, but will also have positive psychological effects on users’ motivation, attitudes, self-efficacy, confidence, and future intentions regarding programming.

Our participants will be approximately 150 5th and 6th grade graders in middle schools across the 4J school district in Eugene, Oregon. Dr. Gerald Tindal, Director of Behavioral Research and the Teaching Research Center in the College of Education, UO and collaborator on this project, has established relationships with teachers in this district, and the teachers are eager to participate in the project (See letters of support). We have been invited to visit and conduct tests in computer classes during the regular school hours in the district. Both girls and boys will participate in the evaluation. Although RAPUNSEL will be designed with girls’ needs and interests in mind, we believe that RAPUNSEL will also appeal to boys. We hope to see no significant differences (or at least a much smaller difference) between girls and boys in programming skills and psychological measures at the end of their RAPUNSEL experience.

Our evaluation will begin with the establishment of a baseline with regard to users’ pre-existing knowledge, attitudes, and intentions. Before participants are introduced to RAPUNSEL, we will ask them in a fun and easy-to-use web-based format about their computer use (e.g. amount of time spent per week, range of activities performed on computers, etc.), their beliefs and attitudes regarding computers (interest, enjoyment, self-reported knowledge, difficulty, gender stereotypes, etc.), their intentions regarding future education and career choices, and will obtain general measures of self-efficacy, self-esteem and confidence. We will also ask them about the beliefs, attitudes, and intentions of their most valued peers regarding computers, because peers exert considerable influence on the thoughts, feelings, and behaviors of middle school girls [44].The same assessment will be carried out at the end of the RAPUNSEL project. We will procure demographic data about our participants from the 4J school district database. This database contains information regarding gender, age, ethnicity, socio-economic status of all students in the 4J school district.

We will conduct a three-pronged evaluation that will investigate the important components of our theoretical model: 1) skill acquisition/learning, 2) psychological processes, 3) software usability and effectiveness. We will assess each of these components using multiple measures to ensure construct validity and at multiple points in time.

1. Skill acquisition/learning will be measured using several different methods: Narrative Tests. Children will be asked after every module to describe in a narrative format

the actions of the characters they created. This method will enable us to assess cognitive and conceptual understanding of programming structures and tasks. Have participants learned the concept of manipulating objects through language? Do they display object-oriented organizational thinking? Do participants demonstrate a depth and breadth of understanding of procedural thinking? We hypothesize that participants will use more metaphoric language in the pre-test, and will involve more “pseudocode” or code-like descriptions in post-test narratives and demonstrate a deeper understanding of conditionals, variables, and objects.

12

Portfolios. We will assess programming skills by collecting what participants build (portfolios) within each module and rating the portfolios according to their complexity [9]. From this research, we will be able to define points of successful implementation as well as factors that hindered success.

Skill Transfer Exercises. After each module, a short “exercise” (i.e. quiz) will be given to participants to assess whether they can apply the skills they learned in the module to a conceptually similar but contextually different problem. These exercises will be scored based on speed/efficacy (length of time to work through the problem), and performance/accuracy (successful task completion, number of errors and corrections).

Task Analysis. We will utilize built-in software monitoring to examine speed/efficacy and performance/accuracy with which participants work with particular tools in each module of RAPUNSEL and on the skill transfer exercises. We will measure efficacy and accuracy on tasks such as creating new objects, adding correct script to the script editor, and correcting code using time-on-task analysis [9]. Our software monitoring will record the speed and number of attempts when participants use a particular tool and when they make a change in the environment.

2. Psychological Processes will be measured using several different methods. Attitudinal/Perceptual Probes. After each module of RAPUNSEL, we will gather

information on attitudes, perceptions, and self-evaluation. Many of these questions will be the same across the modules so we can investigate change in attitudes and perceptions over time. Some of the topics that we will investigate include: Attitude change: are girls more enthusiastic about programming, do they view programming as an area of expertise that is gender-neutral; Self-efficacy and self-esteem: does using RAPUNSEL increase girls’ self-efficacy and self-esteem related to computer programming but also in general, do the least-confident students, i.e. those who report low self-efficacy, become interested in programming; Behavior change: do girls use computers more in their daily lives?

Working group interviews. Module 3 involves multi-user asynchronous collaboration using RAPUNSEL, and we will interview the users of each collaboration to help assess the effectiveness of collaborative learning using RAPUNSEL and the role of peer group social communities in learning programming. A few of the issues we will address in a small focus group format are whether RAPUNSEL’S software environment is socially engaging, and the impact of learning communities on users’ engagement, motivation and learning [7].

3. Software usability will be measured using several different methods. Taken together, these methods will allow us to evaluate: Instantiation: is there a match between the programming concept/skill and its instantiation in the software; User Interface: is the software easy and fun to use; and Operations: are there bugs or other problems with the software that lead to system crashes. Design partners. As described earlier, we will invite two groups of girls, one in New York

and one in Oregon, to serve as design partners. The design partners will help us to create scenarios for the first three modules in RAPUNSEL that are engaging and appealing to middle school girls, and will provide us with pilot data on software usability before we introduce RAPUNSEL in our large sample. Data collection with our design partners will be conducted in one-on-one interviews and focus groups.

User interface probes. After each module, we will ask our participants to rate their difficulty using the software and understanding the tasks to be completed in the module, and to describe any specific problems they encountered. We will also have participants indicate their level of interest and enjoyment.

Engagement and Task Analysis. We will use built-in software monitoring to produce evidence for engagement evaluation, primarily following the evaluation recommendations of Quintana, Fretz, Krajcik, and Soloway in their study of learner-centered tools [52] and time-on-task analysis [9]. Are the program tools easily accessible? If a participant cannot use or

13

access a tool such as the script editor, the tool must be redesigned. We will monitor system tests to see how students progress through the software.

We will include an appropriate control (no-treatment) group into our research design by collecting data on pre-existing knowledge, attitudes and intentions with a sample of 5th and 6th graders from the 4J school district who will not participate in the RAPUNSEL project at multiple points in time. Data collection on the control group will take place during the software construction phase of RAPUNSEL in the first year of the project. The inclusion of this control group will enable us to rule out some alternative explanations for possible results, such as maturation. We will follow all NSF and University guidelines for the protection of human participants, including written informed consent from parents and assent from children.

Management PlanMuch of the infrastructure that is needed to create and support RAPUNSEL has already been established. The research teams led by the PIs have developed an effective set of organizational practices and technologies that lend themselves to a disciplined and intellectually productive collaboration. We will build on this foundation. The PIs will lead their teams at the CAT/MRL and the FUSION labs at NYU and UO respectively, with the technical core of the project being created at NYU and the online survey and evaluation tools being created at UO. We will bring in our design, industry, and educational consultants to kick off the project in an intensive brainstorming session. We will continue to coordinate our work via bi-annual team meetings and frequent “virtual brown bags”, using Internet II videoconference technology. Evaluation will take place in Eugene, and will be led by Dr. Tindal and a doctoral student with prior experience in the evaluation of online learning. We will switch sites for our semi-annual team meetings so that all team members can be at each site once per year. The production team is in place at NYU’s CAT lab, and the 4J school district enthusiastically supports our project and is prepared to move forward with us in evaluation of the project.

Software Development. Rather than follow a traditional software development cycle, we are creating a research software development pattern, incorporating many “mini-research and evaluation cycles” to provide constant feedback on cognitive, pedagogical, and usability issues. In Year 1, we will research, design, and develop the demo and modules 1 and 2, and will gather baseline measures of users’ pre-existing knowledge, attitudes and intentions. In Year 2, we will rework existing modules based on feedback, and develop modules 3 and 4. In Year 3, we will finalize modules based on the research.

Evaluation and Assessment. Because evaluation and assessment are so closely tied to the software development tools schedule, these aspects will closely follow our production cycle. The proposed project will have the following major tasks and associated deliverables:

14

Year 1Development

Meet with consultants and PIs. Design and create back end tools such as the security system, the monitoring technology, the intelligent active script editor, character agents, the graphical user interface (GUI), and pre-coded behaviors and objects. Develop first version of demo module. Meet with design partners and build Modules 1 and 2. Design and create software agents, the online assessment tool, and narrative content. Develop program analysis and measurement tools. Further develop contacts with other researchers and interested teachers.

Year 1 Assessment

Conduct initial middle school user testing including attitudinal/perceptual tests, engagement and task assessment, focus groups, and pretests on demo for performance-based transfer tasks. Pretest measures which will provide a baseline: attitudes, skills, and demographic variables. Collect feedback from first software and demo experiences and tests. Initiate initial prototype rounds of aspects of the RAPUNSEL environment based on cognitive and pedagogical research, and evaluation and user interface testing. We are particularly targeting those users who report low attribution and low self-efficacy [4]. Do students who report low self-efficacy become interested in the project and use it? This first phase will primarily focus on navigation issues, and the usability of the user interface. To evaluate the user interface, we will classify the appropriateness of metaphors, and predictability of the interface, the intuitability of system functions (including the consistency of messages, icons, menus, visual clues), the look and feel of the system, and the feedback which the system provides. Feedback from design partners will be especially important in the design.

Year 2Development

Revise software environment tools, tutorials, and instructional agents based on assessment results. Build Module 3, 4, and database. Refine user interface. Integrate level analysis and measurement tools into programming environment. Fix bugs. Redesign based on results of engagement and task analysis evaluation.

Year 2 Assessment

Continue testing on Modules 1, 2, 3, 4. Analyze data, compile testing, conduct post-tests on demo module feedback. Engagement and task analysis evaluation and attitudinal/perceptual data collection will continue. These measures will help us determine the educational value of the software once functional issues are wrapping up. We will be testing for task persistence in year two, especially interested in low attribution and perception groups. Learning assessment is an important part of this evaluation research methodology: Is the program well packaged, well sequenced, are proper skills taught in order, are girls able to understand tasks and programming principles? We will program screen captures and take timed readings of the use of the

15

software. Narrative tests and portfolios will begin once we introduce the modules in our large sample of middle school students in the 4J school district. We will monitor the types of collaborative activities in which girls engage. We will also begin work group interviews.

Year ThreeDevelopment

Rework and refine environments, testing including post-tests. Project completion, reports.

Year 3 Assessment

We will pursue our last round of data collection across the three content areas: measurement themes from Year 1 and 2 together and will test for cognitive constructions, attitudes, engagement, and transfer tasks

DisseminationWe plan to reach the audience (middle school girls, as well as their teachers, parents, and friends) in a multi-tiered approach. First, we will attract national and international attention for the project based on its innovative design and noteworthy partners and participants. We will also send informative material to major school districts, and our program will be included in the leading technology-based teacher education group, the International Society for Technology in Education's materials and on their website. Second, we will attract media attention to publicize the project to groups who might not normally be interested in utilizing our environment. Third, our findings will be shared at national conferences and in articles and in local schools across the country. Results of this work will be applicable to software development systems, interface research (HCI), software engineering and design, IT literacy, and other areas of information technology. We plan presentations of the work at venues such as SITE, SIGGRAPH, and AECT. Results will also be used as a model for other projects, specifically, a national programming literacy initiative.

The RAPUNSEL environment will be available freely on the web – dissemination of the software will occur via a specially designed website. The software environment in RAPUNSEL will be available almost completely as an opensource environment. It is important to be aware that online spaces can be dangerous to girls, so we must create a secure environment [61]; therefore we have decided to put some restrictions on the core identity servers in order to prevent untoward malevolence towards child users of the project. Each user session will be password protected. Because our architecture is written in Java, we can instantly provide the environment and coding tools to students on the web. The program is also designed to be used in a classroom context. In future developments of this project, we will develop a curriculum guide and conventional instructional activities such as handouts and teacher-led activities to supplement the onscreen environment.

ResultsRAPUNSEL represents an incremental approach to wide-scale cultural change in gender equity issues in math and science. We are conducting this research to afford cultural diversity, individual learner diversity, and cultural change through a relevant and useful programming environment. The program’s goals and open-ended architecture will ultimately encourage others to build communities and artifacts that matter to all individuals and groups.

Not only does RAPUNSEL address issues relating to gender equity and the digital divide, but we also believe that collaborative learning software such as RAPUNSEL will help to facilitate a more fundamental change in society: increasing computer literacy. Computer literacy for all citizens will be imperative for the United States to maintain a diverse, internationally competitive and globally engaged workforce of scientists, engineers and well-prepared citizens. This literacy must include computer programming and computer science fundamentals: and involve both reading (using existing computer applications) and writing (making one's own applications). Literacy, however, does not simply involve technical expertise. Literacy is a widespread and socially engaged system of skills, capabilities, and creativity formed in the context of a material support [17]. Therefore we must not

16

only focus on the material support (software environments) but also on the social: the values, interests, motivations, and community practices of learners. Our project takes small steps towards big changes by addressing the unmet needs of middle school-age girls, but we envision a much larger and more fundamental application for the knowledge that will be gained from this project.

17

References

1. AAUW. (1998). Gender Gaps: Where Schools Still Fail Our Children. American Association of University Women Education Foundation, New York.

2. AAUW. (2000). Tech-savvy:Educating Girls in the New Computer Age. American Association of University Women Education Foundation, New York.

3. AAUW. (2002). “Education and Software Games: Rethinking the “Girls' Game.” The Jossey-Bass Reader on Gender in Education. San Francisco: Jossey-Bass, 345 – 360.

4. Bandura, A. (1997). Self Efficacy: The Exercise of Control. New York: W. H. Freeman Co.

5. Bernstein, D. (1991). “Comfort and Experience with Computing: Are They the Same for Women and Men?” SIGCSE Bulletin 23, 1991, 57 -60.

6. Beyer, S. & Bowden, E. (1990). “Gender Differences in the Accuracy of Self-Evaluations of Performance.” Journal of Personality and Social Psychology, 59, 960-970.

7. Brandon, D. P., & Hollingshead, A. B. (1999). “Collaborative Learning and Computer-Supported Groups.” Communication Education, 48, 109-126.

8. Bruckman, A. & Resnick, M. (1995). “The MediaMOO project: Constructionism and Professional Community.” Convergence, 1(1), Spring.

9. Bruckman, A., Jensen, C. & DeBonte, A. (Jan 2002) “Gender and Programming Achievement in a CSCL Environment.” Long talk. Proceedings of CSCL 2002. Boulder, CO.

10. Carr, H. H. (1992).  “Factors That Affect User-Friendliness in Interactive Computer Programs.”  Information and Management, 22, 137-149.

11. Carter, J. & Jenkins. T. (June 1999). “Gender and Programming: What's Going On?” Bill Manaris, ed. Proceedings of the 4th Annual SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE'99), 1-4, June 1999. ACM SIGCSE/SIGCUE.

12. Catsambis, S. (1994).”The Path to Math: Gender Racial-Ethnic Differences in Mathematics Participation From Middle to High School”, Sociology of Education 67, 199-215.

13. Chmielewski, D. C. (15 Dec 2002). “Online Play to Add New Wrinkle to Oddly Compelling Socially-Interactive Game.” Knight Ridder/Tribune Business News.

14. Chu Clewell, B. (2002). “Breaking the Barriers: The Critical Middle School Years.” 301-313. The Jossey-Bass Reader on Gender in Education. San Francisco: Jossey-Bass.

15. Commission on the Advancement of Women and Minorities in Science, Engineering, and Technology Development (CAWMSET). (Sept 2000) Land of Plenty: Diversity as America’s Competitive Edge in Science, Engineering and Technology.. Available online at http://www.nsf.gov/od/cawmset/report.htm

18

16. de Jong, T.(1991). “Learning and Instruction with Computer Simulations.” Education & Computing 6. 217 – 229,

17. diSessa, A. (2000). Changing Minds: Computers, Learning, and Literacy. Cambridge: MIT Press.

18. Dept. Office of Educational Research and Improvement (DOERI), National Center for Education Statistics. (1997). Findings From the Condition of Education, 1997: # 2, Women in Mathematics and Science Education. Education Department Publication NCES 97-982

19. Druin, A. (1999). “Cooperative Inquiry: Developing New Technologies for Children with Children.” Proceedings of the SIGCHI conference on Human Factors in Computing Systems. New York: ACM Press, 592 - 599.  20. Eckel, B. (2002). Thinking in Java. 3rd Edition. Mindview, Inc. Available online at http://mindview.net/Books/TIJ/.

21. Elliot, C. (1997). “Hunting for the Holy Grail with ‘Emotionally Intelligent’ Virtual Actors.” http://condor.depaul.edu/~elliott/papers/intelligence/.

22. Elliot, C. Lester, J., & Rickel, J. (1997). “Integrating Affective Computing into Animated Tutoring Agents.” In Proceedings of the IJCAI Workshop on Animated Interface Agents: Making Them Intelligent, Nagoya, Japan, 113-121.

23. Flanagan, M. (2000). “Interfacing Differently: Educating Girls in a Changing Digital Landscape.” Proceedings of WebNet 2000 --World Conference of the WWW, Internet, + Intranet. Eds. Hermann Maurer + Richard G. Olson. Charlottesville: Association for the Advancement of Computing in Education (AACE).

24. Furger, R. (1998). Does Jane Compute? Preserving Our Daughters’ Place in the Cyber Revolution. New York: Time Warner.

25. Ward Gailey, C. (1993). “Mediated Messages: Gender, Class, and Cosmos in Home Video Games.” Journal of Popular Culture 27:1, 81-97.

26. Guzdial, M. (1995). “Software-Realized Scaffolding to Facilitate Programming for Science Learning.” Interactive Learning Environments.

27. Guzdial, M., & Soloway, E. (2002). “Teaching the Nintendo generation to program: Preparing a new strategy for teaching introductory computer programming.” Communications of the ACM, 45: 4 New York: ACM Press, 17-21.

28. Haller, S. & Fossum, T. (1998). “Retaining Women in CS with Accessible Role Models.” Proceedings of SIGCSE’98 Conference, Atlanta.

29. Honey, M., Moeller, B., Brunner, C., Bennett, D., Clements, P., & Hawkins, J. (2002). “Girls And Design: Exploring the Question of Technological Imagination.” The Jossey-Bass Reader on Gender in Education. San Francisco: Jossey-Bass, 329-344.

19

30. International Society for Technology in Education (ISTE). (2002). Educational Computing and Technology Standards. Eugene, OR: ISTE Accreditation and Standards Committee, 6. http://www.ncate.org/standard/new%20program%20standards/iste%202002.pdf

31. Kafai, Y. B. (1995).Minds in Play: Computer Game Design as a Context for Children's Learning. Hillsdale, NJ: Erlbaum Associates.

32. Kafai, Y. B. (1998). “Video Game Designs by Girls and Boys: Variability and Consistency of Gender Differences.” Cassel, J. and Jenkins, H. From Barbie to Mortal Kombat: Gender and Computer Games. Cambridge: MIT Press.

33. Laurel, B. (1990). “Interface Agents: Metaphors with Character.” Art of Human-Computer Interface Design. Laurel, B (ed.) Reading MA: Addison-Wesley, 355-66.

34. Lester, J., Stone, B., & Stelling, G. (1999). “Lifelike Pedagogical Agents for Mixed-Initiative Problem Solving in Constructivist Learning Environments.” User Modeling and User-Adapted Interaction, 9:1-2, 1-44.

35. Lewis, S. (2000). “Masculinity and IT: Gender and the IT Industry.” Gender & Cultural Diversity Matters. 1:4.

36. Lonergan, D. (1997). “Network Science: Bats, Birds, Trees: Teaching Science-Related Subjects Through Computers.” Educational Leadership, Nov 1997, 55:3, 34.

37. Lynch, D. (2000). “Purple Moon Setting.” ABC News Internet Edition. ttp://abcnews.go.com/sections/ tech/WiredWomen/wiredwomen990309.html

38. Margolis, J. & Fisher, A. (2001). Unlocking the Clubhouse: Women in Computing. Cambridge: MIT Press.

39. McLester, S. (Oct 1998). “Girls and Technology: What's the Story?” Technology & Learning, 19:3, 18.

40. Miller, L. et. al. (1996). “Girls' Preferences in Software Design: Insights from a Focus Group.” Interpersonal Computing & Technology. 4:2, 27-36.

41. Moller, L. C., Hymel, S., & Rubin, K. (April 1992). “Sex Typing in Play and Popularity in Middle Childhood.” Sex Roles: A Journal of Research. 26:7-8, 331- 354.

42. MUVEES. (2001). http://www.virtual.gmu.edu/muvees/documents/ICLSfinal.pdf

43. Nisbet, J. & Shucksmith, J. (1986). Learning Strategies. London: Routledge Education Books.

44. Orenstein, P. (1995). Schoolgirls: Young Women, Self-Esteem, and the Confidence Gap. New York: Doubleday & Company.

45. Ortega, M., & Bravo, J. (2001). Computers and Education: Towards an Interconnected Society. Boston: Kluwer Academic Publishers.

20

46. Papert, S. (1993). The Children's Machine: Rethinking School in the Age of the Computer. New York: Basic Books.

47. Papert, S. & Harel, I. (1991). “Situating Constructionism.” Constructionism. Ablex Publishing Corporation, Online at http://www.papert.org/articles/SituatingConstructionism.html

48. Pearl, A., Pollock, M., Riskin, E., Thomas, B., Wolf, E., & Wu, A. (1990). “Becoming a Computer Scientist.” Communications of the ACM, 33:11, 47-57.

49. Perlin, K. & Goldberg, A. (1996) “Improv: A System for Scripting Interactive Actors in Virtual Worlds.” Proceedings of SIGGRAPH 96, H. Rushmeier, ed., ACM Press, New York, 1996, 205-216.

50. Petraglia, J. (1998). Reality by Design: The Rhetoric and Technology of Authenticity in Education. Mahwah, NJ: Erlbaum Associates.

51. Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human Computer Interaction. The Open University. London: Addison Wesley.

52. Quintana, C., Fretz, E., Krajcik, J., & Soloway, E. (2000). Evaluation Criteria for Scaffolding in Learner-Centered Tools.” CHI '00 Extended Abstracts on Human Factors in Computer Systems. New York: ACM Press, 189 - 190  

53. Reiser, R. A. & Dick, W. (1990). Evaluating Instructional Software. Educational Technology Research and Development 38 3, 43-50.

54. Resnick, M. (1996). “Towards a Practice of 'Constructional Design.'“ Innovations in Learning: New Environments for Education. Ed. Schauble, L., & Glaser, R. Mahwah, NJ: Erlbaum Associates Publishers, 161-174.

55. Schank, R. C. & Cleary, C. (1995). Engines for Education. Hillsdale, NJ: Erlbaum Associates.

56. Smith, D. C, Cypher, A. & Spohrer, J. (1994). “KidSim: Programming Agents Without a Programming Language.” Communications of the ACM, 37(7):54--67.

57. Smith, T. E. (1992).”Gender Differences in the Scientific Achievement of Adolescents: Effects of Age and Parental Separation.” Social Forces, 71, 469-484.

58. Solomon, G. (Jan 1991). “Mass Multimedia: Classrooms in Shrewsbury, Mass., Use Multimedia Technologies to Add “Real Substance” to Numerous Curriculum Areas.” Electronic Learning, 10:4, 14.

59. Soloway, E., Guzdial, M., & Hay, K. (1994). “Learner-Centered Design.” Interactions of the ACM. 1: 2, 37-48.

60. Soloway, E., & Norris, C. (June 2002). “From Human-Centered Design to Learner-Centered Design.” Proceedings from the Computing Research Association Grand Challenges Conference 2002. June 23-26.  http://www.cra.org/Activities/grand.challenges/solowaynorris.pdf

21

61. Sutton, L. A. (1996). “Cocktails and Thumbtacks in the Old West: What Would Emily Post Say?” Wired Women: Gender and Realities in Cyberspace. Eds. Cherney, L. & Weise, E. Seattle: Seal Press, 169 - 181.

62. Taylor Anderson, B. (1993). “How Can Middle Schools Get Minority Females in the Math/Science Pipeline?” Education Digest. 59:2, 39.

63. Turkle, S. & Papert, S. (1990). “Epistemological Pluralism: Styles and Voices Within the Computer Culture.” Signs 16, 129-157.

64. U.S. Bureau of Labor Statistics. (2002). Report 3915. http://www.bls.gov/rofod/3915.pdf; Accessed 2002. http://www.bls.gov/

65. van Rossum, G. (2001). “Computer Programming for Everybody.” CNRI. http://www.python.org/ doc/essays/cp4e.html

66. Wigfield, A. & Eccles, J. S. (1994). “Children's Competence Beliefs, Achievement Values, and General Self-Esteem Change Across Elementary and Middle School.” Journal of Early Adolescence 14, 107-138.

67. Wilson, B. C. & Shrock, S. (2001). “Contributing to Success in an Introductory Computer Science Course: A Study of Twelve Factors.” ACM SIGCSE Bulletin, Proceedings of the thirty second SIGCSE technical symposium on Computer Science Education, 33(1), 184-188.

22