-Abstract--This paper describes CHAT, educationalsoftware intended for use in introductory college-levelsyntax or general linguistics courses. The software ismotivated by two persistent pedagogical problemsfrequently encountered in scientific teaching: studentmotivation and the abstract, non-procedural characterof formal theories. After describing the system's agent-based architecture and interface, we argue that inquiry-based, collaborative software best supports the teachingof linguistic theory, and consider some broaderimplications for general science teaching in an e-learning environment.

Index terms: collaborative learning, educationalsoftware, inquiry-based learning, science education

I. TEACHING LINGUISTICS AS A SCIENCE

Although the extent to which Chomskyan linguisticsis a science (on a par with natural sciences, likephysics, for example) has been debated onphilosophical grounds [3], it is clear that the teachingof linguistics enjoys many of the same challenges asdo its more traditional sister disciplines. For onething, learning introductory linguistic theory meanslearning grammars–formal systems of rules andprinciples that determine in an algorithmic fashionthe patterns of the languages of investigation. This ispainstaking work both at the theoretical level (atwhich putative universal generalizations must betested) and at the descriptive level (at which often-complicated sets of data must be analyzed). As a

Submitted to Scuola Superiore G. Reiss Romoli (SSGRR)30.May.2001. The authors are faculty and staff of HampshireCollege, Amherst Massachusetts. All correspondence should bedirected to Steven Weisler, School of Cognitive Science,Hampshire College, Amherst Massachusetts 01002 (email:sweisler@hampshire.edu). This work was funded in part by NSFgrant 8-0-0-33901. This effort was sponsored by the DefenseAdvanced Research Projects Agency (DARPA) and Air ForceResearch Laboratory, Air Force Materiel Command, USAF, underagreement number F30502-00-2-0611. The U.S. Government isauthorized to reproduce and distribute reprints for Governmentalpurposes notwithstanding any copyright annotation thereon. Theviews and conclusions contained herein are those of the author andshould not be interpreted as necessarily representing the officialpolicies or endorsements, either expressed or implied, of theDefense Advanced Research Projects Agency (DARPA), the AirForce Research Laboratory, or the U.S. Government.

second challenge, students are often surprised todiscover what the work of theoretical linguists isactually like. Seduced by questions about apelanguage and the thought-influencing lens oflinguistic determinism, students are, to say the least,not eager to accommodate to a regime of datacollection, hypothesis testing, and systematicanalysis. Moreover, grammars are formal systemsthat depend on abstract notation and require analyticprecision. From many students' point of view,learning grammars feels more like learning math andscience than like the excursion into the humanitiesthey were expecting.

In essence, the successful teaching of introductorylinguistic theory requires solving many of thestandard pedagogical problems confronting generalscience teaching, complicated by the student'spervasive sense that the field turns out not to be"what I thought it was." This paper describes theconception and development of CHAT, educationalsoftware that combines an inquiry-driven pedagogicalframework with elements of proceduralized design ina collaborative digital learning environment. The goalof this project is to develop and assess newtechnologically supported teaching paradigms forinquiry-based education in the sciences.

II. STUDENT MOTIVATION

The introductory syntax course is central to theundergraduate linguistics curriculum but it is adifficult course for many students. The problemsstem in part from the course's heavy (and probablyunexpected) use of formal mathematical notation andreasoning. Some degree of mastery of formal syntaxis necessary for students to appreciate the manyspectacular results of modern linguistics, and becausefew entering students have such mastery, theytypically begin with little motivation to shoulder themathematical burdens of the course. The maintenanceof student motivation is therefore an importantchallenge for syntax instructors. To approach thischallenge, CHAT develops three leading ideas: itprovides technical scaffolding for student inquiry, itpromotes a collaborative learning environment, and it

An Inquiry-based Approach to E-learning: TheCHAT Digital Learning Environment

Steven Weisler, Roger Bellin, Lee Spector, and Neil Stillings

develops an intuitive, procedurally implementedapproach for teaching linguistic theory.

The use of technology to enhance student motivationhas a long history and a rich literature. One currenttrend is to provide technological supports for studentsengaged in self-motivated, inquiry-oriented learningexperiences. This approach is reflected, for example,in a recent special section in the Communications ofthe ACM in which the authors write that "Learner-centered, problem-driven approaches to education...are most effective in engagement, motivation, and,through their problem-driven format, in providing asolid conceptual understanding" [8]. This practice isalso consonant with the general pedagogicalenvironment of Hampshire College, where CHATwas developed; Hampshire College has a long-standing and explicitly articulated commitment toinquiry-oriented undergraduate education [7]. Wetherefore took it as our goal to develop a tool thatenhances the motivation and learning of beginningsyntax students by providing technical support forstudent inquiry.

A second insight developed in CHAT is thatmotivation for formal problem solving can often bestrengthened when groups of differentially talentedstudents work in a collaborative learningenvironment. To this end, CHAT employs a chatroom-like design in which students use the grammarsthey develop to "chat." This not only allows for achannel of communication, but also permits studentsto examine the work of others and to test their ownhypotheses in a cooperative manner. To a largedegree, this works against the essentially logico-mathematical character of linguistic theories,anchoring the pursuit of this knowledge in richersocial transactions.

Finally, grammars are normally thought of as theoriesof a speaker's abstract grasp of a language–specimensof linguistic knowledge that underlie the commonability to produce or understand a word or sentence.It is frequently difficult for students to comprehendthe precise ontological status of these so-called"competence theories" [3], which are easily confusedwith the parsing systems and systems of naturallanguage understanding to which they are related, buttypologically distinct. In CHAT, students learn toconstruct grammars by building a sentence generator.This has the joint effect of proceduralizing the task ofgrammar construction from the outset while alsorendering the implementation relationship betweenthe generator and the grammar more transparent. Putmore simply, students can directly grasp the task ofmaking a system that can "chat," which in turn

promotes an intuitive sense of the role of grammarsin sentence production that is nearly impossible toachieve by theoretical description alone.

Several previous projects have provided studentswith general technological supports for the "inquirycycle," for example by providing "lab book"environments integrated with statistical packages andgraphing tools. The goal in the present project was toprovide more direct and domain-specific support forlinguistic inquiry by allowing students to explicitlyconstruct and test linguistic theories in the form ofgrammars and lexicons. The "linguistic theories" inCHAT are active computational systems that producebehavior and data that students can observe tovalidate or falsify their hypotheses about the natureof language. This use of computational "systemconstruction" (or "simulation construction," "modelconstruction," etc.) as an active form of theoryconstruction for inquiry-oriented education also has ahistory in the literature (see, for example, [10]), butto our knowledge CHAT is the first application ofthis idea to the field of linguistics. The linguisticsdomain is a particularly interesting application areafor this approach since by using CHAT, students cantypically understand a theory's strengths andshortcomings simply by examining the sentences itgenerates, using native ability as an English speakerto detect ungrammaticality.

III. THE CONSTRUCTION OF GRAMMARS

A grammar consists of a lexicon (or dictionary) plusa set of context-free rewriting system augmented by acomplex matrix of subcategory features that allowcertain linguistic dependencies to be naturallycaptured. The lexicon contains a list of words alongwith information about syntactic distribution, part ofspeech, and other morphological properties (likesingular/plural, masculine/feminine, mass/count,etc.), each of which must be properly established onpain of over- or under-generating a corpus of thetarget language. For example, the lexical entry for the"cat" would indicate that it is a third-person singularnoun that requires a determiner (e.g., "the") as its"specifier"–that which introduces a common noun.

The phrase structure grammar takes the form of a setof rewriting rules:

S NP VPNP Det NVP V Adv

In this simplified example, assuming a properlexicon, the rule set suffices to generate the followingsyntactic analysis tree:

Fig. 1. A Syntactic Analysis Tree.

Reading the "→" as "consists of," the first rulesanctions sentences (S) that consist of a N(oun)P(hrase) followed by a V(erb) P(hrase). Thecategories "Det," "N," "V," and "Adv" stand forDeterminer, Noun, Verb, and Adverb, respectively.

The central descriptive task is to expand the lexiconand the re-writing system to account for all and onlythe grammatical sentences of English (or any otherlanguage of description), a problem of great scientificmagnitude. Ultimately, we confront aspects of syntaxthat require additional descriptive devices forexample, the case system (e.g., the differencebetween "I" and "me"), the pronoun system, non-declarative sentence types, embedded sentences, andthe helping verb system, to name just a few advancedtopics in English syntax. CHAT, or indeed, any otherlearning environment for grammar construction, mustscale up to permit the description of these morecomplicated data, as well the simpler phrase-structuredependencies exemplified by the grammar above.

More challenging still is the task of writing grammarsthat are independently motivated. Each proposedgrammatical analysis makes a claim about thestructural representations that a speaker assigns to asentence. Therefore, many grammatical analyses that

succeed in generating analysis trees for a range oftarget sentences nevertheless assign implausiblestructures that lose generalizations and ultimately failto extend to provide a general account of thesyntactic structure for the target language. Suchtempting but incorrect alternative hypotheses must berejected, often on the basis of further data. One of themost important heuristics that researchers mustdevelop is a sense of where to look for independentsupport for a promising analysis. Often, the absenceof such intuitions causes great difficulty for the non-vitiate linguist.

IV. OVERVIEW: THE CHAT CONCEPT

CHAT provides a networked "chat room"environment to which students and faculty can postsentences. The sentences, however, cannot be directlycomposed. Instead they are generated by clicking ona "generate" button that produces a parse tree for aparticular sentence form, along with an interface thatallows for selection of particular words for all of thetree's leaves. Only words that satisfy the constraintson the leaf and its position, as specified in thegrammar and the lexicon, are made available forselection. When words for all of the leaves have beenselected the sentence, along with its parse tree, is sentto the public chat room. The sentence generator isdriven by a grammar, a lexicon, and several otherparameters, all of which can be edited by the studentusing the graphical user interface described below.

Students typically begin with empty or primitivegrammars and lexicons. Their goal is to make thesystem capable of generating sophisticatedgrammatical sentences (in some cases specific typesof sentences suggested by the instructor), withoutallowing the system to generate ungrammaticalsentences. In some cases a student with a problematicgrammar may be able to disguise this fact byavoiding the selection of problematic words forparticular tree leaves, but this trick can be easilyexposed; sentences with randomly selected (butconstraint-satisfying) words are also made availableto others who wish to probe the grammatical analysisbehind a particular sentence.

It is also possible to provide students with morecomplex initial grammars and lexicons, and tochallenge the students to change/expand the systemto account for new linguistic phenomena. CHAT'slexicon and grammatical rule-building tool kits arebased on current Chomskyan minimalist syntactictheory [4]. The system allows for the investigation ofa wide range of linguistic phenomena and in somecases allows for different theoretical approaches to

particular problems. As such, it is an open-endedenvironment for inquiry into the nature of humanlanguage. In this way CHAT's structured space forexperimentation can provide a naturally incrementalapproach to theory-building. The student need notunderstanding all of syntax from the start, but can beasked to account for one syntactic phenomenon in thetheory and then move on to another, progressively.

The network features of the system allow forcollaboration among the students and faculty, and acommunity of software agents, described below,provides additional collaborative support and advice.The social nature of the chat room interactions, andof chat rooms in general, also facilitates studentmotivation and engagement. The software is intendedfor use in class or lab sessions, with 5-20 students(one or two students per computer) and with aninstructor or TA present; but it could be used forindividual experimentation or even homeworkassignments without networking. The chat room isbased on TCP/IP networking, so it can be shared overthe Internet among users around the world. CHAT isimplemented in Java to facilitate distribution acrossall popular computer platforms.

V. OVERVIEW: THE AGENT ARCHITECTURE

One challenge in the design of any software forinquiry-oriented education is how to provide usefulfeedback (help, advice, directions, etc.) withoutdestroying the student-initiated character of theinquiry. This is difficult because the software cannotpossibly predict all student actions or intentions whenthe inquiry is truly open-ended. To meet thischallenge we followed another recent trend in theliterature, that of using "intelligent agents" ([2], [1])that function as learning companions ([5], [6]). Theseagents have more autonomy and intelligence thantypical "online help" functions, but they do notattempt to fill the role of a teacher or tutor; each actsmore like a "guide on the side" than a "sage on thestage" ([9]), providing advice only when consulted,signaling discreetly when new information isavailable.

To facilitate the development of robust agents wedesigned a general architecture for inquiry-basedlearning environments, in which students build andmodify computational models (see Figure 2). Theheavy arrows in the figure represent the normalinteractive cycle of a software simulationenvironment: the user initiates actions that influencethe simulation, and the simulation output is providedas feedback to the user. Our agent-based environmentaugments the simulation environment with two

clusters of intelligent agents. One pool of agentswatches the student’s interactions with the system,along with a blackboard to which all agents canwrite, and reports significant patterns and trends tothe blackboard. Some of these monitoring agents mayattempt to model the student’s understanding of thedomain, but simpler agents that, for example, noticerecurring cycles of values for simulation variables,will by themselves provide considerable utility.These "observer" agents can also access externalinformation sources to obtain additional assessmentsof the status of the model. The second pool of agents,the "reporters," watches the blackboard and reports tothe student. The reports may describe noted patterns,trends, suggestions for further experiments, andpointers to additional source materials. The reportingagents may also write to the blackboard so that futurereporting agents can ensure that their reports areconsistent and not redundant.

Fig. 2. Agent architecture for inquiry-based learningenvironments.

As a simple example of the general utility of thisagent architecture, consider an inquiry-based tutor forepidemiology. The tutor might be built around asimulator that models the spread of diseases andimmunity through a population. The simulator mightinclude user-settable parameters such as diseasetransmissibility and average time from infection todeath (for a fatal disease), along with dependent

variables such as the total number of infections.While a student is exploring and experimenting withthe simulator, a simple monitoring agent might noticethat, for example, the disease quickly died out in eachof the student’s simulations, and that the setting fortime from infection to death was in each caseunusually low. A reporting agent might then refer thestudent to source materials on the epidemiology ofquick-killing diseases such as Ebola, while anothermight note other user-settable parameters that couldbreak the noted pattern, and another might suggestexperiments with specific settings of theseparameters.

In many respects these monitoring and reportingfeatures are similar in spirit to the monitoring andreporting features of simulation-based computergames such as SimCity and SimLife. These featuresgive the user high-level status reports about thegame/simulation as it progresses. This approach canbe enhanced significantly through the use of goodgeneral representations, logical inference rules, andan agent architecture. When built around accuratesimulators that are designed for educational ratherthan entertainment purposes, we believe that thesefeatures will provide substantial support for inquiry-based education.

VI. OVERVIEW: CHAT AGENTS

The agents built for CHAT were motivated byobservation of videotaped classroom sessions with anearly (agentless) CHAT prototype. We noted ways inwhich students interacted with one another, and waysin which the instructor would intervene and guidestudents, and we attempted to build agents that act insimilar ways. The architecture was designedspecifically to ease the incremental addition of newagents, and we do not consider the current set ofagents to be final.

The current implementation includes agents thatobserve (e.g., the "Link Parser Agent") and report on(the "Critic," the "Passed Rule Agent," and theBroken Derivation Reporter") a student's progress.The Link Parser Agent generates new sentencessanctioned by a student grammar "behind the scenes,"and ships them off to the CMU online parser to checkfor unnoticed ungrammaticality. The Critic providesfeedback to the students based on an analysis of theCMU feedback, while the Passed Rule Agent reportson phrase structure rules that could not beimplemented because of a conflict in the relevantfeature assignments. The Broken Derivation Reporteris responsible for reporting on circumstances inwhich an attempt to augment the grammar

unwittingly results in giving up previously successfulanalyses of sentences that were generated by priorversions of the student's grammar. This reporter isparticularly useful for students who are making large-scale changes to their grammars, especially in casesin which there are hidden implications of a changethat are hard to deduce.

VII. CHAT'S USER INTERFACE (SELECTED

FEATURES)

CHAT's user interface breaks the task of formalizingEnglish syntax into logical parts, including programnavigation, grammar and lexicon creation andrevision, and windows for analysis tree displays,private, local generation of test sentences, a chatroom, and the CHAT agents. At the beginning ofeach session with CHAT, a login dialogue is shown:

Fig. 3. The Login Dialogue.

The student may enter a name or nickname in thefirst field that will be used to identify the student'scontributions in the chat room. The second fieldcontains the Internet address of a computer that isrunning CHAT's server program. The address may beentered as an IP address (like "127.0.0.1") or a hostname (like "host.college.edu"). If CHAT is unable toconnect to this server because of an incorrect addressor a network problem, or if the computer you enter isnot running CHAT's server application, then the chatroom will not function. (The student will be notifiedof CHAT's failure to connect, and all other parts ofCHAT will continue to work normally.) Studentsusing CHAT on a computer without network access,or not wanting to use the chat room to collaboratewith other CHAT users can uncheck the checkboxlabeled "Use chat room." In that case, no serveraddress is required.

Students who have logged in can use CHAT'sNavigation Palette to bring up the windows of CHAT

in which the various aspects of grammar constructionmay be undertaken.

Fig. 4. The Navigation Palette.

In particular, the Lexicon button makes the Lexiconwindow visible so that the student can create or edit alexical entry, and the Rules button opens theGrammar window with which the student can createor edit a phrase structure rule. Here is a rathersophisticated grammar constructed in CHAT:

Fig. 5. The Grammar window.

Clicking the New button opens the Rule Editorwindow allowing the student to create new phrasestructure rules in the grammar:

Fig. 6. The Rule Editor.

Clicking on any of the pull-down menus allows thestudent to select (or create) different node labels (e.g.,S, NP, etc.). Clicking Preferred increases theprobability that a given phrase structure rule will beused in generation to make it easier to test particularportions of the grammar.

Similarly, clicking the lexicon button pulls up thefollowing window containing a list of previouslyentered lexical items:

Fig. 7. The Lexicon window.

By clicking New, the Entry Editor window isdisplayed, allowing the creation and editing of lexicalitems via a complex series of pull-down menus thatpermit morphological feature assignment.

Fig. 8. The Entry Editor.

In a similar fashion, the Morphology button opens awindow that allows the student to create and editmorphological rules (e.g., 'turn a singular noun into aplural noun by adding "s"'–general (but notexceptionless) rules that can be used to create newlexical items from old entries, thereby simplifying theconstruction of the lexicon. Finally, the Bindingbutton plays a role in accounting for certain advancedproperties of the pronoun system.

There are two options for students to test the outputof their grammars. Clicking Generator opens thefollowing private generation window:

Fig. 9. The Private Generator.

In this mode, only the student (or a group of studentsworking on a single machine) can see the output ofthe grammar. Every time Generate is clicked, arandomly generated sentence compatible with thestudent grammar is displayed in the generation field.By highlighting a generated sentence and clickingDisplay Tree, the analysis tree for that sentence,based on the rules and lexical entries contained in thestudent grammar, is displayed:

Fig. 10. The Tree Display window.

Alternatively, clicking the Chat button opens theChat window through which the student can read thecontents of and contribute to the chat room.

Fig. 11. Chat Room.

Clicking "Send a Sentence" generates new entries inthe chat room. Each successive sentence generated(here, by Jane Doe) is listed sequentially in the Chatwindow along with its source, and is visible to allstudents running CHAT in a given session. Byhighlighting a sentence and clicking "Display Tree,"an analysis tree for the selected sentence is displayed(see above), but in this case, the details for the treefor a given sentence are determined by the grammarthat the sender used to generate it. Students not onlywitness the output of other students, but can alsoexamine the details of the promising approaches offellow students. In this way, CHAT promotes a trulycollaborative learning environment.

The Agents button makes the Agents window visibleso that the student can consult CHAT's agents:

Fig. 12. The Agents window.

The window displays a list of agents, each withindicators and controls below its name. Reporteragents prepare a helpful report to the student basedon information gathered by the observer agents. TheStatus indicator light informs the student of theagent's status. If the agent is stopped, the indicatorwill be red; if the agent is running, the indicator willblink green. The Message indicator light is yellow ifthe agent has a new message to report since the lastclick of the Message button. The Details buttoncreates an Agent Details window showing someunder-the-hood details of the agent's activity, in casethe student is curious or for debugging purposes. TheMessage button opens a Message window showingthe message that the agent is reporting to the student.

VIII. ASSESSMENT

Two different earlier versions of CHAT have beenused in the classroom, each with a different purposein mind. The first deployment involved an agentlessprototype of CHAT written in Director. Fifteenstudents in an advanced cognitive science class used

the software in a section of the course devoted tolinguistic theory. Two special evening sessions werereserved for this trial, which aimed at understandinghow students would interact with CHAT (and atdebugging the application). The two sessions werevideotaped, and later coded, partly to evaluate thesoftware and partly to better understand the inquiryprocess.

There were four main conclusions drawn from thesesessions: 1) Student motivation for the task of writinggrammars seemed considerably higher than thatwhich is normally observed in traditional learningsituations; 2) Students prefer to work in small groupsof two or three at each terminal (and solo chattersseemed to make less progress); 3) It is important toprovide a graded series of problems to guide students'inquiry; and 4) There were certain common kinds offeedback that should be provided by the software tostrengthen the inquiry process.

The evidence for the improvement in motivation was,admittedly, anecdotal. Nevertheless, two separateevaluators were positively impressed by the degree ofstudent involvement, the rate of progress through theassigned work, and the high-spirited atmosphere thatpervaded the sessions. Students evidently enjoyed theexercise–a conclusion they bore out in post-testinterviews. We were not surprised by students' desireto work collaboratively. However, whereas the chatroom was explicitly designed to support groupinquiry, we hadn't anticipated the students' tendencyto work together at each terminal while using the"private" generator. Clearly, students gravitatetowards a collaborative learning environment, and itis a positive design characteristic of the software thatit promotes this style of learning.

The last two conclusions from the first sessionspointed to areas in which the software could beimproved. Paradoxically, some very simple-seemingtarget sentences turn out to be tremendouslycomplicated to analyze syntactically. Moreover,beginning students are not easily able to determinehow difficult a particular construction is likely to beto generate with the resources of CHAT. Since it isdemoralizing to run into serious descriptiveroadblocks early in the inquiry cycle, we determinedthat students should be provided with a graded set ofproblems to support their progress in the preliminarystages of chatting. By carefully selecting the corpusthat the student tries to generate, we can support aproperly incremental learning curve.

Finally, we learned quite a bit about how theinstructor can best interact with students while they

are using the software. Obviously, the role of theinstructor is much different in the guise of CHATfacilitator than in that of lecturer. Although there isthe occasional spontaneous formal lesson that isrequired when large numbers of students bog down,most of the teacher's time is spent moving fromterminal to terminal, checking progress and offeringsuggestions. Since we are stressing the inquirymethod, the instructor must suppress the urge toproffer solutions, attempting instead to steer thestudent down a profitable path. We took note of thefact that many students repeatedly required the sameadvice including help identifying hiddenungrammaticality and unextendable assumptionsabout structure. The desire to incorporate some ofthis scaffolding into CHAT lead to the developmentof the current agent-based model to monitor andprovide systematic feedback on student work.

The second trial of CHAT employed a java-basedbeta version incorporating the aforementioned agentarchitecture. Twenty-two students in an introductorylinguistics course used the software for fourconsecutive course sessions as their only coursematerials for the syntax module of the class. Work onCHAT was guided by a series of ordered problemsentences which students were instructed to try toprogram their systems to generate. At the beginningof each session, students could upload a startinggrammar that incorporated the most successful partsof the previous session's work.

Student progress on required work was excellent.Most collaborative groups were able to successfullycomplete a series of open-ended questions thatrequired applying lessons learned in CHAT to newproblems. Since we have been using similar problemsover the years in traditional syntax courses with onlymixed results of achievement, we found the presentlevel of performance encouraging.

Students were formally surveyed after each class togauge their progress as well as their interest in usingCHAT. Table 1 summarizes the results on sevenstudent-response items that were included in the post-class assessments.

Table 1: Student Response ItemsEach row reports the number of students who gave each responseto the item. Total number of responses vary because items wereadministered on different days.

Item AgreeAgree

SomewhatDis-

agree1.The software

allowed me tobegin to activelyexplore howEnglish syntaxworks.

16 6 0

2.I am learningmore aboutsyntax by usingthe software inclass than Iwould listeningto a good lecturein which theteacher alsoresponded tostudentquestions.

8 10 1

3.I could use thesoftware toexplore syntaxoutside of class,guided byappropriateassignments.

15 2 2

4.During thesyntax classes sofar, I have foundit useful to workwith (or consultwith) a partner orneighbor whileusing thesoftware.

11 2 1

5.My learning ofsyntax andenjoyment of thesoftware hasbeen aided by theCHAT feature.

12 2 0

6.Overall, usingthe CHATsoftware to learnsyntax was asuccessfullearningexperience forme.

18 0 2

7.I find syntaxmore interestingthan I thought Iwould.

14 4 1

Items 1 & 2 confirm the positive response of studentsto the active learning environment. In item 2 studentsappear to judge the learning outcomes in the CHATenvironment and in a good lecture environment to bemore similar than the instructor and evaluator believethey are. The difference may reflect students'overconfidence about what they learn from lectures.Item 3 suggests that CHAT should be evaluated as anout-of-class as well as in-class learning environment,although, again, students may be overconfident abouthow much progress they could make without aninstructor present. Items 4 and 5 confirm students'beliefs that it is useful to collaborate, both directlywith a classroom partner and via the CHAT feature.Items 6 and 7 suggest that CHAT is an effective andenjoyable learning environment overall. Item 7 wasincluded because students typically report that syntaxis the least enjoyable section of a linguistics course.

Two items were included in the assessments to checkwhether students' strong performance within theCHAT environment would show immediate transfer.The first item was: Draw a phrase-structure tree forthe following sentence: Healthy people exercisefrequently. This item required students to draw byhand the tree structures that are automaticallygenerated by the software and to remember the basicstructural generalizations they had learned in the firsttwo classes without referring back to the grammarsthey had saved on the computer. Thirteen of nineteenstudents drew correct trees, and the remaining sixstudents drew trees that were largely correct butcontained one or two errors. In the instructor's andevaluator's past experience students have been unableto perform at this level in lecture-based courses. Thesecond item was: Describe briefly something that youlearned about syntax today. Sixteen of 22 studentswrote answers that incorporated concepts that figuredin their work during the class (e.g. I learned the samegroup of words can mean two different things basedon how constituency is divided.) The remainingstudents commented on their learning process (e.g., Ilearned that syntax is hard) or wrote answers that didnot mention specific concepts (e.g. I learned how toadd additional words to sentences).

There was some evidence that working within CHATled students to focus mainly on specific technicalissues raised by the assignments. This possibility wassuggested by responses to two items that invited themto reflect on their learning: (1) At this point what doyou least understand or find most confusing aboutsyntax?; (2) What aspect of syntax or issue about itwould you most like to work on or have discussed in

the final syntax classes? Nearly all answers focusedon narrow issues that had come up in the day's workrather than on larger questions in linguistic theory.Lectures, readings, or more reflective assignmentsare needed to place the work within CHAT in a largercontext.

Students were also given the opportunity to commenton things they liked best and least about the course.Many cited the textbook as a weak link and CHAT asthe best part of the course (the fact that both weredeveloped in part by this paper's senior author,notwithstanding!). Unfortunately, due to timeconstraints, neither the advanced grammaticalfeatures nor the agent architecture of CHAT weretested or evaluated in this test session. Finally,although this second evaluation is still partial (and amore systematic assessment must be forthcoming),we find that there is mounting evidence that CHATcan raise the levels of student motivation andaccomplishment by successfully promoting aninquiry-based collaborative digital learningenvironment.

IX. CONCLUSIONS

The teaching of science to undergraduate studentsfaces many well-known hurdles, among which arethe apparent lack of applicability of scientific theoriesto the student's practical concerns, a tendency towarddifficult formal methods and notations, theconsiderable abstractness of the problem domain, andthe need for high levels of analytic rigor. Along withmany other teaching scientists, we are committed tothe development of inquiry-driven models of scienceinstruction as a primary strategy for overcominglapses in student motivation and for promulgating thecapacity for scientific method that is most likely tosustain a life-long interest in the sciences [7]. In thispaper we argue that educational software can play animportant role in inquiry-based science teaching. Bycreating a digital learning environment that promotescollaborative work, CHAT manages to transform thenormally unengaging process of grammarconstruction into a highly engaged, socially rootedlearning experience. Perhaps more subtly, CHATencourages precision by drawing on the student users'ability to evaluate their work (individually andcollectively) and grounding that ability in a formal,computational system. It also monitors and reportsback on the student's emerging work by employing aseries of observer and reporter agents that provideuseful feedback to the user.

More than just a static simulation, CHAT offersstudents the means to dynamically build a generativesystem that approximates their own native languageability. Ultimately, we are using a computationalmodel to get across to students the conclusion thatlanguage is a computational system. We areconfident that the transparency of this approach holdspromise not only as the basis of an approach toteaching linguistics, but also as a generalized modelfor instruction across many scientific disciplines.

X. ACKNOWLEDGMENT

The development of CHAT was supported by grant8-0-0-33901.from the National Science Foundationfor "Inquiry-Based Learning: Cognitive Measures &Systems Support."

XI. REFERENCES

G.A. Boy, "Software Agents for Cooperative Learning," inSoftware Agents, J. M. Bradshaw, Ed. Cambridge, Mass.: TheAAAI Press/MIT Press, 1997.

J.M. Bradshaw, Software Agents, Cambridge, Mass.: The AAAIPress/MIT Press, 1997.

N. Chomsky, Aspects of the Theory of Syntax, Cambridge, Mass:MIT Press, 1965.

N. Chomsky, The Minimalist Program. Cambridge, MA: The MITPress, 1995.

B.Goodman, A. Soller, F. Linton, and R. Gaimari, "EncouragingStudent Reflection and Articulation using a Learning Companion,"The International Journal of Artificial Intelligence in Education,Vol. 9, pp. 237-255, 1998.

P.Hietala and T. Niemirepo, "The Competence of LearningCompanion Agents," The International Journal of ArtificialIntelligence in Education, Vol. 9, pp. 178-192, 1998

A.P. McNeal and F.S. Weaver, "Interdisciplinary Education atHampshire College: Bringing People Together Around Ideas," inReinventing Ourselves: Interdisciplinary Education, CollaborativeLearning, and Experimentation in Higher Education,L.S. Smith and J. McCann Eds. Boston, Mass.: Anker PublishingCompany, 2001.

D.A. Norman and J.C. Spohrer, "Learner-Centered Education,"Communications of the ACM, Vol. 39, No. 4, pp. 24-27, 1996.

M. Paolucci, D. Suthers, and A. Weiner, "Automated Advice-giving Strategies for Scientific Inquiry," in Intelligent TutoringSystems: Third International Conference ITS'96, C Frasson, GGauthier, and A Lesgold, Eds. Montreal, Canada, June 1996[Lecture Notes in Computer Science, New York: Springer, pp.372--381, 1996].

A. Repenning, A. Ioannidou, and J. Phillips, "Collaborative Useand Design of Interactive Simulations. In Computer Support forCollaborative Learning (CSCL)," C. Hoadley and J. Roschelle,Eds. UNext.com. Press, 1999.

J.A. Self, "Bypassing the Intractable Problem of StudentModeling," in Intelligent Tutoring Systems: At the Crossroads ofArtificial Intelligence and Education. Norwood, NJ: Ablex, pp.107--123, 1990.

Steven Weisler,Ph.D is Dean of theSchool of CognitiveScience andProfessor ofLinguistics atHampshire Collegein Amherst,Massachusetts. He isalso a co-founderand the Director ofthe InnovativeInstruction

Laboratory, an interdisciplinary center for theconception and development of educational softwarethat incorporates student-active, inquiry-basedapproaches to learning. Dr. Weisler's work inlinguistics concerns the syntax-semantics interface,with supporting work in philosophy of language andlinguistics pedagogy.