1 Labrador: A Tool for Automated Grading Support in Multi-Section Courses Drexel University Programming Learning Experience (DUPLEX) http://duplex.mcs.drexel.edu Christopher D. Cera, Robert N. Lass, Bruce Char, Jeffrey L. Popyack, Nira Herrmann, Paul Zoski Drexel University Mathematics and Computer Science 2 Roadmap l Introduction l Problems and Solution Goals l Labrador l Discussion 3 The Group l Bruce Char – Professor, Computer Science l Nira Herrmann – Professor and Head, Mathematics l Jeffrey L. Popyack – Associate Professor, Computer Science l Paul Zoski – Instructor, Math and Computer Science l Christopher D. Cera – Computer Science Graduate Student l Robert N. Lass – Computer Science Undergraduate Student 4 Who Am I l First TA in MCS to experiment with WebCT in December 2000 l Courses I have TA’ed using WebCT – 3 x Introductory Programming I, II – 1 x Object Oriented Programming l Migrated course content to WebCT from previous course website – HTML assignments and labs into question database – Gradebook maintainer l Wrote demo’s and documentation to train other TAs l Developer of software supplements to WebCT to support course administration 5 The Course l Intro Programming I, II l Variety of Majors – Computer Science – Computer Engineering – Information Systems – Mathematics – Digital Media l Class – 1 x 1 hour lecture – 2 x 1 hour labs l People – 250-300 Students – 2-3 Professors – 10-12 Teaching Assistants 6 The Duplex Project: An Overview l Take advantage of advances in Information Technology to improve instruction and reduce costs for computer programming courses – Modular Structure l Multiple Entry Points l Multiple Audiences l Multiple levels of knowledge (Bloom’s Taxonomy) – Computer Supported Cooperative Work (CSCW) in student labs – Online Services – Today’s Topic
Transcript
1
Labrador: A Tool for Automated Grading Support in Multi-Section
Courses
Drexel University ProgrammingLearning Experience (DUPLEX)http://duplex.mcs.drexel.edu
Christopher D. Cera, Robert N. Lass, Bruce Char, Jeffrey L. Popyack, Nira Herrmann, Paul Zoski
Drexel UniversityMathematics and Computer Science
2
Roadmap
l Introduction
l Problems and Solution Goals
l Labrador
l Discussion
3
The Group
l Bruce Char– Professor, Computer Science
l Nira Herrmann– Professor and Head, Mathematics
l Jeffrey L. Popyack– Associate Professor, Computer Science
l Paul Zoski– Instructor, Math and Computer Science
l Christopher D. Cera– Computer Science Graduate Student
l Robert N. Lass– Computer Science Undergraduate Student
4
Who Am I
l First TA in MCS to experiment with WebCT in December 2000
l Courses I have TA’ed using WebCT– 3 x Introductory Programming I, II– 1 x Object Oriented Programming
l Migrated course content to WebCT from previous course website– HTML assignments and labs into question database– Gradebook maintainer
l Wrote demo’s and documentation to train other TAs
l Developer of software supplements to WebCT to support course administration
5
The Course
l Intro Programming I, II
l Variety of Majors– Computer Science– Computer
Engineering– Information Systems– Mathematics– Digital Media
l Class– 1 x 1 hour lecture– 2 x 1 hour labs
l People– 250-300 Students– 2-3 Professors– 10-12 Teaching
Assistants
6
The Duplex Project: An Overview
l Take advantage of advances in Information Technology to improve instruction and reduce costs for computer programming courses
l Adobe Portable Document Format can be viewed on all major platforms
l With Adobe Acrobat, PDFs can be annotated by graders
l Sony VAIO Slimtop PC (PCV-LX920)
l Wacom Pen Tablet.
27
VAIO / Wacom
28
PDF Markup Example
29
Redistribution
l How can we return annotated PDF’s back to students using WebCT?
l WebCT Mail
l WebCT Groups
l Version 3.8 now addresses this issue
30
Heterogeneous Systems
l Different systems want different formats
l Plagiarism Detection Systems– Moss has been used most extensively– Began experimenting with JPlag more recently
l Future work: Automatic program compiling and testing
31
Automated Plagiarism Detection
l Digital formats make "borrowing" easy
– Browsing similar works needs a simple and quick user interface.
– Careful review by faculty to assess results and present to students
6
32
Moss
l Processes C, C++, Java, ML, Lisp, Scheme, Pascal, and Ada programs.
l Common code feature reduces false positives
33
Moss Interface
34
JPlag
l Processes C, C++, Scheme, and Java programs
l For plain text files, it matches a user specified number of words appearing in succession
l Could be used for any course grading written (text) documents
35
Command Line Invocation
36
Interactive
37
Quiz Download with Config File
7
38
Roadmap
l Introduction
l Problems and Solution Goals
l Labrador
l Discussion
39
Recent WebCT Enhancements
l Relevant to this talk:
– 3.7l Addressed bulk download issue for assignments
– 3.8l Attaching documents to an assignment
40
Future WebCT Enhancements
l Power users will need functionality not yet supported
l Every domain will also require additional functionality
l Not feasible for all domain-specific functionality to run on the WebCT server
41
Wanted
l API for non-administrators
l Stateful protocol so clients can be built by 3rd parties
42
HTTP: Insufficient Protocol
l Inconvenient content interchangel Heavy client interaction
l A HTTP based approach will be sensitive to the exact location of web pages, and format of text within them.
43
Labrador Availability
l Contact Us
l http://duplex.mcs.drexel.edu
8
44
Project Support
l The Pew Learning and Technology Program at the Center for Academic Transformation
l National Science Foundation, Division of Undergraduate Education, DUE- #0089009
l The Ramsey- McCluskey Family Foundation, Margaret Ramsey, '84
l Drexel University
45
References
l [1] Alex Aiken. Moss: A system for detecting software plagiarism (unpublished), http://www.cs.berkeley.edu/ ~aiken/moss.html.
l [2] Jeffrey L Popyack, Bruce Char, Paul Zoski, Nira Herrmann, and Christopher D. Cera. Managing course management systems . In Proceedings of the thirty -third SIGCSE technical symposium on Computer Science Education, Birds-of-a-Feather Sessions, page 423, 2002.
l [3] L. Prechelt , G. Malpohl, and M. Philippsen. Jplag: Finding plagiarisms among a set of programs . Technical Report 2000-1, Fakultat fur Informatik , Universitat Karlsruhe , Germany, March 2000.
l [4] Larry Wall, Tom Christiansen, and Jon Orwant. Programming with Perl . O’Reilly and Associates, 3rd edition, 2000.
l [5] Michael J. Wise. Yap3: Improved detection of similarities in computer program andother texts. In Proceedings of the twenty -seventh SIGCSE technical symposium on Computer Science Education , pages 130–134. ACM Press, 1996.
46
Roadmap
l Introduction
l Problems and Solution Goals
l Labrador
l Discussion
l Bonus Slides
47
Blooms Taxonomy
l Knowledge– remembering of previously learned material; recall (facts or who le theories); bringing to mind.
l Comprehension– grasping the meaning of material; interpreting (explaining or summarizing); predicting outcome and
effects (estimating future trends).l Application
– ability to use learned material in a new situation; apply rules,laws, methods, theories.
l Analysis– breaking down into parts; understanding organization, clarifying , concluding.
l Synthesis– ability to put parts together to form a new whole; unique communication; set of abstract relations.
l Evaluation– Ability to judge value for purpose; base on criteria; support judgment with reason. (No guessing).
