Doc.LifeAddressing the Growing Need for Dissertation Management

Intellectual Product of TSI

A Caveat and Request to the Reader

The document will here forth contain a number of declarations regarding the current state of the dissertation process and perhaps doctoral programs as a whole. These statements reflect the perspectives of the authors; none of whom have experience with the process. In reading this proposal, should you find any great discrepancies between your perceptions and those described in the document, please voice them. The authors’ underlying assumptions are the roots of any proposals contained in the document and we could only be benefited by your insights.

Purpose The purpose of this paper is primarily as a project proposal and secondly as an introductory design document. The proposal piece is for a new approach to dissertation software with an emphasis on collaboration. The latter piece of this document begins an outline of what this tool’s components may be and what existing technologies could be utilized.

The ProblemIn a recent EdLab seminar, concerns with efficiency of the dissertation process were raised. It was suggested that the process:

Lacks a coherent method of scheduling Provides poor structure for the reviewing stages Offers little assistance with writing and analysis Provides few explicit resources such as other exemplary dissertations

These oversights in process create confusion, lead to the disregard of timelines, and depress performance in a critical work for doctoral students. To this, the authors add that the process fundamentally lacks means for collaborating with one’s peers; and suffers little internal coherency with the surrounding processes of publishing, attending conferences and preparing for academic life in whole.

Conceptual SolutionsTo address these discrepancies TSI will design an online system for doctoral students at the university. The system should:

Provide means for scheduling deadlines for oneself and her reviewers Structure the review process so that changes/critiques are immediately

available and prominent Provide resources for developing an idea, writing to standards, and

performing analysis Facilitate collaboration and the forming of review groups Assist in the processes of picking a topic and establishing an agenda

Preliminary Designs: Starting with Ez.D.The initial plans to develop a system aiding the dissertation process were titled “Ez.D.” and a general outline was provided (see Appendix A). The purposes of the Ez.D. documents were (1) to develop a conceptual framework which captures the generic processes of dissertation writing; and (2) to highlight those processes which may be facilitated by computer technology. The system follows six generic stages, and no specific technologies were highlighted in this design.

Widening Scope: Axioms for Dissertation SoftwareIn reviewing the technological requirements of Ez.D., we found the need to more clearly establish an agenda (the “Conceptual Solutions” section) and coherently describe our core principles:

1. The degree of format differentiation both between and within programs requires either a complex decision-tree system to target the applicability of stage x for user y, or an asynchronous system which allows users to move freely between stages. We believe the latter approach will be programmatically more efficient and will afford the user additional benefits. Following trends in language software development, we hold that users rarely work linearly and enjoy having some control over their interactive experience. An asynchronous approach will address both these needs.

2. Providing relevant information requires managing the changing knowledge of the field quickly and efficiently. This task is best served by a knowledge community with tools to collaborate and produce shared content.

3. The relevant field expertise includes much specialized knowledge and widely different information-organizing structures. Additionally, insights into the process of dissertation writing embody a great deal of tacit knowledge. These needs are best met by a system which allows individuals to share knowledge in an unstructured way.

4. Although the dissertation is a single product, its development does not begin until after many years of involvement within the university; this time is spent preparing for and thinking about the dissertation. We believe that even a great dissertation tool will not be used if it is only presented once an individual begins the writing process. It should instead be a resource available and relevant to the student upon their first steps into the university.

Doc.Life OverviewThe shift from Ez.D. to Doc.Life embodies a larger shift in perspective from a technical writing lens to a growth/needs perspective. Doc.Life aims to encourage writers by providing an online environment: which provides a sense of direction; and promotes learning through collaborative feedback and open discussion.

The design goals of Doc.Life are to: Create internal congruence of content by using a node model with Access

Anywhere Components Promote collaboration: encourage group formation and feedback Engender a sense of purpose via asynchronous systems so that users do

not ever feel lost or like they have reached an ending. The dissertation is a living document and Doc.Life will facilitate its development during the six generic stages, but also before, after and between these stages.

Prevent common frustrations by overseeing the review process and structuring review timelines

As a development piece, Doc.Life supports collaboration by utilizing Open Source technology. Where possible, code will be used/recycled from existing Open Source projects to minimize EdLab investment. Additionally, Doc.Life will be released to the Open Source market when fully tested.

A Nodal SystemDoc.Life is designed such that every element in the system is considered a “node.” Node types include uploaded files, suggested readings, blog/wiki posts, events/deadlines, profiles, groups, internal emails/messages. The purpose behind this structure will be to allow for integration of any item type in the site. All nodes will be searchable under a single search and each will be analyzed using latent semantic analysis. This analysis will provide a measure of relatedness-strength which will be used to provide “related content” to each item in the site. For example, when viewing a dissertation on “The Effect of Team Driven Classrooms in Labor Market Success in Twelfth Grade Urban Students,” the system may provide links to the author’s profile, the “It’s All Just Hidden Curriculum Anyway” group, and a blog entry concerning the modern implications of “Learning to Labor.”

Nodes will also serve to divide the site conceptually by program. Each node will be given a program ID and duplicated across programs. Thus when a user clicks

to view “suggested readings,” she will be directed to the readings node associated with her program. This will prevent unnecessary cognitive load on the user, though a drop down menu will provide access to the equivalent nodes of other programs.

Access Anywhere Components (AAC)The nodal system of Doc.Life will introduce the concept of Access Anywhere Components (AAC), which provide a common interface for interacting with node objects anywhere in the site1. Currently, AAC components include tagging, commenting, making suggestions to the administrators, and personal notes.

Public tagging and commenting allow for community interactions anywhere in the site, facilitating a collaborative environment. The “suggestion component” will allow the community to make suggestions regarding content and will only be available for certain nodes. For example, sections of the site will provide a list of “seminal readings” and will contain a link to “suggest additional readings” which updates an administrative queue. To maintain the integrity of these links, only administrators will be able to update the list.

The “personal note” feature is designed as a tagging/commenting/bookmarking feature. This component allows users to associate a node with some text for personal use and is unseen by anyone else. The user profile section of the site will allow users to view their personal notes and provide links to the corresponding nodes. This will allow users to bookmark content for private use and retrieve it later.

Users, Groups and CollaborationWhen users sign up for Doc.Life they supply standard information, their current program, and academic interests. They are also provided with a personal “Biki”. The Biki is a cross between Blog and Wiki, a blog where each entry may optionally be set as a Wiki for anyone else to alter. It will also support video blogging, where users upload audio or video as a blog post, by integrating YouTube2 functionality. These features allow one to create a Doc.Life identity which others will recognize when looking for other collaborators.

1 The technical design of AAC is still being debated. They may include a robust design of “technical components” or they may simply describe a common user interface, not the backend functionality. The latter benefits timelines.2 http://www.youtube.com

Groups are an often overlooked aspect of the writing process. In our interviews we found disparate ideas of how dissertation groups collaborate and varying levels of success. To provide users with a comfortable means of collaborating, each group consists of a user list, photo, description and a Biki. The Biki is unstructured and designed to allow groups to work in the format with which they are most comfortable, blogs, wikis, video all in one. Additionally groups may be public or private. An initial sketch can be seen in Appendix C.

Collaboration will occur as individuals interact with the one or more groups to which they belong, as well as other individuals in the system. Doc.Life supports collaboration with modules for calendars, internal messaging, and public reviews. The calendar system (shown in Appendix D) stores events entered by the user, by their groups and by the community at large to support shared deadlines and community notifications regarding events such as academic conferences. In addition to sharing events, members are able to send messages to one another using a simple internal messaging (email) system. We are also reviewing the capability of an internal chat system for live discourse.

The Dissertation PhasesThe front page of Doc.Life (for an incomplete sketch see Appendix B) will present the six asynchronous stages of the dissertation process with a defense preparation tool and an independent “review cycle” module. Each of these units corresponds to a mini-site integrated with Doc.Life complete with links, search features, maps, and other relevant features.

Additionally, there are certain features found at all stages to be mentioned here: Each program/stage pair (Sociology and Education: Methods) will be

equipped with a Biki linked from the mini-site for community support and discourse. Each stage will be primarily driven by this forum in conjunction with the recommended resources provided at each section (see below).

Each program/stage links to a list of “exemplary papers” from Doc.Life. These papers are found by statistically weighting the responses from the review cycle stage a la Mechanical Turk. (See the “Review Cycle” section below for details.)

Each will link to versions and reviews of the user’s own dissertation for said section.

A small number of short interviews or talks from faculty regarding the criteria for the stage and helpful suggestions.

Picking a Topic/Defining the ProblemThis component is the largest of the eight modules. The conceptual center of this module is a collection of seminal papers uploaded and tagged by administrators. The papers will be tagged with relevant program(s), field(s), keywords, and author(s). They will be presented as a list, be searchable and also viewable within an interactive module which displays the content as a map of field keyword papers (see Appendix E). We have established a good means of creating these graphs dynamically using AJAX and JSViz3.

A second map will describe the relationships between authors using the seminal papers and the WebOfScience4 citation finder. A map will be provided showing the authors of seminal work as central nodes, with adjacent nodes displaying prominent authors who have cited these papers.

A final map will display appropriate methodologies. Since methodologies are stable, we believe the EdLab could produce a list of various methodologies (headed under qualitative or quantitative) with a selection of representative papers tagged by methodology. In the map, the central nodes would be “Qualitative” and “Quantitative” with surrounding nodes of specific methodologies, each surrounded by example journal articles. In addition to browsing the map, each paper would be indexed by Doc.Life such that a user could search by paper content to reveal the associated methodologies. For instance, one could search “Video Game Learning” and find “Virtual Ethnography” but not “Factorial Design.”

Review of the ResearchIn addition to the features found in all stages mentioned above, this section will contain links to faculty suggested search engines. We’ve considered creating technology to provide resource recommendations based off the user’s provided bibliography; however, we feel the technology would be inferior to a human and instead suggest a series of videos discussing proper search methods. Professor/expert opinions are preferred, but a few videos should also center around selected students and their research methods.

3 http://www.kylescholz.com/blog/projects/jsviz/ 4 http://portal.isiknowledge.com/portal.cgi?SID=F2l4MeGj12jhHkdhEEE

MethodsIn addition to the features found in all stages mentioned above, this section will contain links to recommended papers and online courses. This section will also link back to the “methodologies map/search” described in the “Picking a Topic” section above.

DataIn addition to the features found in all stages mentioned above, this section will contain links to recommended papers, datasets, and analysis tools5. Another JSViz map will display the connections between modes of analysis, example datasets and the appropriate tools. For example, the map may contain a node “qualitative” linking to the tool “NVIVO6” as well as several qualitative papers which used the NVIVO software.

ResultsIn addition to the features found in all stages mentioned above, this section will contain links to recommended papers and, if possible, a variety of templates for data reporting to assist young writers in conforming to reporting norms. A video or two should also interview professors concerned with “best practices” in display modes. For example, Aaron Pallas has an interest in these best practices such as: not using pie charts because they have a poor space to content ratio7.

ImplicationsIn addition to the features found in all stages mentioned above, this section will contain a list of resources for writers to engage the field. They could reference grass roots movements, current legal cases, etc.

DefenseThe defense preparation stage should contain a series of videos describing the process and if possible a number of videotaped defenses. Supplemented by tips and the ability to upload self-authored videos of one’s

5 Just a reminder: the datasets are linked as nodes and will be accessible to the “Suggestion Component” outlined earlier in the AAC section, allowing users to recommend additional datasets should they know of any.6 http://www.qsrinternational.com/ 7 For clarification please see me. Unfortunately I am only vaguely familiar with the concepts and do not know the name of the field concerned with these issues but have heard Professor Pallas discuss it.

presentation for review, this section aims to make the user feel confident and well prepared with the presentation.

Review CycleAsynchronous DesignIn keeping with the asynchronous design, the review cycle will be a separate module of the system, accessible at any time. When entering a paper version for review, the system will ask the user to identify the type of review in correspondence with six stages of the dissertation: problem definitional; literature review; methods; data; results; implications; and other. A selection of a stage will cause the version to be displayed under that stage’s page. For example, uploading a “methods” version will cause said version to be displayed (and downloadable) from the Methods page. During this process, users will also be able to assign reviewers, make the document public or private, and provide tags (which default to the tags provided on the prior version).

Technologies in PlayAfter a review of current technologies, we have decided that online paper writing with tools such as Google Documents and ThinkFree8 are not suitable and a file upload system will be used instead. Additionally, the review cycle will be greatly aided by allowing users to immediately see edited/altered content, not just the whole file. Under this scheme, all posted reviews would include a link to “See Revisions” which would display only altered content. We have determined a means of doing this using software to extract text from Word files in conjunction with Open Source versioning software9.

Automated ReviewsAfter a review of automated essay analysis software, we have determined that little of it works well; and that which does, does not do so in a manner applicable to web development. However, we are still investing the use of SAGrader10, and believe that it may be possible to utilize this software to provide limited automated analysis. The current obstacles include finding a command line interface to the software and finding a sound method of developing criteria. Should these problems be solvable, it would be feasible

8 http://docs.google.com and http://thinkfree.com 9 For a nice example, look at the “History” tab in WikiPedia10 http://www.ideaworks.com/sagrader/index.html

to create a system where users selected topics related to the piece and the system returned feedback regarding the writing style and content.

The FrameworkIn keeping with the established agenda of utilizing free software where possible, the review section of Doc.Life will be driven by CuteFlow11. CuteFlow is Open Source, PHP driven and allows users to create a circulation flow, assign readers, and see the current status of review. We aim to integrate this software with Doc.Life by adding the functionality listed above, altering the interface, and adding review deadlines to the calendar system mentioned earlier. We will also add functionality for the reviewer to anonymously assess the quality of the paper as: poor; average; good; or exemplary. This functionality will drive the suggested exemplary dissertations as discussed earlier.

The Next StepsIn finishing this document, three general phases remain before development should begin. First, we eagerly await others speaking to the document. Thus far the project has been the work of a narrow group of individuals and we look forward to the feedback and changes that will come. As discourse continues around this document, we aim to take the feedback of all persons involved and use this to finalize a conceptual design of Doc.Life. One important, and yet unaddressed, aspect of this system is the means to collect the information for suggested resources. This may not be a job well suited to TSI.

During and after this process, we will continue to review relevant technologies and determine ways to make the conceptual real. Once this is done, we will be ready to begin systems and interface development.

11 http://www.cuteflow.org/

Appendix A: Ez.D.

E-Z-D (Easy D) – The Future of Dissertations at Teachers College

The E-Z-D system will be designed to guide TC doctoral students through each phase of the dissertation development process. Students will register as users of the system and make use of it as they plan, execute, and deliver their project.

Our initial ideas for the system include the following:

Phase 1/Chapter 1 – Defining the Problem

1. Video clips of faculty in the program outlining expectations for dissertations in the areas. (0)

2. Video clips of leaders in the field addressing key research issues. (0)

3. Maps of the Literature in the Field showing the key researchers organized around major research themes. (Perhaps drawing on federated search that generates buckets of themes/scholars. (...)

4. Links to Dissertations Completed by Prior Students in the Program with Exemplary Problem Definition Sections (as identified by faculty) (1)

5. Text Review Stage – Capacity to submit a draft problem definition section/chapter to a text analysis tool the provides feedback on writing structure/style/organization (REVIEW CYCLE)

6. Submit to Peer Review – Capacity to send a draft to student peers in the program for their comments (REVIEW CYCLE)

7. Submit to Faculty Review – Capacity to send a draft to faculty in the program for their comments (REVIEW CYCLE)

Phase 2/Chapter 2 – Review of the Research

1. Supersearch (customized to each program) to Identify Relevant Literature (1) 2. Faculty-endorsed sources search (1)

3. Links to Dissertations Completed by Prior Students in the Program with Exemplary Literature Review Sections (as identified by faculty) (1)

4. Tags to comment on prior dissertations (0)

5. Text Review Stage – Capacity to submit a draft problem definition section/chapter to a text analysis tool the provides feedback on writing structure/style/organization (REVIEW CYCLE)

6. Submit to Peer Review – Capacity to send a draft to student peers in the program for their comments (REVIEW CYCLE)

7. Submit to Faculty Review – Capacity to send a draft to faculty in the program for their comments (REVIEW CYCLE)

Phase 3/Chapter 3 – Methods

1. Links to Methods Tutorials/Course Modules (1) 2. Links to Psych Methods Exam Files (1)

3. Commissioned Papers on Major Methods (0)

4. Links to Digital Chapters and Books on Major Methods (1)

5. Links to Dissertation Completed by Prior Students in the Program with Exemplary Methods Sections for Particular Methods (as identified by faculty) (1)

6. Text Review Stage – Capacity to submit a draft problem definition section/chapter to a text analysis tool the provides feedback on writing structure/style/organization (REVIEW CYCLE)

7. Submit to Peer Review – Capacity to send a draft to student peers in the program for their comments (REVIEW CYCLE)

8. Submit to Faculty Review – Capacity to send a draft to faculty in the program for their comments (REVIEW CYCLE)

Phase 4/Chapter 4 – Data

1. Links to library managed datasets relevant to field of study (national, local, TC specific, archives) (1)

2. Commissioned Papers on Major Data Sets (1)

3. Links to Resources on Data Analysis Tools (1)

4. Links to TC Lectures on Data Sets (1)

5. Links to Dissertation Completed by Prior Students in the Program making good use of a particular data set.(1)

6. Text Review Stage – Capacity to submit a draft problem definition section/chapter to a text analysis tool the provides feedback on writing structure/style/organization (REVIEW CYCLE)

7. Submit to Peer Review – Capacity to send a draft to student peers in the program for their comments (REVIEW CYCLE)

8. Submit to Faculty Review – Capacity to send a draft to faculty in the program for their comments (REVIEW CYCLE)

Phase 5/Chapter 5 – Results

1. Links to articles/books with examples of data presentation formats (e.g., tables, figures, etc.) (1)

2. Links to Dissertations Completed by Prior Students in the Program demonstrating good results presentation practices (1)

3. Links to templates (e.g., table shells) fostering sound reporting practices (1)

4. Links to relevant reporting resources (1)

5. Links to visual communication resources (1)

6. Text Review Stage – Capacity to submit a draft problem definition section/chapter to a text analysis tool the provides feedback on writing structure/style/organization (REVIEW CYCLE)

7. Submit to Peer Review – Capacity to send a draft to student peers in the program for their comments (REVIEW CYCLE)

8. Submit to Faculty Review – Capacity to send a draft to faculty in the program for their comments (REVIEW CYCLE)

Phase 6/Chapter 6 – Implications

1. Connections to Alums/Practitioners to Consider Implications (0) 2. Links to Dissertations Completed by Prior Students in the Program demonstrating sound

implications discussion.(1)

3. Text Review Stage – Capacity to submit a draft problem definition section/chapter to a text analysis tool the provides feedback on writing structure/style/organization (REVIEW CYCLE)

4. Submit to Peer Review – Capacity to send a draft to student peers in the program for their comments (REVIEW CYCLE)

5. Submit to Faculty Review – Capacity to send a draft to faculty in the program for their comments (REVIEW CYCLE)

Defense Practice

1) Defense basics – review (0/1)

2) Online recruitment/assignment of defense members (0)

3) Desktop Video of Defense Presentation to share with peers for commentary (REVIEW CYCLE)

4) Submit to Peer Review – Capacity to send a video clip of defense practice to student peers in the program (REVIEW CYCLE)

Tags for the various elements: 0 - little done, basically need to start from scratch 1 - existing, needs major modifications 2 - close to final form, minor modifications 3 - complete X - major hurdle V - pushed to later version ? - needs further clarification

Major issues:

Mapping the literature Supersearch for relevant literature Text analysis tool for review cycle (V?) Entire review cycle process

o Managing users o Managing documents/versioningo Managing workflow

Handling of video & audio for defense practice, and possibly for social commenting on other clips

Social tagging of library resources relevant for dissertation Skeleton social network for TC to determine relationships between faculty and students,

and between students intra- and extra- departmental

Implementation issues:

How will we get all of the users into the system? Can we import some of this from existing CIS systems or is it all going in from scratch?

What's the incentive for peers to review each others work... will there be some sort of recognition system built in?

Consider making the audio/video & commenting a module adapted from Anthony's idea for collaborative class notes

Connecting to alums/practitioners out in the field may take time to roll-in from existing system users (again, what's the benefit for those people)

Things to take a look at: Elgg NDLTD

Appendix B: Rough Homepage Sketch

Appendix C: Sample Group Page

Appendix E: Force Directed Graph of Fields and Content