Date post: | 11-Jan-2016 |
Category: |
Documents |
Upload: | leona-robinson |
View: | 214 times |
Download: | 0 times |
Assessment of Web-based Education Tools: Are your Students Learning Through Technology?
Andrea IrbyUndergraduate Academic Programs
North Carolina State University
Background
NC State is a large, public, research
one institution. 19,000 undergraduates; 9,000 grads. Under pressure in late 1990s to grow Possible expansion of Distance Education Virtual Advising Center created 1999
Why Technology? Enables us to transcend many barriers
Physical campusRaceCountry, nationality“Saving Face”
Student development theory – meet them where they are
Enables us to be more effective and efficient especially in times constrained resources
Why Technology?
Types of Information Exchange
Synchronous vs. AsynchronousData - ways of expressing things Information - the arrangement of data into meaningful patternsKnowledge - the application and productive use of informationInformation versus Knowledge
Technology in EducationCan transform key relationshipsPersonalization: Targeting the student vs. the departmentEngaging the student: making applications interactiveStudent Decision Support: Information becomes knowledgeCreate community: cultivate student relationships
Carl Jacobson, 2000
Engaging Students On-Line
Web-sites On-line Course Management SystemsEmail/ListservsOn line Chat or Instant Messenger type toolsBulletin Boards and BlogsSocial Networking sites
“Students will gravitate toward the technology medium that best meets their needs” Karen Thurmond- Univ of Memphis
Focus on Web-sites2nd Generation Web-sites: Information based3rd Generation are interactive, knowledge based
How do we know if/ how many students come to our website?Are they able to find the information they need?Is it clear and meaningful to them?Do the receive our intended information?
How to Know
Log Data Analysis (data mining)
Usability Studies
Accessibility Testing
Log Data Analysis
Data (log) is created each time a person visits a web-page.
Analytic Tools then either pull data from the server that hosts the web-site or read it from html code on each web-page.
Sample tools Google Analytics (reads code)- free
www.google.com/analytics
SAS Web Analytics (pulls from server)www.sas.com/solutions/webanalytics
TopShareWare (pulls from server)www.topshareware.com/LogSuite-download-38225.htm
Easy to Read Reports (in real life )
Information ReceivedVisitors/UsersRepeat Visitors/Cached VisitorsHow long on siteTraffic by time of day and monthWhere enter, exit, or drop out of siteBrowsers, operating systemsWhere they are logging in fromCampaign/Action Tracking
Virtual Advising Center data mining
2000:
72,419 annual visitors 26,795 repeat visitors176 avg. visitors per day
1,021,947 hits to site
2006:
285,053 annual visitors 108,320 repeat visitors780 avg. visitors per day
8,269,215 hits annually
Virtual Advising Center data mining
Monday-Thursday busiest days. Visitors on site for avg. of 2 minutes now about 5.Path in and out; pages turned.IP locations –whose accessing site on campus, off campus, around the world.Heaviest traffic times per day.
How we used this information
Sent surveys out on busiest daysUsed busiest times of day to determine staff coverage for chatUsed IP address/location information to help IT staff on campus place kiosks in high demand locationsTo enhance relationships with local feeder community colleges
Usability Testing
Observed “Focus” Groups for web-sitesReal data versus self reported dataExamines how people use some human-made object Controlled experimentSmall, cross section of users
Trained users or “Hallway” users
Usability Testing
Pre-test questionnaires get input on subject’s familiarity with web-siteActual test involves a series of short, simple tasks the subject is asked to complete Observations are recorded by “tester”Patterns of how subject completes each task are notedEmotional responses key to watch
Set up
Location can be:In a computer lab with multiple teams
In a usability lab
At your own computer desk
Usability Testing
Post tests – record overall impressions of web-site and experience; reflectionVery Informative Web-site on Usability Testing: www.usability.govIf testing uncovers difficulties, redesignTest in small groups and oftenFor virtual advising center, its how we keep students at the forefront of our design
Accessibility
Inclusive web-communities allow people with diverse abilities to be engaged in on-line learningWeb-Content Accessibility Guidelines
http://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505/#def-checkpoint
3 Priorities for Website Accessibility
Accessibility
Series of CheckpointsPriority 1- Must satisfy these checkpoints or one or more groups will find it impossible to get and use the informationPriority 2- Should satisfy these design elementsPriority 3- May satisfy design elementsResults in A, AA, AAA website ratings
Accessibility
Test your site with on-line toolsTest your site with screen readers/JAWsOn-line web resources to help you:
http://atrc.utoronto.ca/ http://ncsu.edu/it/access/development/software/tools.phphttp://www.accessible.org/bobby-approved.html
Evaluating Learning
Web-site must be interactivetests, essays, portfolios, chats, email
Determine what we want students to learn Get student input Variety of ways: focus groups, check lists, rubrics and content analysis Have more than one staff member working on assessment
What we’ve tried and learned
Focus Groups- what we may see as learning, they see as “help”. Over time, they may recognize what they have learned: raises question of timing of assessmentElectronic communication trails are rich with evidence of learning. Students are readmitting, graduating, learning about themselves, picking majors, staying, expanding their academic portfolios.
Evaluating Learning
Need to develop a rubric for evaluating rich content.
George Mason Business School example
Databases/Microsoft Access tool
Technology is a tool – through which students can learn and for helping us with assessment.
Discussion? Questions?