Snapshot Integration Basics

Kevin CorcoranConnecticut Distance Learning Consortium

http://www.ctdlc.org

Obligatory Slide: Who Am I & Why Should You Listen To Me

CTDLC: Connecticut Distance Learning Consortium

• CT State Agency providing eLearning Services to K-20

for past 12 years

• Hosting, Remote Management, Consulting, Integration,

Block Development, Instructional Design, Help Desk

• Experience supporting Angel, Blackboard, D2L,

Moodle, and Sakai as well as integrating Banner,

Datatel, and Jenzabar to Bb

• Developed the collaborative online tutoring program

called eTutoring.org that serves over 100 institutions

nationally

Administrative Challenges

• Managing Course Creation

• Managing User Creation

• Managing Enrollment/Add/Drop

• Managing Cross-listed Courses

• Managing Branding/Content Copying

• Managing Archive/Removal

The Value of Snapshot

• Automated Processes

– Users– Courses– Enrollments / Assignements

• Deliver Customized Content Based On Bb Role

• Brand Bb To Match Your Campus Identity

Snapshot Concept

Data Extracts

•Users•Courses•Enrollments

SIS

Data Transfer

•Timing•Frequency

S/FTP

Data Processing

•Controller•Properties•Frequency

Blackboard

Integration Methods

• Flat File

– Multiple file method– Each file serves a function– Greater flexibility

• IMS Enterprise XML

– Single XML file– Combined user, course, enrollment management– Based on IMS standards

• Real Time Data Exchange

– Event based data exchange

Sample SIS Query

select REPLACE(course.coursecode, ' ', '') + courseoffering.session + CAST(courseoffering.sessionno AS nvarchar) +

CAST(courseoffering.termid AS nvarchar) AS EXTERNAL_COURSE_KEY, REPLACE(course.coursecode, ' ', '')

+ courseoffering.session + CAST(courseoffering.sessionno AS nvarchar) + CAST(courseoffering.termid AS

nvarchar) AS COURSE_ID, course.coursedescription AS COURSE_NAME, 'N' AS AVAILABLE_IND

from course, courseoffering, term

where courseoffering.session = 'O‘

AND courseoffering.statusid = 1

AND term.termid = courseoffering.termid

AND course.courseno = courseoffering.courseno

/*Pull courses 60 days prior to term begin date and stop 30 days after term start date*/

AND getDate() BETWEEN DATEADD(month, -2, term.termbegindate) AND DATEADD(month, 1,

term.termbegindate)

order by course.coursecode, courseoffering.sessionno

Sample Flat File Feeds

User Feed

EXTERNAL_PERSON_KEY|USER_ID|FIRSTNAME|LASTNAME|EMAIL|INSTITUTION_ROLE|PASSWD

kcorcoran|kcorcoran|Kevin|Corcoran|kcorcoran@ctdlc.org|student|password

wburnes|wburnes|Bill|Burnes|wburnes@ctdlc.org|student|password

Course Feed

COURSE_ID|EXTERNAL_COURSE_KEY|COURSE_NAME

AE000102|AE000102|Algebra I Fall 2006 Term 1 - Session 1

AE000103|AE000103|Algebra I Fall 2006 Term 2 - Session 2

Enrollment Feed

EXTERNAL_COURSE_KEY|EXTERNAL_PERSON_KEY|ROLE

AE000102|kcorcoran|student

AE000102|wburnes|student

Sample IMS Enterprise XML File

File Header

<?xml version="1.0" encoding="UTF-8"?>

<enterprise>

<properties>

<datasource>Institution SIS</datasource>

<datetime>2010-6-15T11:55:0</datetime></properties>

XML Person Record<person>

<sourcedid><source>Institution SIS</source><id>AStudent</id>

</sourcedid>

<userid>AStudent</userid>

<name><fn>Adam Student</fn><n>

<family>Student</family><given>Adam</given><suffix></suffix>

</n></name>

<email>astudent@institution.edu</email>

<extension> <X_BB_INSTITUTIONROLE>Student</X_BB_INSTITUTIONROLE></extension>

</person>

XML Group Record

<group>

<sourcedid><source>Institution SIS</source><id>1003118300230412</id>

</sourcedid>

<description><short>1003118300230412</short> <long>Introduction to Underwater Basket Weaving</long>

</description>

<timeframe><begin restrict = "0">2010-05-10</begin><end restrict = "0">2010-07-09</end>

</timeframe>

<extension><X_BB_GROUP_TYPE>0</X_BB_GROUP_TYPE><X_BB_DURATION>1</X_BB_DURATION>

</extension>

</group>

XML Membership Record

<membership>

<sourcedid><source>Institution SIS</source><id>1003118836909230</id>

</sourcedid>

<member>

<sourcedid><source>Institution SIS</source><id>AStudent</id></sourcedid>

<idtype>1</idtype>

<role roletype="01"><status>1</status>

</role>

</member>

</membership>

Data Source Management

• Data Source Keys

– Allows for multiple data collections

– Associates Snapshot processes to a data collection

– Differentiates automated processes from GUI/System

• DSM Tool

– Command line tool for creating keys

– Provides some global data collection functions

– Uses machine name and domain name

DSM –f CREATE –b Continuing_Education –V server.institution.edu

*Building Block available for fee from Bb

Snapshot Properties File

Properties File

# delimteter used for parsing snapshot filesdata.delimeter=|

# Used to denote datasource to use for operation.data.source.key=institution

# Determines character used to escape delimeterescape.character=^

# Snapshot controlled field settings for Blackboard override of data

# fields for course entitiesgroup.bb.controlled.fields=TEMPLATE_COURSE_KEY,TEMPLATE_ORGANIZATION_KEY,X_BB_TEMPLATEKEY,AVAILABLE_IND

# fields for student entitiesperson.bb.controlled.fields=PASSWD,EMAIL

# Designates flag to enable membership data without "private" data copying for configuration property filemembership.copy=N

# Designates flag to enable "private" membership data copying for configuration property file.membership.exact.copy=N

# Designates whether to delete existing gradebook data or keep it in the targetgradebook.clear=N

Sample Controller Code

'SET YOUR LOCAL SNAPSHOT FOLDER LOCATION - THIS IS WHERE YOUR SHAPSHOT FEED FILES RESIDE

sourceDir = "D:\bbsnapshot\“

'SET YOUR ARCHIVE FOLDER LOCATION - FOLDERS WILL BE CREATED WITH DATE/TIME STAMP

archiveDir = "D:\bbsnapshot\" & DatePart("m",currDate) & "_" & DatePart("d",currDate) & "_" & DatePart("yyyy",currDate) & "_" & DatePart("h",currDate) & "_" & DatePart("n",currDate)

For Each item in FileArray

If item = “students.txt" Then

If objFSO.FileExists(sourceDir&item) Then

'file exists

'Message = "found " & item & archiveDir

'MsgBox Message, vbInformation + vbOKOnly

set Wsshell = createobject("WScript.Shell")

Wsshell.run "D:\blackboard\apps\snapshot\bin\snapshot -f USR_MANUAL -C "&propDir&"snapshot.properties -t " &sourceDir&item&" -V bb.my.edu",1,TRUE

objFSO.MoveFile sourceDir&item,archiveDir&"\"&item

End If

End If

Execution Modes• Snapshot Mode (CRS_SNPSHT or XML_SNPSHT)

– The feed file is the master record

– Any records not contained within the file will be disabled within Blackboard

• Manual Mode (CRS_MANUAL or XML_MANUAL)

– Add/Update functionality

– Accounts will be disabled only if the record indicates it

• Remove Mode (CRS_REMOVE or XML_REMOVE)

– Data purge mode

– Only records contained in the file are processed

• Course Copy (CRS_ COPYINTO or XML_ COPYINTO)

– Specifies a source and destination course for content, enrollment, and/or settings to be migrated

Ex: Snapshot –f CRS_MANUAL –t c:\bbsnapshot\fallCourses.txt –C c:\bbsnapshot\

snapshot.properties –V servername.institution.edu

Error Logs

Invalid Text Files (Snapshot root)

Cannot insert duplicate key row in object 'dbo.course_users' with unique index 'course_users_ak1'.

Bb Logs (\\blackboard\logs\snapshot)

2010-07-09 12:10:19 - VL000184|ffd_tasker|student|disabled - blackboard.persist.PersistenceException: com.inet.tds.be: Msg 50000, Level 18, State 1, Procedure delete_cu_snap_proc, Line 18, Sqlstate 01000

[SQLBB5]Record does not exist.

at blackboard.admin.snapshot.persist.impl.SnapshotRemoveProcedureQuery.executeBatchQuery(SnapshotRemoveProcedureQuery.java:350)

at blackboard.admin.snapshot.persist.impl.SnapshotDbPersister.remove(SnapshotDbPersister.java:217)

at blackboard.admin.persist.course.impl.EnrollmentDbPersister.remove(EnrollmentDbPersister.java:152)

at blackboard.admin.snapshot.authority.Authority.delegatePersistence(Authority.java:678)

Suggested Integration Committee

• Registrar’s Office

• Student Information System Representative

• Academic Dean or decision maker

• Blackboard System Administrator

Business Rules To Consider

• Account Creation – Non-SIS accounts

• Manage Add/Drop - DISABLE vs. DELETE

• Respond to Financial or Administrative Holds

• Manage Incompletes

• Automation of Course Availability & Creation (blank or template generated shells)

• Cross-listed Courses

• Manage User Departure (transfer, resignation, etc)

• Deletes vs. Disables

• Workflow and Lifespan Policies around Users, Courses, & Enrollments

• Frequency of Data Flow

Sample Planning Guide - Courses

Availability:

• When will courses be made available to faculty?When will courses be made available to students?Do you wish to make the courses not available to students after the semester ends?If so, how many days after the end date?

Content:

• Will courses be generated from a template?If so, one template for all courses or a predictable algorithm for subjects?

Cancellation:

• How will you handle courses cancelled before the start date? (Unavailable vs. Delete)How will you handle courses cancelled after the start date? (Unavailable vs. Delete)

Archiving/Exporting:

• Do you want to archive or export your course content? (Archive vs. Export)If so, when do you want to process this?

Retention/Deletion:

• How long do you want to keep your courses on the Bb server?

Sample Planning Guide - Users

Creation:

• Will you be creating all your user accounts (faculty & students) or just actively enrolled users?

• When will users be created?

• How will you handle Non-SIS/Guest accounts?

Updating:

• What information do you want to update for users? (Email address? Password? Last name? User ID?)

Retention/Deletion:

• How long will you keep inactive user accounts?

How will you determine which accounts to remove? (Deletion vs. Disabling) 

Sample Planning Guide - Faculty

Assignments:

• When do you want faculty to have access to their course shells?

Drops:

• How do you want to handle a faculty assignment being changed? (Old

assigned faculty being dropped)

Sample Planning Guide - Students

Enrollments:

• When do you want students to have access to their courses?

When do you want to start processing enrollments?

Drops:

• How do you want to handle student drops? (Deletion vs. Disabling)

Sample Planning Guide – Secondary Roles

Secondary Institution Roles:

• Do you have a need to delivery role-based information within Blackboard?

(Student vs. Faculty)

Sample Process Outline

• Create courses -60 days from start date

• Copy content – 60 days from start date

• Faculty User Creation & Assignments – 30 days from start date

• Student User Creation & Enrollments – 10 days from start date

• Course made available to students – 4 days from start date

• Course made unavailable to students +10 days after end date

• Courses archived/exported + 12 days after end date

• Courses deleted from server + 365 days after start date

Future of Integration – XEI?

References

• Behind the Blackboard

– Advanced Integration and Data Management Guide

• IMS Enterprise XML Binding Specification

– http://www.imsglobal.org/enterprise/enbind03.html

Questions & Contact Information

Any Questions?

Any Brain Lock-up?

Contact Me:

Kevin Corcorankcorcoran@ctdlc.org860-832-3889

Please provide feedback for this session by emailing DevConFeedback@blackboard.com. 

The title of this session is:

Snapshot Integration Basics