505 Final Evaluation Project

8/2/2019 505 Final Evaluation Project

1/35

E D T E C H 5 0 5 - 4 1 7 3

D E C E M B E R 9 , 2 0 1 0

Submitted by: Latanya Blakley

A short-term evaluation of the professional development

workshops integrated into the companys Annual Franchise

Conference for 2010.

Evaluation of the

Franchise ConferenceProfessional

Development

Workshops


2/35

1

Table of Contents

Learning Reflection 2

Executive Summary

....3Purpose of the Evaluation .4

Background Information5

Evaluation Design Description.9

Results. ...12

Discussion of the Results.. ..16

Conclusions and Recommendations .18

Appendix .20


3/35

2

Learning Reflection

Evaluation is a part of our daily lives. We informally evaluate products and programsprior to making decisions about them. We also evaluate those same things after weve

engaged them to determine if it was worth our time, money, effort and so on. Therefore,when we shift our focus of evaluation to the implementation of products, programs andinstruction that typically affect more than one individual and utilizes significantly higherinvestments of time and money, it is understandable that deliberate and systematicprocesses should be employed.

This class helped to reinforce that evaluation is an important part of educationaltechnology and many other disciplines. While several aspects of an evaluation weresomewhat familiar to me, there were a few areas that I really learned a good deal ofnew concepts. First of all, the evaluators program description as explained in the textwas an eye-opener in terms of the way to initiate the evaluation process. In terms ofincorporating this into my existing schema, I likened it to a needs analysis in theinstructional design process. I learned to rely on the EPD as essential to beingknowledgeable about the program, understanding the objectives, the audience and thequestions that the client wants answered. It is a major launching point for the evaluatorand now that I understand its function, I cant imagine completing a comprehensiveprogram evaluation without starting with this step.

I also understand that data collection and analysis plays a key role in supporting andsubstantiating the conclusions and recommendations. Coupled with a strong base ofevaluation questions answered by the analysis and recommendations, those aspectslend a great deal of credibility to the evaluator. In my experience, many in-houseevaluations, especially those of instructional programs tend to stop at evaluations that

only gauge the attendees reaction.

Knowledge of evaluations can be useful when writing proposals to implementtechnology or instructional solutions. Educational technologists must research anddetermine the appropriate technology needs to best support instructional methods. Inorder to do that with some degree of certainty, a short-term evaluation such as the onein this report may need to be conducted before making a decision. One would also beexpected to request funding for technology resources and many times this also means a

justification for allocating resources. Therefore, educational technologists have aresponsibility to measure the effectiveness, efficiency and if possible, the impact oftechnology resources, instruction and programs to know whether the resources utilized

have or will yield the desired results or met the specified standards.

Overall, I learned that evaluation is a both a science and an art. Though data collection

and analysis is methodical, it can be challenging. After this process, I understand data

analysis beyond what I learned in my Undergraduate studies, but I will benefit from

additional specialized training on statistics and analysis. I have a strong desire to

strengthen my skills in this areas, especially learning to measure correlations, standard

deviation and performt tests.


4/35

3

Executive Summary

The National Franchise Conference provides an annual professional developmentopportunity for the franchisees of our organization. The purpose of this event is to

positively provoke and educate the attending franchisees. The event is funded by apool of royalty payments from franchisees, so they come with high expectations forlearning about how to improve their business position and execute better instruction.

Recently, a new conference committee acquired the responsibility for planning theconference and selecting the content for the professional development workshops. Thiscommittee was charged with first evaluating the effectiveness of the 2010 conferenceworkshops before moving forward with any planning activities and thus any funding.

The evaluation program description uncovered a few deficiencies in the workshopplanning process including the fact that standards had not been developed prior to theworkshop development. As a result, the evaluation was executed as goal-free with the

intent of obtaining a picture of how the program currently operates and how it could beimproved as planning ensues to meet the needs of the audience.

The 2010 conference attendees numbered over 1300 and immediately following theconference an online survey was deployed to the over 700 franchisees who attendedthe professional development workshops. Later, a brief small scale group interview wasalso done to supplement the results of the survey. There were both qualitative andquantitative aspects to this survey which had a high respondent rate of approximately33%. However, the questions on the survey lacked the necessary depth and range andserved to provide one-dimensional ratings of the workshops and a host ofuncategorized comments.

While the results of the quantitative analysis did not produce very significant or distinctresults, the qualitative analysis revealed the details needed by the committee to improvethe process of selecting, developing, executing and evaluating the professionaldevelopment workshops for the 2011 and 2012 conferences.


5/35

4

Purpose of the Evaluation

This evaluation will assess the usefulness of professional development workshopsoffered at our 2010 annual franchise conference. As the franchisee royalties are the

primary source of funding for this event, it is vital to ensure that their needs are beingmet. However, the evaluation itself is being funded by the corporation with oversightbeing provided by the Conference Planning Committee (here forward known as thecommittee). The primary purpose of the evaluation is to examine what happened whenattendees participated in the 2010 conference workshops as well as report theattendees reactions. The secondary purpose of this evaluation is to inform futureplanning decisions of the 2011 and 2012 conferences being made by the committee.

The following questions were asked of the committee during the evaluators programdescription meeting:

1. Overall, what would the committee like to accomplish with this evaluation?

2. Are the any historical data that can be used in the evaluation?3. Who is ultimately accountable for the outcome of this program?

The committee shared the program information that is found in the background sectionof this evaluation report. In addition to understanding the outcome of the 2010evaluation, the committee will use this evaluation to make decisions about the planningof the 2011 and 2012 conference workshops.

Central Questions

The central questions this evaluation will attempt to answer are:

According to the franchisees, how useful were the workshops? What were the reactions of the franchisees to the workshops?

What improvements should the committee consider for the future conferenceworkshops?

There are also several ancillary questions, subdivided under the top three questionsthat will also be answered in the scope of this evaluation. These are found under theevaluation design description section.

Evaluation Results Audience

The committee and the corporate university department leaders are ultimately

responsible for the planning and accountable for the outcome of this program. The twokey audiences that will be most impacted by the results of this evaluation are thecommittee and, most notably, the attendees of the 2011 and 2012 conferences.

Challenges and limitations of this evaluation include: There were no formal instructionalobjectives determined prior to the planning of the 2010 conference. Of the six membersfrom the 2010 conference committee, five of them are in other roles or no longer withthe company because of a recent reorganization. Since the inception of the conference,


6/35

5

a formal evaluation of this caliber has yet to be completed, though there were surveysdone in at least the last five years. Due to time constraints, the evaluator could notobtain access to the previous years data for a comparative analysis.

Background Information

Program Origin

Each year over a thousand people from the Canada, Mexico and the U.S. gather inselect cities for the companys largest franchisee conference. Hundreds of thousandsof dollars are invested into the development of this event for the purpose of educatingand motivating franchisees. The North American Franchise Conference has been anannual professional development event for franchisees and their families since itsinauguration in year 2000. The first conference was held in a small hotel in downtown

Chicago and was a fairly intimate event with a few main stage guest speakers andbreakout sessions. At that time the conference lasted approximately two days and hadonly about 300+ attendees. Over the next 9 years, the conference grew to reflect thecompanys growth. The 2010 conference spanned 3 full days, hosted 1301 guests,including over 700 franchisees, and was held at a large conference hotel in Manhattan.

Conference Structure/Standards

The 2010 conference structure included the following key elements: a welcomereception, 1-2 keynote speakers, 1-2 franchisee main stage presenters, 2 studentspeakers, 8-10 workshops, an extended conference experience including city tours andvisits to local franchises, addresses from the both the North American and Worldwide

Presidents, 2 awards ceremonies, software demonstrations and numerous othernetworking opportunities.

Franchise attendees were allowed to register for four workshops, two academic and twocenter management workshops. The minimum number of seats in a workshop was 150and the maximum was 300. Company associates were allowed to register afterfranchisee registration was complete, but attended the same workshops. Registration isvery strictly controlled. Once an individual has registered for the four workshops, nochanges could be made. This is important to control the amount of constant or lastminute changes that would potentially be made by registrants. Strict guidelines werealso necessary to support the logistics of sending confirmations and printing badgeswith all the guest event details for such a large group.

The Committee

The conference committee in 2010 was comprised of 6 company associates. Planningfor the conference usually takes a little over a year from venue booking through the daybefore the event. The new committee, associates from the companys corporateuniversity, believes that conference content and production should be developed withthe following goals in mind:


7/35

6

Educate franchisees on ways to improve their business and increase profits

Share inspirational stories of highly successful franchisees Present findings of student case studies and research-based VSGs (voluntary

study groups)

Increase participation and attendance at the conference each year

The company recently announced a new expansion initiative that was discussed withthe committee to determine if there was a potential impact to the conference content.The committee agreed that the new performance indicators should be considered whendeveloping the future conference workshop content.

The Workshops

The workshops being evaluated can be grouped into two main categories, academicand business management workshops. An academic workshop can best becharacterized by content that is student-centered and focused on helping the franchiseeimprove instruction. A center management workshop can best be characterized by

content that is business-centered and focused on helping the franchisee grow his or herbusiness. Presentations in either category can also be the result of a Voluntary StudyGroup (VSG). These presentations tend to be research-based and include case-studiesof franchisee practices and student instruction methodologies.

Each of the workshops in the program is no longer than one hour in length. Workshopsare delivered in person via a classroom style lecture format using digital media such asMicrosoft PowerPoint. The presenters include successful franchisees, companyassociates and outside consultants with current corporate affiliations. In past years, thecommittee has selected the workshops for the conference by meeting with potentialpresenters who have been nominated by leaders within the company. The concept ispresented and the committee typically reads them and chooses the workshops basedon the diversity of the content and prestige of the franchisees involved. The presentersmust submit a synopsis of their workshop for inclusion in the conference registrationwebsite.

The following are descriptions of the 10 workshops conducted at the 2010 conference.Please note that some names and terms have been omitted to protect the privacy of thecompany and its franchisees.

Academic Workshops

Workshop A: Learning from Students with Special Needs

Students with special needs often provide Instructors extraordinary learningopportunities. The Instructors in this St. Louis-based VSG have worked withmany children with special needs those with specific diagnoses, such as alearning disability and ADHD, autism, and developmental and emotional delayand recognize that each student and situation is unique, but that what we learnfrom these students has broad application. The Instructors will share their

journeys with three students with special needs. They will discuss lesson


8/35

7

planning, parental involvement, and specific accommodations, which havehelped these students to reach their potential.

Workshop B: Simple Steps for Retaining Students

Learn three of the keys to student retention. The Phoenix Branch Retention VSG

has spent its time focusing on the orientation and enrollment process, effectiveongoing communication with students and parents, and developing Centerassistants. Come and see how focusing on these three keys as well as otherfundamentals has helped this group of Instructors to retain students longer and tobring them to higher levels of achievement, while at the same time improvingtheir retention rates, increasing ASHR percentages, and raising their IARrankings.

Workshop C: Title Withheld

Members of the renowned Gateway VSG will share their experience andtechniques for successfully guiding students to reach the highest levels while inelementary school. The workshop will focus on the dynamics of working withstudents in three key areas:

Instruction tips to develop students XXXX skills Communication that emphasizes the benefits of advanced study

Motivation strategies based on goal sharing and self-evaluation

These talented Instructors will share their experiences and knowledge related tothe building of skills and confidence in their students. Join this workshop to learnmore about how to instruct excellent elementary-aged children with focused hintsand how to help them to succeed with the right amount of motivation.

Workshop D: I Can: Assistant and Student Development for the ReadingProgram

Esteemed veteran Instructors began the I Can VSG with the goal of clarifyingand refining what their assistants assess when working with Reading studentsand grading their worksheets. As the VSG continued, they turned toward theirlesson planning, seeking new ways to streamline lesson planning based on theirVSG findings. Join this workshop to learn firsthand how these excellentInstructors are continuing to perfect their lesson planning, instruction techniques,and assistant training through their ongoing collaboration.

Workshop E: Title Withheld

XXXX said, Each instructor may have his or her preconceived beliefs, and eachmay reach a point where they fee l something is good enough. However, thetruth is, there is nothing good enough, but always something better. Thispresentation will explore how four Los Angeles Branch Instructors who felt theyhad reached good enough discovered through collaboration that by pursuing thepotential of their own students, their ability as Instructors blossomed, too. These


9/35

8

four Instructors will share with you how learning from their excellent students andfrom each other has impacted how they communicate with, observe, and lessonplan for all of their students.

Center Management Workshops

Workshop F: Maximizing Your Employees Potential to Maximize Your Potential

Are your employees giving you and your students what you need and expectfrom them? Have you tried to provide sufficient training, but things still dontseem to be working out? If you are at all like the accomplished veteran ChicagoInstructors who took part in this Voluntary Study Group (VSG), you may beexperiencing some of the same staff challenges they have begun to addresseffectively: lack of initiative or confidence, ineffective interaction with students,lack of customer service skills, frequent employee turnover all of which lead toproblems within the classroom, as well as have an impact on your own ability todetermine compensation and manage payroll. These Instructors are eager toshare how they utilized employee surveys and worked together to overcomethese challenges, while also increasing their self-awareness and development asleaders. By sharing the path they have taken and the resultingaccomplishments, these Instructors hope to motivate you to begin yourleadership self-development anew, strengthen your staff development process,and work towards building a strong, successful team.

Workshop G: Growing Your Business from the Grassroots

Any small business owner will tell you that the best way to build awareness is

through grassroots marketing. But what does this mean and how do you do it?This presentation offers examples of well-executed, community-focusedinitiatives, and outlines the tangible benefits of non-traditional marketing that notonly drives leads, but also delivers value back to the community. Join thisworkshop to learn how Instructors in the U.S. and Canada have enhanced thereputation of the brand and their Centers, and how you can, too!

Workshop H: Measuring Instructor Achievement with New Center PerformanceAnalysis Reports

Join this workshop to discover the new tools to help Instructors and Associatesevaluate Center performance. The Center Performance Analysis (CPA) is a

newly developed web-based report examining several key enrollment, retention,and student achievement indices. The CPA graphically depicts statistics found onthe existing CMFA Report, as well as many new informative measurements, suchas Student Inflow/Outflow, Student Turnover, Student Attainment within 12-Months, and more. Discover how these new online reports can aid in youranalysis of Center operations, including the ability to compare ones achievementwith company goals and other benchmarks. With this new way of examining


10/35

9

Center performance, Instructors can become more empowered to improveinstructional quality and Center operation.

Workshop I: Operating an Efficient and Profitable Franchise Business

Take a good, hard look at the way youre conducting your Center. Consultants

from Coleman, Chappel & Company, one of the Top 50 public accounting andbusiness consulting firms in the U.S., will offer expert advice on how to operate asuccessful franchise business. This workshop will provide tips on how franchiseowners can operate their businesses more efficiently and profitably, includinghow to overcome the unique challenges franchise owners face throughout the lifecycle of their business operations and how to boost profits.

Workshop J: Understanding and Effectively Communicating With ProspectiveParents

Just who is our prospective parent? Why do they matter to us? And what does ittake to communicate more effectively the extraordinary benefits of XXXX tothem, ensuring they not only enroll their child, but remain committed for asignificant length of time? Join us for an informative session that is sure tochange the way you think about parents and which will give you the tools youneed to reach and retain families.

Audience Characteristics

The physical audiences for the conference workshops include associates, franchiseesand their family members. However, the clients who are most affected by this programinclude the franchisees of this international supplemental education company. They areadult learners with at least a bachelors degree, although several have post -

baccalaureate degrees. Over 60% of the learners speak English as a second language.Also about 70% of have owned their franchise for 3 or more years.

Description of Evaluation Design

The committee acknowledged not being aware of any objectives, explicitly written orotherwise stated, that were determined in advance of the 2010 conference workshopselection and development. Therefore, the evaluation design is based on anamalgamation of the Goal-free Model with some components of the Decision-making

Model. Both of these models will yield a summative evaluation of the conferenceworkshops.

As stated in the purpose section, the answers to the following questions will providethe information necessary to determine the useful of the workshops in the manner thatthey took place. A portion of the questions are also included to gather what franchiseeswould prefer to see in conference workshops.


11/35

10

1. Overall, how did attendees rate the 10 workshops?2. Which three workshops received the top ratings? Why?3. Which three workshops received the lowest ratings? Why?4. How do the results for center management workshops compare to academic

workshops?

5. What are the attitudes of franchisees about the content for the conferenceworkshops? (ordinal data ranking)6. What are the attitudes of franchisees about the format of the conference

workshops?o What aspects of the workshops did franchisees like most?o What improvements did franchisees suggest?

Data Collection

The main set of data was collected using an online survey instrument (Appendix A) withboth a quantitative and qualitative element. A series of questions with a Likert scalewas employed to gather a general rating of the workshops. Questions 1-4 in the list

above, which were derivatives of the evaluators program description, were the impetusfor the use of the Likert scale in the evaluation design. The ratings were used toaddress the first two central questions from the purpose section about usefulness andfranchisee reaction.

The survey also utilized open-ended survey questions to elicit more specific feedbackrelated to each workshop that the respondent attended. It was necessary to gather thisqualitative data to answer the third central question about considerations of futurecontent. Questions 5-7 provided the framework for incorporation of an open-endedquestion format for each of the workshops on the survey.

Approximately one week following the conference, the survey was distributed to the 707

franchises who attended the conference. Due to the nature of the business, it is typicalthat each attendee be given an opportunity to respond to the survey and themanagement was not willing to forgo that format at the risk of upsetting franchisees.Therefore, the data collection effort initially included the entire attendee population withthe expectation that about 25% would respond. These respondents would constitute a25% purposive sample in the evaluation design. The survey was made available forabout three weeks before it was closed for reconciliation and preliminary data analysis.

After the primary data collection activities, the evaluator was able to briefly meet withtwo separate groups of franchisees to conduct group interviews. These wereunplanned opportunities coordinated with the approval of the committee that allowed for

discussion about the workshop selection process. The first group consisted of 7franchisees and the second group had 8 franchisees all of whom attended theconference and went to the workshops.

A series of questions were asked to the groups to capture supplemental detail aboutwhy they chose the workshops for which they were registered.

Did you register for all four workshop slots?


12/35

11

Did you attend the workshops for which you were registered?

What was the main reason you registered for the workshops you chose?

Data Analysis

In the quantitative analysis, the evaluator used measures of central tendency and

variability to categorize most of the data sets. In order to make clearer comparisons,some data was converted to percentages where necessary.

Qualitative data which consisted of many different comments was reviewed and codedto produce comparable nominal data such as positive, negative, neutral as shown inTables 5a and 5b. The comments were also categorized as a suggestion or non-suggestion. The suggestions were then coded to determine what type of suggestionand if there were similarities they were lumped together and discussed in the resultssection.

Part of the analysis also included an interpretation of the comments to determine theaspects of the workshops most enjoyed by the respondents. This was done by tallyingthe coded comments in the prevalent categories.


13/35

12

Results

Survey

The online survey instrument generated responses from 239 respondents out of 707

attendees in the total population, yielding a 33.8% response rate. This section of thereport will show how the data provides the answers to the questions and establish towhat extent the attendees thought the workshops were useful.

1. Overall, how did attendees rate the 10 workshops?

The mean of Likert scale ratings for each workshop was calculated. Becauseeach workshop had a different number of attendees and respondents, nvaries in the data set. The range of the scores (3.31 2.51) is 0.8.

Workshop n= Mean

A 80 2.93

B 143 2.97

C 98 2.74

D 137 3.31

E 53 2.81

F 145 3.28

G 79 3.09

H 68 2.71

I 126 2.51

J 92 2.88

Table 1. Mean of Likert Scale Ratings per Workshop

2. Which three workshops received the top ratings? Why?3. Which three workshops received the lowest ratings? Why?

The top three workshops are D,F,G and the bottom three are C,H, I.

Workshop n= Mean

D 137 3.31

F 145 3.28

G 79 3.09

B 143 2.97

A 80 2.93J 92 2.88

E 53 2.81

C 98 2.74

H 68 2.71

I 126 2.51

Table 2. Highest & Lowest Ratings


14/35

13

4. How do the results for center management workshops compare to academicworkshops?

The Academic workshops received a slightly higher average rating thanthe Center Management workshops. However, it is noted that two of the

top three workshops were from the Center Management category.

Academic n= Mean

A 80 2.93

B 143 2.97

C 98 2.74

D 137 3.31

E 53 2.81

2.95

Center

Mgmt. n= Mean

F 145 3.28

G 79 3.09

H 68 2.71

I 126 2.51

J 92 2.88

2.89

Table 3. Workshop Category Comparison

5. What are the attitudes of franchisees about the content for the conferenceworkshops?

6. What are the attitudes of franchisees about the format of the conferenceworkshops?o What aspects of the workshops did franchisees like most? (next section)o What improvements did franchisees suggest? (next section)

Respondents were aware that the numeric rating for each scorecorresponded to a description. Over 70% of respondents felt theworkshops were either good or excellent. Appendix A illustrates theresults of each individual workshop. Below is the combined result of thoseresponses. (Colors correspond to the chart colors in Appendix B.)

All WorkshopsResponse Number Percent

1 Poor 65 6%

2 Fair 215 21%

3 Good 445 44%

4 Excellent 296 29%

1021 100%

Table 4. Overall Attitudes of Respondents


15/35

14

Following each rating question, respondents were also asked to shareopen-ended comments about the overall content, format and delivery ofthe workshops. This table, which represents the total of all commentsentered, shows the number of responses coded with a positive, negativeor neutral comment. Each comment was also examined for suggestions

made to improve the content of the workshops (Appendix C). Asummarization of the 69 comments is provided in the next section.

Positive Negative Neutral Total

Content

Suggestion

A 2 8 5 15 7

B 14 4 5 23 7

C 3 15 4 22 14

D 25 1 4 30 3

E 0 3 1 4 1

F 19 5 4 28 5

G 7 2 4 13 1

H 2 8 5 15 6

I 9 26 6 41 17

J 10 16 1 27 8

91 88 39 218 69

42% 40% 18%

Table 5a. Respondent Attitudes Reflected in Open-Ended Answers

Workshop Positive Negative Neutral Total

A 13% 53% 33% 100%

B 61% 17% 22% 100%C 14% 68% 18% 100%

D 83% 3% 13% 100%

E 0% 75% 25% 100%

F 68% 18% 14% 100%

G 54% 15% 31% 100%

H 13% 53% 33% 100%

I 22% 63% 15% 100%

J 37% 59% 4% 100%

Table 5b. Percentage of Attitudes Reflected in Open-Ended Answers

7. What topics/content do franchisees want to see for upcoming conferences?

There were 153 comments sharing suggestions for upcoming topics. Forthe purposes of the evaluation, a sample equal to the response rate, or33% (51), was randomly taken of the comments.


16/35

15

Interview

The online survey instrument generated responses from 239 respondents out of 707attendees in the total population, yielding a 33.8% response rate. This section of thereport will show how the data provides the answers to the questions and establish towhat extent the attendees thought the workshops were useful.

1. Did you register for all four workshop slots?

Question 1

Yes 14 93%

No 1 7%

15 100%

Table 6. Interviewees with Complete Registrations

2. Did you attend the workshops for which you were registered?

Question 2

Yes 15 100%

No 0 0%

15 100%

Table 7. Interviewees with Correct Attendance

3. What was the main reason you registered for the workshops you chose?

Question 3

The synopsis 7 47%

The presenters 5 33%

Influence from others 2 13%

Nothing else available 1 7%

15 100%

Table 8. Reasons for Registration

In the next section, these evaluation results will be interpreted.


17/35

16

Discussion of the Results

This evaluation was employed to answer three central questions posed by thecommittee responsible for planning the future conferences and designing workshops:

According to the franchisees, how useful were the workshops?

What were the reactions of the franchisees to the workshops?

What content should the committee consider for the future conferenceworkshops?

Results of the data analysis show that on a scale of 1-4, with 4 being the highest rating,the mean rating that respondents gave to the workshops overall was 2.92. This numberwas calculated by taking the average of the averages in Table 1. The ratings had avariance of 0.8 which means that the ratings varied moderately. As the overall average

is closer to a 4 than a 1, and is closest to a rating of 3, we can interpret that therespondents collectively felt the workshops were good as established by our ratingdescription.

Analysis of data in Table 2 reveals that the top 3 workshops are D, F and G. The topicsof these workshops are reading instruction, employee training/leadership andgrassroots marketing respectively. Two out of three of these workshops are on CenterManagement and two of them are based on VSG research. The same analysis revealsthat the least liked workshops are C, H and I. The topics of these workshops areinstruction tips for elementary school students, the new center performance analysisand operating efficiently and profitably. Two of these workshops are also Center

Management and only one is based on VSG research. The respondents who attendedthe top three workshops constitute 35% of the sample population, compared with 29%of those who attended the bottom three workshops. Upon examination of the qualitativedata in Tables 5a and 5b, or the comments, the top three workshops D, F and G share56% of the comments coded as positive. There seems to be a strong correlationbetween the ratings for D and F to the number of positive comments received. Thecomments themselves (Appendix C) also show that respondents appreciated thepractical nature of the workshops and tools demonstrated from the top three workshopsoverall.

In terms of accomplishments, the workshops yielded 42% positive comments fromrespondents versus 40% negative. These numbers could each be higher, but with thesame ratio, because some of the neutral comments were coded as such because theyincluded both a positive and negative aspect, which was deemed neutral overall. Alsoover 70% of the respondents rated the workshops good or excellent as shown inTable 4. Appendix A will also show that in each individual rating table, the percentageof good and excellent outweighs the percentages of fair and poor ratings withoutexception.


18/35

17

There key areas that stand out as aspects of the workshops preferred most by therespondents (Appendix C) can be summarized as follows:

Workshops that offer specific strategies or recommendations

Helpful and practical tools that have been tested and are available for use

Strategies and ideas that can be immediately implemented Information that can impact instruction or help students more directly

Hearing from their peers rather than from an outside speaker

Having more informative and accurate workshop descriptions that reflect thetarget audience

The last point is also supported by the results of the small group interview where 47% offranchisees said their main reason for choosing their workshops is because of thesynopsis they read on the website.


19/35

18

Conclusions and Recommendations

Immediate Conclusions1. There appears to be no clear distinction between the effectiveness of CenterManagement workshops and Academic workshops. Therefore, the committeeshould continue to offer a balance of these workshop categories.

2. Develop program objectives/standards Prior to beginning the planning andselection of professional development workshops, the committee should meetand determine specific and measurable objectives for this aspect of theconference. Having benchmarks will help guide the activities of the committeeand set standards that the attendee surveys can be measured against.

3. Use a criteria checklist when choosing workshops to ensure there are objectivestandards used that meet the needs of the audience. The preferred contentmentioned in the survey should be considered:

Workshops that offer specific strategies or recommendations

Helpful and practical tools that have been tested and are available for use Strategies and ideas that can be immediately implemented

Information that can impact instruction or help students more directly

Hearing from their peers rather than from an outside speaker

Having more informative and accurate workshop descriptions that reflectthe target audience

According to the committee, the new company initiatives should be included inthe future conference content. The criteria can include these initiatives to ensurethat new content meets the needs of the company and franchisees.

4. The workshop synopses appear to have a large bearing on the expectations thatattendees have for the workshops. Designate a strong writer and someone fromthe marketing team to make sure the synopses include information such as thetarget audience and an accurate description of the workshop content and format,so that attendees can choose the best workshops suited to their experience leveland dont feel misled because of unmet expectations.

5. Consider involving an advisory committee comprised of franchisees in theplanning of the conference professional development workshops. The committeecan and should control the extent to which they have input, but franchisees mightfeel better that their interests are represented in the planning of event funded bytheir royalties.


20/35

19

6. Hire a professional survey writer to redesign the survey used by franchisees toevaluate the workshops and write a survey to request content suggestions for theupcoming conference workshops. The new survey should include demographicdata and be much more specific in terms of categorizing comments. Thedemographic data can be used to cross tabulate with survey ratings. It is

important to determine the experience of franchisees and whether or not anycorrelation exists with their level of satisfaction regarding workshop content.Include location, number of students, number of previously attendedconferences. The standards and criteria established from points 1 and 2 shouldalso be included in the new survey and used to confirm the standards are beingmet.

Long-Range Planning

This evaluation should be used as a benchmark for future evaluations. The committeeshould discuss ways to improve these statistics and work to track this data over time ifthese workshops will continue to be produced for the conference.

Objectives-based point of view choose a couple of workshops with clear cut learningexpectations write actual learning objectives for them. In the future, the committee canemploy a focus group or small group interview to gain insight into how well the learningobjectives were met which could aid in assessing effectiveness. May need to randomlysample from franchisees either on-site at the conference or a short-time after. Forexample, the committee could collect a list of participants and then send them aquestionnaire immediately after the workshop followed by an interview schedule.


21/35

20

Appendix A Individual Ratings with Percentages

Response Number Percent Response Number Percent Response Number Percent

1 Poor 7 9% 1 Poor 6 4% 1 Poor 8 8%

2 Fair 15 19% 2 Fair 28 20% 2 Fair 24 24%

3 Good 35 44% 3 Good 73 51% 3 Good 51 52%

4 Excellent 23 29% 4 Excellent 36 25% 4 Excellent 15 15%

80 100% 143 100% 98 100%

Workshop F

Response Number Percent Response Number Percent Response Number Percent1 Poor 1 1% 1 Poor 4 8% 1 Poor 1 1%

2 Fair 19 14% 2 Fair 14 26% 2 Fair 18 12%

3 Good 54 39% 3 Good 23 43% 3 Good 66 46%


137 100% 53 100% 145 100%

Response Number Percent Response Number Percent Response Number Percent

1 Poor 3 4% 1 Poor 5 7% 1 Poor 24 19%

2 Fair 15 19% 2 Fair 22 32% 2 Fair 35 28%

3 Good 33 42% 3 Good 29 43% 3 Good 46 37%


79 100% 68 100% 126 100%

Workshop J

Response Number Percent

1 Poor 6 7%

2 Fair 25 27%

3 Good 35 38%

4 Excellent 26 28%

92 100%

Workshop I

Workshop A Workshop B Workshop C

Workshop D

Workshop HWorkshop G

Workshop E


22/35

21

Appendix B Survey Questions and Graphic Results


23/35

22


24/35

23


25/35

24


26/35

25


27/35

26

Appendix C Open-ended Comments

Workshop A: Learning from Students with Special Needs

Each special need student is different and I am still wondering how to better help my special need

student.

I was hoping to get information on definition of different disorders and how to handle it. Did not attend

I did not attend

This really was three case studies; would have preferred to see a presentation on specific strategies

for students with specific types of needs. The supplements showcased are not XXXX's -- why can't

our corporation adapt to the needs of our communities by growing our supplemental tool arsenal?...

Well done.

Needed more specifics. I would better know how different between normal students and special

need students to XXXX study plan and goal.

not attend

There were no specific recommendations from the VSG. I expected to get more information on

symptoms and characteristics of specific learning differences and a way to deal with them. NA

More in depth tips of identifying and working with these students would have been helpful. XXXX

needs a forum where instructors can go to talk & get help for these students...there are so many

and the need is so great.

Could have been better presented. It was kind of obvious this isn't something they do regularly.

There was not enough deep content in this workshop. I left with a sincere appreciation of what

these Instructors are doing for their students but without anything substantial to take back to my

Centre.

I was expecting specifics of working with these students. e.g. what approach to take, what works,

tips for various needs etc.

Wish we can have more Q&A time. Love the discussions.

Workshop B: Simple Steps to Retaining Students

Good but a bit basic for my needs

Not much additional info. Just seemed to be recycled info.

I received a lot of good ideas. This was very well done.

Nice format, the speakers were very good.

Did not feel that we got any new information, XXX's presentation was very good, however, the ideas

of conferences etc were the same old stuff.

Good enough just reinforces what I do

Excellent

Liked the ideas of conferencing with parents of new students after 3, 6, 9, and 12 months. Also likedtalking about assistants and the role they play in retention.

Lots of good ideas to try out over time.

Nothing new.

I liked this - similar to other IMMs but some new ideas also.

I liked XXXX - gave some specific suggestions. Would have liked even more.

This VSG did a great job of providing actionable tips to increasing ALS.

Excellent


28/35

27

Great new ideas to take back and use.

After talking with other Instructors, we came to consensus that these were the steps we are

currently taking to retain students and communicate with them. These were also the steps

continuously discussed throughout the initial training before we became Instructors. It would be

nice to see actual samples of the newsletters, how a parent conference is held, and some of the

logistics of it. I think Instructors have many fancy ideas, but executing the plan and coming up with

the materials is always the hardest.

I wanted something more concrete.

This workshop should have been entitled, "What everyone should already be doing". I was very

disappointed with the superficial nature of this workshop. These were retention strategies - they

were basics. I was however, pleased that these "simple steps" have lead to increased retention and

subsequently, increased XXXX.

Interesting and useful to hear about approaches used by successful instructors.

It will be more meaningful if all the presentations and material are available to us.

Gave me some ideas that I will incorporate in my center.

It is helpful for the new instructors.

To balance listening to the instructors reading from their pages, it would've been nice to have more

visual aids or handouts for reference.

Workshop C: XXXX

Very disappointing.

"I liked this the least. Mrs. XXXX (sorry not to spell correctly) has a wealth of information butdelivery is poor.

Everyone states that observation is important. I prefer more how to than results presentation.

I wish I had more content. More specific details.

Loved it

Very, very good.

The blurb should have made it clear that the workshop was Gateway related. Because I thought it

would be more curriculum based I was disappointed.

Didn't learn much new but gave me new impetus to XXXX as quickly as possible.

too cold

Was not what I expected- expected to hear more about how to equip my students . Mrs. XXXX spokeentirely too fast- could not keep up nor follow.

key concepts / XXXX were breezed through too quickly

need more advance notice of session content not really geared to instructors with more than 1 year

experience

XXXX was the only presentation that I could understand

Poor/ If I knew it was a gateway meeting, I would have selected something else.

I need to learn more about how I can help students better by showing them examples from previous

sets that explain new concepts. I wish there was a cumulative reference print-out.

Very informative and motivating

The presentation was difficult to understand as the slides were moved thru quickly. However, I will

view it again on XXXX once it is available. There was a great deal of use info.

Expected more solutions used to problems students encountered....

What they are talking about does not fit this topic.

Although the content was very good, the material was presented too quickly for satisfaction.


29/35

28

The presentation was presented in a very boring manner and didn't really give me any concrete tips

for helping the average student to program completion.

Disappointed. Didn't seem to really give any concrete, hands-on ideas. Seemed mostly to overviewthe XXXX instead of how to adequately and/or uniquely plan for them.

Workshop D: I Can: Assistant & Student Development

I hope XXXX can share the guides and material with us.

Wish there had been a Q&A. I really thought this was a helpful workshop and I look forward to trying

out their materials.

Found the 'contract' approach something I wouldn't use, liked the booklets with the oral checks, too

much paper for each reading student in my opinion

Wish I had seen this. I will be anxious to watch DVD.

New tools to help communication with staff. Great work!

I liked the presentation, I feel XXXX has to put the booklets together and make it available for

instructors as a non-chargeable item.

Reinforces all that I do

The tools they have developed were just the thing we have been looking for!

Excellent. Eye-opening.

The reading department should take the material from this VSG and apply it to all the XXXX of the

reading program. XXXX should also provide the student and assistant reading guide for all the

reading XXXX.

WOW! Lots of info and am very thankful they are willing to share their findings with us. The best

workshop I attended.

Although the process seems a bit cumbersome/redundant, I love the XXXX-by-XXXX customization.

Again, it would be nice to see XXXX adopt some of these tools presented by franchisees as the

"standard" available to all franchisees.

Practical and informative. Helpful to go away with tools to use in the Centre.

Well done.

Wow -- I love those booklets. Seems like this was a fantastic VSG. Can't wait to see that

presentation on-line.

Extremely interesting. I would like to hear more from this group.

Very useful and practical. We need this ASAP. This is a missing link in the reading program.

"We need more summary the other XXXX, too. They provide only XXXX. We need the chart in

irregular past verbs."

Great- would have liked to walk away with the Reading XXXX summary for XXXXs.

Interesting replacement for the orange book and good staff training

Don't need more paperwork. Would like to see video in real time with the normal "chaos" offeedback, speaking to the students, etc. It always seems that these video clips show the ideal

situation; no other students or interruptions while the instructor is focusing on ONE student.

I hope the little booklets will eventually become a non-chargeable order. This is a great tool and is

extremely useful for bringing staff to a higher XXXX of understanding of the reading program

concepts.

Love this tool. Please help the VSG complete the project and get them online for us to use. I would

love to see these available through XXXX in the future.

These Instructors from the VSG presented concrete materials that they used in their centers which

make this presentation more interesting and useful. Especially - these are Instructors that are

experienced and have large centers.


30/35

29

These instructors have obviously presented before and they did a good job.

I am hoping there will be a posting of the "I Can" printouts and I look forward to developing more.

Thank you for sharing your hard work, XXXX and team.

This was the very best. I am very grateful to the instructors who put so much work into their VSG,

were so creative in generating the Student and Assistant Reading guide booklet, and most of all, for

being so generous in sharing their insights. I am touched, impressed and inspired by this group ofexcellent instructors.

The presentation was good, but will only be helpful if all the reading booklets are made available to

us.

I want to thank these instructors for all their hard work

Very helpful and it was delivered by platinum instructors...

Workshop E: Learning from XXXXs

Everyone states that observation is important. I prefer more how to than results presentation.

"I doubt even though young students are excellent in academic, how many students handle the endof our XXXX? I chose one of students in my center, parent reject to work the completion XXXX."

Didnt include reading completers

More of the same from general assembly.

Workshop F: Maximizing Your Employees Potential

I learned many practical hints I can use right away.

Addressed many of my concerns and was very motivational.

Well thought out, well presented, good materials, hope soft copies of their ideas will be made

available

Looking forward to implementing a lot of these.

Awesome new materials!

There were nuggets of information that we could take away. I am waiting for the posting of the

presentation. If the information is available to all on the forum it would be easy and customizable.

Being a part of every VSG may not be viable for everyone. I am currently in a VSG in my branch.

Adding one more will distract me from that one.

Did not feel I got a whole lot from this presentation as well.

Insightful. Every presenter had much to offer.

A little too general. It would be better if they address some of the more specific concerns. But I dounderstand the time constraint.

I hope that I can receive some record from the presentation.

Well done- but needed to get to the point sooner.

This was the most valuable part of the conference for me. I got MANY concrete ideas to improve my

center at this workshop. XXX is such a wonderful asset. The instructor-presenters did a fantastic job.

Best workshop this year.

I was expecting something a little more specific. It would be good to know what a VSG has decided is

a best ratio staff:student or how specialized XXXX staff arrangements compare with "Jack/Jill of all

XXXXs"- in terms of efficiency.

Wonderful presentation, I wish, we could get started with such VSG in all branches.

I walked away with practical steps and have already implemented two in my center! I would haveloved to have more time with Dr. B!

This was great.


31/35

30

Lots of great work done by VSG. I am looking forward to contacting them to obtain templates, etc.

produced to tackle this huge component of running a XXXX franchise. I think these should be posted

on XXXX under some forum of 'best practices'.

"A head hanging for me really since I should have already been doing most of what they talkedabout and either haven't or haven't done it consistently. I have already written the long overdue job

descriptions although I still need to tweak them."

great ideas

Came away with a few good ideas. Very well done.

Great ideas presented. Very clear and to the point w/ great ideas.

Some good tools and strategies.

I am hoping the criterion for employee evaluation papers will be posted on line.

I would like copies of the employee, student checklists

I will be giving the employee survey to my employees and hopefully will develop my own concrete

policies and procedures.

Still wished they could share the PD's they developed in their VSG.

Forty-five minutes of the hour was spent talking about the leader's VSG. The only helpful portion

was XXX's Q&A at the end. I was especially disappointed because I took the pre-conference survey

and expected to have some of those topics covered. However, handouts really would have been

great here.

Workshop G: Grassroots Marketing

Realized that many ideas need to follow and keep trying every year.

Very varied and lot of really good options.

Good. I got some ideas that I may implement.

Excellent.

I am biased on this one because I was one of the presenters. Unfortunately I was not able attend

another workshop on Day 2 because of the time conflict.

I wish I'd gone to this one....

Great ideas. I particularly liked the idea of a Parent Education Night. That's on my list for later this

summer.

Nothing new - really a redo of a monthly IMM.

Did not attend

Good ideas... why did we not hear from all participants in the VSG?

I really like that a variety of instructors were used. XXXX had a smaller number of students than

many center instructors that present. Still she had a good turnout for her family night and she feels

it really helped her. Sometimes we need to learn from the big, excellent centers and sometimes it is

helpful to see what will work for a smaller center.

Good ideas and presentations.

Didn't really hear anything new.

Workshop H: Center Performance Analysis Report

Although many of us thought it was our IPA opportunity, it was still interesting, just not what I

expected. XXXX should get the XXXX equivalent of an Oscar--should we call it a XXXX? Same for XXXX

for that video in day one.

Thought it was about IPA

I would like to see this replace XXXX with full capability. This way staff time will be spent in

discussing solutions as opposed to a lengthy, you did not do this got to change. The information


32/35

31

presentation need not take a negative tone that something is deficient hence improving with this

proposal. Instead it can give time for the instructor to digest and approach the meeting in a 'How

can we take care of this' mode.

Presentation explained the new reports, but presenters should not have avoided issue of outgoing /incoming transfers. There is a student ID. Why can't the student be tracked across different

centers?

For the first time in many years, I fully understand a report before I have to analyze it! The

lighthearted presentation kept my interest.

The content of the workshop was very interesting but I felt the presentation was a bit too silly for

me.

It was useful to see how the data will be provided and how to interpret it. XXXX and XXXX provideda light-hearted presentation to a topic that could have been dull and stale. =)

The presenters did a great job, but this just didn't answer the questions I needed answered. For

example, when a student comes to me -- knowing up-front they will only be able to enroll for 3-4

months, but really want to get started doing something... Why am I penalized for working with

them? I enroll them because it's the right thing to do, but my ALS is trashed by it.

Far too basic

Having my own report as a reference would have been ideal

Another session for new instructors with silly role play

No new material

There was no report. It was an overview of CPA. Most of people who attended the workshop had an

interest how to figure out CPA report points.

This information was already presented by the branch to each of us individually. I felt the majority ofthe audience was misled to believe the presentation to be about IPA.

The questions left me unsure on several areas. Can this be explained on a smaller venue such as

IMM?

Workshop I: Operating an Efficient & Profitable Franchise

Disappointing - way too basic!

Did not find any new info for me.

Gave me some good ideas to move ahead with, simple, direct and to the point

This was a very good workshop. Main speaker very effective.

Should have been called Running a Business 101...

Good explanation of basic accounting and financial analysis. Would like to have presentation onbusiness valuation (equity) next time.

It was an accounting lesson -- where was the part about becoming more profitable -- not just

tracking our costs and preparing a budget etc.

Should not be part of workshops could be part of general assembly

This presentation was not what it was advertised to be. Most disappointing element of the

conference for me. The presenter gave tips and strategies for how a new business owner shouldhandle budget management, not how an existing business owner (the VAST MAJORITY of the

audience) could adapt the business to become more efficient/profitable than it is today. Make sure

your workshop titles accurately reflect the presentations being given.

I wish I'd gone to this one too...

It was very basic. It would be nice to have different XXXXs for the workshops so that long term

instructors can get more out of it.

Nothing new


33/35

32

Very good info - this should be part of training and IMMs. We need this kind of additional support in

order to run a business. XXXX treats us like educators instead of business owners.

Very good.

Having attended training in the past year, I don't feel I was the target for this presentation.

Far too basic. This should have been flagged as most appropriate for new instructors.

Far too basic "This was basic info that should be presented at training and not at the conference."

This was essentially a review from training

Did nothing to provide help with operating any kind of Centre Basic accounting 101was useless to

even the newest instructor let alone those with 30 years business experience 10 of which running

million dollar budgets. Printing balance sheets and P&L statements and sticking them in a book

hardly rates up there with state of the art financial guidance. I sure hope the advice and counsel

provide XXXX on their money matters is of significantly higher quality

This was very good.

Entertaining and informative

This was too rudimentary for me...I was looking for guidance on analysis of the BS/PL or tips on what

areas should be watched to manage expenses better, etc. I probably should have attended H

instead.

He was excellent, answered key questions, and covered what's involved. VERY CLEAR.

Did not learn anything new

Good basic business understanding. We need more of this type of session. It would be more helpful

to drill it down even further rather than making it so broad.

This is more so a presentation on how to keep a good bookkeeping system. What are some

suggested percentages of our monthly/annual budget should be spent on what areas? What are

some successful marketing techniques that Instructors have used to PR their center? How do you

run a profitable franchise? These were questions I had prior to the presentation yet still remained

unanswered.

While I thought he was entertaining, I didn't think this would be Accounting 101. I have a tax

advisor, use Quick Books, know how to produce P&L's & Balance Sheets, etc. I would have liked to

hear about what XXXX thinks is an optimal staff to student (subject) ratio. Also, what is an optimal

payroll percentage vs. net profit, etc. Also, what is the formula for determining what your center is

worth, should you decide to sell it.

Too basic.

Not enough XXXX specific recommendations just general business tips.

Coming from a business background before being an instructor - I truly believe XXXX is doing a

disservice to new franchisees in not giving them enough information about running a business. The

business plan helps but it is a drop in the ocean compared to reality.

The speaker was good but this was an accounting workshop.

This had nothing about efficiency and profitability. It was an hour recap of my grade 10 accounting

course. I already have balance sheets and P&L statements - I was looking for ways of increasing thebottom line on these, not just a workshop on actually doing them. The speaker, however, was very

good.

Waste of time. I had hoped this would talk more about student:staff ratio, reasonable rent to pay in

relation to tuition, 12 month commitment or not, tuition payment plans, time management on the

part of the instructor, what I can or should not delegate... XXXX has a great printout available online

and I am studying it but want to learn more.

Seemed pretty basic.


34/35

33

Not bad but too basic for me. I wonder if you can give instructors the XXXX of lecture e.g.

beginner/intermediate/advanced

This was just a lesson in accounting jargon. The presentation should deal with cost cutting/control,mistake avoidance, ways to boost profit, etc.

The title is deceiving...please use better descriptions next time and it will help if you include what

type of instructor could benefit from it; i.e. brand new, veteran, etc. Helped me realize how important financial statements are.

Topic was misleading. I thought it was going to show you ways to efficiently run a profitable

business; instead it was more like an accounting class. It was OK.

Good, basic info. No really interesting tips though.

Workshop J: Understanding and Effectively Communicating with Prospective XXXX Parents

This session alone made the conference worth it, even if there had been nothing else. XXXX rocks.

I'm now focusing more on the "Involved Mom."

Very well done! Enjoyed this and obtained a lot of info from workshop.

Exciting prospects with the new ad company!!!

It was all about XXXX and social media. I expected something more comprehensive -- perhaps info

from exit surveys that we could turn around and use with prospective parents?

Did not get a lot from this...

I thought it was for the communication with perspective parents but somehow it covered more

marketing which we already attended in other workshop. I feel the name for the workshop was very

misleading.

Should not be a work shop should also be part of general info. I feel I missed out on opportunities to

learn from other instructors

I thought I was going to learn techniques to make me better at understanding and communicating

with parents not listen to the new marketing direction.

Interesting information. Well presented

I am looking for great things from this new agency. I hope they really understand the need to cut

down on the length of material we give to families. I really believe it rarely gets read.

Super plans on revamping web site and targeting our advertising. Can't wait to see it roll out.

Informative

It was mostly about understanding, not communicating.

Excited about going digital

Very excited to see new information that is coming soon.

I do not like the vendor to take one of our conference sessions for their advertisement. I would like

to learn more from instructors.

It's a good start, I was happy to see the staff wanting some feedback.

Hitherto, XXXX in NA has barely scratched the surface of the non-immigrant market. This serious

attempt by a commercial advertising company to target that market will begin a campaign to make

some inroad in that direction. Let's hope there will be positive and significant results!

Excellent. This workshop actually excited me and I look forward to the new websites and the ability

to communicate with parents more effectively.

My expectation was different from the presentation. The title of this presentation is misleading. Iexpected to learn how to effectively communicate with parents, not what's in store for us from

marketing perspective. However, I am glad I took this session. This was informative in another way

and I am excited to see the good thing coming to us.


35/35

Wasn't what I expected - thought it would provide more concrete info to give parents as well as

answering questions during enrollment session

Did not care for data regarding educated moms, etc....expected handouts on sample "here is how(centers)communicate with prospective parents"

I was turned off with the idea of just marketing to mothers earning $75000 and more. My town

doesn't have many of these mothers. We are looking for more specific communication skill improvement.

"This workshop was informative, but it was mistitled. It was all about the new advertising campaign.

I expected some additional advice about communicating with prospective parents who actually

come in to the center to see me."

Was disappointed with information.

The title was a bit misleading...the actual instructor communication w/parents was not addressed

fully

Date post:	06-Apr-2018
Category:	Documents
Upload:	latanyab1001
View:	218 times
Download:	0 times

505 Final Evaluation Project

Documents