___________________________________________________________________________
Implementation of Blackboard 9.1 (learn) on the Police Academy of the
Netherlands
Reinder Vrielink, MSc
Police Academy of the Netherlands
July 22nd, 2016
_________________________________________________________________________
Abstract
This research covers appreciation of the use of Blackboard among students and teachers on
the Police Academy of the Netherlands from 2011 up to and including 2015 with the focus on
the educational implementation of Blackboard 9.1(learn). The Technology Acceptance Model
(TAM) was used as a research method. The results of this research show due to the
educational implementation of Blackboard 9.1, appreciation in the use of Blackboard by
teachers has increased. Appreciation among students has decreased, possibly due to faltering
technology. The results furthermore demonstrate that implementation of ICT in education is
difficult and complex and arriving at a programme approach is therefore recommended.
Highlights
• Due to the educational implementation of Blackboard 9.1(learn), appreciation in the
use of Blackboard by teachers has increased.
• Appreciation among students has decreased, possibly due to faltering technology.
• Usefulness is the strongest predictor of the intention to start using Blackboard, and
enjoyment has a great impact on this as well. Enjoyment and Ease of use positively
effects behavioural intention through usefulness.
• Implementation of ICT in education is difficult and complex and arriving at a
programme approach is therefore recommended.
Keywords: Blackboard; behavioural intention; enjoyment; implementation; usefulness
___________________________________________________________________________ Corresponding author: email: [email protected]:
1
1. Introduction
1.1. The use of Blackboard on the Police Academy of the Netherlands/Previous research
The Police Academy of the Netherlands offers a basic training course (initial) and a further
training course (post initial). The institute covers six schools and nine trainings locations.
The School of Policing is one of the six schools and offers middle vocational education
and training, leading on to all-round Police Officer (initial, level 4). The use of
Information and Communication Technology (ICT) plays an important role in the course.
Education and training is supported by Blackboard (since 2002).
The appreciation of the use of Blackboard by students and teachers is not great. There is
grumbling that the use is difficult and student unfriendly. This is endorsed by previous
research (Vrielink, 2015). This previous research covers appreciation of the use of
Blackboard among students and teachers on the Police Academy of the Netherlands in the
period from 2008 up to and including 2011, using the Technology Acceptance Model
(TAM). From questioning the students and teachers, it became clear that the unclear
structure of Blackboard was a problem. Teachers scored (in 2011) significantly lower at
behavioural intention than students.
In 2013 Blackboard 9.1 (learn) was introduced. The reasons for this included: Blackboard
8 would no longer be supported; Blackboard 9.1(learn) would have a user-friendly
interface; the connection to Single Sign On would be easier, and the design and create a
better search function was simplified. The introduction comes with an implementation
plan (teaching and learning with Blackboard), in which there is specific attention for the
professionalization of management and teachers. The research was repeated at the end of
2014 and at the beginning of 2015, a year after the implementation of Blackboard
9.1(learn).
2
1.2. Review of Literature
Implementation of e-learning
There are relatively few scientific articles on the implementation of e-learning (Cross, J.
& Dublin, L., 2002; Netteland G., Wasson B., and Anders I., 2007).
Attwell (2005) claims that the restricted empirical research in the e-learning field has
mainly focused on the development of technology or product evaluation, and not on “what
works and what does not [work in a workplace environment]”. Rosenberg (2006)
underlined the importance of implementation. “It is not the technology which makes
learning challenging, but the way it is used and implemented. It’s the highway, not the
destination; the means rather than the end”.
The Four in Balance model (Kennisnet, 2008)
Ever since its creation, the Four in Balance model has proven its added value in the
implementation of ICT in Dutch educational institutions. The Four in Balance model
shows how schools can maximise the value of ICT. The model describes the preconditions
(building blocks) required to successfully use ICT in education.
Figure 1 The Four in Balance model
3
These preconditions (building blocks) include;
Vision: About the undeniable importance of leadership and vision during a change process
are many books written, e.g. by Kotter (1996). The view regarding high-quality and efficient
education and the position ICT occupies here. The vision comprises the overarching
ambitions. The direction is known. The organisation is managed on the basis of this vision.
Not only words, but also deeds. Displaying exemplary behaviour, participating in online
activities, support (resources, interventions), staying on course, and not losing sight of the top
of the mountain. Showing how important e-learning is through behaviour (Rubens, 2013).
Expertise: ICT expertise of teachers. Knowledge, skill and attitude in respect of ICT, in
pedagogical-didactic practice (Selim, 2007), working in the school context as well in
individual professional development. Are teachers capable of using technology?
According to Kirschner (2013), teachers become unsure if ICT ‘has’ to be deployed in
education.
• Teachers have no experience with ICT.
• Teachers do not have a good concept of what you can and cannot do with it.
They lack examples from their own training.
• What’s in it for me? If you have to explain why this innovation is required, this
is the deathblow to innovation.
• Teachers often had and still have insufficient facilities to use ICT in education.
• ICT is still too often regarded as an addition, instead of a normal tool to
provide education.
According to Kirschner (2013), the biggest challenge for teachers in this field is to properly
reflect on what type of education should be provided, and what learners have to learn. What
are the learning goals, how can they be achieved, and how do you test? This requires teachers
who are experts in the field of content, didactics and learning technologies. Teachers are
afraid of things they do not know and with which they have no experience. Having to explain
it is the kiss of death for educational innovation. Success factors are the necessary facilities,
with failure factors coming down to not having a clue. For example, if, in the case of
cooperative learning, the assignment is too simple, the student will prefer to do it by
themselves.
4
Content and applications: The information, educational content and software used in an
educational institution, such as:
• Digital learning material. The biggest challenge for teachers is determining
learning goals, including the didactics and technology used. A triangle between
teacher, student, and content. What content, which pedagogical approach and with
what technology. Why and how? (Hudson, 2008). (Hudson, 2008). A teacher is
excellent if they possess superb domain knowledge and understand how didactics
are to be applied. They are then capable of differentiating and providing
customised education (Kirschner, 2013).
• Educational software packages, such as an electronic learning environment,
student tracking system and student administration.
• Timetabling packages, apps, and so on.
Infrastructure: The availability and quality of hardware, networks and connectivity. The
hard ICT side, the infrastructure and systems, the design of the network. Wireless technology,
creating enough exit points, which facilitates working wireless with large groups of learners
as well. Interconnectivity between systems, so that certain data will not need to be entered
again in various systems such as an electronic learning system. Are enough suitable spaces
available in a blended situation in the event of a mixture of face-to-face and online learning?
Rubens (2013) adds the following building blocks to the Four in Balance model:
Curriculum and content development: what impact does e-learning have on the
curriculum? Will the existing curriculum be digitalised or will other learning activities be
developed now we have new technologies at our disposal?
Culture: one of the four Cs (Culture, Champions, Communication and Change) critical for an
implementation’s success in the introduction phase (Rosenberg (2001).
Does the culture focus on collaboration, on online learning together? In a hierarchical
organisation, attention will first have to be paid to cooperation, social learning and
collaborative learning.
Power relations and interests: it will have to be clear beforehand what roles there are during
the implementation. What role does ICT management have? Do we have to support existing
5
processes or do we have to adapt them first? What is the relationship between the ICT
department and the business plan? Who is the owner of the e-learning application? What is
the position of staff? Who directs who? Do staffs direct education or does education direct
staff?
Context: the implementation occurs within a certain context. What is the view in respect of
learning? Is it traditionally focused on courses and training programmes? What is the impact
of various social developments and the network society? What is the incentive to be involved
in e-learning?
1.3. Problem statement/Research question
As described in the introduction, there is little appreciation for the use of Blackboard among
students and teachers. With the introduction of Blackboard 9.1(learn) is the opportunity taken
to the focus on the educational implementation. The introduction comes with an
implementation plan (teaching and learning with Blackboard), in which there is specific
attention for the professionalization of management and teachers.
A project was designed for the introduction of Blackboard 9.1 (learn) steering group and a
project group were established. For the implementation, a leader and two key users were
appointed for every school. The aim of implementation of the latest version of Blackboard is
for application and updating of Blackboard to occur in such a way that the new educational
view is supported in the best possible way, i.e. better use of Blackboard in education with an
improved return on investment. Project teams were established for every school.
In order to realise a better application and use of Blackboard for every school as well as
Policy Academy-wide, the following results are aimed for:
1. The vision of Blackboard’s application in education being translated into objectives
and policy
2. Improved design, functionality and performance of Blackboard
3. Improved content management and functional management
4. Professionalization and ‘change’ for teachers, educational supervisors, experts and
managers
5. Guidance, organisational assurance and support: ownership, roles, responsibilities,
6
anchorage in the HR cycle and HR policy, guaranteeing continuous development.
6. Blackboard 9.1(learn) implemented according to a coordinated development plan
7. Implementation of new functions, in consultation with the schools:
a. Installed module, Blackboard Mobile
b. Installed module, Blackboard Collaborate (optional)
Research question:
Does the suggested project-based approach to the implementation of Blackboard 9.1(learn),
with the formulated results, produce more appreciation of the use of Blackboard among
students and teachers in 2014/2015 compared to the results of the comparative research from
2011 (Vrielink, 2015)?
This research, as well as previous research (Vrielink, 2015), was carried out among students
and teachers of the School of Policing (initial).
7
2. Material and methods
2.1. The Technology Acceptance Model (TAM)
In this paragraph the Technology Acceptance Model (TAM) is described, which Davis (1993)
introduced and which was modified by Yi and Yuong Hwang (2003). The TAM encompasses
a wide range of research objects, a brief list: website usage (Selim, 2003); the use of
Blackboard (Yi ad Yujong Hwang, 2003); ICT acceptance by therapist (Schaper and Pervan,
2004); online banking (Pikkarainen 2004); online shopping (Banus and Baptist, 2005); mobile
telephone (Aversano, 2005), and mobile commerce (Wu & Wang 2005). TAM has proven to
be a powerful model for explaining and predicting usage intentions and acceptance behaviour.
Legris (Legris et al, 2003) concluded that TAM can be regarded a useful model, but it has to
be integrated into a broader one which would include variables related to both human and
social change processes, and to the adoption of the innovation model. TAM does not
explicitly include any social variables. TAM provides a quick and inexpensive way to gather
general information about individuals’ perceptions of a system. TAM provides an information
representation of mechanisms by which design choices influence user acceptance, and should
therefore be helpful in applied contexts for forecasting and evaluating user acceptance of
information technology (Davis, 1993). TAM introduced two new constructs: perceived
usefulness (the belief that using an application will increase one's performance) and perceived
ease of use (the belief that one's use of an application will be free of effort). TAM theorizes
that an individual’s actual system usage is determined by behavioural intention, which is
determined by perceived usefulness and perceived ease of use.
8
Figure 2. Davis’ Technology Acceptance Model (TAM).
Recent findings on intrinsic motivation in social psychology indicate that enjoyment plays an
important role in determining a person’s behaviour. Yi and Yujong Hwang (2003)
investigated this feature (enjoyment) as external variables and their research shows that the
motivational variable enjoyment plays an important role in influencing the individual’s
decision whether or not to use a Web based technology.
According to Yi and Yujong Hwang (2003) the ease of use perceptions are influenced by the
degree to which people perceive using the system to be personally enjoyable. Agarwal and
Karahanna (2000) propose that the individual traits of playfulness and personal
innovativeness are important determinants of cognitive absorption. Performance is reached
when it joins up with enjoyment and learning in the same triangle. When there is enjoyment,
this will positively influence learning, which in its turn will lead to performance and so on.
Most of us know by experience that performance enhances when you are enjoying yourself.
In this manner, you are able to develop competencies (Gallwey, 1999).
external variables
behavioural intention
actual system
use
perceived usefulness
perceived ease of use
9
Criticism
According to Chuttur (2009) TAM has been widely criticised, despite its frequent use, leading
the original proposers to redefine it several times. Criticism of TAM as a "theory" include its
lack of falsifiability, questionable heuristic value, limited explanatory and predictive power,
triviality, and lack of any practical value. Benbasat and Barki (2007) suggest that TAM ‘has
diverted researchers’ attention away from other important research issues and has created an
illusion of progress in knowledge accumulation. Furthermore, the independent attempts by
several researchers to expand TAM in order to adapt it to the constantly changing Information
Systems (IS) environments has led to a state of theoretical chaos and confusion. In general
TAM focuses on the individual 'user' of a computer, with the concept of 'perceived
usefulness', with extension to bring in more and more factors to explain how a user 'perceives'
'usefulness'. TAM ignores the essentially social processes of IS development and
implementation, without question where more technology is actually better, and the social
consequences of IS use.
Despite the criticm above, TAM was chosen in this research because it is used worldwide.
TAM gives easy and quick results, partly because there is only one external variable chosen
(enjoyment) in this research. It was decided to use only one variable because this way the
questionnaire remains short. Experience from previous research shows that a longer
questionnaire consisting of questions where the link with Blackboard is not clear to students is
confusing to students.
Moreover, it is about a comparative research on the appreciation of the use of Blackboard
under different conditions (e.g. improved structure). A previously carried out comparative
research showed that this method is suitable for this (Vrielink, 2006).
2.2. Questionnaire
The preparation of the Blackboard research resulted in constructing a questionnaire in Dutch
for students, the same as the examination of Vrielink in 2008 and 2011 (Vrielink, 2015) is
used, consisting of 14 questions and the opportunity to make comments. All the questionnaire
items used an 11-point Likert-type scale where 0=completely disagree, 5=neither agree nor
disagree, and 10=completely agree.
The questionnaire consisted of three questions to measure the enjoyment construct; four
questions were used to measure the ease of use construct; four questions were used to
10
measure the usefulness construct and three questions were used to measure the behavioural
intention construct.
The actual use of Blackboard was measured through the course statistics of the control panel
by counting the number of times (frequency) a student logged into Blackboard courses in the
period of 3 months.
2.3. Data analysis
The reliability of the questionnaire was measured by examining the internal consistency. ICR
= Internal Consistency Reliability (similar to Cronbach’s alpha). The correlation coefficient
(Pearson’s product-moment correlation coefficient) is measured to show the connection
between the questions.
A T-test was carried out to find if there was a significant difference between the samples. The
T-test is based on an a-select random sample survey and on the assumption that the random
sample survey distribution is normal divided.
The structural model and hypotheses were assessed by examining the significance of the ß-
coefficients and the variance accounted for by the antecedent constructs. The standardized ß
coefficients (the coefficient of the independent variables when all variables are expressed in
standardized form) were presented. Multiple regression analyses were employed to adjust for
the influence of behavioural intention on use the ß-coefficient is measured with use as
dependent variable.
All statistical analyses were performed with the SPSS version 18.0. (SPSS Inc.).
11
3. Results
3.1. Respondents
In 2011, 213 initial students from six locations (average age 24.5, 32% female) and 71
teachers (average age 45.6, s.d. = 8.8, 32% female) completed the questionnaire. In 2015, this
involved 208 students (average age 25.2, 31% female) and 36 teachers (average age 49.7, s.d.
= 7.6, 28% female) respectively.
Not all respondents answered all questions (e.g. name, age, gender), as a result of which N
may deviate from the numbers above in the tables below.
3.2. Reliability of the questionnaire
The reliability of the questionnaire is expressed in ICR (ICR=Cronbach’s alpha). All ICRs
vary between 0.70 and 0.92. Internal consistencies (comparable to Cronbach’s alpha) of 0.70
or higher are considered adequate (Barclay et al., 1995).
3.3. The correlation between the questions
With correlation are the strength and direction of the relationship between two (or more)
interval/ratio variables expressed. Pearson's Product-moment correlations are expressed in
correlation coefficient r. The value of r is always between -1 and + 1. The higher the absolute
value, the stronger the relationship. Pearson’s product-moment correlation coefficient across
the board, r, varies from 0.44 to 0.84. This may be qualified as a high interconnectedness
(Lund and Lund, 2010).
While an ICR of 0.70 or higher is considered adequate, in relation with Pearson r, it can be
concluded that the questionnaire is a good tool for this comparative research.
12
3.4. Comparative research into the scores of students and teachers in 2015 and with
the help of the independent T Test.
Table 1: Comparison of the scores from students in 2015 and 2011. It shows that students in
2015 scored significant lower on the construct behavioural intention than students in 2011
Students 2015
N = 208
Students 2011
N = 213
Construct mean s.d. mean s.d. t Sig.
(2-tailed)
Enjoyment 4.6 2.1 4.8 1.8 - 0.8 ns
Ease of use 5.8 1.8 5.7 1.7 0.3 ns
Usefulness 5.5 1.9 5.8 1.6 - 1.7 ns
Behavioural
intention
4.5 2.0 6.1 1.7 -8.5 P < 0.001
Actual use 138 106 111 81 * *
*Because of a total change in the curriculum is a meaningless comparison. The training
duration has been shortened from four to three years. Also the rhythm changed.
Table 2: Comparison of the scores between students and teachers in 2015. It shows that
teachers scored significantly higher on usefulness and behavioural intention than students.
Students 2015
N = 208
Teachers 2015
N = 36
Construct mean s.d. mean s.d. t Sig.
(2-tailed)
Enjoyment 4.6 2.1 4.9 1.6 -0.7 ns
Ease of use 5.8 1.8 5.5 1.7 1.0 ns
Usefulness 5.5 1.9 6.3 1.7 -2.7 P<0.01
Behavioural
intention
4.5 2.0 5.9 1.7 -4.2 P < 0.001
13
Table 3: Comparison of the scores between teachers in 2015 and in 2011. It shows that
teachers in 2015 scored significantly higher on the items enjoyment and behavioural intention
than teachers in 2011.
Teachers 2015
N = 36
Teachers 2011
N = 71
Construct mean s.d. mean s.d. t Sig.
(2-tailed)
Enjoyment 4.9 1.6 4.0 2.0 2.0 P<0.05
Ease of use 5.5 1.7 5.2 2.0 0.6 ns
Usefulness 6.3 1.7 5.5 2.2 1.8 ns
Behavioural
intention
5.9 1.7 4.9 2.4 2.2 p < 0.001
14
3.5. Regression analysis for testing the suggested model
Figure 4: Regression analysis for testing the suggested model
Figure 4 and table 4 show the results of the regression analysis
6
Usefulness
Behavioural Intention
Ease of Use
Enjoyment
Use
2
5
1
3
4
15
Table 4: Comparison of beta coefficients. Table 4 shows the results of the measured β
coefficients. Behavioural intention has a significant effect on Use. Usefulness has a
significant effect on Behavioural intention and Ease of use has a no significant effect on
Behavioural intention . Enjoyment has a significant effect on Usefulness and Ease of use has a
significant effect on Usefulness. Enjoyment has a significant effect on Ease of use.
2015
N = 137
2011
N = 186
1 0.19* 0.29*
2 0.73*** 0.59*
3 ns ns
4 0.35*** 0.22*
5 0.53*** 0.52**
6 0.53*** 0.58***
|ns = not significant| |* p<0.05 | | ** p<0.01| | *** p<0.001|
3.6. Comments from students in Blackboard research (2015)
Table 5: Comments from students categorised. Table 6 indicates that in 28% of cases students
complain about poor displaying of the schedule. Subjects 1 and 6 are to do with technology.
Comments from students Percentage
1 Display of the schedule is problematic 28
2 No structure, difficult to find things 23
3 Announcements are not always for one’s own group 17
4 Search function is missing 8
5 Teachers make insufficient use of Blackboard 8
6 Slow, freezes, system error 6
7 Good program 6
8 Other 4
16
4. Discussion
The aim of the comparative research was: Does the suggested project-based approach to the
implementation of Blackboard 9.1(learn), with the formulated results, produce more
appreciation of the use of Blackboard among students and teachers in 2014/2015 compared to
the results of the comparative research from 2011 (Vrielink, 2015)?
From table 1 it becomes clear that students from the 2015 population score significantly lower
at behavioural intention than students from the 2011 population. The scores in 2015 could be
attributed to faulty technology (see the comments by students, table 5). There were problems
with the Single Sign On making the applications on the eCampus, including Blackboard. The
problems are now solved.
Table 2 shows that students in 2015 intended to use Blackboard less often than the teachers in
2015! Students in 2015 scored lower on usefulness than their teachers. Teachers in 2015
intend to use Blackboard more often than the teachers from 2011 and they scored better on the
construct enjoyment too (table 3). The implementation of Blackboard 9.1 (learn) had a
positive effect on teachers and a negative effect on students. In General on the goals and
results (paragraph 1.3), they are somewhat vague and may be more specific. In the goals is
involvement of the student in the design and use of Blackboard missed. Teachers and
managers are explicitly mentioned (paragraph 1.3, result 4). Involvement of the student is not
mentioned.
Teachers give scores of 5.9 (11-point scale) for the intention to use Blackboard in 2015. This
is encouraging. Perhaps this concerns a select vanguard of teachers.
The attitude of the teacher plays an important role Albirini (2006). Whether an innovation
succeeds, strongly depends on the question if teachers are willing and able to further develop
their competencies, and if teachers are able to promote the competency development of their
teachers (Miedema and Stam, 2008). This is also in accordance with the observation by
Ertmer (Ertmer et al., 2012) that the personal convictions and attitudes of teachers regarding
the relevance of technology for students were considered to be the biggest impact on their
success.
It appears from table 4 that the construct ‘usefulness’ is the most powerful one for the
intention to use Blackboard. This is in line with the results of the research by Pei-Chen (Pei-
17
Chen et al., 2008) where perceived usefulness was also one of the critical success factors in
perceived satisfaction for students. In addition, the construct ‘enjoyment’ scores high.
Improvements in the use of Blackboard therefore need to be sought in the areas of
‘usefulness’ and ‘enjoyment’. In general, students consider Blackboard to be user-unfriendly,
as a result of which they no longer experience the usefulness and enjoyment, as a result of
which they quit. This corresponds with the findings of Timothy and Jones (2011): “It is
important to ensure that teachers acquire the necessary technical skills for the performance of
their role as a teacher, and it is important for teacher trainers to focus on the development of a
positive perception of usefulness and the extent to which technology is easy to use; both these
elements are strongly influenced by perceived enjoyment.”
It appears from table 4 that Ease of Use does not affect the intention to start using Blackboard.
Only perceived ease of use is not recognised as one of the critical factors affecting learners'
perceived satisfaction. This result is in line with the findings of the studies of online banking
(Pikkarainen 2004), and mobile commerce (Wu & Wang 2005). Perceived ease of use is less
likely to be a determinant of attitude and usage intention. According to of Donghue (2009) the
effect of ease of use on intention to use and actual use was mediated by perceived
usefulness. Ease of use is not the students’ major concern for deciding to use the e-resources
because they had some experience of using e-resources and knew how to use them. In the
study of Okantey and Addo (2016) it was also observed that significant relationship existed
between Perceived Usefulness and e-learning, while the weakest relationship existed between
e-learning and Perceived Ease of Use.
Usefulness is the strongest predictor of the intention to start using Blackboard, and enjoyment
has a great impact on this as well. The results of the study clearly point out the important roles
of enjoyment and usefulness in positively influencing the discussion to use Blackboard and
subsequent actual use. These findings significantly extend prior research on user acceptance
of web-based technology (Yi and Yujong Hwang, 2003; Selim, 2003; Shu-Sheng Liaw, 2008)
and empirically validating the relationship with enjoyment as the external variable.
Enjoyment is a powerful external factor which positively effects behavioural intention
through usefulness. This corresponds with the findings of Yi and Yujong Hwang (2003) and
Goetz et al. (2006). Enjoyment might play a more influential role than ease of use in
determining the usefulness perception within the Web-based IS context.
18
From table 5 – comments by students – it appears that students complain about technology
(schedule, announcements, failure), about the faulty structure and about the fact that teachers
do not work with Blackboard often enough. This possibly explains the low scores from
students. Moreover, table 1 and 5 shows that improve the performance of Blackboard are not
entirely successful. The development of policy at central level (project team) and then hand
over to the schools (key users) fails often because schools (managers) also have their own
agenda. Therefore, members of the project team have to remain present in the
implementation, in order to support managers and key users, answer questions, motivate and
so on. Salmon (2005) notice “the difficulty of truly ‘embedding’ e-learning into everyday
practice including uncertain leadership, lack of true innovation and lack of professional
expertise”.
Technical problems were also observed in the research by Al-Drees (Al-Drees et al., 2015)
which mentions technical problems in relation to Blackboard. The finding of this study
showed a poor utilization of Blackboard learning features. Students faced technical
difficulties while using Blackboard. The findings of this study indicate the need of
compulsory students training of any newly introduced learning management systems
including Blackboard in the educational institute.
In relation to the Four in Balance Model the balance is disturbed. The project approach of the
implementation of Blackboard 9.1 (learn) with project teams for every school, in such a way
that the new educational view is supported in the best possible way, has not led to a better use
of Blackboard in education by students’. This approach goes only on for the teachers’
population. The implementation of learning technology is apparently more complex than it
was thought.
According to the “Four in Balans Monitor 2013” three quarters of the schools in the
Netherlands indicates that they are changing the education and search for the possibilities that
ICT has to offer. The need where the education as a whole stands for is: minimize wrong
choices using knowledge about what works with ICT. The wrong expectation that young
people, many with ICT, also spontaneously have the skills to find information, to understand
and use (Kirschner, 2013). Lack of understanding of what works with ICT and what not,
hampers schools to make good choices.
According to Rubens (2015), the introduction of technology enhanced learning
19
is a complicated process, in which a large number of factors and actors play an important role.
It is primarily a process in which the human factor plays an important role, like dynamics,
interests, relationships and shared leadership. The multitude of actors makes education
difficult to predict and plan an innovation dynamic and fascinating process”.
Rubens (2013) recommends tackling the implementation process on a project basis. This
conflicts with what Assen & van der Pol (2008) argue: “In our experience, what successful
implementations have in common is that they do not work in terms of a project, but in terms
of a multi-year programme. E-learning involves organisational change and this cannot be
achieved with one project.” They also argue that “while it is important to have ICT on board,
e-learning is often overlooked on the strategic ICT agenda. As a result, the development of
internal ICT expertise for e-learning often lags behind and the tasks to be performed by ICT
tend to incur delays”.
According to Jacobs (2013), in the Netherlands, the use of ICT in learning environments often
depends on improvisation, coincidences, individual commitment and isolated projects.
Teachers and education managers often do not have an interconnected vision on the
application and impact of ICT in education. Viewed in light of the intensive use of ICT in
society, education lags behind.
Students abandon Blackboard and they are using other tools like Facebook, WhatsApp and
Twitter. There are becoming more free cloud solutions as alternative for Blackboard. This
allows the Police Academy called for a new dilemma. Continue with Blackboard? The cloud
is safe enough for the Police Academy? The finding of Maleko (Maleko et al., 2014) was that
Facebook attracted more students (over Blackboard) due to its social and community learning
benefits, encouraging students to support one another. Blackboard was viewed as the
authoritative and valid medium for official course material. Social media is not the answer to
every problem, but it’s an undoubtedly a useful and powerful advance in learning’ (Clark,
2011). Moreover, is there a growing gap between learners and teachers in the way they use
the Internet? (Atwell, 2011).
20
5. Conclusions and recommendations
The introduction of Blackboard 9.1 (learn) had a positive impact on the intention of teachers
to use Blackboard. Significant progress may be perceived in the intention of teachers to use
Blackboard.
The results show that, among students, the appreciation of the use of Blackboard significantly
decreased in 2015 compared to 2011. Is this only due to technical problems when showing the
timetable? Have students been sufficiently involved in the design and formulation of the
results?
What did the Police Academy of the Netherlands learned from this research?
• There has been an insufficient focus on implementation at the level of the students.
Involve students in setting up and using Blackboard
• It is recommended to spread to large changes in the time. Too many changes,
(introduction Blackboard 9.1(learn), Single Sign On, new student administration
system, new curriculum, new schedule program, many management changes) at the
same time addressing are asking for trouble.
• Make a programme for the implementation of ICT.
• Teachers who form a vanguard may be used as ambassadors within the school by
sharing good practices
• Members of the project team have to remain present in the implementation, in order to
support key users, answer questions, motivate and so on.
• Looking at "four in balance model", it can be stated that the balance is disturbed. It is
recommended to invest in the professional skills of managers and teachers. It is also
recommended to use instructions in the teacher’s manuals for Blackboard use. Above
all, technology has to be perfect
Although there are many more questions than answers regarding teaching with Blackboard as
learning management system, it is hoped that these results and experiences may encourage
further pedagogical dialogue, and empirical results about how to deliver and organize courses
in this technological environment effectively and successfully.
21
References
Agarwal R and Karahanna E. (2000). Time Flies When You're Having Fun: Cognitive
Absorption and Beliefs about Information Technology Usage. MIS Quarterly, Vol. 24, No. 4
(Dec., 2000), pp. 665-694
Albirini, A. (2006) Teachers’ attitudes toward information and communication technologies:
the case of Syrian EFL teachers. Computer & Education. Volume 47, Issue 4, December
2006, Pages 373-398
Al-Drees, A., Khalil, M.S., Meo, A. and Abdulghani, H.M. (2015). Utilization of blackboard
among undergraduate medical students: Where we are from the reality? Journal of Taibah
University Medical Sciences. Volume 10, Issue 1, March 2015, Pages 16–20.
doi:10.1016/j.jtumed.2014.07.002
Assen, D. en van der Pol, R. (2008). Duurzaam implementeren van e-learning.
Leerinfrastructuur realiseren voor lerende organisaties. Thema Nieuwe Technologie
DEVELOP NR 4-2008 blz. 44 – 55.
Attwell, G. (2005), “E-learning and sustainability”, available at: www.ossite.org/Members/
GrahamAttwell/sustainibility/attach/sustainibility4.doc (accessed 1 September 2006).
Atwell, G. (2011) Pedagogic Approaches to using Technology for Learning – Literature
Review Last accessed at June 12 2011 at URL:
http://www.pontydysgu.org/2011/05/pedagogic-approaches-to-using-technology-for-learning-
literature-review
Aversano, N. (2005). Technology rejection of mobile telephones. Case Western Reserve
University.
Benbasat, I. and Barki, H. (2007). Quo vadis, TAM? Journal of the Association of
Information Systems 8 (4)): 211–218.
Banus S. and Baptist T. (2005). De Acceptatie van Online Shoppen. Bachelor Thesis
Marketing. Radboud Universiteit Nijmegen. Netherlands
22
Barclay, D., Higgins, C., Thompson, R. (1995). The partial least squares approach to causal
modelling: personal computer adoption and use as an illustration. Technology Studies 2, 285-
309.
Chuttur M.Y. (2009). Overview of the Technology Acceptance Model: Origins,
Developments and Future Directions. Indiana University, USA . Sprouts: Working Papers on
Information Systems, 9(37). http://sprouts.aisnet.org/9-37
Clark, D. (2011) 7 objections to social media in learning (and answers) Last accessed at June
12 2011 at URL: http://donaldclarkplanb.blogspot.com/2011/06/7-objections-to-social-media-
on.html
Cross, J. & Dublin, L. (2002). Implementing e-learning. Alexandria: ASTD
Davis. F.D. (1993). User acceptance of information technology: system characteristics, user
perceptions and behavioural impacts. International Journal of Man–Machine Studies 38, pp.
475–487.
Donghua T. (2009) Intention to Use and Actual Use of Electronic Information Resources:
Further Exploring Technology Acceptance Model (TAM). AMIA Annu Symp Proc. 2009;
2009: 629–633. Published online 2009 Nov 14.
Ertmer , P.A., Ottenbreit-Leftwich, A.T., Sadik, O., Sendurur, E., and Sendurur, P. (2012).
Teacher beliefs and technology integration practices: A critical relationship. Computers &
Education Volume 59, Issue 2, September 2012, Pages 423–435
Gallwey, W. T. (1999). The Inner Game of Work. Random House, New York.
Goetz, T., Hall, N. C., Frenzel, A. C. and Pekrun, R. (2006) A Hierarchical
Conceptualization of Enjoyment in Students. Learning and Instruction, v16 n4 p323-338 Aug
2006
23
Hudson, B. (2008). Didaktik Design for Technology Supported Learning. Zeitschrift fur
Erziehungswissenschaft, Vol. 9. pp.139-158
Jacobs, F. (2013). Slagvaardig met ICT. Proefschrift TU Delft
Kennisnet (2008/2013). Vier in Balans Monitor 2008/2013. Kennisnet
Kirschner, P. (2013). Knopvaardig is wat anders dan digitaal geletterd. In: 4W. Weten wat
werkt en waarom (2-2013).
Kotter J.P. (1996) Leading Change. Havard Business School. Press, Boston, Massachusetts,
USA.
Legris, P., Inghamb, J. and Collerette P. (2003). Why do people use information technology?
A critical review of the technology acceptance model. Information & Management 40 (2003)
191–204.
Lund, A and Lund, M. (2010). Laerd Statistics. Lund Research Ltd 2010 Last accessed on
13 December 2011 on URL: http://statistics.laerd.com/statistical-guides/pearson-correlation-
coefficient-statistical-guide.php
Maleko M., Hamilton M., D'Souza D. and Scholer F. (2014) Facebook versus Blackboard for
Supporting the Learning of Programming in a Fully Online Course: The Changing Face of
Computing Education. Learning and Teaching in Computing and Engineering (LaTiCE)
International Conference
Miedema, W.G. and Stam, M. (2008). Leren van innoveren: wat en hoe leren docenten van
het innoveren van het eigen onderwijs? Thesis. Universiteit van Amsterdam
Netteland G., Wasson B., and Anders I. (2007) E-learning in a large organization A study of
the critical role of information sharing. Journal of Workplace Learning Vol. 19 No. 6, 2007
pp. 392-411
24
Okantey M and Addo H. (2016) Effect of theoretical and institutional factors on the
adoption of e-learning. European scientific Journal Vol 12, No 16. Last assessed on 5 July
2016 at URL: http://www.eujournal.org/index.php/esj/issue/view/235
Pei-Chen Sun , Ray J. Tsai, Glenn Finger, Yueh-Yang Chen and Dowming Yeh (2008). What
drives a successful e-Learning? An empirical investigation of the critical factors influencing
learner satisfaction. Computers & Education Volume 50, Issue 4, May 2008, Pages 1183–
1202
Pikkarainen, T.; Pikkarainen, K.; Karjaluoto, H. (2004). Consumer acceptance of online
banking: An extension of the Technology Acceptance Model. Internet Research-Electronic
Networking Applications and Policy 14 (3): 224–235,doi:10.1108/10662240410542652
Rosenberg, M. (2001), E-Learning: Strategies for Delivering Knowledge in the Digital Age.
McGraw-Hill, New York, NY
Rosenberg, M. (2006) Beyond E-learning: Approaches and Technologies to Enhance
Organizational Knowledge, Learning and Performance. Last accessed on 16 July 2004 at
URL: http://www.learningcircuits.org/2006/March/rosenberg.htm
Rubens, W. (2013). E-learning. Trends en ontwikkelingen Uitgeverij InnoDoks
Rubens, W. (2015). Geleerde lessen op het gebied van onderwijs innovaties. Blog:
Technology Enhanced Learning. Last accessed on 23June 2015 on URL:http://www.te-
learning.nl/blog/geleerde-lessen-op-het-gebied-van-
onderwijsinnovatie/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3
A+typepad%2FfIYD+%28Wilfred+Rubens+TE-
learning+centrum%3A+technology+enhanced+learning%29
Salmon, G. (2005) Flying not flapping: a strategic framework for e-learning and pedagogical
innovation in higher education institutions. ALT-J, Research in Learning Technology Vol. 13,
No. 3, October 2005, pp. 201–218
25
Schaper, L and Pervan, G. (2004). A Model of Information and Communication Technology
Acceptance and Utilisation by Occupational Therapists. The IFIP TC8/WG8.3 International
Conference 2004.
Selim, H. M. (2003). An empirical investigation of student acceptance of course websites.
Computers & Education 40, 343-360.
Selim, H. (2007). Critical success factors for e-learning acceptance: Confirmatory factor
models. Computers & Education, 49, 396–413. doi:10.1016/j.compedu.2005.09.004
Shu-Sheng Liaw (2008). Investigating students’ perceived satisfaction, behavioural intention,
and effectiveness of e-learning: A case study of the Blackboard system. Computers &
Education Volume 51, Issue 2, September 2008, Pages 864–873
Timothy T., and Noyes J. (2011). An assessment of the influence of perceived enjoyment and
attitude on the intention to use technology among pre-service teachers: A structural equation
modeling approach. Computers & Education, Volume 57, Issue 2, September 2011, Pages
1645-1653
Vrielink, R. (2006). Predicting the use of Blackboard and predicting the use of a Personal
Digital Analyser with the Technology Acceptance Model. Current Developments in
Technology-Assisted Education. Formatex 2006. Vol. 1. pp. 591 - 595.
Vrielink, R. (2015). Appreciation of the use of Blackboard at the Police Academy of the
Netherlands using the Technology Acceptance Model. The Journal of Technology Enhanced
Learning, Innovation & Change, 1 (1)
Wu J.H. and Wang S.C (2005) What drives mobile commerce? An empirical evaluation of
the revised technology acceptance model Information & Management 42 (2005) 719–729
Yi, Mun Y. and Yujong Hwang. (2003). Predicting the use of web-bases information systems:
self-efficacy, enjoyment, learning goal orientation, and technology acceptance model.
International Journal of Human-Computer Studies. 59, 431-449.