¿Hacia un nuevo paradigma de comunicación en colaborative learning?

Gustavo Zurita1, Nelson Baloian2, Oscar Jerez3 and Sergio Peñafiel2

1 Department of Information Systems and Management Control, Faculty of Economics and Business, Universidad de Chile, Diagonal Paraguay 257, Santiago, Chile.

gzurita@fen.uchile.cl2 Department of Computer Sciences, Universidad de Chile, Beaucheff 851, Santiago, Chile.

nbaloian@dcc.uchile.cl,spenafie@dcc.uchile.cl 3 Teaching and Learning Centre. Economics and Business Faculty, Universidad de Chile, Di-

agonal Paraguay 257, Santiago, Chileojerez@fen.uchile.cl

Abstract. Reading comprehension is essential for students, because it is a predictor of their academic or professional success, however, it is challenging for many students, even more if they are part of large classrooms. This paper presents a work which uses …. … a collaborative learning activity to develop the skills of reading comprehension of 12th

grade students and, along with an application for iPads supporting teacher and students in the execution of this activity. …

Keywords: Reading comprehension, Collaborative Learning, Multiple Selection.

1 Introduction

During the last years, statistics show a constant decline in reading and writing illiteracy in the whole world, and especially in developing countries. For example, the webpage https://our-worldindata.org/literacy/ contains several charts showing how the rate of illiteracy has been falling in many some countries in all continents. The tendency turns dramatically sharp during the second half of the last century.

However, although more people can read a text, the reports are so encouraging when it comes to reporting results about the understanding that people in their school ages achieve about the texts they read [1]. Also, in developed countries we can see same concerns about people understanding the content of what they read. As an example in [2], it is reported in that USA "far too many American students remain poor readers"

In the past, computer systems have been developed to support training of reading com-prehension with reported good results. In [3] authors identify the advantages of using com-puter-assisted programs in educational programs for children as greater motivation [4] and a higher level of attention [5]. CA reading programs are mostly aimed at supporting the learning of strategies that the experts in reading comprehension learning have identified as conducive to improve it [6]. Some of the most used are to have students read a text trying to determine the main message of this by means of summaries or keywords, have the students construct al-ternative representations of the text, such as drawings, conceptual maps, mental images) and answer questions about the text [7].

2

The authors of the present work have also developed a work which takes this second di-rection and was based on the strategy of training the reading comprehension by highlighting the words inside a piece of text which represent the key idea exposed by it [8]. However, this experience has shown that the approach of using constructed development responses has its disadvantages, especially when applied to mass courses.

In this sense, an activity in which the students can respond through the selection of multi-ple alternatives, would make the whole process easier; since the evaluation of the correctness of the answers are easy to validate. Multiple choice based tests have some advantages [9] And critics [10] related to the fact that a constructed response is supposed to require more complex skills on the part of the student; and therefore allows a student to perform a more elaborate learning activity.

However, in [11] and [12] the authors show that there is equivalence between constructed responses, to those of alternatives. The justification for the above is that for these authors, one way to achieve response is that the student must first build a response, then verify/check against possible alternative responses.

In various papers, authors describe experiments in which students answering a multiple-choice questionnaire are asked to justify their decision for different reasons, like detecting false positive (choosing the right answer for a wrong reason) [13] or for stressing the reason-ing process of the student [14].

In this work, we wanted to explore how does justifying the answer to a multiple-choice questionnaire affects the performance of the students and if the quality of the comment pro-vided by them is relevant. For this, we designed a learning activity in which students have to read texts and answer a multiple-choice questionnaire. They also have to provide a short text justifying their choice for the right answer.

Based on [15], we can derive that this activity can be done collaboratively to take advan-tage of the benefits offered by Collaborative Learning (CL) not only in the academic but also in the social and psychological spheres. Therefore, this activity also includes a collaborative component in which students have to share and analyze their own and their classmates’ an-swers as well as the justification text.

In order to support this learning activity, we developed a technological tool named Red-CoMulApp by a Multiple Choices (described in detail in section 3).

The content of this article is organized as follows. Section 2 explains the relevant the-ories, methods and techniques used as design requirements for the reading comprehension activity converted in a formative activity based on the Chilean national selection test called PSU. Section 3 details the design of the reading comprehension activity, along with the description of the RedCoMulApp application. Finally, section 4 describes the prelimi-nary experiments and section 5 concludes the paper.

2 From Multiple Choice to reading comprehension learning activity

Aa previously sad, in the past we developed a The authors of the present work have also developed a work which takes this second direction and was based on the strat-egy of training the reading comprehension by highlighting the words inside a piece of

3

text which represent the key idea exposed by it [6]. However, this experience has shown that the approach of using constructed development responses has its disadvan-tages, especially when applied to mass courses. Some of them are the following:

- I it is complicated for students to develop a response built in mobile computer systems., - I it is expensive for the teacher to evaluate the answers developed by all the students. - It is difficult to monitor the degree of progress of the answers given by the students: a)

how many have responded, and if b) they have responded well.- It is costly in effort and time for the teacher to give appropriate feedback to each stu-

dent based on their answers.

In this sense, an activity in which the students can respond through the selection of multiple alternatives, would make the whole process easier; since the evaluation of the correctness of the answers are easy to validate. In fact, according to [10], the ad-vantages of multiple choice tests are:

- They are easy to apply. - Their results are trustworthy. - Because they are standardized, they are applied in the same way to all students. - They are objective.

However, this and other authors [19] [9] also express some critics

- It is possible to choose the right answer without real knowledge. - Normally, students do notr receive specific feedback específico, appartapart from the

general results. - The examinator evaluator could be a non-expert in the subject and decide to take out

questions without a right criteriaa right criterion.

The critics are related to the fact that a constructed response is supposed to require more complex skills on the part of the student; and therefore, allows a student to perform a more elaborate learning activity. However, in [11] [16] and [12] [17] the authors show that there is equivalence between constructed responses, to those of alternatives. The justification for the above is that for these authors, one way to achieve response is that the student must first build a response, then verify/check against possible alternative responses. En [13], [21] los autores argumentan que en una prueba de multiple choice no solo se debe marcar la alternativa que el estudiante considera correcta sino que además hay que justificarla.

In Chile there is a single test of income selection for almost all universities, whether pri-vate or state, called PSU. An important part of it consists of reading comprehension, measured by multiple-choice questions. This evaluation is summative, that is, it is meant to measure what the students know. Based on [23] [14], It is possible to convert an activity with summa-tive evaluation into a formative one if the evaluation is used as feedback for the student to re-flect on and reformulate their original answers. Moreover, this can be done collaboratively to take advantage of the benefits offered by Collaborative Learning (CL) not only in the aca-demic but also in the social and psychological spheres [15][22]:

Social benefits:;

- CL helps to develop a social support system for learners; - CL leads to build diversity understanding among students and staff;

4

- CL establishes a positive atmosphere for modelling and practicing cooperation, and; - CL develops learning communities.

Psychological benefits:;

- Student-centered instruction increases students' self-esteem; - Cooperation reduces anxiety, and; - CL develops positive attitudes towards teachers.

Academic benefits:;

- CL Promotes critical thinking skills. - Involves students actively in the learning process. - Classroom results are improved. - Models appropriate student problem solving techniques. - Large lectures can be personalized. - CL is especially helpful in motivating students in specific curriculum.

Given the usability problems that we had in the first version of this system (described in [10] [8]) in this work we report a finished study of the usability and perception of utility that the students found in the system

1.1 Reading Comprehension

Students who use reading comprehension strategies (such as prediction, think-aloud, text structure, visual representation of text, key words selection, etc.), improve their under-standing of the message read, identify the essential and relevant message of the text, and/or are able to express opinions [16].

According to the reasons mentioned above, the RedCoMulApp design that we pro-pose (see section 3) will:

1) Use the advantages of the use of short messages (microblogging). 2) Implement real-time monitoring to manage the follow-up of the elementary

stages. 3) Implement collaborative learning with groups of 2 to 5 students who will work

together to answer the multiple choices questions. 4)

In an educational context, short messages (microblogging, or tweets) can be used to express ideas, paraphrase or critique a concept, [17]. Short messages provide support for the collaborative work of the students, as they facilitate posing questions, share ideas and send answers.

One of the main contributions of software applications as a scaffolding for learning activities is the real-time monitoring that the teacher can have on the level of progress

5

and achievement of his students, allowing her to act as a catalyst to produce changes in the educational activity or in pedagogy [18].

Nowadays, university leaders are recognizing the need for collaborative learning inside of classroom, to bolster student success [19]. The goal of collaborative learning technique is to support learning for a specific educational objective through a coordinated and shared activity, by means of social interactions among the group members [20]. Research has shown that the proper design of collaborative learning tasks can improve motivation levels, facilitate communication and social interaction [21], support coordination and increase the level of students’ learning achievement [22, 23], and facilitate face-to-face work supported by mobile devices [21, 24, 25].

3 Design of the reading comprehension activity: RedCoMulApp

This section describes the design of the collaborative application RedCoMulApp to sup-port of reading comprehension, which can be used under two roles: teacher (see section 3.1) and student (see section 3.2).

1.2 Teacher’s role

The teacher’s role allows creating the learning activity, and a real-time monitoring of the task development. This allows the teacher to monitor the status of progress of the learning activity that students are performing at “Iindividual”, “Aanonymous” and “Tteam Wwork” phases (see these phases at the second line of the top of the main interface in Fig. 1).

For the creating of the learning activity, a teacher performs following actions using the “Editor” option of the RedCoMulApp (see this option at the top of Fig. 1):

- Input the title of the activity and writing a text specify the general instructions. - Upload a text used as a context for the multiple selections questions. - Introduce the multiple selection questions with their corresponding right answers. - Using the “Users” option, a teacher assigns the students that will participate in the ac-

tivity. - With the “Groups” option, the teacher assigns the task to work teams, each one com-

posed of three or two students.

For the real-time monitoring of the task development, the teacher has access to rel-evant information during the execution of the learning activity in order to identify the state of progress of the students in each of the three phases by using the "Dashboard" op-tion. For example, the teacher can know in the “Iindividual” phase, how many students have chosen the correct answers (see the bar diagrams of Fig. 2); or in the “Tteam work” phase how many work teams have already completed all multiple selections questions (see Fig. 3). The information presented to the teacher will be shown by easy to understand graphic interfaces, such as comparative tables or matrices, bar charts, etc. (see Fig. 2 and Fig. 3), that is used by the teacher to decide whether to move to the next phase or wait for a significant number of students to complete the current activity stage. In addition, this in-

6

formation allows the teacher to identify the level of achievement of students in each phase, according to answers correctly chosen by the students. For example, if at the “Iin-dividual” phase there less than 1/3 of the students have successfully completed to answer their questions, the teacher may proceed to intervene the class, offering feedback to ex-plain the about the questions, explain the context of the texts, etc.

Using the options "Configuration", "Individual", "Anonymous" and "Team Work", the teacher manages the development of the application. In the initial phase of "Configu-ration”, the teacher can create the learning activity.

Fig. 1. View of the interface for the teacher’s role, showing the tile of the activity the instructions, the text and the multiple-choice questions with their answers. Using the “Users” option the task is as -signed to the students and the “Groups” option is used to configure the groups. Using the “Dash-board” option the teacher monitors the activity (see Fig. 2). The second line shows the stages of the activity “Configuration”, “Individual”, “Anonymous”, “Team Work”, and “Finished”. The current one is highlighted in yellow: “Finished” en este casoin this case. The “Go to Next” option advances from one stage to another.

Then, when the teacher selects the option "Go to Next" (see Fig. 1), the phase changes to "Individual", which is when the students receive the text to be read and the questions to answer in an individual way. Once all the students have finished responding, with the "Go to Next" option the teacher changes the task the status from "Individual" to

7

"Anonymous"; where the students answer the same questions once again but having anonymous access to the answers of other two of their classmates (see Fig. 4), according to the groups that were formed in the "Groups" option. Then, with the "Go to Next" op-tion changes the teacher the status from "Anonymous" to "Team Work". In this stage stu-dents meet face to face and together they answer the same multiple-choice questions, on the base of the answers previously given in the "Individual" and "Anonymous" phases and the conversation, exchange of opinions that the students have to make in order to agree on a single answer.

Fig. 2. A view of the monitoring tool showing the performance of the students with the correct an-swers in green and the wrong answers in red. This is from a real activity performed on august 22 of 2017 by 22 students, section 1. This view shows the results of the "Individual" phase, where stu-

8

dents got an achievement of mostly 40%-60%. Although this view shows that the RedCoMulApp has finished (status "Finished" in the label in yellow), it is possible for the teacher to access to see the students’ performance, since the view is in with "Dashboard" option activated, and has chosen the "Individual" phase or iteration. In Fig. 3 are shown for this same state, the phases "Anonymous" and "Team Work"

Fig. 3. Two views of the monitoring tool showing the performance of the students with the correct answers in Green and the wrong answers in red for “Anonymous” (top view covered partially) and “Team Work” (bottom view superimposed) phases. This is from a real activity performed on august 22 of 2017 by 42 students. The views show the activity has finished. The upper view are the results corresponding to the "Anonymous" stage and the third one of the "Team Work" phase. We can note

9

an increment in the performance of the students from phase to phase, reaching from 40% - 60% and 60% - 80% mostly in the “Individual” phase (see Fig. 2), 60% - 80% in the “Anonymous” phase to an increment of 60% - 80% and 80% - 100% of correct answers in the “Team Work” phase.

In each of the stages "Individual", "Anonymous" and "Team Work", the teacher can monitor and manage in real time the activity and performance of students through the "Dashboard" option (see Fig. 2 and Fig 3.).

1.3 Stages in the learning activity – Student’s role.

This section describes the three phases “Individual”, “Anonymous” and “Team Work” students should go through with RedCoMulApp to accomplish the reading comprehen-sion learning activity in strictly that order. The teacher determines the moment when to change from one stage to the other.

“Individual” phase. In this stage, each student individually reads the text provided by the teacher (see the left view of Fig. 4). As each student answers the questions, the teacher can see the answers (in green if correct, and in red if it is incorrect) in real time in RedCoApMulApp using the "Dashboard", as seen in the left view of Fig. 2.

“Anonymous” phase. In this stage, students do the same as in the previous stage (Reading the text and answering the multiple-choice questions) with the difference that in this stage they can see the answers of the other two students of their groups without knowing who they are (see the view on Fig. 4 at the right). Students have then to confirm their previous answer or change them based on what their groupmates have answered.

“Team Work” phase. In this stage, students see the names of their groupmates and they meet face-to-face to choose a single option together. They can talk to each other, Ex-change opinions and discuss their disagreements. All students of the group have to select the same option as answer; otherwise, they receive a message from the system to do so. This stage introduces the collaborative learning modality to the learning activity.

10

Fig. 4. Two views of the of RedCoMulApp interface in the role of a student from a session called "August 22, 2017". At the left, the status in "Individual" phase is highlighted. In the following line labels with numbers 1 to 6 correspond to the six multiple-choice questions that for this session. They change color as they answer them. The text to be read is shown in the middle, followed by the first of the 6 questions, along with 5 response options labeled with letters A, B, C, D and E. In this case, the student has selected option C, and in the line below has written a brief justification of his answer. The view on the right corresponds to the "Anonymous" phase, which contains the response of the same student from the view on the left, together with the answers and justifications of his col-leagues from whom he receives this information anonymously.

4 Evaluation of the reading comprehension learning activity

Subjects and settings. The evaluation took place at the Faculty and Business Faculty of the Universidad de Chile, with 12th grade students from nine mid-income high schools of Santiago de Chile, from July to October (6 sessions in total, of 90 minutes each). There were 46 students in total, divided in two sections (22 and 24 students), counting finally with 42 students (22 in the control group, 24 in the experimental group) due to some of them did not participate in all sessions. Students were ranging from 15 to 16 years.

11

Procedure. In each session, the students worked in a regular classroom during the language class time. At the first session, during 5-min the teacher gave basic instructions about the collaborative activity to both control and experimental groups. Students per-formed a first test activity for 15 minutes for students in order to learn how to use the ap-plication. This activity consisted in reading a simple short text and answering three ques-tions. After this, students performed the proper activity (intervention), which was recorded with the presence of the teacher and three teaching assistants.

Description of the Reading comprehension activity and learning contents. The activity was designed according to the objectives of reading comprehension of the lan-guage area for students of 12th grade of high school level. Before starting the activity, the teacher explained the methods and techniques of reading comprehension of texts in a the-oretical and practical way. Later, during the intervention sessions, the students received a text between 800 to 900 words that they had to read in order to develop the learning activ-ity, based on answering multiple selection questions that evaluate the level of achieve-ment achieved in the comprehension of reading. In each of the six session, they received a different text to read. The 6 texts chosen (2 to address the teaching of reading comprehen-sion, 2 for texts of literary type and 2 for journalistic texts), along with multiple-choice questions associated with measuring the level of achievement of reading comprehension of texts, were all extracted from the curricular content of the Chilean Ministry of Educa-tion.

Each of the six sessions held, for the two sections of students who participated in the evaluation, the activity started providing the students with iPadssd with network access to the RedCoMulApp application, and on which they had to log in with their personal ac -count and password. Then, the teacher started the activity in the iPads in his role as teacher, activating the "Individual" stage for approximately 20 minutes, where students individually accessed and read the text. After reading the text, they answered between 12 and 14 associated multiple-choice questions to measure the level of reading comprehen-sion of the text they had read. Once all the students finished this phase, they continued to the next face "Anonymous" for approximately 30 minutes, where again each student ac-cessed individually the text to answer again the same multiple selection questions as be-fore but having access to the answers of their two groupmates. In this stage, students could confirm their previously selected option or re-elaborate their answer choosing an-other one. Finally, in the "Team Work" stage, which lasted approximately for another 30 minutes, the students received the names of their colleagues from RedCoMulApp who in the previous phase saw their answers, and with whom they had to meet face to face to an -swer in jointly form the same multiple-choice questions. In this phase, students are ex-pected to converse among themselves, exchange opinions, and discuss their disagree-ments with a view to agreeing on a single answer.

One section corresponded to the control group, who did not have to write a justifica-tion for the selection of their response; and the other section corresponded to the experi-mental group, who had to justify the answer with a short text. In the case of the sections that had to justify their selection, they had to write briefly the reason or argument for which they had answered.

12

Results. Comparing the performance regarding the correct answers given by of the group students who had to justify their answers with a short text (experimental group) with those who did not have to (control group), the difference is statistically significant at 0,000 as shown in Table 1. This means that those students who justified the selection of the answer option obtained better results.

Moreover, when comparing the correct answers in all sessions for each stage “Individ-ual”, “Anonymous” and “Team Work”, the difference is also statistically significant at 0,000 in all cases as shown in Table 1. This implies that when having to justify the given answer during the “Individual” stage the student could elaborate with more detail the comprehension of the text, which resulted in better answers.

Table 1. Description of the results, comparison between answers without and with justifica-tions, and comparison between answers without and with justifications by phases.

Descriptive statisticsJustifications Answers Frequency Valid Percentage

Without justifications Incorrect 781 45,2 (Control Group) Correct 946 54,8 Total 1.727 100,0With justifications Incorrect 597 32,7 (Experimental Group) Correct 1.226 67,3

Total 1.823 100,0

Comparison of correct answers between control and experimental groupsContrast statistics Correct answers

U of Mann-Whitney 2755577,000W de Wilcoxon 8722362,000Z -8,763Sig. asintot. (bilateral) ,000

Comparison of correct answers between control and experimental groups by phasesPhase Contrast statistics Correct answers

“Individual” U of Mann-Whitney 418677,000W de Wilcoxon 1389598,000Z -7,248Sig. asintot. (bilateral) ,000

“Anonymous” U of Mann-Whitney 206804,000W de Wilcoxon 452154,000Z -5,360Sig. asymptote. (bilateral) ,000

“Team Work” U of Mann-Whitney 255944,500W de Wilcoxon 1182785,500Z -3,541Sig. asymptote. (bilateral) ,000

On the other hand, during the “Anonymous”, stage the justification text helped the stu-dents understand more accurately the selection of the answers of their groupmates, with

13

whom they still had not talked with. Finally, during the “Team Work” stage the collabora-tive work with their groupmates, from whom they already knew their justification en-riched even more their own decision when selected the correct answer.

We also analyzed the quality of the justification written by the students of the experi -mental group was carried out; that is, the degree to which they related the reasons or argu-ments written in relation to whether it covers all the dimensions required for the selection of their answers, whether correct or incorrect. For this analysis, we rated the quality of the justifications to support the selected answers in all 6 sessions, and in all the stages "Indi-vidual", "Anonymous" and "Team Work" of the experimental group.

The justification was rated in one of three categories (see Table 2): a) Insufficient: It fails to cover any of the dimensions or elements of the response or selection generated, or simply, there was no evidence of coherence between the central idea and the answer de-livered. b) Partially Sufficient: Covers some elements or dimensions of the response or selection generated. It is on the right track, but it does not give an adequate justification that covers the entire response delivered. c) Sufficient: The justification given is good enough to justify the selected answers. They are in total coherence and can give relevant reasons for their selection.

Table 2. Classification of quality of the justifications written by the students of the experimen-tal group on all the stages. It is important to note that the number of comments will not coincide

with the total of justified answers, because only non-repeated justifications are taken into ac-count, since several were re-used and others were new in subsequent stages such as the

"Anonymous" and "Team Work".

Session #Comments Insufficient Partially sufficient Sufficient1 592 93% 4% 3%2 397 89% 6% 5%3 433 88% 9% 3%4 405 90% 7% 3%5 449 83% 13% 4%6 265 86% 10% 4%

Given the results shown in Table 2, we can conclude that the justifications written by the students do not sustain (or are insufficient) for building an argument that justifies the decisioón made when selecting an option. However, it seems the intent of doing it posi-tively influenced the selection.

5 Conclusions

The results of the experiment confirmed that the learning activity supported by the Red-CoMulApp presented in this paper is an effective instrument to support the improvement of the reading comprehension of the students. Its design was based on converting an eval-uative learning activity into a formative one using a mechanism that is easy to implement and to use on tablets.

14

El solo producir un comentario, sin importar su calidad y naturaleza, impacta positi -vamente en el desempeño. Al parecer, el ejercicio neurocognitivo de esforzarse por pro-ducir mensajes, si genera mejores elecciones en los estudiantes. ¿pensarán más sus res-puestas? ¿movilizan procesos cognitivos de niveles superiores? Sin embargo, no se obser-va una relación entre el proceso cognitivo que provoca el producir textos cortos y la cali -dad de éstos textos. Pero si existe relación entre el proceso cognitivo y la producción tex -tual.

Es interesante ver respuestas tales como "es lo que yo creo".... "es lo que yo pienso"... "me quedo con mi opinión...". Lo anterior no hace otra cosa de reafirmar la premacía de la postverdad; es decir solo pensar o creer o sostenerse en "mi opinión" como base o fundamento de todas mi decisiones. Lo anterior no hace otra cosa de comprender nuevos escenarios educativos centrados en la pedagogía de la incertidumbre y posverdad o mi opinión es lo que importa y crea realidad, [26].

Por otra parte, la gran cantidad de "emojis" está abriendo las puertas a una manera di-versa de comunicación, que no son posibles de desifrar ni comprender bajo los marcos tradicionales de producción textual.

Nunca antes las personas, gracias a los dispositivos móviles, habíamos tantas perso-nas escrito muchos caracteres ¿pero implica una mejor forma de producción escrita tradi-cional? ¿o simplemente estamos bajo un nuevo paradigma?

Acknowledgements. This paper was supported by Fondecyt Regular 1161200.

References

1. Mullis, I.V., et al., PIRLS 2011 International Results in Reading. 2012: ERIC.2. Slavin, R.E., et al., Effective reading programs for middle and high schools: A best‐

evidence synthesis. Reading Research Quarterly, 2008. 43(3): p. 290-322.3. Rosas, R., et al., Impact of a computer-based intervention in Chilean children at risk

of manifesting reading difficulties/Impacto de una intervención basada en ordenador en niños chilenos con riesgo de manifestar dificultades lectoras. Infancia y Apren-dizaje, 2017. 40(1): p. 158-188.

4. Wild, M., Using computer‐aided instruction to support the systematic practice of phonological skills in beginning readers. Journal of Research in Reading, 2009. 32(4): p. 413-432.

5. Karemaker, A.M., N.J. Pitchford, and C. O’Malley, Does whole-word multimedia software support literacy acquisition? Reading and Writing, 2010. 23(1): p. 31-51.

6. Dole, J.A., et al., Moving from the old to the new: Research on reading comprehen-sion instruction. Review of educational research, 1991. 61(2): p. 239-264.

7. Sung, Y.-T., K.-E. Chang, and J.-S. Huang, Improving children’s reading compre-hension and use of strategies through computer-based strategy training. Computers in Human Behavior, 2008. 24(4): p. 1552-1571.

15

8. Zurita, G., et al. Practice of Skills for Reading Comprehension in Large Classrooms by Using a Mobile Collaborative Support and Microblogging. in CYTED-RITOS In-ternational Workshop on Groupware. 2017. Springer.

9. Gunderman, R.B. and J.M. Ladowski, Inherent limitations of multiple-choice testing. Academic radiology, 2013. 20(10): p. 1319-1321.

10. Park, J., Constructive multiple‐choice testing system. British Journal of Educational Technology, 2010. 41(6): p. 1054-1064.

11. Rodriguez, M.C., Construct equivalence of multiple‐choice and constructed‐response items: A random effects synthesis of correlations. Journal of Educational Measure-ment, 2003. 40(2): p. 163-184.

12. Bennett, R.E., D.A. Rock, and M. Wang, Equivalence of free‐response and multiple‐choice items. Journal of Educational Measurement, 1991. 28(1): p. 77-92.

13. Marsh, E.J., et al., Using verification feedback to correct errors made on a multiple-choice test. Memory, 2012. 20(6): p. 645-653.

14. Tamir, P., Some issues related to the use of justifications to multiple-choice answers. Journal of Biological Education, 1989. 23(4): p. 285-292.

15. Laal, M. and S.M. Ghodsi, Benefits of collaborative learning. Procedia-Social and Behavioral Sciences, 2012. 31: p. 486-490.

16. Duke, N.K. and P.D. Pearson, Effective practices for developing reading comprehen-sion. The Journal of Education, 2008. 189(1/2): p. 107-122.

17. Prestridge, S., A focus on students’ use of Twitter–their interactions with each other, content and interface. Active Learning in Higher Education, 2014. 15(2): p. 101-115.

18. Dufresne, R.J., et al., Classtalk: A classroom communication system for active learn-ing. Journal of computing in higher education, 1996. 7(2): p. 3-47.

19. Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., and Ananthanarayanan, V, NMC Horizon Report: 2017, in NMC Horizon Report, H.E. Edition, Editor. 2017, The New Media Consortium.: Austin, Texas.

20. Zurita, G. and M. Nussbaum, Computer supported collaborative learning using wire-lessly interconnected handheld computers. Computers & education, 2004. 42(3): p. 289-314.

21. Baloian, N. and G. Zurita, MC-Supporter: Flexible Mobile Computing Supporting Learning though Social Interactions. J. UCS, 2009. 15(9): p. 1833-1851.

22. Blasco-Arcas, L., et al., Using clickers in class. The role of interactivity, active col-laborative learning and engagement in learning performance. Computers & Educa-tion, 2013. 62: p. 102-110.

23. Carpenter, J.P., Twitter’s capacity to support collaborative learning. International Journal of Social Media and Interactive Learning Environments, 2014. 2(2): p. 103-118.

24. Zurita, G., N. Baloian, and F. Baytelman. A face-to-face system for supporting mobile collaborative design using sketches and pen-based gestures. in Computer Supported Cooperative Work in Design, 2006. CSCWD'06. 10th International Conference on. 2006. IEEE.

25. Zurita, G. and N. Baloian. Handheld-based electronic meeting support. in Interna-tional Conference on Collaboration and Technology. 2005. Springer.

16

26. Aparici, R. and D.G. Marín, Comunicar y educar en el mundo que viene. 2017: Edito-rial GEDISA.