Coordination in the formation of the transversal skills needed for University-level assignments and support tools for their assessment

TEEM’16 - Track 1November 3, 2016 · Salamanca

Authors

M. DOLORES CEPERO-ASCASO

F. JAVIER GARCÍA-RAMOS

PABLO MARTÍN-RAMOS

Escuela Politécnica Superior de HuescaUniversidad de Zaragoza

dcepero@unizar.es

fjavier@unizar.es

pmr@unizar.es

Presentation templateby SlidesCarnival

IntroductionImportance of the formation and assessment of transversal skills

1

“

“When a thing has been said and said well, have no scruple. Take it and copy it”

Anatole France (1844-1924)

“To steal ideas from one person is plagiarism; to steal from many is research”

Steven Wright (1955 - )

Transversal skills

◉ “A complex system of knowledge, integrated by the cognitive, affective and procedural dimensions, and which may be implemented in different contexts”

◉ At a University-level, their importance is comparable or even higher than that of the specific competencies!

… but whereas we thoroughly plan the syllabus of our courses, we seldom plan and train in transversal skills at the same level →responsibility of teachers at Secondary or Pre-University level, of the family or of the education system in general

Transversal skills

◉ Need for a training program in relation to transversal skills…• Corporate-level planning of the formation of transversal

competences: educational guides for each skill, with indicators, learning outcomes and courses in the study plan of the degrees (e.g., University of the Basque Country)

• No institutional guidelines at Universidad de Zaragoza or at our Center (Higher Polytechnic School of Huesca).

How to train transversal skills?

◉ For example, by conducting academic assignments…• Contribute to the development of the analysis and

synthesis ability, one of the most valued skills by employers, teachers and graduates.

◉ In a high percentage of our Bachelor’s degree courses students must complete assignments

There is a need to provide specific training in the involved transversal skills at basic, intermediate and advanced levels in a coordinated manner…

How to evaluate transversal skills?

… but the abundance of information sources available to the student favors plagiarism if a solid awareness and education against its practice is missing!

◉ Need for plagiarism detection tools to support faculty in the assessment of the students’ assignments and to promote good intellectual property practices

◉ Need for an up-to-date panorama of these tools, focusing on free ones, compatible with Spanish language

Plagiarism detection tools

Studies in other Universities say that…

◉ They significantly reduce the overall frequency of plagiarism violations (Trident University, Cypress, CA, USA)

◉ They enhance academic integrity: incidence of plagiarism offences reduced by half in 3 years (A.T. Still University, Mesa, AZ, USA)

◉ Students consider it “helpful in learning about appropriate source use” and wished it had been introduced earlier in their degree course (University of Wolverhampton, UK)

◉ Very few of their students express objections to the use of these software tools (University of Southern Mississippi, Hattiesburg, MS, USA)

Goals of this study

◉ Provide an example of how program chairs, faculty and academic support services can collaborate to reduce the incidence of plagiarism through the use of detection software and skill building.

◉ To offer an outline of the features of different freeplagiarism detection tools and compare them with Turnitin®

• Most popular commercial software in other recent academic dishonesty studies

MethodologyKey points of the study

2

Methodology

◉ Design of an action plan in specific courses aimed at developing transversal competences.

◉ Training activities within the courses framework.

◉ Assessment of different software tools for plagiarism detection, both free and paid (Moodle integration?).

◉ Analysis of the results obtained upon application of the plagiarism detection software to a sample of assignments.

◉ Analysis of the results obtained by the students in the assessment of those assignments.

ResultsSkill building activities and assessment of plagiarism detection tools

3

Skill building activities

◉ Activities tracked by the Coordination of the Degrees.

◉ At least a faculty member responsible for a course taught in each of the 4 years of each of the 2 degrees participated.

◉ Active collaboration of the Library of the Center:• Information and materials on intellectual property available at their

webpage• Intellectual property support service at AyudICa• Training plan in informational competences, which covered the topic of

plagiarism and strategies to avoid it

1st year 1st year

2nd year 2nd year

3rd year 3rd year

4th year 4th year

Rural and Agri-food Engineering degree

(7 courses)

EnvironmentalSciences degree

(10 courses)

Optative: databases, search strategies, analysisand treatment of results, reference management…

Searches, academicassignments structure,

formatting…

Survey for environmentallegislation references

Academic assignmentstructure, citing, plagiarism

FALL SEMESTER

SPRING SEMESTER

FALL SEMESTER

SPRING SEMESTER

Informationalcompetences(basic level)

8-10 h

FALL SEMESTER

SPRING SEMESTER

FALL SEMESTER

SPRING SEMESTER

Bibliographyworkshop

2 h

FALL SEMESTER

SPRING SEMESTER

FALL SEMESTER

SPRING SEMESTER

Bibliographicalsurveys.

Scientific-technical writing

FALL SEMESTER

SPRING SEMESTER

FALL SEMESTER

SPRING SEMESTER

Informationalcompetences

(advanced level)

Symposiumsimulation

Assessment ofplagiarism detection tools

◉ Most are web-based tools that operate on a credit-based system, in which the user pays per number of analyzed words or per document.

◉ They allow free use, but with severe restrictions in terms of the number of words per analysis and the number of analyses per day.

Assessment ofplagiarism detection tools

Tools tested:

◉ Viper (Scanmyessay): • Service restrictions that prevent its normal operation in Spain

(and in other EU countries) → US proxy needed.• Affects normal operation of Microsoft Word, causing

unexpected application closures.• Slow and imprecise results.• Terms of use involve violations of privacy that are not

compatible with the policy of our Institution → information could be sold to third parties!

Assessment ofplagiarism detection tools

Tools tested:◉ http://www.duplichecker.com/: usefulness limited to the

occasional analysis of some paragraphs, not for an entire assignment

◉ PlagScan software (http://www.plagscan.com): effectiveness seemed to be very inconstant

◉ http://www.plagiarismsoftware.net/, http://www.checktext.org/, http://smallseotoolz.com/plagiarism-checker/, http://www.plagium.com/, www.copycheck.cc, http://checkplagiarism.org/, http://www.plagiarismsoftware.net/: very restrictive limits (words/day)

Assessment ofplagiarism detection tools◉ https://noplag.com/, https://www.grammarly.com/,

https://www.paperrater.com/, https://www.plagtracker.com/upload/: do not allow the analysis of texts in Spanish language, only in English.

◉ http://plagiarisma.net/: Trojan in the desktop application!◉ Crot (http://crotsoftware.com): not really free. $300/250

searches and $1000 for unlimited searches.◉ Anti-Plagiarism (Check on plagiarism) by mdop: not free to use,

in spite of the fact that the tool is hosted in an Open Source repository!

Assessment ofplagiarism detection toolsUseful tools:◉ WCopyfind (http://plagiarism.bloomfieldmedia.com/

wordpress/software/wcopyfind/): this free tool allows the user to compare a document against another. Free alternative to BeyondCompare®.

◉ JPlag (https://jplag.ipd.kit.edu/): free tool, can only analyze programming code. Similar to Moss software (http://theory.stanford.edu/~aiken/moss/).

Assessment ofplagiarism detection toolsCommercial tool: Turnitin®

◉ Widely used at an international level (in over 15,000 institutions) and in other Spanish centers (21 Spanish universities in 2015).

◉ Indexes over 60,000 million web pages, 600 million student papers and 150 million publications and articles

◉ It can be integrated in Moodle.◉ Involves a cost of ca. 1 € per student and

academic year.

Results of plagiarism analyses

◉ Considering that the percentage of text detected as plagiarized was on average 24.4% (median 20%), even exceeding 50% in some cases, the results were worrisome! →Importance of student sensitization: user awareness programs and training programs needed

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

<5%

6-10

%11

-15%

16-2

0%

21-2

5%26

-30

%31

-35%

36-4

0%

41-4

5%46

-50

%51

-55%

56-6

0%

61-6

5%66

-70

%>7

0%

No.

of a

ssig

nmen

ts

Similarity index

General impressions aboutTurnitin

◉ It detects books, scientific articles, reports, web pages, etc. in Spanish with precision.

◉ Analyses are not immediate → not critical.◉ Excellent integration with Moodle.◉ User-friendly and comfortable interface.◉ Similarity index never 0, because of references section text.◉ The software does not discriminate whether the ‘copied’

text is properly referenced or not, so a subsequent analysis by the teacher is ‘a must’.

Grades and impressions aboutthe plagiarism results

◉ Pass rates over 90% for the assignments in all the courses for which the faculty had provided data…• … but they were tutored assignments in which the most serious errors

had already been detected in mid-semester revisions◉ As regards plagiarism, there were differences in the criteria

used by the faculty in terms of the percentage that could be treated as ‘fair’, hence acceptable.

◉ But all agreed that an entirely plagiarized text should be immediately rejected.

◉ None of the participants had previously used plagiarism detection software.

ConclusionsKey outcomes of the study

4

Conclusions

Specific training activities aimed at the development of the transversal skills necessary to successfully complete academic assignments have been carried out. Activities certainly contributed to improving the acquisition of such skills.

Having a map of activities aimed at attaining the transversal competences of the degrees is very useful for the different agents involved in the Internal Quality Assurance System.

It is essential to continue working on the specific training in the context of the courses: although pass rates in academic assignments were high, there is the perception that there is plenty of room for improvement in the acquisition of these skills.

Except for occasional searches, aimed at analyzing specific paragraphs, it is not possible to use free software to detect plagiarism in assignments written in Spanish language. However, the most popular paid tools are affordable.

From the analysis of academic work with Turnitin, an average percentage of plagiarized text of ca. 25% was found. Need to sensitize the students of the importance of avoiding it and correctly citing their sources.

Turnitin tool is comfortable and intuitive, integrates well into Moodle and perfectly fulfills its function. Can be deemed as highly recommendable.

Any questions ?You can find me at◉ pmr@unizar.es

Thanks!