1P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g

This presentation is designed as support for a

PISATranslation Training Session

(after customization and practice)

<Customizable Parts>are presented in <angle brackets>

Delete this slide before use

PISA 2015 FIELD TRIAL

TRANSLATION TRAINING <Place><Date>

3P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPracticalia

<This could be the right place to discuss practical arrangements, e.g.:

Estimated length of the session, breaks

Security (signing of confidentiality agreement)

Reimbursement of expenses and/or other administrative matters

Etc. >

4P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gAbout PISA

Programme for International Student Assessment

Administered to 15-year-olds in schools,

who are tested in their own language

Purpose: inform education policy-making

Carried out in 2000, 2003, 2006, 2009, 2012 (3-year cycle)  

At the moment, PISA 2015 is at the stage of item development for the field trial

5P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gParticipating Countries – 1/2

6P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gParticipating Countries – 2/2

7P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPISA Materials

PISA materials are designedto measure “literacy” in 15-year-old students

to collect contextual data about the students

Translated PISA instruments should make it possibleto collect data in <our country>

that will be comparable to data collected in all other countries

8P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPISA Trend Test Units

Test materials for PISA 2015 include ‘trend units’ from previous PISA cycles

Our country, <Country>, <participates in PISA for the first time> <did not participate in <2003>/<2006>/<2009>>, so we have some catching up to do: for us, <some of> these trend units are like new materials

9P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPISA Trend Test Units

What do these units look like?

The next 2 slides show an example of a Science unit last used in 2006

10P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gSample PISA unit, page 1

Introducing some terminology, useful when referring to

parts of a unit

Unit heading

Stimulus: text and/or

graphics, which students must

read and understand to

answer questions

11P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gSample PISA unit, page 2

Item heading Item code

Item stem (the part that has the actual question). Can be followed by multiple choice

options, a Yes/No table(as here) …

… or a space for student to answer (‘open response’ type

item)

Another item in the same unit (referring to the same stimulus

12P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPISA Trend Test Units

And the next two slides show screenshots of the same unit in the PISA 2015 computer-based testing environment

<Delete this slide and the next two if you are not doing computer-based testing>

13P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gSample PISA unit, screen 1

Question pane Stimulus pane

Directions are suitably modified,

e.g. “circle” becomes “click”

14P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gSample PISA unit, screen 2

Same stimulus pane, different question pane

15P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gORDER OF TRANSLATION

Now that you know what a stimulus and an item are:

Always translate the stimulus first, and then the questions

Regardless where items appear in your translation environment

16P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gSo what is special about translating PISA units?

17P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gLocalization of test materials:

key ‘mission statement’

“In order to collect internationally comparable data, the focus is on maintaining the highest possible level of equivalence across countries and across languages. …”

18P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g

… Each item should examine the same skills and invoke the same cognitive processes as the

original version, while being culturally appropriate within the target country.”

19P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gNot an easy mission…

Let’s look at an example of a PISA Reading item, and observe the correspondences between item and stimulus and other item characteristics

which could easily be

lost in translation

20P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g

21P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gTo achieve this mission

Survey-specific translation design which involves:

double translation (two independent translations)

reconciliation (merging into a version that combines the best elements)

further quality control steps: verification, final check, layout check

22P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gOverview of the Process in PISA

Source(English and French)

Comes with general translation/adaptation

guidelines and monitoring instrument including item-specific

guidelines (TAS)Countries can choose

between double translation from

English and French or from English only

Translation 1 Translation 2

This is where you come in!

Reconciliation

Merging the 2 translations into a

version that takes the best elements from

each one

Verification

Independent review by specialist trained to

detect survey-specific defects

National review

Revision of materials taking into account

verification feedback

Final check

Layout and content-related checks before going to press / before

finalizing the test delivery software

23P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gIntroducing the TAS

(Test Adaptation Spreadsheet)Item-specific guideline for the attention of translators, reconciler, verifier

Space for country to report adaptations made to source (to conform to local usage or other reasons) and to justify/explain cases where a guideline could not be respected or leads to a difficulty

Name too rare, we used ‘leopard’

OK

OK

OK

Empty rows for documenting additional issues (not all cases can be foreseen in advance)

grasslands We use “savannah”

NOTE: It will be the Reconciler’s responsibility to fill

in the TAS for the processes that follow, but the TAS can be

a practical way for the translators to communicate

with the Reconciler (to explain difficulties encountered, choices made, possible

alternatives…)

24P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPreparation before translating

PISA test materials

Become familiar with the Translation & Adaptation Guidelines, and keep them to hand when translating

Let’s take them out now and run through key points

Please take notes, ask questions

25P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gTranslation traps

Respect layout of the original Best to translate by overwriting the source

Respect typographical cues: titles, subtitles, numbering of lines/ paragraphs/ items; position indications, emphasis (bold, italics, underline)

Attention to graphic elements: in particular keys, captions, scales, units of measure

This applies to paper-based. For computer-based, text elements are exported and then re-imported after translation into a template which

maintains the layout

26P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gLinguistic difficulty

Avoid complicating or simplifying the vocabulary or syntax

Attention to length of sentences “Common” vocabulary vs. “scientific”

terminology: avoid translating difficult words by using easier words and vice versa

Degree of abstraction: do not modify by e.g. using nouns where the source uses verbs, or vice versa

27P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gLinguistic difficulty

Active versus passive turn of phrase: do not modify unnecessarily

Attention to negations, especially double negations

Echo reference chains (use of pronouns, synonyms)

Attention to rendering the English ‘indefinite’ article

28P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPsychometric traps

General principle: avoid providing clues that direct the respondent to the correct answer or that make a distracter (wrong answer) more attractive

Example

These two words APPEAR several times

in the article

While these two words DO NOT

APPEARin the article

Reminder: consult item-specific

guidelines in the TAS

29P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPsychometric traps

Attention to length of correct answer and distracters in multiple-choice items

ExampleOption D ‘stands out’

because it is the longest. This should

not be lost in translation

Reminder: consult item-specific

guidelines in the TAS

30P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPsychometric traps

Echo literal matches and synonymous matches Example

Reminder: consult item-specific

guidelines in the TAS

31P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gPsychometric traps

Plural versus singular wording of questions, e.g. “which ...” (in multiple choice questions, this is always a singular “which”)

Respect the order of information. Conventions with respect to word order differ, but various stylistic devices often make it possible to enhance this or that part of a question

Small details : attention to “easy” questions, use copy/paste with care Copy/paste risk…

32P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gAdaptations

When translating a PISA unit, it’s a good idea to:

Read the entire unit carefully before you start translating it

• then first resolve any adaptation issues

ADAPTATION:Intentional deviation from the source version made for cultural reasons or to conform to local usage.

33P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gAdaptations

General principles:

An adaptation is needed when there is otherwise a risk that respondents would be put at a disadvantage (or at an advantage!)

Adaptations should be limited to those cases in which they are truly required

34P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gAdaptations

Do NOT adapt: format of an item (e.g. inverting Yes/No or True/False categories)

Do NOT add explanatory notes, additional instructions

DO adapt: spelling and typographic conventions mathematical symbols, abbreviations, formulas units of measurement, time and date notation,

number formats Currency and prices Etc. (unforeseen needs may arise)

Only in a few cases, with

precise instructions in the

TAS. Mostly a fictional currency,

“zeds” is used

35P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gSolving adaptation issues

Consult the TAS Guidance on adaptation issues

36P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gA final cautionary remark on

the guidelines When one tries to apply ALL the rules

– Observe literal and synonymous matches– Keep the same register– Avoid giving (or removing) clues to the

correct answer– Keep the same relative lengths of

distracters– Find suitable adaptations

All this within a tight timeline...

37P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g

–… One will observe that a really goodtranslation of PISA units

–may at times be the result of a compromise

–and might strike the best balance between faithfulness and fluency

–at the cost of violating one or the other rule from the PISA Translation and Adaptation Guidelines

38P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g OK, about time for a coffee break, no?

Questions?

39P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gBefore Translating PISA materials

<Ideally, this should be prepared beforehand by the reconciler and handed out to the translators>

– If recent spelling reforms for your language, a list of up-to-date, authoritative reference works

– Decide on form of address: polite form or familiar form of address used to address students

40P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gBefore Translating PISA materials

The Contractor for Core 3 will provide a list of recurring expressions

Before the work begins, agree on translations for recurring prompts:

– Prepare these beforehand and hand out to the translators

– Or, ask each of the translators to propose entries in a blank list, then determine one by one the agreed translation

41P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise or demo

<Several modalities can be envisaged:

Participants are given a sample (short) unit plus related TAS, to translate on the spot

Participants present their translation of a sample unit distributed ahead of the session, and these are discussed

A volunteer participant or the presenter does a translation on the PC connected to the beamer>

42P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g

<The following slides are for a paper-based translation exercise session Delete them if you are doing computer-based PISA>

Delete this slide

43P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / Translation

We will now translate the unit <XXX>

Open the <English> or <French> source version (MS Word file), open the TAS (Excel file)

Translate by overwriting the source

44P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / Reconciliation<This part can be skipped in the Translator

Training session, though it may be of interest to translators>

After translations are discussed and two are chosen (if needed)

The Reconciler demonstrates the “merging” of two translations into a third one and the correct documentation of adaptations/issues in the TAS

45P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / ReconciliationPractical tip: ideally the reconciler should

use 2 screens while consulting a paper source version (may not be possible in training session conditions)

S/he needs to consult Translation1, Translation2, and (possibly) the TAS’s with translators’ annotations

Reconciliation is carried out by overwriting one of the two translations, taking elements from the other one, and possibly making further changes

46P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / ReconciliationDocumentation in the TAS:All adaptations (reminder: deliberate

deviations from source) must be recorded in the TAS

It is very helpful to record also unresolved difficulties

To limit the workload: no need for elaborate comments when compliance with a T&A guideline was not a problem. A simple 'OK‘ will do. Also, minor and ‘natural’ changes do not need to be repeatedly justified.

47P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gBefore we part

<This could be the right place to present work volume, fees, deadline>

<As regards the delivery mode: translators can be instructed here about uploading and releasing translated units on the PISA Portal (and credentials distributed) OR advised that this information will follow>

48P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g

<The following slides are for a computer-based translation exercise session Delete them if you are doing paper-based PISA>

Delete this slide

49P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / Translation

Preparations: you will need

o The OLT software installed <the link can be provided in advance of the training session, or the software copied to USB sticks for distribution at the session>

oCredentials for the PISA 2015 Portal <now may be a convenient time to distribute them>

oThe PISA 2015 Translator’s Guide <provided in advance of the training session, or distributed now in electronic and/or printout form>

50P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / Translation

We will now translate the unit <XXX>

Follow the instructions in the Translator’s Guide to

oLog on to the Portal, download a file for translation and launch a preview of the source version

51P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gLog in, download from the Portal,

launch preview of source

Log in, navigate to the Translation Monitoring Report

Select a cluster Click the link to view the files for the

cluster Save an XLIFF file to your computer Launch a preview of the source Keep preview screen open for

consultation while translating Open the TAS

51

52P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gTranslate using OLT

oFollow PISA 2015 Translator’s Guide step by step

oFor example: in Options menu, Update Tags and Autopropagate unchecked, all other options checked. To be done only once.

oOther example: If a dialogue box asks whether you want to automatically translate segments in the file, select NO

52

53P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gTranslate using OLT

Screen divided in 3 sections: source pane, target pane and match pane

Translate segments: oOverwrite text in target paneoMove to the next segment:

Confirm and Translate Next button oThis enters translation in Mini-TM

53

54P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gTranslate using OLT

During your translation task, consult the item-specific translation and adaptation guidelines in the TAS

You can also use the ‘Country Comments’ column of the TAS to write any comments you may have to Reconciler who will be reviewing and using your translation

54

55P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gTranslate using OLT

In the XLIFF files, items may appear before the stimulus

In the Excel TAS, stimulus always appears first

Please translate in the order of appearance in the TAS: stimulus first, then items.

Focus on the semantic and formal relationship between items and stimulus (e.g. literal matches)

55

56P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gTranslation Memory

As you translate, Mini-TM builds up When software detects a match:

previous translation(s) display in match pane

Transfer button is enabled If appropriate, select a match and

click on button If edits needed, make them + click

on the button Update Mini-TM Usually you will want to Overwrite

but sometimes Add 56

57P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gUpload and Preview

Once you are (provisionally) satisfied with the translation of a unit, save and close

Answer ‘Yes’ if asked “Do you wish to save the Mini-TM?”

Login to the Portal (if you are logged out), navigate to the Translation tab

Preview your translated unit

57

58P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gRevise, Finalise

If you change your mind about any of your translation choices after previewing–Re-open XLIFF in OLT, make

corrections–Re-preview on the Portal

That’s it! (for now)

58

The Guide includes instructions for finalizing units, but this

should be done on an entire batch of units and it is an

irreversible step – best not done during training session…

59P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise: Reconciliation<It may be unpractical to demonstrate

reconciliation of a computer-based unit, because this would require two released translations on the Portal>

<But the process can always be explained, as it may be of interest to translators>

<If possible>: The Reconciler demonstrates the “merging” of two translations into a third one and the correct documentation of adaptations/issues in the TAS

60P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / Reconciliation

Practical tip: ideally the reconciler should use 2 screens (may not be possible in training session conditions)

S/he needs to open OLT, consult previews of source and two targets, consult the TAS guidelines and (possibly) consult translators’ annotations in two TAS

61P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / Reconciliation

Reconciliation is carried out in OLT by selecting for each segment either T1 (from target pane) or T2 (from match pane) and making further changes as needed

It is essential to preview a reconciled version before finalising

The Reconciler also reports any layout or format issues in the dedicated FORMAT & LAYOUT CHECK form (FLC, one per cluster or batch)

62P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gHands-on exercise / ReconciliationDocumentation in the TAS:All adaptations (reminder: deliberate

deviations from source) must be recorded in the TAS

It is very helpful to record also unresolved difficulties

No need for elaborate comments when compliance with a T&A guideline was not a problem. A simple 'OK‘ will do

Minor and ‘natural’ changes do not need to be repeatedly justified

63P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

gBefore we part

<This could be the right place to present work volume, fees, deadline>

<As regards the delivery mode: translators can be reminded here about uploading and releasing translated units on the PISA Portal, using the “I’m done” button>

64P

ISA

20

15F

T T

ran

sla

tion

Tra

inin

g

Thank you for your attention!