Luigi MuziiThe

Rumble Seat

Agenda

Standards

Quality and the translation industry

Quality management

ISO/DIS 17100

Requirements

ISO/TS 11669:2012

KPIs and capability statements

The Rumble Seat

Technical standards Make life easier

The Rumble Seat

Technical standards

Ensure safety and reliability, reduce costs by minimizing waste and errors, and increase productivity

The Rumble Seat

Process standards

Ensure repeatability, efficiency, consistency, cost reduction, and safety

The Rumble Seat

Quality and the translation industryThe fallacy of an all-time concept

The Rumble Seat

Quality in translation

The unique selling proposition of the entire industry, a sort of life vest to rely on, pretending confidence in safety instructions

The Rumble Seat

Three basic questions

What do translation buyers care about?

What is the impact of technology on translation quality?

Is the current TEP translation model still making sense?

The Rumble Seat

A buyer’s focal point

Buyers want to know what they spend for and for what it’s worth

The Rumble Seat

Same ingredients

To most customers, quality is a feature, just like color in beer

The Rumble Seat

Quality as USP

Commodification of translation Priced on average and market conditions: the lower

the expectations for differentiation, the lower the willingness to pay

Impact of technology

The Rumble Seat

Translation technology’s impact is lower than the spreading of computers and the Internet

Machine translation

Online machine translation engines and freely available open-source machine translation engines have actually been disrupting the translation industry

The Rumble Seat

Translation quality standards

The general and stubborn application of unsuccessful, overcomplicated traditional theories and models, prone to subjectivity and fallacy

Differentiation Flexibility is key

Agile

leansimple

explicitpractical

collaborativeadaptive

The Rumble Seat

TranslationA century-old practice: everybody knows it, very few know how it works; with standards claiming to rule all aspects under strict tradition

The Rumble Seat

TEP

outdatedserial

stiffcompartmentalized

sluggishinefficient

The Rumble Seat

Quality managementBasics

The Rumble Seat

ISO 8402: 1987 The totality of features and characteristics of a

product or service that bears its ability to satisfy stated or implied needs.

ISO 9001:2005 Degree to which a set of inherent characteristics of an

object fulfils requirements Requirement

Need or expectation that is stated, generally implied or obligatory.

Characteristic Distinguishing feature

Definitions of quality

Basic quality management

In 10 minutes1. Write down what you do2. Do what you have written3. Substantiate what you have done

After the presentation4. Reflect on how to improve it

Process approach

Breaking down work into interrelated tasks Each task serving a specific goal

Repeatability Maximizing efficiency in using resources Reducing variations

Pros and cons

Repeatable quality Repeatability is not quality

–High degree of control Heavy initial investments

–Reduced impact Major maintenance costs

–Continual improvement –Bureaucratization

–Greater confidence Reduced flexibility

The Rumble Seat

Streamlining Removing unnecessary or harmful activities Prevent errors rather than correcting them

Process efficiency

Translation quality managementAnother story

The Rumble Seat

Translation standards

UNI 10574 Italy 1996 Requirements for translation and interpreting services

ATA Taalmerk The Netherlands 1997 Requirements for translation services

DIN 2345 Germany 1998 Requirements for translation contracts

ÖNORM D 1200 Austria 2000 Requirements for translation and interpreting services

ÖNORM D 1201 Austria 2000 Contracts for translation and interpreting services

SAE J2450 U.S.A. 2001 Translation quality metric

ISO 12616 International 2002 Translation-oriented terminography

GB/T 19682 China 2005 Target text quality requirements for translation services

GB/T 193636.1 China 2008 Specification for translation services

EN 15038 E.U. 2006 Requirements for translation services

ASTM F2575-06 U.S.A. 2006 Quality assurance in translation

ASTM F2809-01 U.S.A. 2006 Requirements for interpretation services

CAN/CGSB-131.10 Canada 2008 Requirements for translation services

ISO/TS 11669 International 2012 General guidance for translation projects

ISO/DIS 17100 International Requirements for translation services

ISO/WD 14080 International Assessment of translations

ISO/CD 18587 International Requirements for machine translation (MT) and post edition levels

Aspects ofquality (Melby)

First Linport Symposium (2011) Translation quality is relative

1.Transcendent quality Accuracy and fluency

2.Manufacturing quality Compliance with specifications

3.User quality End-user satisfaction (dissatisfaction expressed in terms of

specifications)

4.Value quality Price comparison only meaningful between two providers who can

deliver according to same specifications

5.Social quality Societal harm (related to specs)

The Rumble Seat

Translation quality(Melby)

ASTM F2575 – 6 (2006) The degree to which the characteristics of a

translation fulfill the requirements of the agreed-upon specifications

First Linport Symposium (2011) Translation quality is relative

A quality translation follows specifications that are appropriate to end-user needs, avoiding social harm, at the lowest available cost, without compromising needed accuracy and fluency

MQM (2012) A quality translation (1) demonstrates required

accuracy and fluency (2) for the audience and purpose and (3) complies with all other negotiated specifications, taking into account end-user needs

The Rumble Seat

Error typology is seen as being static and unable to respond to new text types or varying communicative situations (Sharon O’Brien)

Subjectivity Time Inappropriate use of linguistic resources Learning curve Technology

The error-catching approach

Quality is when the buyer or customer is satisfied, yet quality measurement in the translation industry is not always linked to customer satisfaction, but rather is managed by quality gatekeepers on the supply and demand side who have specific evaluation models based on counting errors, applying penalties and maintaining thresholds with little, if any, interaction from customers.

Quality measurement(TAUS)

Error Deviation from a benchmark for rule violation

Metric A method of measuring errors

Quality Degree of deviation from a predefined threshold of

errors

Defects, errors, metrics and quality

Six Sigma

Level Acceptability Defects per million

1 30,9% 691.462

2 69,1% 308.538

3 93,3% 66.807

4 99,38% 6.210

5 99,977% 233

6 99,99966% 3,4

2 incorrect landings per day at major airports

7-hour power failure per month

15,000 urgent packages lost per week

5 minutes of only non-drinkable water every day

5,000 incorrect surgical operations per week

200,000 wrong prescriptions per year

1% defects

Caution!In a 10.000 word projects, the seemingly minute difference between 99,38% and 99,99% means 62 errors, 2 errors every three pages, compared to only 1 in total

The Rumble Seat

Subtractive Counting errors with possible final score

Error subtraction Rating percentage

Gravity

Weight

Additive Counting items that meet requirements

Metrics

ISO/FDIS 17100Translation Services — Requirements for translation services

The Rumble Seat

Vienna Agreement

Between CEN and ISO Primacy of international standards

To avoid duplication of potentially conflicting standards

ASTM not an ISO member ASTM F 2575-14 still in force

The Rumble Seat

General structure

Exclusively focusing on the implementation of the conventional translation process

Revision by a second person

Requirements considered in the TSP’s view

The Rumble Seat

Requirements vs.

specifications

Requirements Provisions of this standard, other standards or legal

regulation

Specifications provisions involving the translation service or project

originating from the client, the TSP or other sources

The Rumble Seat

Scope

Requirements for all aspects of the translation process directly affecting the quality and delivery of translation services

Management of core processes Minimum competence requirements Availability and management of resources Other actions

Outside the scope

Use of raw output from machine translation plus post-editing

Basics

TSPs must demonstrate conformity of specified translation services to the standard and capability of theirs processes and resources to deliver a translation service that will meet the client’s and other applicable specifications.

A documented process in place to ensure that the people selected to perform translation projects have the required competences and qualifications

Record the basis upon which the professional competences of translators, revisers, reviewers and other professionals have been demonstrated

A process in place to document that the competences of their translators, revisers, reviewers, project managers and other professionals are maintained by continuing practice, and regularly updated by training or other means

Human resources

Competences Translation competence Linguistic and textual competence in the source and the

target language Competence in research, information acquisition and

processing Cultural competence Technical competence Domain competence

Credentials A recognized graduate qualification in translation from an

institution of higher learning A recognized graduate qualification in any other field from

an institution of higher learning plus two years’ full-time professional experience in translating

Five years’ full-time professional experience in translating A certificate of competence in translation awarded by an

appropriate government body

Competences and

credentials

Identifying the key requirements and translation project specifications during the pre-production process and following the procedures and specifications throughout its production

Supervising and monitoring the translation project preparation process

Assigning a competent translator or translators to the translation project

Assigning a competent reviser or revisers

Disseminating information, issuing instructions related to the assignment and management of the translation project to all parties involved

Monitoring to ensure compliance with agreed schedule and deadlines

Monitoring constant conformity to the client-TSP agreement, project specifications and, where necessary, communicating with all parties involved in the project, including the client

Managing and handling of feedback

Verifying that the translation service specifications have been complied with before approving the translation and giving clearance for its delivery to the client

Delivery of the service

Assigning a competent reviewer or reviewers to the translation project

Implementing corrective measures and/or corrective action

Monitoring to ensure the project does not exceed the agreed budget

Preparing and issuing the invoice

Completing other activities or tasks agreed with the client

Project management

Translation project management competence can be acquired in the course of formal or informal training

Basic understanding of the translation services industry

Thorough knowledge of the translation process Mastering project management skills

Project managers

Technical equipment required for the efficient and effective completion of translation projects, and for the safe and confidential handling, storage, retrieval, archiving and disposal of all relevant data

Communications equipment including appropriate hardware and software

Information resources and media

Tools such as translation technology, translation management systems, terminology management systems, and other systems for managing translation-related language resources

Technical resources

All agreements

Each accepted translation project

A log of each project

All assignments

Information concerning linguistic specifications

Record

Translation workflow (Annex A)

Enquiry and feasibility

Quotation

Client-TSP agreement

Handling of project-related client information

Project preparation

Translation

Check

Revision

Review

Proofreading

Final verification and release

Post-production processes

Feedback

Closing administration

Account manager Project manager

Linguists Various

Weak points

Reaffirmation of the serial traditional process

Strong emphasis on ISO/TS 11669

Not applicable to processes involving MT

Responsibility for sub-contracted processes

No clue on how to achieve quality and measure performance

No metrics whatsoever

Only vague reference to SLAs

Collection and detailing

Requirements

Clients and projects

Customers/projects are what providers/managers get from them (and vice versa)

The Rumble Seat

A bidirectional route

The final outcome of a project depends on the quantity and quality of information exchanged by the customer and the vendor

The more the vendor understands the customer’s needs and expectations, and translate them into requirements, the more it will be able to meet them

In 5 minutes In 20 words

Name and describe your service Name and describe your customer’s product/service

The Rumble Seat

A document containing a detailed indication of the characteristics required to satisfy a contract.

Requirement specification

Goal

To form the basis of the agreement between the customer and the supplier

Allow the supplier to use most suitable resources and prepare a workable plan

Determine whether the delivered product meets the customer’s needs

A clear and comprehensive description of the basic conditions for the service

Workload Expected results Characteristics of deliverables

Content

Information, objectives and customer requirements Project summary

Closed-ended questions Key points

Keywords

Guiding questions Directing search for understanding

Bringing the interviewee to tell you what you want to know (goal)

Do not repeat the same questions (wrong communication) Combine analogical language (suggestion) and digital

language (precision)

Previous unsuccessful solutions

Developing a requirement specification

Information for the assessment of deliverables Metrics and scorecard

Criteria (KPI) Weights (to settle assessment deviations) Measurements Scores Comments

QA

In 5 minutes Pick up a scorecard Name 5 basic translation requirements

From the buyer’s perspective From the provider’s perspective

The Rumble Seat

Sample scorecard

La

ng

ua

ge

pa

ir

Su

bje

ct

Pu

rpo

se

Co

nte

nt

typ

e

Sp

eci

al

reg

ula

tio

ns

Au

die

nce

Sty

le

Fo

rma

t

Te

chn

olo

gy

Weight Score Requirement

The Rumble Seat

ISO/TS 11669:2012Translation projects — General guidance

42 pagesLength

Goal Improve communication among all relevant stakeholders in a translation project

The target content [must] be readable in the target language and correspond in some way to the source content

Basic and implicit specification

Requesters and TSPs should work together to determine project specifications

Basic assumption

The quality of a translation — from a workflow and final delivery perspective — can be determined by the degree to which the target content adheres to the predetermined specifications

Quality translation projects and quality translation products result from developing and following appropriate project specifications

Quality

Scope Guidance concerning best practices for all the phases

of a translation project

A framework for developing structured specifications

Use of specifications

Attached to a legally binding contract or to a purchase order or other document supporting the request to define the work to be done

The starting point for all assessments, both qualitative and quantitative

Translation parameters Set of key factors, activities, elements and attributes

of a given project used for creating project specifications

Vendor selection

Requesters should first define the preliminary project specifications, then choose an appropriate TSP that meets the needs of the project (i.e. applying a structured specification approach to translation projects and having the necessary competences)

Competences Language proficiency Professional experience Technology proficiency

Credentials Degree in translation Certification from a recognized body Language proficiency certification Language immersion Evidence of ongoing professional development References from previous work Samples of previous work

Competences and

credentials

Roles

Translators

Revisers

Reviewers

Project managers

Translation parameters

21 key factors, activities, elements and attributes of a given project used for creating project specifications

Parameters

Linguistic (13) Source content information

Source characteristics Specialized language Volume Complexity Origin

Target content information Target language information Audience Purpose Content correspondence

Covert vs. overt

Register File format Style Layout

Production (2) Typical production tasks Additional tasks

Environment (3) Technology Reference materials Workplace requirements

Relationships (3) Permissions Submissions Expectations

Process Traditional TEP

Prone to introduce new errors at any step rather than removing them

The Rumble Seat

Terminology work

Pre-production Ends when both the requester and the TSP have agreed upon the final

project specifications

Source content identification and preparation

Preliminary project specifications Translation parameters

TSP selection

Final specifications

Approval of specifications

Production Preparation Initial translation In-process quality assurance

Self-checking (or post-editing), revision, review, final formatting or compilation, and proofreading

Delivery

Post-production Requester-managed post-delivery inspection Acquiring end-user feedback Incorporating end-user feedback into the specifications

Phases of a translation project

Strengths Shared specifications

Legally binding specifications

Accessible specifications

Assessments based on specifications

Pitfalls

A legacy of the typical academic training in translation

Restating the traditional information asymmetry between customers and suppliers

The customer is supposed have the same skills interest as the vendor for translation and its technicalities

No indications for developing binding agreements No reference to SLAs or SoWs No hints on developing quantitative measures No guidance for review

Classical trial and error approach The long list of parameters makes room for introducing

surreptitious, vague, and subjective evaluation criteria

The result Mashed by stiffness and overhead

The Rumble Seat

Requirements Simple, straight, realistic, exhaustive, verifiable to use as checklists

The Rumble Seat

KPIs and capability statementsFlyers and landing pages

The Rumble Seat

CSF for the localization industry

CSF Area Variable

Price Production Cost of service

Delivery Production Shipping capacityProject management

Quality of service Production InvestmentsAnalytics

Customer service ProductionCustomer service

Quality of serviceMaintenance costs

Flexibility VendorsProduction

Peaks of demandPlanning ability

Financial independenceFinanceProductionSales

Cash flow

Image Staff Resource developmentHuman capital

The Rumble Seat

KPIs in a showcase

EVA (Economic Value Added) A measure of real economic value a vendor adds to its

business

Customer satisfaction Customer satisfaction index Corporate reputation Customer retention rate

Quality index DIFOT (Delivery In-Full, On-Time) rate

A measure of the vendor’s delivery reliability FPY (First Pass Yield)

The percentage of units coming out of a process with no rework

Rework level Percentage of units requiring rework

HCVA (Human Capital Value Added) A measure of the extent to which staff add value to the

business

The Rumble Seat

Capability statements

Snapshot of your company Introduction to your business

Client needs to be able to see what you do at a glance Highlights your companies unique features

Stick to the facts – Right, tight, and bright – KISS

Flyer 1 A5 page (two-sided)

Constantly updated Target audience

Landing page

The Rumble Seat

KPI

Capability statement (front)

The Rumble Seat

Logo

Contact details

Company profile

Capabilities

Facilities, equipment and resources

Performance

Main customers

Finance Processes

Sales Human Resources

Areas of expertise, services, subject field(s)

Technologies, certifications, accreditations, licenses, clearances, awards

Vision, mission and value statements, competitive advantage (how the company distinguishes itself and why it is better positioned in the market against competitors)

According to subject field

Capability statement (rear)

The Rumble Seat

Logo

Management policies

Social responsibility statement

Strategic partners

Contact details

Testimonials

Management

Financial policy, innovation strategies, data security plans

Environmental practices, employee relations, community participation