Integrating Open-Source Solutions in your Processes

Barcelona – June 2011

Stefan Pries - Andrä AGYves Savourel - ENLASO

Why FOSS Should Interest You?Many Proven Solutions

• Open source products widely used in many domainso The LAMP stack (Linux, Apache, MySQL and

PHP/Perl/Python)o Lucene, Drupal, Zope, SVN, Firefox, OpenOffice, etc.

    Why not use the ones in the translation domain too?

Why FOSS Should Interest You?

They are used already

• FOSS projects use FOSS translation tools

• Some LSPs do use open-source translation tools

• Some translators as well (and not just to translateFOSS projects)

Why FOSS Should Interest You?

Maturity

• Some projects have been established for a long time

• User-driven improvements over the years

Why FOSS Should Interest You?

Innovation

• The Bazaar model often leads tomore creative and innovative solutions

• Example: Collaborative translation in FOSSlong before it was in the "industry"

Why FOSS Should Interest You?

Interoperability

• Important part of FOSS projects, because they often are meant to be used along with others FOSS projects

• Wide support for TMX, PO• Increasing support for XLIFF, SRX• Web services• Documented APIs• Command-line interfaces

Some Advantages and Drawbacks

+ You don't have to pay for software licenses- You still have to spend time to install and maintain things

- There is no technical support+ There is almost alway a users group that can help (for free)  and some projects have a commercial side too

- What if the main developers of the project leave?+ You still have the source and possibly a community

- Developers may not implement what you need+ Money can be an incentive, or branching a solution

Let's Explore...

• Only active projects related to translation• Not even all of them, just a selection

Categories:

• Libraries and tools sets• Translation management systems• Translation editors• Machine translation systems• Checkers and QA tools

Demonstrations

The best way to discover the tools is to use them.Two scenarios:

1. Integration with proprietary system2. Integration with other open-source applications

Scenario 2

• Archibald the PM uses Rainbow to pre-translate MIF, Word and XML files from a GlobalSight TM and send them for translation to a Transifex server.

• Pedro the translator uses his own Translate Toolkit TM server to provide additional leverage, then uses OmegaT, an Okapi plugin, Hunspell, LanguageTool and Apertium to finish the translation of the Transifex files.

• Archibald post-processes the translated documents.

9 different open-source tools

And Many More...• Anaphraseus (translation editor in OpenOffice)• Boltran (online translation editor)• Bitext2Tmx (aligner)• Gaupol (sub-title translation)• KBabel (PO editor)• Lokalize (translation editor)• M4Loc (Moses integration tool)• POEdit (PO editor)• Open Language Tools (XLIFF editor)• OpenLogos (MT system)• Qt Linguist (translation editor)• TinyTM (TM server)• Zanata (translation management system)  …Etc.

Summary• Many different tools available for several tasks

o Editingo Quality assuranceo Project managemento Translation assistance (Terminology, TM, MT, etc.)

• Open standards allow complementing existing processes with open-source solutions

• Entire enterprise processes basedon open-source tools are possible

• Not a solution for everyone,but one worth exploring

For More Information

• Apertium – apertium.org• GlobalSight – globalsight.com• Hunspell – hunspell.sourceforge.net• LanguageTool – languagetool.org• Moses – statmt.org/moses• Okapi Framework, Rainbow & CheckMate – okapi.opentag.com• OmegaT – omegat.org• OpenTM2 – opentm2.org• Transifex – transifex.org• Translate Toolkit, Pootle & Virtaal – translate.sourceforge.net

FLOSS Manual on open translation tools:en.flossmanuals.net/open-translation-tools/

 

Thanks!

Any Questions?

stefan.pries@andrae-ag.deysavourel@translate.com

Credits for this presentation:

• Indifex provided the Transifex server access• Jordi Serratosa (@jordis) did the Catalan translations• Universitat d'Alacant provided the Apertium server access• Welocalize provided the GlobalSight server access

And many thanks to all the developers of the various open-source projects for their hard work.

Appendix

 

Started in… License Platform / Language

Remarks

Translate Toolkit

October 2003

GPL Independent / Python

Initial build around the Gettext facilities and PO files,Working with more formats now.Works best with software strings (vs. documentation)Integration: Python API, command-line toolset, Web service (tmserver)

Okapi Framework

Open-sourced in November 2005

LGPL Independent / Java

Components: filters, connectors, steps, etc.Rainbow and Tikal toolsIntegration: Java API, command-line tools, Rainbow functionalities available through Longhorn Web services

Libraries and Tools Sets

Translate Toolkit

Okapi Framework

Started in… License Platform / Language

Remarks

Pootle(Web-based server application)

December 2004

GPL Independent / Python

Existed long before "crowd-sourcing" was a term.Example of use: the translation system of Launchpad, the backbone of Ubuntu translations.

Transifex(Web-based server application)

Summer 2007 as a GSoC project

GPL Independent / Python

Mainly PO and TS filesIntegration with many SCMsIntegration: API, Web services, command-line client.

GlobalSight(Web-based server application)

Open-sourced in January 2009

Apache 2.0 Windows, Linux / Java

Complex system, many featuresIntegration with TDA TMsIntegration: Client tool, Web service API

Translation Management Systems

Pootle

Transifex

GlobalSight

Translation Editors

Started in… License Platform / Language

Remarks

OmegaT 2002 GPL Independent / Java

Very active communityOther projects use OmegaT at their core: e.g. Boltran, BentenIntegration: Plugin API, commend-line mode.

Virtaal 2007 GPL Independent / Python

Uses Translate Toolkit

OpenTM2 Open-sourced in June 2010

EPL Windows OS/2 type user-interfaceLISA wanted to make it a way to provide reference implementation for standards

OmegaT

Virtaal

OpenTM2

Machine Translation Systems

Started in… License Platform / Language

Remarks

Apertium 2005 (OpenTrad project)

GPL Linux, Windows / C++

Rule-based systemSome of the language pairs: Catalan<->Spanish,Spanish<->English,, etc.Integration: Web services, command-line

Moses 2006(EuroMatrix, etc.)

LGPL Linux, Windows / C++

Statistical-based systemOther projects are related:- Moses For Localization (M4Loc)- Moses for Mere Mortals- Do Moses Yourself CEIntegration: Web services, command-line.

Apertium

Moses

Checkers and QA ToolsStarted in…

License Platform / Language

Remarks

LanguageTool

May 2004 LGPL Independent / Java

Grammar and QA checkerRule based, extendableHas defaults for many languagesIntegration: Java API, Web services

Hunspell July 2005 GPL, LGPL, MPL

Independent / C++

Spell-checkerUsed in many other toolsIntegration: API

POFilter 2004 GPL Independent / Python

Part of the Translate ToolkitSoftware string verificationIntegration: Command-line

CheckMate 2010 LGPL Independent / Java

Part of the Okapi projectIntegration: Java API

LanguageTool

Hunspell

PO Filter

CheckMate