AVTOdyssey:VoyagetotheFutureAnnaMatamalaUniversitatAutònomadeBarcelonaTransMediaCataloniaresearchgroupanna.matamala@uab.catIntermedia,14-16April2016FFI2015-62522-ERC,2014SGR0027,FFI2015-64038-P
Myview
• Audiovisualtransla2on(includingmediaaccessibility):accessthroughtransfer
• Tothelanguageoftheoriginal,toaudioandvisualelements
• Variouscontent,modali2es/services,plaAorms,agents,andusers
• Mul2pleapproaches
SomefirstthoughtsonAVTresearch
• AVTmodali2es:dubbing/sub2tlingversusmediaaccessibility
• Methodologicalapproaches:descrip2ve,movingtowardsmoreempiricalandapplied.
• Topics:moretechnologyandrecep2onresearch.
Isthisreallyso?
ResearchonAVT:data
Jumpback10yearsto…– InSoManyWords.London,2004.– BetweenTextandImage.Forlì,2005.– MediaforAllI.Barcelona,2005.
ResearchonAVT:data
…andbacktothepresent– Intermedia.Lodz,2016.– Linguis2candculturalrepresenta2oninaudiovisualtransla2on.Rome,2016.
– MediaforAll.Sydney,2015.
Audiovisualtransfermodes
BTS:AVTmodes
Approach
Topics• Manydivergingtopics• Similarinterest– Culturalapproaches– Training,didac2cs– Specificprac2ces
• Slightdrop– Linguis2capproaches(s2llanimportanttopic)
Topics
• Increasein:– recep2onofcertainfeaturesbyend-users– fantransla2on,crowdsourcing– mul2lingualism
FocusonInterMedia16
• Technologies:MTandTM,cleanaudio,secondscreens,SR(liveinterlingualsub2tling),computa2onalstylis2cs,collabora2ontools
• Recep2onofAD(divergingstrategies,amountofinforma2on)
• Processresearch(respeakingcrisispoint)• Differentgenres(emergencynews,TVads),paratexts(taglines)andstrategies
FocusonInterMedia16
• AVTtrainingandAVTinteachingforeignlanguages
• Humourandinterculturalelements• Filmfeatures:mul2lingualismandpar2alsub2tling,textonscreen
• Sub2tlingfeatures(segmenta2on)
FocusonInterMedia16
• Filmadapta2on• Touchtours• Fansub2tling• Legalaspects
• LessWestern-European-centricapproach:Russian,Persian,Arabic,Indian,Belarus,etc.
AVTResearch&Technology
Theroleoftechnology
• Intheprocessofcrea2ng• Intheprocessofdelivering• Intheprocessofreceiving
• Intheprocessofresearching
Speechrecogni2on
• Respeakinginsub2tling:fromintralingualtointerlingual
• RespeakinginotherAVT-relatedtasks– Anexample:documentarytranscrip2on(ALST,withLukaszDanilukandPabloRomero-Fresco)
• Automa2ctranscrip2on(ALST,withHéctorDelgadoandJavierSerrano)– Domain-specificissues,speakerdiariza2ontools,languagedifferences,meaningoferrorrate
Machinetransla2on
• Extensiveresearchonsub2tling
• Newresearchonothertransfermodes:ADandVOinALST
• Focus:translator/describerandend-user
Machinetransla2on:example1
12AVTMAstudentsHumantransla2onvspost-edi2ngTwoshortwildlifedocumentaryexcerpts(EN>ES):voice-overandoff-screendubbingTemporal,technical,cogni2veeffort(Inputlog)
Or2z-Boix&Matamala(forth)
Machinetransla2on:example1
• Post-edi2ng,faster;lesstechnicalandcogni2veeffortthantransla2on
• Outputqualityanalysisbyexperts(differentevalua2onrounds),indubbingstudio,andbyend-users
• Transla2ons,slightlybefer(trainedtranslators)
Machinetransla2on:example2
• Butwhatabouttranslator’sfeelings?
• MTinAD– 12AVTMAstudents,EnglishintoCatalan– crea2on,humantransla2on,post-edi2ng– Temporal/technical/cogni2veeffort– Subjec2veopinions
Machinetransla2on:example2
• Temporal:nosta2s2caldifferences,butADcrea2on(55.95’),postedi2ng(44.44),transla2on(48.66)
• Technical:post-edi2ngwassta2s2callytheleastkeyboardintensivetask.
• CogniSve:post-edi2nglessthanADcrea2on(onlysta2s2caldifference,althoughvaluesalwayslowerinpost-edi2ngintermsofeffortinvolved)
Whataboutperceivedeffort?
Machinetransla2on:example3
• WhydoweneedMTfor?Automa2c?Partofaprocess?
– Tes2ngonautoma2csub2tlingwithinHBB4ALLproject,provedusefulonlyforcertainusers(Matamala,Oliver,Álvarez,Azpei2a2015)
Text-to-speech
• TTSADinPoland(Szarkowskaetal),spokensub2tleswithTTS
• TTSADinCatalan(Fernández-Torné&Matamala)
• TTSVOinwildlifedocumentariesinCatalan(Or2z-Boix&Matamala)
Othertechnologiesandtools
• Textsimplifica2on– Simplextproject,Able-to-include
• Textcompression– Aziz,deSousa&Specia(2012)
• Textsegmenta2on– Scaianoetal(2010)andÁlvarezetal(2016)
Othertechnologiesandtools
• Automa2cimagerecogni2on– SeeingFB
hfp://newsroom.n.com/news/2016/04/using-ar2ficial-intelligence-to-help-blind-people-see-facebook/
• Automa2clipsynch/dubbing hfp://aescripts.com/auto-lip-sync/hfp://mrl.nyu.edu/~bregler/videorewrite/hfp://videodubber.com/
Othertechnologiesandtools
• AVTasbigdata
– Textalignedwithaudio– Imagesalignedwiththeirdescrip2on
• Informa2onretrieval
Edi2ngtools&thecloud
• Professionaltechnology:– Sub2tlingsopwarevsdubbingsopware
• Collabora2vetoolsforprofessionalsandthecloud:cohesion
Collabora2veplaAorms
• Crowdsourcing,fan(sub/dubb)ingoruser-generatedtransla2ons(O’Hagan,Dwyer)
• Dubbing(Dubroo,Dubjoy)• Sub2tles(Amara,Viki)
Collabora2veplaAorms
• Currentframework:collabora2veconsump2on(airbnb)andcrea2on(wikipedia),immediacy,personalisa2on,consciouseconomy?
• Legalissues• Whatisquality?Userexpecta2ons?• Endangered/minoritylanguages• Professionalvsuser-generatedcontent
Toolsfortheend-user
• Consump2onacrossdevices• Apps:– MovieReading,Whatscine,ArtAccés,etc.– MyLingoApp.
Dotheyreallywork?(Walczak’sresearch)
Toolsforresearch
• (Mul2media)corpusanalysistools– POStagging,seman2ctagging(inAD)
• Keyloggingandscreenrecordingsopware• Eye-trackingandelectrophysiologicalmeasures• Sen2mentanalysisinsocialmedia– hfp://www.laurenceanthony.net/sopware/fireant/
• Crowdsourcing(micro)tasks(AMT)– Example:Transcriu-me(BibliotecaCatalunya)
Technologyimpactsonresearch
• Wes2llneed:– Concepts(theory)– Stateoftheart(descrip2on)
• Butwecanalsofocusonpeople(translators,end-users)– Processresearch– Recep2onresearch
Butneedtogoonestepfurther
Reachingout
Researchreachingpeers
• Anexample:theVIWproject
hfp://pagines.uab.cat/viw• Openaccess
• publica2onsandpresenta2ons• audiovisualcontent(copyrightissues)• datasetsavailable
Reachingtheend-user
• Sciencedissemina2on
– Raisingawarenessaboutourresearch– Goingbeyondthescien2ficarena
• Asaserviceincorporatedbytheindustry
Parallelworlds
• Virtualvoice-overtransla2onatBBC
h"ps://www.journalism.co.uk/news/how-the-bbc-uses-virtual-voiceover-transla9on-to-reach-a-mul9lingual-audience/s2/a620320/
Reachingtheindustry
• Standardiza2onwork
• Anexample:ISOWork– ISO/IECJTC1/SC35Userinterfaces(WG6:accessibility)
• Anewstandardonaudiosub2tles– Differentbackgroundsandinterests– Reachingconsensus(content,terminology,etc.)
Thefuture?
• Evolu2onfromdescrip2vetoothertypesofresearch(notforgetngconceptualframeworks)
• Adap2ngtoneweconomiesandtechnologieswithoutforgetngthepast
• AlignmentwithEUpoliciesversusindependentresearch?
• Transferingknowledgetosociety• Coopera2on:interna2onal,interdisciplinary
AVTOdyssey:VoyagetotheFutureAnnaMatamalaUniversitatAutònomadeBarcelonaTransMediaCataloniaresearchgroupanna.matamala@uab.catIntermedia,14-16April2016FFI2015-62522-ERC,2014SGR0027,FFI2015-64038-P