High-Performance Distributed High-Performance Distributed Multimedia Computing Multimedia Computing

Frank Seinstra, Jan-Mark GeusebroekFrank Seinstra, Jan-Mark Geusebroek

Intelligent Systems Lab Intelligent Systems Lab AmsterdamAmsterdam

Informatics InstituteInformatics InstituteUniversity of AmsterdamUniversity of Amsterdam

MultimediaN (BSIK MultimediaN (BSIK Project)Project)

MultimediaN and DAS-3MultimediaN and DAS-3

Van Essen et Van Essen et al. Science al. Science 255, 1999.255, 1999.

MultimediaN and high-MultimediaN and high-performance computingperformance computing

A Real Problem, part 1…A Real Problem, part 1…

News Broadcast - September 21, 2005 (News Broadcast - September 21, 2005 (see see

video1.wmvvideo1.wmv))

Police investigating over 80.000 (!) CCTV recordingsPolice investigating over 80.000 (!) CCTV recordings First match found no earlier than 2.5 months after July First match found no earlier than 2.5 months after July

7 attacks7 attacks

automaticautomatic

analysis?analysis?

Image/Video Content AnalysisImage/Video Content Analysis

Lots Lots ofof research + benchmark evaluations: research + benchmark evaluations:– PASCAL-VOC (10,000+ images), TRECVID (200+ hours of video)PASCAL-VOC (10,000+ images), TRECVID (200+ hours of video)

A Problem of scale:A Problem of scale:– At least 30-50 hours of processing time per hour of video!At least 30-50 hours of processing time per hour of video!

Beeld&Geluid => 20.000 hours of TV broadcasts per yearBeeld&Geluid => 20.000 hours of TV broadcasts per year NASA => over 850 Gb of hyper-spectral image data per dayNASA => over 850 Gb of hyper-spectral image data per day London Underground => over 120.000 years of processing … !!!London Underground => over 120.000 years of processing … !!!

High Performance High Performance ComputingComputing Solution:Solution:

– Very, very large scale parallel and distributed Very, very large scale parallel and distributed computingcomputing

New Problem:New Problem:– Very, very complicated softwareVery, very complicated software

Solution:tool to make

parallel & distributed computing

transparent to user

- familiar programming- easy execution

Wide-Area Grid Systems

UseUserr Beowulf-type Beowulf-type

ClustersClusters

Since 1998:

“Parallel-Horus”

Parallel-Horus: Features (1)Parallel-Horus: Features (1)

Parallel-HorusParallel-Horus

Parallelizable PatternsParallelizable Patterns

Sequential programming:Sequential programming:

Sequential APISequential API

Seinstra et al., Seinstra et al., Parallel ComputingParallel Computing, 28(7-8):967-993, August 2002, 28(7-8):967-993, August 2002

+/- 18 patterns +/- 18 patterns (MPI)(MPI)

Parallel-Horus: Features (2)Parallel-Horus: Features (2)

Lazy Parallelization:Lazy Parallelization:

Seinstra et al., Seinstra et al., IEEE Trans. Par. Dist. Syst.IEEE Trans. Par. Dist. Syst., 15(10):865-877, October , 15(10):865-877, October

20042004

Don’t do this:Don’t do this:

ImageOpImageOpImageOpImageOp ScatterScatterScatterScatter GatherGather GatherGather

Do this:Do this:

ImageOpImageOpScatterScatter Avoid CommunicationAvoid Communication ImageOpImageOp GatherGatherOn the fly!On the fly!

Extensions for Distributed Extensions for Distributed ComputingComputing

Parallel

Horus

Client

Parallel

Horus

Client

Wide-Area Multimedia Services:Wide-Area Multimedia Services:

Parallel

Horus

Server

Parallel

Horus

Servers

Parallel

Horus

Servers

User transparency?User transparency? Abstractions & techniques?Abstractions & techniques? Grid connectivity problems?Grid connectivity problems?

Color-Based Object Recognition Color-Based Object Recognition (1)(1)

Our Solution:Our Solution:– Place ‘retina’ over input imagePlace ‘retina’ over input image– Each of 37 ‘retinal areas’ serves as a ‘receptive field’Each of 37 ‘retinal areas’ serves as a ‘receptive field’– For each receptive field:For each receptive field:

Obtain set of local histograms, invariant to shading / lightingObtain set of local histograms, invariant to shading / lighting Estimate Weibull parameters ß and Estimate Weibull parameters ß and γγ for each histogram for each histogram

– Hence: scene description by set of 37x4x3 = 444 Hence: scene description by set of 37x4x3 = 444 parametersparameters

+ =

Geusebroek, British Machine Vision Conference, 2006.Geusebroek, British Machine Vision Conference, 2006.

Color-Based Object Recognition Color-Based Object Recognition (2)(2)

Learning phase:Learning phase:– Set of 444 parameters is stored in databaseSet of 444 parameters is stored in database– So: learning from 1 example, under single So: learning from 1 example, under single

visual settingvisual setting

Recognition phase:Recognition phase:– Validation by showing objects under at least 50 different Validation by showing objects under at least 50 different

conditions:conditions: Lighting directionLighting direction Lighting colorLighting color Viewing positionViewing position

““a a hedgehog”hedgehog”

Amsterdam Library of Object Images Amsterdam Library of Object Images (ALOI)(ALOI)

In laboratory setting:In laboratory setting: 300 objects correctly recognized under all (!) visual conditions300 objects correctly recognized under all (!) visual conditions 700 remaining objects ‘missed’ under extreme conditions only700 remaining objects ‘missed’ under extreme conditions only

Geusebroek et al., Geusebroek et al., Int. J. Comput. Vis..Int. J. Comput. Vis.. 61(1):103-112, January 61(1):103-112, January 20052005

See also: See also: http://www.science.uva.nl/~fjseins/aibo.htmlhttp://www.science.uva.nl/~fjseins/aibo.html

Example: Object RecognitionExample: Object Recognition

Example: Object RecognitionExample: Object Recognition

Demonstrated live (a.o.) at ECCV 2006, June 8-11, 2006, Graz, Demonstrated live (a.o.) at ECCV 2006, June 8-11, 2006, Graz, AustriaAustria

(see video2.wmv)(see video2.wmv)

Performance / Speedup on Performance / Speedup on DAS-2DAS-2

0

8

16

24

32

40

48

56

64

0 8 16 24 32 40 48 56 64

Nr. of CPUs

Sp

eed

up

linear

client

Recognition on single machine: +/- 30 secondsRecognition on single machine: +/- 30 seconds Using multiple clusters: up to 10 frames per Using multiple clusters: up to 10 frames per

secondsecond Insightful: even ‘distant’ clusters can be used Insightful: even ‘distant’ clusters can be used

effectively for close to ‘real-time’ recognitioneffectively for close to ‘real-time’ recognition

0

16

32

48

64

80

96

0 16 32 48 64 80 96

Nr. of CPUs

Sp

eed

up

linear

client

Single cluster, client side Single cluster, client side speedupspeedup

Four clusters, client side Four clusters, client side speedupspeedup

Current & Future WorkCurrent & Future Work

Very Large-Scale Distributed Multimedia Computing:Very Large-Scale Distributed Multimedia Computing:– Overcome practical annoyances:Overcome practical annoyances:

Software portability, firewall circumvention, authentication, …Software portability, firewall circumvention, authentication, …

– Optimization and efficiency:Optimization and efficiency: Tolerant to dynamic Grid circumstances, …Tolerant to dynamic Grid circumstances, … Systematic integration of MM-domain-specific knowledge, …Systematic integration of MM-domain-specific knowledge, …

– Deal with non-trivial communication patterns:Deal with non-trivial communication patterns: Heavy intra- & inter-cluster communication, …Heavy intra- & inter-cluster communication, …

– Reach the end users:Reach the end users: Programming models, execution scenarios, …Programming models, execution scenarios, …

Collaboration with VU Collaboration with VU (Prof. Henri Bal)(Prof. Henri Bal) & GridLab & GridLab– Ibis:Ibis: www.cs.vu.nl/ibis/ www.cs.vu.nl/ibis/– Grid Application Toolkit: Grid Application Toolkit: www.gridlab.orgwww.gridlab.org

ConclusionsConclusions

Effective integration of results from two largely Effective integration of results from two largely distinct research fieldsdistinct research fields

Ease of programming => quick solutionsEase of programming => quick solutions With DAS-3 / StarPlane we can start to take on With DAS-3 / StarPlane we can start to take on

much more complicated problemsmuch more complicated problems But most of all:But most of all:

– DAS-3 very significant for future MM researchDAS-3 very significant for future MM research

The EndThe End

(see video3.avi)(see video3.avi)