Robust Mesh WatermarkingRobust Mesh Watermarking

Emil PraunEmil Praun

Hugues HoppeHugues Hoppe

Adam FinkelsteinAdam Finkelstein

Emil PraunEmil Praun

Hugues HoppeHugues Hoppe

Adam FinkelsteinAdam Finkelstein

Princeton UniversityPrinceton University

Microsoft ResearchMicrosoft Research

Princeton UniversityPrinceton University

Princeton UniversityPrinceton University

Microsoft ResearchMicrosoft Research

Princeton UniversityPrinceton University

Watermarking ApplicationsWatermarking ApplicationsWatermarking ApplicationsWatermarking Applications

•Authentication / localization of changesAuthentication / localization of changesFragile watermarksFragile watermarks

•Ownership protectionOwnership protectionRobust watermarksRobust watermarks

•Tracing of distribution channelsTracing of distribution channels FingerprintsFingerprints

•Authentication / localization of changesAuthentication / localization of changesFragile watermarksFragile watermarks

•Ownership protectionOwnership protectionRobust watermarksRobust watermarks

•Tracing of distribution channelsTracing of distribution channels FingerprintsFingerprints

Watermarking ApplicationsWatermarking ApplicationsWatermarking ApplicationsWatermarking Applications

•Authentication / localization of changesAuthentication / localization of changesFragile watermarksFragile watermarks

•Ownership protectionOwnership protectionRobust watermarksRobust watermarks

•Tracing of distribution channelsTracing of distribution channels FingerprintsFingerprints

•Authentication / localization of changesAuthentication / localization of changesFragile watermarksFragile watermarks

•Ownership protectionOwnership protectionRobust watermarksRobust watermarks

•Tracing of distribution channelsTracing of distribution channels FingerprintsFingerprints

3.3. How can Alice prove ownership?How can Alice prove ownership?(and make Bob pay her a lot of money)(and make Bob pay her a lot of money)

3.3. How can Alice prove ownership?How can Alice prove ownership?(and make Bob pay her a lot of money)(and make Bob pay her a lot of money)

2.2. Bob sells it as his own.Bob sells it as his own.2.2. Bob sells it as his own.Bob sells it as his own.

Motivating ScenarioMotivating Scenario

1. Alice creates a 3D shape,1. Alice creates a 3D shape,and publishes it on the web.and publishes it on the web.

1. Alice creates a 3D shape,1. Alice creates a 3D shape,and publishes it on the web.and publishes it on the web.

Digital WatermarksDigital Watermarks

originaloriginaldocumentdocument

watermarkwatermark

kept secretkept secret

insertioninsertioninsertioninsertion

Hidden in data!Hidden in data!Hidden in data!Hidden in data!publishedpublishedpublishedpublished

extractionextractionextractionextractiondetecteddetected

watermarkwatermarkdetecteddetected

watermarkwatermark

suspectsuspectdocumentdocumentsuspectsuspect

documentdocument

““attack”attack”““attack”attack”

??

• Filtering & smoothingFiltering & smoothing

• A/D & D/A conversionsA/D & D/A conversions

• ScalingScaling

• RotationRotation

• CroppingCropping

• Filtering & smoothingFiltering & smoothing

• A/D & D/A conversionsA/D & D/A conversions

• ScalingScaling

• RotationRotation

• CroppingCropping

Incidental AttacksIncidental Attacks

Malicious AttacksMalicious Attacks

• Adding noiseAdding noise

• Adding another watermarkAdding another watermark

• ResamplingResampling

• Statistical analysisStatistical analysis

• Adding noiseAdding noise

• Adding another watermarkAdding another watermark

• ResamplingResampling

• Statistical analysisStatistical analysis

Our GoalOur Goal

Watermarking scheme for 3D models:Watermarking scheme for 3D models:

• Robust against attacks Robust against attacks

• Works on arbitrary meshesWorks on arbitrary meshes

• Preserves original connectivityPreserves original connectivity

• Imperceptible Imperceptible

Watermarking scheme for 3D models:Watermarking scheme for 3D models:

• Robust against attacks Robust against attacks

• Works on arbitrary meshesWorks on arbitrary meshes

• Preserves original connectivityPreserves original connectivity

• Imperceptible Imperceptible

Previous WatermarkingPrevious Watermarking

[Cox et al. ’97][Cox et al. ’97] Introduce spread-spectrum for imagesIntroduce spread-spectrum for images

[Ohbuchi et al. ’98][Ohbuchi et al. ’98] 3 schemes fragile under resampling3 schemes fragile under resampling

[Kanai et al. ’98][Kanai et al. ’98] Requires subdivision connectivity meshesRequires subdivision connectivity meshes

[Benedens ’99][Benedens ’99] Redistributes face normals by moving verticesRedistributes face normals by moving vertices

[Cox et al. ’97][Cox et al. ’97] Introduce spread-spectrum for imagesIntroduce spread-spectrum for images

[Ohbuchi et al. ’98][Ohbuchi et al. ’98] 3 schemes fragile under resampling3 schemes fragile under resampling

[Kanai et al. ’98][Kanai et al. ’98] Requires subdivision connectivity meshesRequires subdivision connectivity meshes

[Benedens ’99][Benedens ’99] Redistributes face normals by moving verticesRedistributes face normals by moving vertices

Spread-Spectrum WatermarkingSpread-Spectrum Watermarking

Transform to frequency spaceTransform to frequency space

[Cox et al. ’97][Cox et al. ’97]

Transform to frequency spaceTransform to frequency space

[Cox et al. ’97][Cox et al. ’97]

frequencyfrequency domaindomainimageimage

DCTDCT

Spread-SpectrumSpread-Spectrum

Image basis function Image basis function DCT coefficient DCT coefficient Image basis function Image basis function DCT coefficient DCT coefficient

Salient features Salient features largest coefficients largest coefficients

Perturb coefficients slightly to embed signalPerturb coefficients slightly to embed signal

Salient features Salient features largest coefficients largest coefficients

Perturb coefficients slightly to embed signalPerturb coefficients slightly to embed signal

Our ApproachOur Approach

Extend spread-spectrum method to meshesExtend spread-spectrum method to meshes

Problem: no DCTProblem: no DCTSolution: multiresolution representationSolution: multiresolution representation

Problem: no natural samplingProblem: no natural sampling

Solution: registration & resamplingSolution: registration & resampling

Extend spread-spectrum method to meshesExtend spread-spectrum method to meshes

Problem: no DCTProblem: no DCTSolution: multiresolution representationSolution: multiresolution representation

Problem: no natural samplingProblem: no natural sampling

Solution: registration & resamplingSolution: registration & resampling

Replacing DCT Basis FunctionsReplacing DCT Basis Functions

Multiresolution Multiresolution frequency information frequency information

Progressive mesh [Hoppe ’96]Progressive mesh [Hoppe ’96]

Multiresolution Multiresolution frequency information frequency information

Progressive mesh [Hoppe ’96]Progressive mesh [Hoppe ’96]

cosine basiscosine basis

imageimage meshmesh

??

Multiresolution NeighborhoodsMultiresolution Neighborhoods

Naturally correspond to important featuresNaturally correspond to important features

Provide hints on allowable perturbationProvide hints on allowable perturbation

Naturally correspond to important featuresNaturally correspond to important features

Provide hints on allowable perturbationProvide hints on allowable perturbation

correspondingcorrespondingmesh regionmesh region

vertexvertexneighborhoodneighborhood

Scalar Basis Function i Scalar Basis Function i

amplitudeamplitude ii directiondirection

ddii

radius

disp

lace

men

t

Watermark InsertionWatermark Insertion

Construct basis functions Construct basis functions 11 … … mm Construct basis functions Construct basis functions 11 … … mm

Watermark InsertionWatermark Insertion

Construct basis functions Construct basis functions 11 … … mm

Perturb each vertex:Perturb each vertex:

Construct basis functions Construct basis functions 11 … … mm

Perturb each vertex:Perturb each vertex:i

m

1i

ijjj w '

idvv i

m

1i

ijjj w '

idvv

watermark coefficientwatermark coefficientwatermark coefficientwatermark coefficient

basis function coefficientbasis function coefficientbasis function coefficientbasis function coefficient

watermark directionwatermark directionwatermark directionwatermark direction

wBvv ' wBvv 'Matrix system:Matrix system:Matrix system:Matrix system:

Watermark ExtractionWatermark Extraction

Get points Get points v*v* on attacked mesh on attacked mesh surfacesurface corresponding to original mesh corresponding to original mesh verticesvertices vv

Use Use samesame basis functions basis functions 11 … … mm

and hence same matrix and hence same matrix BB

Solve least-squares system for w*:Solve least-squares system for w*:

Get points Get points v*v* on attacked mesh on attacked mesh surfacesurface corresponding to original mesh corresponding to original mesh verticesvertices vv

Use Use samesame basis functions basis functions 11 … … mm

and hence same matrix and hence same matrix BB

Solve least-squares system for w*:Solve least-squares system for w*:)( vvB w

False-Positive ProbabilityFalse-Positive Probability

Correlation Correlation = < w*,w > = < w*,w >

PPfpfp computed from computed from and and mm

using Student’s using Student’s tt-test -test

Declare watermark present ifDeclare watermark present if PPfpfp < < PPthreshthresh ( e.g. ( e.g. PPthreshthresh = 10 = 10-6-6 ) )

Correlation Correlation = < w*,w > = < w*,w >

PPfpfp computed from computed from and and mm

using Student’s using Student’s tt-test -test

Declare watermark present ifDeclare watermark present if PPfpfp < < PPthreshthresh ( e.g. ( e.g. PPthreshthresh = 10 = 10-6-6 ) )

ProcessProcess

(1) original mesh (2) watermarked (exaggerated)

(3) suspect mesh (4) registered (5) resampled

Registration & ResamplingRegistration & Resampling

Registration:Registration:

• [Chen & Medioni ’92][Chen & Medioni ’92]

Resampling choices:Resampling choices:

• Closest point projectionClosest point projection

• Ray-casting along local normalRay-casting along local normal

• Global deformation of originalGlobal deformation of original

Registration:Registration:

• [Chen & Medioni ’92][Chen & Medioni ’92]

Resampling choices:Resampling choices:

• Closest point projectionClosest point projection

• Ray-casting along local normalRay-casting along local normal

• Global deformation of originalGlobal deformation of original

Global DeformationGlobal Deformation

Deform original mesh to fit suspect meshDeform original mesh to fit suspect mesh

Minimize:Minimize:

+ Inter-mesh distanceInter-mesh distance( vertex springs )( vertex springs )

+ DeformationDeformation( edge springs )( edge springs )

+ Penalty for flipped trianglesPenalty for flipped triangles

Accurate, but slowAccurate, but slow

Deform original mesh to fit suspect meshDeform original mesh to fit suspect mesh

Minimize:Minimize:

+ Inter-mesh distanceInter-mesh distance( vertex springs )( vertex springs )

+ DeformationDeformation( edge springs )( edge springs )

+ Penalty for flipped trianglesPenalty for flipped triangles

Accurate, but slowAccurate, but slow

OptimizedOptimized meshmesh

Suspect meshSuspect meshSuspect meshSuspect mesh

ResultsResults

1010-29-291010-7-7

similaritysimilarity1/2 faces1/2 faceswatermarked meshwatermarked mesh

1010-6-6 1010-7-7

watermarked meshwatermarked mesh 22ndnd watermark watermarknoisenoise

ResultsResults

1010-12-12 1010-2-2

smoothingsmoothing all attacksall attackswatermarked meshwatermarked mesh

watermarked meshwatermarked mesh croppedcropped1/8 faces1/8 faces

1010-13-13 00

SummarySummary

Robust watermarking for 3D meshesRobust watermarking for 3D meshes

• Spread-spectrumSpread-spectrum

• Basis functions from multiresolution analysisBasis functions from multiresolution analysis

• Resampling as global optimizationResampling as global optimization

Resilient to a variety of attacksResilient to a variety of attacks

Robust watermarking for 3D meshesRobust watermarking for 3D meshes

• Spread-spectrumSpread-spectrum

• Basis functions from multiresolution analysisBasis functions from multiresolution analysis

• Resampling as global optimizationResampling as global optimization

Resilient to a variety of attacksResilient to a variety of attacks

Future WorkFuture Work

Consider other attacks:Consider other attacks:

• General affine and projective transformsGeneral affine and projective transforms

• Free-form deformations! Free-form deformations! [StirMark by Petitcolas] [StirMark by Petitcolas]

Explore other basis functionsExplore other basis functions

• e.g. [Guskov et al. ’99]e.g. [Guskov et al. ’99]

Fast mesh recognition Fast mesh recognition web crawler web crawler

Consider other attacks:Consider other attacks:

• General affine and projective transformsGeneral affine and projective transforms

• Free-form deformations! Free-form deformations! [StirMark by Petitcolas] [StirMark by Petitcolas]

Explore other basis functionsExplore other basis functions

• e.g. [Guskov et al. ’99]e.g. [Guskov et al. ’99]

Fast mesh recognition Fast mesh recognition web crawler web crawler