WATER LEVEL DETECTIONWATER LEVEL DETECTION FOR FUNCTIONALLYFOR FUNCTIONALLY LAYERED VIDEO CODINGLAYERED VIDEO CODING

ICIP 07 2007.9.17ICIP 07 2007.9.17MP-P3: Scalable Video Coding , September 17, 14:30-17:30

M. IWAHASHI S. UDOMSIRI Y.IMAI S.MURAMATSU*Nagaoka Univ. of Technology, *Niigata University, Japan

http://tech.nagaokaut.ac.jp/index_en.html

Purpose of the ResearchPurpose of the Research

Water Level of Rivers via WEB siteWater Level of ivers via WEB siteMinistry of Land, Infrastructure and Transport Government of Japan

http://www.river.go.jp/

▲ safe▲ caution▲ dangerous× closed

Before…Before… After…After…● by Government● large scale, high cost

● by individual● friendly, ubiquitousg g y

ＩＰネットワークIP network

observation point

requirement to the system1. Transfer Water level regularly2. Video signal when necessaryg y

W t L l D t tiWater Level Detection from videofrom video

“Existing Approaches”Existing Approaches

Existing Method #1

A “board” in the water !

Detect these points by image recognition

W l l i d d

1 Y T k i t l “D l t f t t li id l l i t i i

Water level is detected. Takagi Lab.

1. Y. Takagi, et.al, “Development of a non-contact liquid level measuring system using image processing”, Water science and technology, vol. 37, no.12, pp.381-387, 1998.

2. Y. Takagi, et.al, “Development of a water level measuring system using image processing”, IWA conf. instrumentation, control and automation, pp.309-316, 2001.

Existing Method #2

tical

nt

iatio

n

tect

ntal

line

s

Stain on the wall

Vert

diffe

ren

Det

Hor

izon Water level

Verticaldifferentiation

SubtractionExisting Method #2

* i t i

of framesExisting Method #2

* running water remains* Sensitive to rain drops

N. Tsunashima, M. Shiohara, S. Sasaki, J. Tanahashi, “Water level measurement using image processing”, Information processing society of Japan, Research report, Computer vision and image media, vol.121, no.15, pp.111-117, 2000.

Previously Proposed Methods #1 & #2 by the authors

Verticaldifferentiation

Horizontaldifferentiation

Wavelettransformd e e a o d e e a o a s o

Frame subtraction

ExistingMethod #2

No good

Frame

addition Pretty goodPreviouslyProposed

PreviouslyProposedaddition Method #1 Method #2

1. M. IWAHASHI, “Water Level Detection from Video with FIR filtering”, Sixteenth International Conference on Computer Communications and Networks (ICCCN), Aug. 2007.

2 M IWAHASHI S UDOMSIRI YIMAI S FUKUMA “Water Level Detection for River2. M.IWAHASHI, S.UDOMSIRI, Y.IMAI, S.FUKUMA, Water Level Detection for River Surveillance utilizing JP2K Wavelet Transform”, IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), pp.1766-1769, Oct. 2006.

Proposed Method- abstract -

“Functionally” Layered Coding

CWater levelWater levelComponent 1 2 3

Choos

マイクロビジョン（株）

Component 1 2 3

1st priority ●

se nece

Thumb nailThumb nail

p y

2nd priority ●

essary

non-priority ● ● ●

y compponent

videovideo

t(s)

ウェブドゥ（株）

Previous method #2“recognition”

MPEG“compression”x =recognition

Video Input

“compression”

SynchronousFrame Addition

Motion Compensationd P di ti

Recognition Compression

Frame Addition

WaveletTransform

and Prediction

Discrete CosineTransform

ensor node

Water LevelDetection Encoding Encoding

Se

Priority Layer Non Priority LayerSensor node

Network

DecodingDecoding

Inv.Trans.Inv.Trans. eceivers

Water Level

Inv.Trans.Inv.Trans.

High QualityOutput Video

Low QualityOutput Video

Re

Proposed Methodoposed e odVideo InputS d

TemporalHaar Transform

V deo putSensor node

e

1) frame addition &

2) motion compensation

SpatialHaar Transform

ensor node

) p

temporal Haar trans.

Encoding EncodingEncoding

kSe

3) Wavelet trans. &

4) DCT2nd Priority Layer Non Priority Layer1st Priority Layer

Network

spatial Haar trans.

Water LevelDetection

DecodingDecoding

Inv.Trans.Inv.Trans.

Decoding

Receivers

Detection

Water LevelHigh QualityOutput Video

Low QualityOutput Video

Recognition

Proposed Method- details -

temporal Haar transforminput frames time -->

TemporalHaar Trans.

X(t1) X(t2)

TemporalHaar Trans.

X(t3) X(t4)

TemporalHaar Trans.

X(t5) X(t6)

TemporalHaar Trans.

X(t7) X(t8)

1) priority layer

same effect as the Haar Trans.

L(t1) H(t1)

Haar Trans.

L(t3) H(t3)

Haar Trans.

L(t5) H(t5)

Haar Trans.

L(t7) H(t7)

T l T l

frame addition

TemporalHaar Trans.

LL(t1) LH(t1)

TemporalHaar Trans.

LL(t5) LH(t5)

2) priority layer +

non priority layer

TemporalHaar Trans.

= perfect reconstruction

Haar Trans.

LLL(t1) H(t1) H(t3) LH(t1) LH(t5) H(t5) H(t7)LLH(t1)

non-priority layerpriority layers

“spatial” “spatial” transformWhich band is effective

2HＬ2LＬ HＬ22LＬ

Which band is effective for water level detection?

Land

Water level

ＬＬ HＬ2ＬH 2HH ＬＬ 1HＬ2ＬH 2HH

(a): original video

WaterＬH HH

(d): band signals (e): 1st priority layer (g): 2nd priority layer

1ＬH HH1

temporal transform spatial transform(thumb nail video)

ML estimation

Water level

(b): priority layer(c): non-priority layer (f): discrimination result (h):feature value

level

ML estimation & Water level detectionWater level detection

Spatial ML Temporal

DetectedWaterp

transform LL HL estimationp

transform level

After frame addition LH HH

Feature vector Discrimination Feature value D t ti

Feature vectorland water

result of each line Detection errorDiscrimination

error:land:water

Teacher signals Feature vector

Dimension of the Feature Vector= The number of band signals

b d f ML i i

IdealWater

to be used for ML estimation level

ExperimentalExperimental

resultsresults

Sample #2No.2

(a) original frame (c) after spatial transform(b) after temporal transform

ππ πNo.2 1HH1LH

ω

2ω 2ω

π π00 ω 1ω

2ω

π0

1HL2HH2LH

2HL

1ωπ π00 1ω 1ω π0(a) Land region (b) Water region (c) Difference

Which band should be includedWhich band should be included into the 1st layer?

One dimensional feature vector

30

35

%]

1LL2LL3LL

rror

rror Existing…

discrimination only

20

25

Err

or [%

1LH

3HL

tion e

tion e discrimination only

Proposed

10

15

20

min

atio

n 1HL

1HH2LH2HL3LH

minat

minat Proposed…

discrimination & Data size

5

10

Dis

crim

1 stage2 stage3 stage

1HH

2HH3HH

Discrim

Discrim

00 20 40 60 80 100 120 140

Data Size [KB]

3 stageDD

Data Size [KB]Data sizeData size This criteria is added.

Multi dimensional feature vector

65

70

75

Discrimination Error→ c6 is the minrr

or

rror

55

60

65

Erro

r

→ c6 is the min.(existing)ｃ７

tion e

tion e

45

50

orm

aliz

ed

c2

3

c4 Data Sizec7 is the minm

inat

minat

30

35

40N c3c6

existing

c1proposed

→ c7 is the min.

Discrim

Discrim

25

30

25 30 35 40 45 50 55 60 65 70 75Normalized Data Size

Both of them→ c1 is the min.

DD

Normalized Data Size(proposed)Data sizeData size

Multi dimensional feature vector

sample 2

# b dnormalized normalized normalized

130 

%%# bandnormalizeddata size

normalizederror

normalizeddistance

NB NR NDstdev 10.00 10.00 5.79mean 50 00 50 00 50 62

110 

120 

t3.7

%

32.5

%

mean 50.00 50.00 50.62min 34.54 40.45 40.36max 64.63 70.81 62.10

c1 2HH 38.70 41.95 40.36 ← d

80

90 

100 

emer

it

Mer

it3

c2 3HH-2HH 39.95 43.71 41.87c3 2LH-2HH 44.70 43.25 43.98c4 2LH-3HH-2HH 45.96 44.85 45.41c5 2LH-3HH 40.54 56.71 49.29

proposed

60 

70 

80 

DeM

c6 2HH-1HH 57.37 40.45 49.64c7 3HH 34.54 63.03 50.82c8 3HH-2HH-1HH 58.62 42.25 51.10c9 3HH-1HH 53 21 50 35 51 80

← existing

50 

normalized data size

normalized error

normalized distance

c9 3HH 1HH 53.21 50.35 51.80c10 2LH-2HH-1HH 63.37 41.15 53.43c11 2LH-3HH-1HH 59.21 49.29 54.48c12 2LH-3HH-2HH-1HH 64.63 43.05 54.9113 2LH 39 29 67 13 55 00

proposed/existing

c13 2LH 39.29 67.13 55.00c14 2LH-1HH 57.96 52.01 55.06c15 1HH 51.95 70.81 62.10

Sample #1No.1

75

(a) original frame (b) after temporal transform (c) after spatial transform

130

60

65

70

75

100

110 

120 

130 

No.1

45

50

55

60

rmal

ized

Erro

r

c4 70

80 

90 

100 

30

35

40

No

c2c3c4

c1proposed=existing

50 

60 

70 

normalized  normalized  normalized 

2525 30 35 40 45 50 55 60 65 70 75

Normalized Data Size

p p gdata size error distance

(b) proposed/existing

Sample #3No.3

75 130 

N 3

(a) original frame (c) after spatial transform(b) after temporal transform

60

65

70

ror 100 

110 

120  No.3

45

50

55

orm

aliz

ed E

r

c3c1 70 

80 

90 

30

35

40No

c2

c3

c4proposed

c10existing

50 

60 

normalized d t i

normalized  normalized di t

2525 30 35 40 45 50 55 60 65 70 75

Normalized Data Size(b) proposed/existingdata size error distance

Sample #4No.4

75 130 

(a) original frame (c) after spatial transform(b) after temporal transform

60

65

70

or 100 

110 

120  No.4

45

50

55

orm

aliz

ed E

rr

c2 c3c4

70 

80 

90 

30

35

40No

c1proposed

c5existing 50 

60 

normalized d

normalized  normalized d

2525 30 35 40 45 50 55 60 65 70 75

Normalized Data Size

data size error distance

(b) proposed/existing

ExpectedExpected

resultsresults

What is expected in general ?p g

65

70

75 Proposed method- nearest to the origin.rr

or

rror

55

60

65

d Er

ror Existing method

- lowest vertically

ｃ７

tion e

tion e

1050 −⋅+=

mRNX X45

50

Nor

mal

ized

c2

3

c4Proposed m

erit

- lowest vertically.

minat

minat

}errordetection_data_size,{

,1050

∈

⋅+=

X

NXXσ

30

35

40N c3c6existing

c1proposed Existing

Merit

Dem

Discrim

Discrim

2

22 NBNRND +=25

30

25 30 35 40 45 50 55 60 65 70 75Normalized Data Size

MeritDD

Normalized Data Size

Data sizeData size

Conventional methods ...iti l / i lrecognition only / compression only

75 - recognitionn

50 -

(sacrifice)

%29.2911100 −=⎞⎜⎛ −o

gnitio

proposed●

50 %29.2912

100⎠

⎜⎝

reco

●●

existing25 - compression

(improved)ood←

50 -

25 -

75 - %36.35

221100 +=⎟

⎠⎞

⎜⎝⎛go

good← compression

Proposed method ...U ifi ti f iti & iUnification of recognition & compression

both of75 -n both of

recognition & compression50 -o

gnitio

compression(improved)

122 ⎞⎛proposed

●

50

reco

%23185

1221100 ⎟⎠

⎞⎜⎜⎝

⎛ −−

●●

existing25 -

ood←

%23.18+=

50 -

25 -

75 -

go

good← compression