Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Convolutional Neural Network forAppliance Recognition in Energy

Disaggregation (NILM)

Anthony FAUSTINE(NM-AIST), Prof.Nerey Mvungi(UDSM), Dr. KisangiriMichael(NM-AIST) and Dr. Shubi Kaijage(NM-AIST).

Data Science in Africa Workshop (DSA2017) - Arusha,Tanzania.

21 July 2017

NMAIST PhD 21 July 2017 1 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Outline

1 Introduction and Motivation

2 Hybrid DNN-HMM

3 CNN Appliance Recoginition

4 Data Set Development

5 Conclusion

NMAIST PhD 21 July 2017 2 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Presenter Bio

• PhD student at Nelson Mandela African Institution ofScience and Technology,

• Research : Applied machine learning and signalprocessing for computational sustainability.• Hybrid HMM-DNN for energy dis-aggregation problem.

• co-founder pythontz [https://pythontz.github.io]• ass.Lecturer : the University of Dodoma• blog : [https://sambaiga.github.io]

NMAIST PhD 21 July 2017 3 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Energy Disaggregation Problem.

FIGURE – Huss A.(2015)

A source separation problem (signalprocessing problem)⇒ Separateaggregate power signal

y(t) =∑

t∈{1,...,T}x(t) + σ(t)

into all source (appliance) signals.x(t) : t ∈ {1 . . .T}

survey-paper :https://arxiv.org/abs/1703.00785NMAIST PhD 21 July 2017 4 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Outline

1 Introduction and Motivation

2 Hybrid DNN-HMM

3 CNN Appliance Recoginition

4 Data Set Development

5 Conclusion

NMAIST PhD 21 July 2017 5 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

NILM Algorithm Development

State-of-the art NILM algorith : Hidden Markov Model (HMM) vsDeep neural networks (DNN)

HMM 1 DNN 2

+ suitable for controlled multi-state loads + easier to generalize to similar appliances+ easy to train and can work in real-time +very powerful- difficult to generalize to similar appliances - require lots of data for model training- limited to few appliances - training sensitive to hyperparameters

Open-Issue : Combine DNN and HMM for real-time and generalizedenergy disaggregation.

1. Makonin S., eta.l (2015), Exploiting HMM Sparsity to Perform OnlineReal-Time Nonintrusive Load Monitoring.2. Kelly J. eta.l (2015), Neural NILM : Deep Neural Networks Applied to

Energy Disaggregation.NMAIST PhD 21 July 2017 6 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

What Next

Hybrid DNN-HMM model

NMAIST PhD 21 July 2017 7 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Hybrid CNN-HMM

Appliance Modeling

For each appliance k , the HMMparameters are :

λ(k) = {π(k),A(k), θ(k),B(k)}

• π(k) ⇒ the initial probability of anappliance state sk (1) at time t = 1.

• A(k) = P(sk (t) = i|sk (t−1) = j)⇒the transition probability.

• θ(k) ∼ N (µsk (t), σsk (t))⇒ theappliance model.

• B(k) = P(∆yt |sk(t) = j)⇒estimated from CNN⇒ P(sk (t)|∆yt )

P(sk (t)).

The CNN gets a window of 2B + 1 of inputfeatures such that : ∆yt = [∆ymax (0, t −B), . . .∆yt , . . .∆ymin(T , t + B)]

NMAIST PhD 21 July 2017 8 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Hybrid CNN-HMM

Appliance Modeling

For each appliance k , the HMMparameters are :

λ(k) = {π(k),A(k), θ(k),B(k)}

• π(k) ⇒ the initial probability of anappliance state sk (1) at time t = 1.

• A(k) = P(sk (t) = i|sk (t−1) = j)⇒the transition probability.

• θ(k) ∼ N (µsk (t), σsk (t))⇒ theappliance model.

• B(k) = P(∆yt |sk(t) = j)⇒estimated from CNN⇒ P(sk (t)|∆yt )

P(sk (t)).

The CNN gets a window of 2B + 1 of inputfeatures such that : ∆yt = [∆ymax (0, t −B), . . .∆yt , . . .∆ymin(T , t + B)]

NMAIST PhD 21 July 2017 8 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Hybrid CNN-HMM

Appliance Modeling

For each appliance k , the HMMparameters are :

λ(k) = {π(k),A(k), θ(k),B(k)}

• π(k) ⇒ the initial probability of anappliance state sk (1) at time t = 1.

• A(k) = P(sk (t) = i|sk (t−1) = j)⇒the transition probability.

• θ(k) ∼ N (µsk (t), σsk (t))⇒ theappliance model.

• B(k) = P(∆yt |sk(t) = j)⇒estimated from CNN⇒ P(sk (t)|∆yt )

P(sk (t)).

The CNN gets a window of 2B + 1 of inputfeatures such that : ∆yt = [∆ymax (0, t −B), . . .∆yt , . . .∆ymin(T , t + B)]

NMAIST PhD 21 July 2017 8 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Hybrid CNN-HMM :Learning and Inference

Joint probability probability of all sequences :

P(Y ,∆Y ,Sk |λ) = π(k)sk (1)

· B(k)sk (1)

· P(xk (1) ≤ y1|sk (1), λ)

T∏t=2

Ask (t),sk (t−1) · P(xk (t) ≤ yt |sk (t), λ) · Bsk (t)

Training :

• Bauch-welch algorithm underMLE to train initial GMM-HMM.

• Stochastic Gradient Descent(SDG) to train initial CNN.

• Embedded-Viterbi-algorithm to

train CNN-HMM

Inference and Signal Extraction :

• virtebi algorithm :sk = arg max

s[P(Y ,∆Y ,Sk |λ)]

• Power estimation : xk (t) = µsk (t)

NMAIST PhD 21 July 2017 9 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Hybrid CNN-HMM :Learning and Inference

Joint probability probability of all sequences :

P(Y ,∆Y ,Sk |λ) = π(k)sk (1)

· B(k)sk (1)

· P(xk (1) ≤ y1|sk (1), λ)

T∏t=2

Ask (t),sk (t−1) · P(xk (t) ≤ yt |sk (t), λ) · Bsk (t)

Training :

• Bauch-welch algorithm underMLE to train initial GMM-HMM.

• Stochastic Gradient Descent(SDG) to train initial CNN.

• Embedded-Viterbi-algorithm to

train CNN-HMM

Inference and Signal Extraction :

• virtebi algorithm :sk = arg max

s[P(Y ,∆Y ,Sk |λ)]

• Power estimation : xk (t) = µsk (t)

NMAIST PhD 21 July 2017 9 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Outline

1 Introduction and Motivation

2 Hybrid DNN-HMM

3 CNN Appliance Recoginition

4 Data Set Development

5 Conclusion

NMAIST PhD 21 July 2017 10 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Appliance recognition

Appliance recognition is an important sub-task of the NILMproblem.

• Several approaches for thissub-task 3, 4

• Deep-learning have receivedlittle attention.

3. Gao, Jingkun, et al (2015). "A feasibility study of automated plug- loadidentification from high-frequency measurements."4. Karim Barsim, et al 2016. "Neural Neural Network Ensembles to

Real-time Identification of Plug-level Appliance Measurement"NMAIST PhD 21 July 2017 11 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Objective

Goal Appliance recognition :Apply CNN to recognize the labeled appliances once they areswitched on.

Data : Plug load Appliance Identification Dataset (PLAID 5).

• 55 households in USA

• 11 different appliances

• sub-metered on events ofthe appliances (1074 total).

• Sampled at 30 kHz.

5. http ://plaidplug.com/NMAIST PhD 21 July 2017 12 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Appliance Signature

VI Binary Image Voltage-Current (VI) image feature during thesteady state operation 6.• Obtained by converting the VI trajectories into binary image.

−100 0 100Voltage

−0.4

−0.3

−0.2

−0.1

0.0

0.1

0.2

0.3

0.4

Curr

ent

VI trajectory of a Fridge

0 5 10 15

0

2

4

6

8

10

12

14

VI binary image of a Fridge

−100 0 100Voltage

−1.0

−0.5

0.0

0.5

1.0

Curr

ent

VI trajectory of Laptop

0 5 10 15

0

2

4

6

8

10

12

14

VI binary image of Laptop

6. Gao, Jingkun, et al (2015). "A feasibility study of automated plug-loadidentification from high-frequency measurements."

NMAIST PhD 21 July 2017 13 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Model Definition

NMAIST PhD 21 July 2017 14 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Experment

• Training : Leave-house-outcross validation.

• Metrics : Precision (PR),Recall (RE), and F-Measure(F-1 score).

PR =TP

TP + FP(1)

RE =TP

TP + FN(2)

FM =2× (PR × RE)

PR + RE(3)

where :

• TP⇒ correct claim thedetected event wastriggered by an appliance.

• FP⇒ incorrect claim thatdetected event wastriggered by an appliance.

• FN⇒ indicates thatappliance used was notidentified.

NMAIST PhD 21 July 2017 15 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Results

FM Recall Precision0.0

0.2

0.4

0.6

0.8

1.0

Sco

res

(Hig

her

isb

ette

r)Evaluation Results

RandomForest

Convolutional

code : https://github.com/sambaiga/cnn-appliance-detector

NMAIST PhD 21 July 2017 16 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Outline

1 Introduction and Motivation

2 Hybrid DNN-HMM

3 CNN Appliance Recoginition

4 Data Set Development

5 Conclusion

NMAIST PhD 21 July 2017 17 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Energy Data set Development

Develop tool and establish resource pertaining to residentialelectrical energy consumption-data set in Tanzania.

• RF-based WSN for individualappliances and aggregatepower monitoring

• LUKU-pulse-sensor to collectaggregate power consumptionusing LED pulse found onexisting LUKU meter.

• Experiment the tool in somebuildings for one year⇒establishment of the energyconsumption data-set.

NMAIST PhD 21 July 2017 18 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Outline

1 Introduction and Motivation

2 Hybrid DNN-HMM

3 CNN Appliance Recoginition

4 Data Set Development

5 Conclusion

NMAIST PhD 21 July 2017 19 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

Conclusion

Open Challenges & Opportunities :• Data, Data, Data• NILM in renewable sources⇒ Improve battery energy

storage.• NILM to predict electrical fires accidents or solve electricity

theft problem.• Develop realistic simulators for simulating disaggregated

electricity data.• Explore different Deep Learning architecture for NILM

problem.

NMAIST PhD 21 July 2017 20 / 21

Introduction and Motivation Hybrid DNN-HMM CNN Appliance Recoginition Data Set Development Conclusion

THANK YOU

NMAIST PhD 21 July 2017 21 / 21