Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes

transcript

Energy Cost Optimization in Water Distribution Systems Using Markov Decision ProcessesPaulo T. Fracasso, Frank S. Barnes and Anna H. R. Costa

University of Sao PauloDepartment of Electrical and Computer Engineering

Intelligent Techniques Laboratory

University of Sao PauloDepartment of Electrical and Computer Engineering

Intelligent Techniques Laboratory

AgendaAgenda

• Anatomy of Water Distribution Systems

• Problem relevancy

• Markov Decision Process

• Modeling a Water Distribution System as an MDP

• Monroe Water Distribution System

• Experiment results

• Conclusions

Water distribution system Water distribution system • It is a complex system composed by pipes, pumps and other

hydraulic components which provide water supply to consumers.

Focus of my work

Problem relevancyProblem relevancy• About 3% of US energy consumption (56 billion kWh) are used

for drinking water (Goldstein and Smith, 2002).

$2 billion/year

Source: Electric Power Research Institute,1994.4

MDP is a model for sequential decision making in fully observable environments when outcomes are uncertain.

Advantages of MDP compared to other techniques: Real world – operates in uncertain and dynamic domains Planning – generates control policies to sequential decisions Optimal solution – guarantees to achieve a higher future payoff

Disadvantages of MDP: Discrete domains (state and action) Course of dimensionality

Markov Decision Process - MDPMarkov Decision Process - MDP

MDP is defined as a tuple where: S is a discrete set of states (can be factored in Nv features):

A is a discrete set of actions:

T is a transition function where

R is a reward function where

Markov Decision Process - MDPMarkov Decision Process - MDPRTAS ,,,

ttt assPT ,|'',, 1

),...,(),...,,...,(,..., 11

ttt asrR ,|,

1,0: SAST

Solving an MDP consists in finding a policy , which is defined as a mapping from states to actions, s.t.

Bellamn’s equation allows to break a dynamic optimization problem into simpler sub-problems:

The optimal value of the utility is:

The optimal policy are the actions obtained from :

Markov Decision Process - MDPMarkov Decision Process - MDP

** '',,,maxarg

** '',,,max

Water Distribution System modeled as an MDPWater Distribution System modeled as an MDP

HHTS ,...,, 1

maxmin ,TTT

maxminmin ,...,, TTTTT

maxmin ,HHH

maxminmin ,...,, HHHHH

UUA ,...,1

)(),(),(1 tAtDtHftH

1,...,,0 UU

DC CCR

OPC PTtPwPTtPwC )()(

D PtPwxmaC )(

MarkovDecision

Processes

Constraints

Control policy

Final result:

Energy priceschema

Understand MDP resultsUnderstand MDP results

Control policy: Maps state variables into a set

of actions

States variables: everything that is important to control (tank level and time)

Set of actions: what you can manipulate (pumps)

Indicates controllability (avoid black region)

Correlated to demand curve

Understand MDP resultsUnderstand MDP results

Controller: Uses control policy map to

produce actions

Actions are based just on tank level and time

Easy to implement and fast to run in PLC (lookup table)

Monroe Water Distribution System

Characteristics: 11 pumps

1 storage tank

4 pressure monitoring

40k people served

182 miles of pipes

Diameters varyingfrom 2 to 42 inches

Demand curve (during summer season):

Average: 6 700 GPM

Minimum: 4 188 GPM

Maximum: 8 389 GPM

Pressure restrictions (in PSI): J-6: 65 ≤ P ≤ 70 ▪ J-131: 45 ≤ P ≤ 55 J-36: 50 ≤ P ≤ 60 ▪ J-388a: 40 ≤ P ≤ 90

Pumps (E2, E3, E4, E5, E6, E7, W8, W9, W10, W11 and W12):

Energy price schema: On-peak (09:00 – 20:59): $0.04014/kWh Off-peak (21:00 – 08:59): $0.03714/kWh Demand (monthly): $13.75/kW

MDP apply to Monroe WDSMDP apply to Monroe WDS

Mathematical model:

Set of states: where and

Set of actions:

Transition function:

Reward function:

Data flux diagram:

)(),(),()1( tAtdtHftH

demandt

peakofft

peakon tPwtPwtPwTtR $)(max$)($)(30)(00:9

EPANETDLL.INP FILE MATLAB

24,0T 25.33,1H

111,...,UUA

MDP results in Monroe WDSMDP results in Monroe WDS

Expected electrical power :

E5 and E7 consume 144.3kW W11, E2 and E6 consume 320.4kW

Number of activated pumps (27 possibilities):

on={E2,E6} on{E5,E7}

on={W12,E3,E4,E5} on={E2,E3,E4,E5}

SCADA records: obtained from historical data (July 6th, 2010) 75% of WTP consumption is considered to be used in pump

One day is extrapolated to one billing cycle (30 days) Both approaches started in the same level (19.3 ft)

Energy expenses SCADA records MDP Difference Off-peak energy [$/month] 3 210.57 2 608.32 -23.1% On-peak energy [$/month] 3 750.78 3 768.51 +0.5% Demand [$/month] 3 836.25 3 603.67 -6.5% Total [$/month] 10 797.60 9 980.50 -8.2%

ConclusionsConclusions

MDP avoids restrictions (level, pressure, and pumps) and reduces expenses with energy

To reduce energy consumption is different to reduce expenses with energy (demand is the biggest villain)

Summer season imposes small quantity of feasible actions Verify if it is possible to reduce the number of pump combination MDP policy is easy to implement in a non-intelligent device (PLC)

ContactContact

Thank you for your attention

PAULO THIAGO FRACASSO paulo.fracasso@usp.br

Av. Prof. Luciano Gualberto, trav.3, n.158, sala C2-50CEP: 05508-970 - São Paulo, SP - Brazil

Phone: +55-11-3091-5397

Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes

Documents