Competence Center Agent Core Technologies
Or: what agents can be good for
DATA SIM Summer School 2015
Electric Vehicles vs. (Micro) SmartGrids
Dr. Marco Lützenberger
3. September 2015
23. September 2015
►Technische Universität Berlin – Berlin Institute of Technology
►DAI-Lab
►~150 employees
Post-docs, PhD Students, undergraduates
►Separated into competence centres
IRML, NEMO, SEC, COG, EDU, ACT
Aims of this Talk
33. September 2015
►Some optimization problems
►Science
►Agents
Agenda
43. September 2015
►Why are Electric Vehicles important for us (as a researcher)?
►Part One: The Driver of an Electric Vehicle
A user-centric approach
►Part Two: Electric Vehicle Fleets
A provider-centric approach
A Demonstration
The Electric Vehicle
53. September 2015
►the (electric) vehicle
►regular vs. electric
► Interesting:
the (flaw of the) battery makes the difference
Optimise range and charging intervals
the features of the battery
Utilisation of renewable energy, grid load balancing, minimising emissions
►different stakeholders
Mini Cooper Mini E (V2G)
Power 90 kW/122HP 150 kW/204HP
Torque 160 Nm 220 Nm
Weigth 1090 kg 1.465kg
Acceleration 9.1 s 8.5 s
Maximum Speed 203 km/h 152 km/h
Range 740 km 250 km
Battery 40 l 35 kWh
Charging 1-2 min 2.4 h (230V, 50A)
3.8 h (230V, 32A)
10.1h (230V,12A)source: www.mini.de
Stakeholders and their interest in EVs
63. September 2015
►„Common“ stakeholders
The driver
The vehicle manufacturer
►„Uncommon“ stakeholders
The battery manufacturer
The charging station operator
The energy provider
The government
►Conflicting interests!
Driver Battery/Vehicle Charging station Energy provider Government
mobility lifetime money grid safety image
money service availabilityexploiting renewable
energy
reducingemissions
imagereducing
emissionsutilisation
regulatoryenergy
distribution
image
The researcher
73. September 2015
►The challenge: different stakeholders, different interests
►The task: bring them together
►The aim: maximise the stakeholders profit
Driver
Battery/Vehicle
Manufacturer
Charging Station
Operator
Energy Provider
Government
Developing a Solution - Variables
83. September 2015
►Driver
scheduled and unscheduled appointments
►Vehicle/Battery manufacturer
charging profile, feeding profile, CO2 fingerprint and consumption
►Charging station operator
amount, characteristics, local grid infrastructure (LLM)
►Energy provider
local and global grid infrastructure, availability prognoses, CO2 fingerprint
►Government
amount of vehicles, CO2 emissions
The Problem... In a nutshell
93. September 2015
►Different stakeholder
driver, vehicle/battery manufacturer, charging station operator, energy provider, (government)
►Different interests
mobility, CO2 efficiency, lifetime, service, utilisation of infrastructures, utilisation of wind energy, money, image, ...
►The aim: maximise the stakeholders profit
►Developing such system is…
Modelling
Implementation
Deployment
Monitoring
► logical distribution, autonomy, reactivity, proactivity, interaction
► { The | A } solution: The agent paradigm
►Consider the stakeholders as (software-) agents
Easy/difficult
Agentoriented Software Engineering
103. September 2015
► „The Agent“ as constituting concept
► What is the definition of an Agent? There is no (common) definition!
Wooldridge and Jennings (1995): […] the term agent is used to denote a hardware or (more usually) software-based computer system that enjoys the following properties:
Autonomy: agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state
Social ability: agents interact with other agents (and possibly humans) via some kind of agent-communication language
Reactivity: agents perceive their environment, (which may be the physical world, a user via a graphical user interface, a collection of other agents, the INTERNET, or perhaps all of these combined), and respond in a timely fashion to changes that occur in it;
Pro-activeness: agents do not simply act in response to their environment, they are able to exhibit goal-directed behaviour by taking the initiative.
► Why agents?
AOSE Methodologies, Documentation, Development Tools, Frameworks, Monitoring Tools
JADE, JACK, Jason, JASDL, Janus, Jadex, JIAC, 3APL, Cougaar, …
► Back to the problem: autonomy, reactivity, proactivity, interaction, logical distribution
The W2V2G System I - Design
113. September 2015
► User Agent(s) Accesses mobility patterns (derived, upcoming), detect derivations
► Car Agent(s) Current state, vehicle and battery characteristics and constraints,
charging and feeding control► Charging Station Agent(s)
Local grid management, infrastructure information, charging and feeding control
► Energy Provider Agent Information about (global) grid load and available wind energy
► System Functionality Energy Management (W2V, V2G, Controlling)
► Additional functionality: Route Planning, Booking, etc.
The W2V Algorithm
123. September 2015
►Backend Software
►Triggered by Vehicle Agent (VA) and User Agent (UA)
Calculated consumption (VA) < Minimum SOC (UA)
The W2V Trigger I
133. September 2015
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Expected SOC
Minimum
The W2V Trigger II
143. September 2015
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Expected SOC
Minimum
The W2V Algorithm
153. September 2015
►Backend Software (BS)
►Triggered by Vehicle Agent (VA) and User Agent (UA)
Calculated consumption (VA) < Minimum SOC (UA)
►Preceeding Time intervals are examined
Effect of charging time on energy progression (BS – UA – VA)
Grid state (BS – Energy Provider Agent (EA))
W2V - Filtering
163. September 2015
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Expected SOC
Minimum
The W2V Algorithm
173. September 2015
►Backend Software (BS)
►Triggered by Vehicle Agent (VA) and User Agent (UA)
Calculated consumption (VA) < Minimum SOC (UA)
►Preceding time intervals are filtered
Effect of charging time on energy progression (BS – UA – VA)
Grid demand (BS – Energy Provider Agent (EA))
►Remaining time intervals are assessed
Wind energy (BS – EA)
Utilise renewable energy
Local grid state (BS – Charging Station Agent)
Grid load balancing
W2V - Filtering
183. September 2015
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Expected SOC
Minimum
0
2
1 3 5 7 9 11 13 15 17 19 21 23
W2V - Charging
193. September 2015
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Expected SOC
Minimum
The W2V Algorithm
203. September 2015
►Backend Software (BS)
►Triggered by Vehicle Agent (VA) and User Agent (UA)
Calculated consumption (VA) < Minimum SOC (UA)
►Preceding (violation) time intervals are filtered
Effect of charging time on energy progression (BS – UA – VA)
Grid state (BS – Energy Provider Agent (EA))
►Remaining time intervals are assessed
Wind energy (BS – EA)
Local grid state (BS – Charging Station Agent (CA))
►Vehicle is mainly charging renewable energy
►Additional consumption serves for load peak grading
The V2G Algorithm
213. September 2015
►Backend Software
►Triggered by User Agent or by Vehicle Agent (BS – UA – VA)
Detected change in mobility pattern
►Potential time intervals are analysed (BS – EA – UA – CA)
Grid load expected wind energy (EA)
Quotient > 0.9 discarded
Availability (CA)
►Constraint check for identified feeding intervals (UA – VA)
SOC violation?
If not possible compensation by charging (W2V)
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Expected SOC
Minimum
W2V - Charging
223. September 2015
Original W2V Strategy
V2G Feeding Intervals
V2G Compensation
Results
233. September 2015
►Field test Evaluation
►No SOC violation
►A few grid violations (our fault)
►Mini Cooper S CO2 Emissions (estimated): 18.126 gram
►Mini E (user controlled charging): 4.283,53 gram
►Mini E (W2V2G application): 2364,57 gram
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
Mini E (W2V2G) Mini E (User Controlled) Mini Cooper S
Implementation Details
243. September 2015
►Java Intelligent Agent Componentware V (JIAC V)
►Framework with a focus on industrial applications/projects
►Reliability, robustness, scalability, modularity, reusability
►Merging agents and services
►Third party API integration
Java Intelligent Agent Componentware
253. September 2015
► features
reliable communication, extensibility, reuse, performance, monitoring, maintenance, documentation, comprehensive tool support, state-of-the-art concepts/paradigms
►project requirements
robustness, scalability, support for service management, monitoring, extensibility, SOA, Cloud, webservices, OSGiBundles, ...
►extensibility by modular assembly tailored solutions
component based architecture (agent/node beans)
►state-of-the-art libraries and languages
Java, Spring, ActiveMQ, JMX, …
The JIAC V Framework – Architecture
263. September 2015
►Agent platform agent nodes (+node beans) agents agent beans
►Runtime deployment (Spring)
►Java based implementation
►agent interaction by (ActiveMQ)
service invocation (SOA), messages, custom protocols
►Knowledge
tuple-space based memory
►runtime monitoring (JMX), ASGARD
JIAC Applications - Nodes
273. September 2015
►Default Nodes
JMX, Secured JMX, Service Directory, Registry
►Component Specification in Spring parent description
JMX capable parent node
Agent references
Agent description
JIAC Applications - Agents
283. September 2015
►Default Agents
Simple Agent, non-blocking agent, custom
►Component specification in Springparent description
non-blocking parent agent
bean reference
JIAC Applications – (Agent) Beans
293. September 2015
►Specification in Spring
► Implementation in Java (extends AbstractMethodExposingBean)
fully qualified java name
Bean attribute
JIAC V – Agents
303. September 2015
►agent standard components
execution cycle, local memory, communication adaptors
►component based architecture
agent behaviours and capabilities in AgentBeans
► flexible activation schemes
regular, life cycle, observers, action methods
►AgentBeans and NodeBeans
►available AgentBeans (and NodeBeans)
communication, JADL++ interpreter, Drools rule engine, migration, persistence, load measurement and –balancing, user management, human agent interface, webserver, webservice gateway, OSGi gateway
Lessons Learned
313. September 2015
► It worked!
►CO2 was decreased
►Agent technology supported:
Transparent distribution
Distributed development
Programming behaviour
►Not everything was good!
Communication
Planning performance
► It was NOT a trivial problem!
2 Years of research, 1.5m€ funding
Lessons Learned
323. September 2015
►Stakeholder
Driver Battery/Vehicle Charging station Energy provider Government
mobility lifetime money grid safety image
money service availabilityexploiting renewable
energy
reducingemissions
imagereducing
emissionsutilisation
regulatoryenergy
distribution
image
Electric Vehicles and Micro SmartGrids
333. September 2015
►Wish list:
Not one but many (electric) vehicles
Valid information on vehicle utilisation
Consuming AND producing infrastructure
Ability to (temporarily) store electric energy
► (Electric) car sharing + the Micro SmartGrid
►The vision
Use EVs and local storage to ‘buffer’ surpluses of energy
Make the grid autarkic
Area of application: Companies, car sharing enterprises
Test Site Setup
3. September
2015ISGT 2012 34
▶ Real-life test system of ‚Micro Smart Grid‘
Photovoltaic 50 kWp
Wind Turbines 5 kWp
Hydrogen Fuel Cell 1 kWel, 1 kWth
Stirling Engine 1 kWel, 16 kWth
Grid Buffer Battery 140 - 160 kWh; 18 kW
13 Electric Vehicle charging stations with distinct specifications,mostly 16 A, 400 V
Single Point of Common Coupling at 630 kVA transformer
The (Second) Problem
353. September 2015
►Factors:
Consumption
Production (wind and solar energy)
Vehicle utilisation
►The challenge:
Not one but many vehicles
Vehicles are REALLY required (time critical)
Minimise grid procurement
Maximise utilisation of renewable energy minimise C02
emissions
Counter the Second Problem
363. September 2015
►W2V2G Approach
Inapplicable time critical environment
►Deterministic optimisation
Brute force?
Complex (NP-hard problem) time critical environment
►Stochastic optimisation
Possible! …but is it good?
Evolution strategy
Modelling – Formulate the problem
373. September 2015
►Consider charging schedules (the arrangement of charging and feeding processes) as ‘population’
►Populations are moduled as follows:
►A ‘process graph’ contains all energy consumption and generation processes
►These are modelled as activities (duration+energy demand)
►Activities are linked to ‘inventory resources’ (EVs or charging stations)
Constraints: maximum load, minimum capacity, ...
►Minimize the externally procured energy
Depending on a (dynamic) tariff
Optimsation Algoritm (Evolution Strategy)
383. September 2015
► (μ/ρ + λ) strategy
►Generate initial population of μ individuals
►Based on these μ ‘parents’, λ ‘offsprings’ are generated...
by recombining the best measured selection of ρ parents
and slightly altering ‘mutating’ the result
► Initial population is created by very simple scheduler (charge when vehicles return)
►Mutate by shifting individual or groups of activities to another place in the process plan
►Recombination difficult due to many dependencies
►Measure the quality and mutate, again
►Terminate when there is no increase in quality
Problems
393. September 2015
►Results are ok!
►Well know problem of stochastic approaches
Local optima
Algorithm gets ‚stuck ‘
►Solution: Not one but many problem solvers (agents)
Develop interaction protocol
Distribute agents on local (multi-core) machine
Optimisation protocol
403. September 2015
►Optimisation client
Proposes optimisation job
►Optimisation server
Accepts (or rejects) optimisationjob
Performance
413. September 2015
►Quality versus populations
►Quality versus time
Conclusion
423. September 2015
►Agents can not only be used for physical distribution, but also for logical distribution
►Avoided well known problem of stochastic optimisation
►Exploited multi-core architecture
►No ‘real’ agency
autonomy? social ability? reactiveness? pro-activeness?
►recombination
Demonstration 2
433. September 2015