1 Multi-Agent System Control and Coordination of an Electrical Network Mark E. Collins Abstract—Multi-Agent Systems (MAS) have the potential to solve Active Network Management (ANM) problems arising from an increase in Distributed Energy Resources (DER). The aim of this research is to integrate a MAS into an electrical network emulation for the purpose of implementing ANM. Initially an overview of agents and MAS and how their characteristics can be used to control and coordinate an electrical network is presented. An electrical network comprising a real-time emulated transmission network connected to a live DER network controlled and coordinated by a MAS is then constructed. The MAS is then used to solve a simple ANM problem: the control and coordination of an electrical network in order to maintain frequency within operational limits. The research concludes that a MAS is successful in solving this ANM problem and also that in the future the developed MAS can be applied to other ANM problems. Index Terms—Multi, Agent, Control, ANM I. I NTRODUCTION T HE electrical grid of Great Britain is changing. This change is mainly due to an increase of distributed renewable generation [1], a decrease of centralized traditional generation (due to EU and UK emission targets [2][3]) and an increase in consumers taking a more active role in the management of their consumption [4]. These changes have driven the need for an intelligent and more robust grid. Bringing this intelligence to the grid may require a departure from classic control regimes such as automation techniques that make use of SCADA [5] and the introduction of software based distributed autonomous agent control [6]. Active Network Management (ANM) is a field of control described as the use of IT, automation and control to manage grid constraints associated with the integration of distributed energy resource (DERs). One way of solving ANM problems is via a Multi-Agent System. A. Multi-Agent Systems There has been much debate in recent years as to what a software agent actually is (as discussed by Wooldridge and Jennings [7]) and what separates it from a program (as discussed by Franklin and Grasser [8]). What has become a popular definition is “a software process that exhibits the properties of autonomy, social ability, reactivity and pro- activeness”, as defined below. • Autonomy: agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state • Social ability: agents interact with other agents (and possibly humans) via some kind of agent communication language • Reactivity: agents perceive and react to their environment • Pro-activeness: agents are capable of exhibiting flexible problem-solving behaviour in pursuit of their design objectives This gives an overview of what an agent is. A more detailed look into agents can be found in a comprehensive paper by Woolridge[7]. These characteristics of an agent system would make such a system a good solution for the ANM problem. An agent is rarely used as a single entity but rather in a conglomerate or a multi-agent system (MAS). A MAS contains a number of agents which interact with one another through communication acts. The typical structure as developed by Jennings [6] is shown in figure 1. Figure 1. Fundamental Multi-Agent System The system contains multiple agents that can communicate with each other and act on the environment. An agent com- munication language (ACL) provides agents with a means to exchange information between each other [9]. There has been work done before on this area such as Intelligent Distributed Autonomous Power Systems IDAPS which presents itself as an all-encompassing MAS system for control and coordination [10]. There has also has been devel- opment on controlling emulated electrical networks by Feroze on integrating MAS with MATLAB/Simulink [11] as well as MAS integration into OPF software such as PowerWorld [12]. This paper describes the construction and testing of a real time, dynamic system, controlled by a MAS. This brings together all of the aforementioned work and allows for an empirical evaluation of MAS solutions to ANM problems in both simulated and live environments. II. CONSTRUCTING A SUITABLE MULTI -AGENT SYSTEM The Multi-Agent system hierarchy is designed to maintain frequency operating limits in the system. The agents are defined in a similar manner to A.Dimeas’ virtual power plant control system [13]. However, their control and coordination
Transcript
1
Multi-Agent System Control and Coordination of anElectrical Network
Mark E. Collins
Abstract—Multi-Agent Systems (MAS) have the potential tosolve Active Network Management (ANM) problems arising froman increase in Distributed Energy Resources (DER). The aim ofthis research is to integrate a MAS into an electrical networkemulation for the purpose of implementing ANM. Initially anoverview of agents and MAS and how their characteristicscan be used to control and coordinate an electrical network ispresented. An electrical network comprising a real-time emulatedtransmission network connected to a live DER network controlledand coordinated by a MAS is then constructed. The MAS isthen used to solve a simple ANM problem: the control andcoordination of an electrical network in order to maintainfrequency within operational limits. The research concludes thata MAS is successful in solving this ANM problem and also thatin the future the developed MAS can be applied to other ANMproblems.
Index Terms—Multi, Agent, Control, ANM
I. INTRODUCTION
THE electrical grid of Great Britain is changing. Thischange is mainly due to an increase of distributed
renewable generation [1], a decrease of centralized traditionalgeneration (due to EU and UK emission targets [2][3]) andan increase in consumers taking a more active role in themanagement of their consumption [4]. These changes havedriven the need for an intelligent and more robust grid.Bringing this intelligence to the grid may require a departurefrom classic control regimes such as automation techniquesthat make use of SCADA [5] and the introduction of softwarebased distributed autonomous agent control [6].
Active Network Management (ANM) is a field of controldescribed as the use of IT, automation and control to managegrid constraints associated with the integration of distributedenergy resource (DERs). One way of solving ANM problemsis via a Multi-Agent System.
A. Multi-Agent Systems
There has been much debate in recent years as to whata software agent actually is (as discussed by Wooldridgeand Jennings [7]) and what separates it from a program (asdiscussed by Franklin and Grasser [8]). What has becomea popular definition is “a software process that exhibits theproperties of autonomy, social ability, reactivity and pro-activeness”, as defined below.
• Autonomy: agents operate without the direct interventionof humans or others, and have some kind of control overtheir actions and internal state
• Social ability: agents interact with other agents (andpossibly humans) via some kind of agent communicationlanguage
• Reactivity: agents perceive and react to their environment• Pro-activeness: agents are capable of exhibiting flexible
problem-solving behaviour in pursuit of their designobjectives
This gives an overview of what an agent is. A more detailedlook into agents can be found in a comprehensive paperby Woolridge[7]. These characteristics of an agent systemwould make such a system a good solution for the ANMproblem. An agent is rarely used as a single entity butrather in a conglomerate or a multi-agent system (MAS). AMAS contains a number of agents which interact with oneanother through communication acts. The typical structure asdeveloped by Jennings [6] is shown in figure 1.
Figure 1. Fundamental Multi-Agent System
The system contains multiple agents that can communicatewith each other and act on the environment. An agent com-munication language (ACL) provides agents with a means toexchange information between each other [9].
There has been work done before on this area such asIntelligent Distributed Autonomous Power Systems IDAPSwhich presents itself as an all-encompassing MAS system forcontrol and coordination [10]. There has also has been devel-opment on controlling emulated electrical networks by Ferozeon integrating MAS with MATLAB/Simulink [11] as well asMAS integration into OPF software such as PowerWorld [12].
This paper describes the construction and testing of a realtime, dynamic system, controlled by a MAS. This bringstogether all of the aforementioned work and allows for anempirical evaluation of MAS solutions to ANM problems inboth simulated and live environments.
II. CONSTRUCTING A SUITABLE MULTI-AGENT SYSTEM
The Multi-Agent system hierarchy is designed to maintainfrequency operating limits in the system. The agents aredefined in a similar manner to A.Dimeas’ virtual power plantcontrol system [13]. However, their control and coordination
2
technique is different, being more closely based upon agenttrading algorithms [14].
The system is split into enterprise areas, each enterprise areawill have an agent which will undertake power commoditytrades within a market - note that this is not based on theNETA/BETTA trading arrangements under which the GB gridcurrently operates [15]. This example market is presented inorder to test the system and the agent’s ability to undertakecontrol and coordination of a network.
Figure 2. Multi-Agent System Hierarchy
The MAS is broken up into three levels which interact toprovide control of the electrical network, which is shown infigure 2. The agents’ task is to maintain the system frequencywithin operating limits.
• Enterprise Agent: The role of this agent is to implementtrades within its area and with other areas in order tomaintain operational frequency.
• Management Agent: The role of the management agent isto monitor and store information on the area assets (gen-erator, loads) that the enterprise agent exhibits influenceupon
• Field Agent: The role of this agent is to interact withthe electrical environment and send measurements to themanagement agent as well as send control signals to theelectrical environment
The MAS, figure 2, is constructed in JADE/Java [16] usingthe Foundation for Intelligent Physical Agent-Agent Commu-nication Language (FIPA-ACL) as its means of agent to agentcommunication [17].
III. EMULATION OF TRANSMISSION AND DISTRIBUTIONNETWORK
The emulation of a transmission network and a DER-richdistribution network is required in order to allow the MAS tocontrol and coordinate the system and to allow for observationof any electrical dynamic effects within the emulated system.
A. Emulated Transmission Network
The transmission network design, shown in figure 3, theparameters for generators, line characteristics and controllercharacteristics are based upon a GB representation [18]. Itis divided into three areas in order to represent Scotland,Northern England and Southern England.
Connected to the emulated transmission network is anemulated distribution network
12
3 4
56
1 2 3
Scotland
North England
South England
DER-Rich Network A
Synchronous Generator
Line Impedance
Bus Bar
Load
Inverter ControlledPower Source/Sink
Key
1 2 3 DER-Rich Network B
DER-Rich Network
Transmission Network
Figure 3. Developed Transmission Network Connected to a Developed DERNetwork
B. Emulated Distribution Network
Distribution Networks with significant Distributed EnergyResources are becoming more important in the analysis ofpower networks. This is due to the increase in the use ofhighly variable small scale generation (such as PV) [1] andthe introduction of large new loads (such as EV). As thesenetworks develop, it is predicted that the classic characterof power flow in distribution systems will change, eventuallyleading to bidirectional power flow. The emulated distribu-tion network includes DER and variable loads to allow theelectrical dynamic behaviour of these future networks to beinvestigated. Within the DER network constructed, the datafor the passive elements that make up the network comes fromUK-Generic Distribution System data[19], and the demandprofiles featured are based on historical data from the NationalGrid [20]. The PV sources and EV’s controllable source/sinksare able to be controlled by passing their output to an inverterand using PQ [21] controllers that have been designed in orderto fully control real power (P) and reactive power(Q) flows toand from the assets.
IV. INTEGRATION OF EMULATED SYSTEMS WITH LIVESYSTEM AND MAS
A live electrical network was constructed and integrated intothe emulated electrical network to allow for MAS control andcoordination. The live section of the network allowed for ob-servation of electrical dynamic affects caused by disturbanceswithin the network.
A. Emulated Transmission with Live Distributed Network
The previous emulation of the transmission network con-nected to the emulated DER-rich network was developed toremove the emulated DER-rich network and to replace it witha DER-rich network constructed within a laboratory. This isshown in figure 4.
3
Figure 4. Integration of Emulated Transmission Network with LaboratoryDER Network
The link between the emulated transmission system andthe physical DER network was created using a back-to-backinverter fed from the local mains supply. This inverter iscontrolled by an emulated transmission system model thatsets the the bus voltage and system frequency provided to thelaboratory DER network. When a disturbance occurs in theemulated transmission system, this appears on the output ofthe inverter as a change in voltage and frequency. The physicalDER network reacts to this change and the resulting P and Qflow measurements made by the inverter are fed back to thetransmission model which reacts to the changes and adjusts thevoltage and frequency set points of the inverter accordingly.
B. MAS integration with Emulated Tranmission and LiveDistributed Network
Shown in figure 5 is the full realisation of the network.The emulated transmission network is shown as within thedashed line, it is the MATLAB Real-Time SimPower model ofthe transmission network discussed in Section 3 A, this areais assigned an Enterprise Agent. The laboratory DER-Richnetwork A shown within the dashed line is the interconnectionbetween the emulated model and the laboratory DER-richnetwork as explained in Section 4, it is assigned and EnterpriseAgent. the MAS discussed in Section 2 is also shown in themodel. In order for the field agents to interact with the physicalhardware of the laboratory, an I/O protocol was developedbased on TCP/IP[22][22]. The emulated DER-Rich network Bis shown within the dashed line, it is a MATLAB Real-TimeSimPower model of the DER network discussed in Section 3B, it is assigned an Enterprise Agent.
Finally, the agents outside the areas constitute the rest of theJava MAS, the Management Agent and The Enterpirse Agentdiscussed in Section 2.
V. TESTING AND RESULTS
A frequency test scenario was developed in order to test theMAS control and coordination as an ANM solution
A. Test of MAS solution to the ANM problem of maintainingfrequency within operating limits
The frequency test scenario used the MAS developed in sec-tion 2 to maintain the system’s (figure 5) operating frequency
Discrete,Ts = 8.333e-005 s.
powergui
A
B
C
Syncronous Generation North
A
B
C
Synchrnous Generator Scotland
A
B
C
Synchrnous Generation South
A
B
C
Load Demand North
A B C
Load Demand 2 South
P
Q Frequency
Vo_dq
A
B
C
DER Network Area A
Frequency
Voltage
P
Q
DER Network (LABORATORY)
A
B
C
DER Area B
ABC
ABC
B9-B11
ABC
ABC
B8-B10ABC
ABC
B6-B5
ABC
ABC
B5-B7
A B C
A B C
B3-B5
ABC
ABC
B3-B4
A B C
A B C
B1-B2
A
B
C
a
b
c
2000 MVA-50 Hz400/11kV AreaA
A
B
C
a
b
c
2000 MVA-50 Hz400/11kV Area B
Freq
Vo_dq
KeyField Agent
Management AgentEnterprise Agent
Comms LinkEnterprise Area
Emulated Transmission Network
Emulated DER-Rich Network B
LaboratoryDER-Rich Network A
Figure 5. MAS control and coordination an Emulated Transmission NetworkConnected to a Emulated DER Network and a Laboratory DER Network
between 50.2 and 49.8 Hz. The test scenario implemented afault to open on a load within the transmission network (Tx)to show how the MAS coordinates and controls the DER-richnetworks (Dx) involved to solve the frequency problem thisfault causes.
The MAS responds as such, first all the Enterprise Agents(EA) determine whether they are the cause of the problem thathas occurred in the system by requesting to their ManagementAgents (MA) if any faults have occurred in the EnterpriseArea. The Management Agent can determine this by callingto its Field Agents (FA) to see if any faults have been flagged.If the Management Agent discovers a problem did occurwithin its Enterprise Area it must rectify the problem by firstdetermining whether its own area can solve the problem. Ifit can not the Management Agent will inform it EnterpriseAgent who can call to the other Enterprise Area’s EnterpriseAgents to provide services to move the frequency back intooperating limits while it remedies the fault.
In this test the transmission network is at fault and itsEnterprise Agent calls for services from the two DER-richnetwork’s Enterprise Agents to help. The DER-rich network’sEnterprise Agents calculate the amount of available powerthat is available to trade within their area and then theycalculate a cost for this service. The Enterprise Agents of theDER-rich networks then reply to the transmission network’sEnterprise Agent requiring this service with a bid and theamount of power associated with that bid. Once all the bids arereceived the transmission network’s Enterprise Agent using anarg min function determines the cheapest offer. Once this iscomplete the transmission network’s Enterprise Agent informsthe bidding Enterprise Agents of the winning bid or bids. Oncethe bidding Enterprise Agent has received this notice theyinform their Management Agents to dispatch power with theirnetwork accordingly, thus solving the operational frequencyproblem. When the fault within the transmission network issolved the transmission network’s Enterprise Agent informsthe DER networks to end the need for the service allow themto return to pre-fault operation.
The test scenario explained was carried out on the system infigure 5. With the aid of figure 6 it was analysed. The analysis
Figure 6. Top:Frequency Dynamics of the Electrical Network Under MASControl and Coordination Bottom:Agent Messages During Operation
looked at the messages being passed from the agents and theeffects the decisions made on the system frequency.
1) The system was balanced at 50Hz in section one. TheField Agents inform the Management Agents of theareas electrical information such as voltages and powerflows etc.
2) The transmission network faults to open. The inertiawithin the network stabilises the frequency however itis outside of operating limits. The Management Agentsinform their Enterprise Agents whether they are at faultor not. The Enterprise Agent at fault (transmissionnetwork) then begins the trading protocol, explainedin the previous paragraph, to begin between the otherEnterprise Agents (DER-rich networks).
3) The transmission network’s Enterprise Agent has in-formed which DER-rich network’s Enterprise Agent ithas won the bid to offer the service causing the DERnetwork’s Management Agent to tell the Field Agentsto re-despatch its assets.
4) The fault in the transmission network is cleared.5) The transmission network’s Enterprise Agent informs
the DER network’s Enterprise Agent it no longer re-quires its services. The DER-rich network’s Manage-ment Agent then informs the Field Agents to dispatchtheir electrical assets back to their pre fault states.
The results show that the MAS successfully exhibits controlover the developed electrical network for a simple ANMproblem. This concludes that the MAS system is success-fully controlling and coordinating the constructed electricalnetwork.
VI. CONCLUSIONS
The aim of the work conducted in this paper was to allow forthe empirical evaluation of MAS solutions to ANM problems.It was demonstrated that MAS control and coordination couldcontrol an electrical network to remedy undesired operatingconditions in both simulation and live environments. Theresults shown in this paper presents the ANM problem ofmaintaining system frequency within operational limits solvedvia the integration of an MAS system for the control and
coordination of an emulated transmission network connectedto a live DER electrical network. It is planned that thesystem will be used to empirically evaluate other multi-agentsystems which will be developed to address Active NetworkManagement problems such as voltage constraints [23] orpower system restoration [24].
REFERENCES
[1] IET, “Distributed generation a factfile by the iet,” Institute Of Engineer-ing and Technology, 2010.
[2] E. Commission, “Directive 2001/77/ec of the european parliament andof the council,” 2001.
[3] D. of Energy and C. Change, “Low carbon community challenge,” 2010.[4] A. N. P.Vytelingum, S.D.Ramchurn, “Decentralised demand side man-
agement in the smart grid,” Proc. of 10th Int. Conf. on AutonomousAgents and Multi-agent systems - Innovative Application Track (AAMAS2011), pp. 10–14, 2010.
[5] S.A.Boyer, SCADA:Supervisor Control and Date Acquisition. Con-necticut, USA: ISA, 2009.
[6] N. R. Jennings and S. Bussmann, “Agent-based control systems,” IEEEControl Systems Magazine, vol. 23, pp. 61–74, 2003.
[7] M.Wooldridge, Multi Agent Systems. Chichester, England: John Wileyand Sons Ltd, 2002.
[8] S. Franklin and A. Graesser, “Is it an agent, or just a program?: Ataxonomy for autonomous agents,” Third International Workshop onAgent Theories, Architectures, and Languages, pp. 1–10, 1996.
[9] Y. Labrou, T. Finin, and Y. Peng, “The current landscape of agentcommunication languages,” Intelligent Systems, vol. 14, pp. 45–52,1999.
[10] S. Rahman, M. Pipattanasomporn, and Y. Teklu, “Intelligent distributedautonomous power systems (idaps),” in Power Engineering SocietyGeneral Meeting, 2007. IEEE, june 2007, pp. 1 –8.
[11] M. Pipattanasomporn, H. Feroze, and S. Rahman, “Multi-agent systemsin a distributed smart grid: Design and implementation,” in PowerSystems Conference and Exposition, 2009. PSCE ’09. IEEE/PES, march2009, pp. 1 –8.
[12] T. T.Logenthiran and D.Wong, “Multi-agent coordination for der inmicrogrid,” IEEE International Conference on Sustainable Energy Tech-nologies, pp. 77–82, 2008.
[13] A. Dimeas and N. Hatziargyriou, “Operation of a multiagent systemfor microgrid control,” Power Systems, IEEE Transactions on, vol. 20,no. 3, pp. 1447 – 1455, aug. 2005.
[14] S. A. P.Vytelingum, T.D.Voice and N.R.Jennings, “Agent-based micro-storage management for the smart grid,” Proc. of 9th Int. Conf. onAutonomous Agents and Multi-agent systems (AAMAS 2010), pp. 10–14, 2010.
[15] H. of Commons Committee of Public Accounts.[16] D. G. Fabio Luigi Bellifemine, Giovanni Caire, “Developing multi-agent
systems with jade.”[17] B. Jennings, R. Brennan, R. Gustavsson, R. Feldt, J. Pitt, K. Prouskas,
and J. Quantz, “Fipa-compliant agents for real-time control of intelligentnetwork traffic.”
[18] J. S. Hughes.M., Anaya-Lara.O., “Influence of wind farms on powersystem dy- namic and transient stability.”
[19] G. Ault, “United kingdom generic distribution system (ukgds).”[20] N. G. G. S. Y. Statement, “National grid.”[21] Q. Rahim.A.N, “Iodeling and analysis of a feedback control strategy for
three-phase voltage source utility interface systems.”[22] W. R. Stevens, “Unix network programming: The sockets networking
api.”[23] M. Baran and I. El-Markabi, “A multiagent-based dispatching scheme
for distributed generators for voltage support on distribution feeders,”Power Systems, IEEE Transactions on, vol. 22, no. 1, pp. 52 –59, feb.2007.
[24] T. Nagata, Y. Tao, K. Kimura, H. Sasaki, and H. Fujita, “A multi-agentapproach to distribution system restoration,” in Circuits and Systems,2004. MWSCAS ’04. The 2004 47th Midwest Symposium on, vol. 2,july 2004, pp. II–333 – II–336 vol.2.
