Abstract—The purpose of this paper is to stress how the

development of real-time platforms for smart grid software services can be propelled by software agility. The Agile software development methodology promotes simplicity and rapid adaptability to evolving requirements and standards. Integrating software tests systematically from the onset of product development to the final release ensures robustness. The Software Solutions Group within GE Energy adopted software agility to ensure that software service developments are driven by quality.

Many steps are involved in the development of a comprehensive real-time software services platform, and it is tempting for stakeholders to over-specify the initial requirements at the expense of a design that addresses how failures are handled. In this paper, the authors describe basic architectures and methodologies in the development of real-time software services to support specific smart-grid business needs.

Index Terms—Agile, Software Development Smart Grid, Software Services, Design for Failure, Real-Time.

I. INTRODUCTION MART GRID real-time applications address

different utility needs such as volt-var optimization, fault restoration, and demand response management. For each need, smart grid software utilizes different components such as power flow, visualization clients, optimization engines, model converters, geographical information systems (GIS), and load forecasting. A number of components such as power flow or load forecast are shared across many applications. To maximize code reuse across a smart-grid portfolio, unify the way shared components communicate with each other and with each application, and improve interoperability between similar software components, GE Energy is embracing the software-as-a-service (SaaS) approach. Software services communicate using a unified interface known as the Software Services Interface (SSI). Eventually, software components such as load flow, state estimation, two-way communications to a control center or to a SCADA system, load forecast, and so on become software services.

The expected quality and robustness of smart grid products are the same as mission-critical software in other industries. Although smart grid software failures may be a simple inconvenience for households, they may result in incorrect data, inaccurate load forecast, and lack of situational awareness. Incorrect operating conditions may cost utilities millions of dollars. In this paper, the authors illustrate a

All authors are with GE Energy Atlanta (email: lance.gleason@ge.com; binh.dam@ge.com, vaughn.mcmullin@ge.com).

design-for-failure approach for smart grid software, and they describe how software agility can provide forward-looking steps (and pitfalls) in the development of a real-time platform for power systems software services.

This paper is organized as follows:

• Robustness and design-for-failure approach, • Software agility, • Simple examples of software services platforms

(fallback architecture, VIP advantages and drawbacks,

• SSI advantages and drawback, and • Real-time tradeoffs.

II. AGILITY AND SMART GRID SOFTWARE SERVICES

A. Smart Grid is Software and Communications Smart grid technology involves an unprecedented amount

of software: exchange and storage of power systems data, advanced power flow, statistical, and optimization algorithms, and communications between devices and software services. Communications are mostly performed based on the Internet Protocol (IP) which already provides a large bandwidth and large scale deployments. Traditional IT and telecommunications companies are offering the software and technology to support the smart grid, and newly developed devices such as advanced meters, smart appliances, GPS clocks, and wireless terminals are designed to work together to optimize grid operations. In the U.S., these smart grid developments have been supported by the American Recovery and Reinvestment Act of 2009 (ARRA, the “Stimulus” bill), with funding available starting October 2009 [1, 2].

B. Software Agility The Agile methodology promotes software development

using an iterative process; Agile was first conceived in 2001 [3]. Iterative software development that follows customer needs and allows fast response to changes distinguishes Agile software development from approaches such as the waterfall model. The Software Solutions Group (SSG) within GE Energy combines Agile development and test-driven development to deliver software that meets customer requirements and company quality standards [4]. Quality assessment includes unit tests and functional tests. The most apparent aspect of the Agile methodology is the iteration process, where features are broken down into “user stories” that are the most basic unit of functionality. The impact of user stories (such as “Publish an optimization

S

Using Agility to Build Robust, Real-Time Platforms for Smart Grid Software Services

Q. Binh Dam, Member, IEEE, Lance Gleason, and Vaughn McMullin

978-1-61284-788-7/11/$26.00 ©2011 IEEE

profile”) and iterations are particularly visible at the user interface level [5]. Nonetheless, user stories also drive core application functionality. A product owner defines user stories and sets the priority order in which they are developed and tested. In each iteration or “sprint,” a limited set of user stories is committed for development. At the end of each iteration, the newest user stories and features developed are demonstrated to customers and product owners. Work can be refocused based on how fast functionality is coded and tested (the development velocity) and on the latest customer feedback. The software is released to customers multiple times a year. An illustration of a typical software Agile process is shown in Fig. 1. Features backlog (product requirements and other customer requests)

Features due in Release n

Release process (every 3-6 months)

Features due in Iteration i

Iteration process (every two weeks)

Development

Fig. 1. Overview of the Agile development process

III. AGILITY AND SOFTWARE SERVICE ARCHITECTURE DESIGN

Most smart grid software products and services are new developments that implement new algorithms or new technology not available on existing systems. It may seem very appealing to include large requirements on new software, especially if one is targeting full compliance with standards such as CIM (Common Information Models) or IEC; however, excessive requirements increase testing complexity. The approach adopted at GE Energy is to build software services gradually using software agility to deliver applications that meet customer and business requirements in the shortest time possible and to ensure that the software exceeds quality standards during all phases of the software development. Robust smart-grid software relies on a gradual implementation on a proven foundation.

In the following sections, we apply the same Agile principle to client-server architectures. Different client-server layouts are presented, the next one being gradually more complex than the previous one. In an Agile development process, customers provide the impulse to adopt a more complex architecture. Eventually, all aspects of software services can be embraced to meet customer needs.

A. Basic Client-Server Architecture (No Redundancy) The most simple client-server layout is between a single

client and a single server (Fig. 2). By its simplicity, this layout is clearly the easiest and fastest to implement; however, it is completely vulnerable to a server failure. In the following sections, we present layouts that provide redundancy so that a server failure does not result in the unavailability of an application.

Client Serverrequest

Fig. 2. Simple client-server layout.

B. Fallback Architecture The fallback approach is a simple approach in the context

of a multi-client, single-server architecture, and it represents typical web architectures. The fallback approach is suited for high-level applications (e.g. energy management systems (EMS) and distribution management systems (DMS)) that utilize primary and backup servers. Software clients (e.g. web portals and user-interface-based applications) communicate with a single primary server. Backup servers provide redundancy and avoid application outages, and they are transparent to the clients and data-synchronized with the primary server. An example of a simple fallback architecture is illustrated in Fig. 3. In Fig. 3, Server 1 acts as the primary point of contact. Server 2 handles all requests when Server 1 is unavailable; Server 3 handles all requests when Server 2 is unavailable. Data is synchronized when a server is brought back to service, and changes initiated from the client are propagated in real-time to Servers 2 and 3.

Server1 Server2 Server3Client1

Requests

Data synchronization

Backup serversPrimary serverClient2

Fig. 3. Illustration of the fallback architecture.

C. Virtual Internet Protocol and Parallel Processing The Virtual Internet Protocol (VIP) is a simple and efficient

solution for a multi-client, multi-server architecture that involves mobile and clustered computing [6, 7]. VIP is compatible with traditional IP as it provides a single IP address for different clients to call a specific service. The VIP architecture dispatches the call to an available server that processes the call. The physical location (physical IP) of the server that actually handles the call is irrelevant. As a result, VIP is suitable for fundamental smart grid services, such as power flow, load forecast, and state estimation. The implementation of the VIP is transparent to the clients and provides service continuity in case of a server failure.

Typically, pooled agents equally share the load (server requests) for a specific service such as computing a power flow solution. Agents are first-come, first-serve, may be located on several servers for redundancy, and shared among different clients. Some demanding processes may require more than one agent. Note that the agents do not hold any application state, as application states should reside on the high-level application servers. This architecture loses real-time characteristics when all agents are busy; in such cases, the VIP architecture places requests in a queue and processes the queue as agents complete previous tasks. An example of a

VIP architecture is illustrated in Fig. 4 where Server1, Server2, and Server3 are the clients of Agent 1, Agent 2, and Agent 3.

VIP: My Service

Server1 Server2 Server3

Agent1 Agent2 Agent3

Server1 request is handled by Agent3

Fig. 4. Illustration of the VIP architecture.

The combination of the fallback and the VIP architecture creates a pattern of multi-client, multi-server architecture with redundancy that provides service continuity if one server fails at any layer.

D. Design for Robustness and Adaptability The fallback and VIP server architectures are simple

enough to not interfere with software quality and robustness: (i) It is easier for developers to develop simple software faster than it is to develop software with complex requirements. Moreover developers enjoy the rewards of seeing their software modules at work, and simple software brings such rewards early in the development process. (ii) Simple software makes testing easier, and the overall quality of the software increases with simple initial requirements that have provisions to expand functionality. (iii) Both architectures may be expanded and modularized according to the specific needs of each application. (iv) Both architectures have support for the failure of one server so that client requests can continue to be honored.

E. Agility in Software Services Architecture Design At this point, we can draw a close analogy between the

design of smart-grid software services and the design of the software itself. Both software and software architecture are designed for failure, with simplified testing and quality assessment. Simple architectures for software services help deliver applications in the shortest time possible, with the development team focusing on functionality requested by customers. Simple architectures can also be expanded and tailored to specific customer needs.

The second author argues from the reasoning above that the development of robust architectures for smart-grid software services should follow the Agile methodology as well. Specifically, developing complex architectures for software services should be done to respond to a customer need

There are instances, however, when the development of a comprehensive architecture for software services is a necessity. We will illustrate that point with the case of the software service interface (SSI).

IV. BUILDING A UNIFIED SOFTWARE SERVICES INTERFACE The Software Service Interface (SSI, or Software Systems

Interface) is a middleware between applications, fundamental

smart grid services, and third-party services. Typically, SSI is implemented as a bus where different applications and services can post and exchange messages. Examples of applications include intelligent VAR control systems, demand response programs, and substation automation. Fundamental application services include load flow, state estimation, and load forecasting, among others.

Because SSI is a comprehensive architecture, SSI is an enabler for communications between smart grid applications and services. The ambitious goal set forth by SSI proponents is to achieve a universal communications platform between applications and basic services. SSI cannot be a product or an application by itself. A typical SSI interface is illustrated in Fig. 5.

SSI Bus

App1 (Varcontrol)

App2 (OMS)

App3 (Peak control)

Service1 (Power flow)

Service2 (Network

trace)

Service3 (Load

forecast)

App1 request is posted to the SSI bus and handled by Service3

Fig. 5. Illustration of an SSI architecture. The need to know about the types of data transmitted

through the SSI bus (expected inputs and outputs in the appropriate format) may lead to over-engineering and distract a software developer from the initial implementation of the communications platform.

A. Acknowledging Needs for SSI Depending on individual situations, SSI may prove superior

to other architectures: - When multiple services and applications need to

communicate with each other, - With large DMS/EMS applications deployments that

utilize multiple services and multiple applications, - When modules need to listen to all system events in a

way that is similar to the IEC 61850 substation event bus.

B. Software Services Communications Standards The IEC 61850 [8] standard defines how substation

equipment should communicate among each other. CIM [9] standardizes how models are coded to facilitate exchange of model data between GIS, DMS, and EMS. Existing communications standards include the DNP protocol in substations, IEEE 802.11 [10] (ethernet for wireless devices), the Zigbee protocol for communications between energy devices in a wireless home area network (IEEE 802.15.4) [11], and, more recently, 3G and 4G cell phone standards.

C. Development and Testing Challenge Because each standard defines its own requirements, an

Agile development process to support all the standards must

implement and test requirements in a priority order that allows customer functionality to be implemented. Following such a gradual approach increases the quality and robustness of each supported aspect of a standard.

Because SSI aims at comprehensiveness, the testing of individual functionality is more challenging than with simple architectures. Quality is more difficult to assess with more complicated services. However, using an approach such as Agile helps determine the development steps to gradually implement SSI depending on business needs.

V. REAL TIME REQUIREMENTS Smart grid software must process models and

measurements so that utilities and consumers can have access to power data in real time. Smart grid technology is expected to optimize large distribution networks within minutes and obtain the system state several times per minute. In this section, we examine the various implications for the development of real-time smart-grid software services.

A. Speed Requirements Recent distribution automation applications must process

circuits in real time to provide operating recommendations. Distribution optimization applications scan feeder data and measurements for improvements and should process several million nodes within minutes. Such desired processing speeds impose strict time requirements to the distribution load flow, state estimation, and optimization processes. Load flow may be needed a few dozen times per second, and state estimation up to ten times per second. Architectural overhead increases the total processing time depending on architecture complexity. The total processing time is tied to architectural choices as well as the performance of the individual components of the chosen architecture.

B. Programming Platforms Programming platform come with tradeoffs between

performance control and ease of development. Platforms such as Java can be mastered quickly; however, automatic garbage collection is often times a handicap in real-time programming [8, 9], especially when programmers have no control on the duration of a garbage collection cycle. As an alternative, the Ruby platform offers control over garbage collection. In contrast, C/C++ is usually fastest as it compiles directly to CPU instructions. Granularity of control to the developers is greater than with Java or similar languages; however, memory management is the full responsibility of the developer.

C. Software Complexity Most of the time, software complexity and layers such as

security layers reduce real-time performance. Simple software service architectures are easier to fine-tune for performance than large architectures such as SSI. The different software layers in large architectures increase overhead processing without adding value, but are useful if disparate communication protocols exist between software services.

D. Communications Delays/Lags Communications delays and lags caused by long

communications lines must be accounted for. Time stamps may be applied to mark time-sensitive data. Also, exchanging megabytes of data several times a second between servers may raise questions about where the data should be handled, as fewer batches of data exchanged also mean less time spent encoding or decoding the data.

E. Server Requirements Communications in real-time services should be limited to

the originating and destination servers. Compared to the VIP architecture, and depending on implementation, additional redundant servers may be required to dispatch the messages posted on the SSI bus while maintaining service continuity in the event of a server failure.

F. Security and Data Integrity Utilities take security seriously, given the potential for a

hacker to penetrate utility servers and issue commands that could take an electric network out of control. The time needed to authenticate, process the security layer, and to encrypt/decrypt data with a strong cipher must be accounted for in an application.

VI. CONCLUSION The authors emphasize the importance of simplicity and

adaptability in the implementation of smart-grid software services. Simplicity and adaptability define Agile development processes, and both software development and software service architecture development benefit from Agile development methodologies.

Because smart grid technology has a tremendous impact on millions of people, simplicity is instrumental to make testing easier and facilitate quality assessment. In an Agile process, unit tests from developers, tests conducted by expert quality analysts, and performance benchmarks from customers constitute acceptance criteria for each functionality or user story.

Beyond simplicity, the most important aspect of GE’s smart grid software technology is its ability to adapt to and ultimately anticipate changing needs and requirements of utility customers.

REFERENCES [1] A. Greenberg, “Smart grids get a pay raise,” Forbes, May

18, 2009, [Online] http://www.forbes.com/2009/05/18/smart-grid-electricity-technology-internet-infrastructure-smart-grid.html

[2] R. Smith, B. Worthen, “Stimulus funds speed transformation toward 'Smart Grid',” Wall Street Journal, September 28, 2009, [Online] http://online.wsj.com/article/SB125409459487544787.html

[3] Agile Manifesto: http://agilemanifesto.org/ [4] GE Quality:

http://www.gepower.com/commitment/en/quality.htm

[5] J. Ferreira, J. Noble, R. Biddle, “Agile development iterations and UI design,” Proc. AGILE 2007 Conference, pp. 50-58, Washington, DC.

[6] F. Teraoka, F. Claffy, M. Tokoro, “Design, implementation, and evaluation of Virtual Internet Protocol,” Proc. 12th Conf. On Distributed Computing Systems, pp 170-177, June 1992, Yokohama, Japan.

[7] “Oracle GoldenGate on Sun Oracle Database Machine Configuration,” Oracle Maximum Availability Architecture White Paper, February 2010 [Online] http://www.oracle.com/technetwork/database/features/availability/maa-wp-gg-oracledbm-128760.pdf.

[8] Communication Networks and Systems in Substations, IEC Standard 61850, 2004.

[9] Common Information Model, IEC Standard 61970-301 and 61968-11, 2009.

[10] IEEE Standard for Information technology--Telecommunications and information exchange between systems--Local and metropolitan area networks--Specific requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, IEEE Standard 802.11, 2007.

[11] IEEE Standard for Information Technology- Telecommunications and Information Exchange Between Systems- Local and Metropolitan Area Networks- Specific Requirements Part 15.4: Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for Low-Rate Wireless Personal Area Networks (WPANs), IEEE Standard 802.15.4, 2006.

[12] D. Dvorak et al., “Project Golden Gate: Towards real-time Java in space missions,” NASA [Online] http://trs-new.jpl.nasa.gov/dspace/bitstream/2014/37998/1/04-0051.pdf.

[13] Greg Bollella et al, The Real-Time Specification for Java, Addison-Wesley, 2001, http://rtj.org

BIOGRAPHIES Q. Binh Dam (StM’05, M’09) is from Paris, France. He received the E.E. Diploma in 2003 from the National Polytechnic Institute of Toulouse, France; the M.S.E.E. and Ph.D. degrees from the Georgia Institute of Technology in 2003 and 2009, respectively. He is now with GE Energy in Atlanta. His interests include advanced, data-intensive smart grid software applications, protective relaying, and new testing tools and methodologies to assess power systems reliability.

Lance Gleason holds a Bachelor’s in Computer Science from the State Univerity of New York, Brockport and a Master’s in Computer Science from the Southern Polytechnic Institute in Atlanta, Georgia. He has over 12 years of experience working as a software engineer and architect for a number of large and small projects in a variety of domains including digital imaging, advanced defense projects, financial services broadcast video delivery. Some of his clients have include firms such as Eastman Kodak, Lockheed Martin, McKesson, CNN and Turner Broadcasting

Vaughn McMullin is the lead architect for advanced software applications at GE Energy. He has 20 years of expertise in communication networks, modular architectures, cloud computing, and information security. He holds a Bachelor’s in Mathematics and Physics and a Master’s in Computational Physics from Virginia State University.