Abstract The paper describes an approach to allow dynamic reconfiguration of mobileagents. Both pure software agents—as the ones usually supported by multi-agent systemsframeworks—as well as pure hardware agents, running in FPGA hardware, are supported.Agents can migrate among nodes or move from software to hardware (or vice-versa) usinghigh-level services. The proposed approach even supports that a software agent migrates toanother node and becomes a hardware agent in the target node. This opens a wide spec-trum of possibilities for system designers, allowing dynamic reconfiguration of distributedapplications based on current system requirements. Communication among agents followsthe same schema usually adopted by multi-agent frameworks and the fact that the calledagent is implemented as hardware or software is totally transparent to the calling agents.This is made possible due to the development of the an agent wrapper layer, on which proxyagents, implemented in software, are responsible for handling all necessary information totheir hardware counterparts. The proposed approach was implemented using the Jade frame-work, a Java-based multi-agent systems framework which is widely adopted in several ap-plications. In order to assess the timing performance and memory overhead of the proposedapproach, a case study including two computationally intensive agents—a Floating PointUnit (FPU) and a Kalman Filter agent—were developed and implemented both as softwareas well as hardware agents. Obtained results that allowed the validation of the proposedideas are presented in the paper.
Keywords Multi-agent systems · Hardware agent · Mobile agent · Reconfigurablecomputing
D. Cemin · M. Götz (B) · C.E. PereiraElectrical Engineering Department, Federal University of Rio Grande do Sul, Porto Alegre, Brazile-mail: [email protected]
Multi-agent systems (MAS), considering mobile agents, are a promising technology fordistributed computing applications in dynamic scenarios [10]. MAS consist of a set of au-tonomous, intelligent, and goal oriented units that efficiently cooperate and coordinate theirdecision making to reach a higher-level or global goal. In MAS groups of agents are or-ganized according to specific, precisely defined principles of community organization andoperation (such as messages and negotiation protocols), being supported by an adequateagent platform [23, 27].
Dynamic scenarios can be also found in modern industrial automation systems due toseveral required aspects like: effective enterprise integration, handling of cooperative and ag-ile processes, scalability, distribution, fault tolerance, interoperability, among others. Mod-ern Evolvable Assembly System (EAS) is one new paradigm for production system, whereconcepts of MAS and embedded distributed systems are applied [34].
In the higher levels of the automation pyramid, real-time requirements tend to becomesofter and knowledge intensive decision processes are performed by agents. MAS can alsobe used in other applications in industrial automation [28], such as: (i) real-time manufac-turing control; (ii) production management level (encompassing problems such as planning,scheduling, orders preprocessing, etc.) and (iii) Virtual Enterprises (VE) that will integratemanufacturing, sales networks, suppliers, distribution channels. Other examples of agent-based industrial applications can be found in [4, 28, 38, 44].
At the real-time manufacturing control level, agents have usually a simple reactive behav-ior, whilst other higher levels might apply a deliberative behavior based on complex modelsand proactive strategies. At this level, agents generally have an one-to-one correspondencewith individual resources (machines and other devices) and orders (raw materials, parts, andproducts).
Hybrid systems, comprising general purpose processor (GPP) and Field ProgrammableGate Array (FPGA), are very attractive as they can provide flexibility as well as high compu-tational performance. Particularly, the capability of a FPGA to be reprogrammed at run-timeemphasizes these characteristics [14]. Even more attractive is the usage of such hybrid ar-chitectures in changing (dynamic) environments where applications may enter of leave thesystem during run-time. Such architectures may be used as basic elements in building exe-cution platforms for reconfigurable manufacturing systems.
In this direction, the work presented in this paper proposes a reconfigurable agent, able tobe executed both as a pure software agent as well as a hardware agent. The agent is capableof migrating from hardware to software (or vice versa) based on a decision that is madeinside the application or even by an external trigger, for example. The focus of this work,however, is on the infrastructure to enable such migration between hardware and software,and not on the decision algorithms. The intention hereby is to enable a MAS to be builtwith both mobile hardware and software agents, and also with dynamically reconfigurableagents. Reconfiguration in this context is the capacity of an agent changing itself betweenhardware and software during a migration of this agent, from one node to another.
For this work, JADE was used as a MAS Framework. Although JADE [43] was usedas implementation choice for the proposed ideas, the concepts here presented might be im-plemented using other MAS framework technologies as well. Moreover, the interaction andcommunication between other agents and reconfigurable agents occur in the same way asfor non-reconfigurable agents. Actually, other agents are unaware of current execution na-ture (if under software of hardware execution) of a given reconfigurable agent. This enablesthe necessary transparency to incorporate reconfigurable agents in a smooth way into thenetwork.
Dynamically reconfigurable hardware/software mobile agents
This paper is organized as follows. Section 2 introduces definitions and relevant tech-nologies focusing on the aspects treated in this work. Section 3 gives an overview of theproposed architecture. The main parts of the architecture are depicted in the three subse-quent sections. Experiments based on case studies are explained and results are discussed inSect. 7. Other approaches related to the work here presented are discussed in Sect. 8. Finally,Sect. 9 draws conclusions and signals future work directions.
2 Theoretical background
2.1 Mobile agents
In highly dynamic distributed environments, mobile agents are a suitable paradigm for sup-porting the development of distributed applications, services, and protocols [49]. Agents areprocesses capable of reasoning without external intervention.
The use of an architecture with mobile agents is a natural way of developing distributedapplications, such as in distributed automation components in an industrial environmentor as in sensor network applications. Agents can migrate with the goal of, for example,increasing the overall system performance by dynamically distributing the computation loadacross the nodes. Sensor networks typically have different types of hardware as well asdifferent types of operating systems. The use of mobile agents in heterogeneous executionplatforms brings to context the problem of having the same code running in different typesof systems, which can be solved by using Virtual Machines (VMs) as an abstraction layer.
Another degree of flexibility can be assigned to a sensor network by considering themobility of the network’s nodes. So, for instance, a sensor network might be composed bystatic and mobile nodes. These approaches have shown great results in some applicationsexamples like presented in [7, 8] and in [35].
2.2 Reconfigurable hardware
Tasks in embedded systems are usually implemented in a single processor or microcontrollerdue to their flexibility. For computational intensive tasks, a more powerful processor, or evena hardware co-processor, might be chosen.
The implementation in dedicated hardware processors of computationally intensive tasks,such as manipulation of large matrices, becomes an important alternative to cope with timingrequirements of real-time applications. A limitation related to dedicated hardware, such asApplication-Specific Integrated Circuits (ASICs), is that they are usually not reconfigurable,and changes in system’s requirements can not be appropriately handled. ASICs are also veryexpensive to develop, which can increase significantly the final cost of the system.
Reconfigurable hardware appears in this context filling the gap between the softwareflexibility and high computational power of dedicated hardware. FPGAs are the devicesmost currently in use that provide reconfigurable elements, different types of system blocksand embedded soft processors. The inherent reconfigurability of FPGAs enables changingsystem’s functionality after the deployment [12].
It is possible to change FPGA’s bitstream during its execution. It is possible to changeFPGA’s bitstream (a FPGA’s configuration file) during its execution. In its common use,there is only one bitstream that configures the whole device, and by reprogramming it, allthe tasks being executed must be restarted. In order to overcome this limitation, some FPGAmanufacturers, for example Xilinx, offer a solution to reprogram small portions from thedevice instead of reprogramming the whole chip [47] called reconfigurable modules.
D. Cemin et al.
FPGA’s partial reconfiguration also enables new types of designs that would otherwise beimpossible to implement. Considering that a FPGA might comprise several tasks, and eachtask is enclosed in a reconfigurable module, these tasks (hardware ones) might be changedat runtime. That means, this approach allows hardware tasks to be handled in a similar wayas software tasks [45, 46]. Moreover, some further approaches enable a system to choosethe right version of a task (hardware or software) depending on the execution platformavailable, or even to dynamically change its task version depending on some applicationdecisions [15].
2.3 JADE
Jade is an acronym to Java Agent Development, and it is a software framework used in MASwhich enables controlling multiple agents accordingly to Foundation for Intelligent PhysicalAgents-Agent Common Language (FIPA-ACL) specifications, which is responsible for reg-ulating the agent’s communication language. Jade creates the concept of multiple containersto agents, which are spread throughout the network and together they form the network.Each platform is connected to a single main container which is responsible for, among otherthings, creating and destroying agents as well as keeping agent’s services updated. The im-age depicted in Fig. 1 shows a MAS from the Jade’s perspective. All agents are directlyconnected to an specific container, and agents can exchange messages with other agents aswell as with the containers.
Jade agents communication is performed through message passing, where FIPA-ACLis the language that represents messages. The agent platform can be distributed in severalhosts each one with a single JVM. Each JVM is basically a container of agents that pro-vides a complete run-time environment for agent execution and allows several agents toconcurrently execute in the same host.
A single Jade agent can encapsulate several behaviours, as for example cyclic behaviours,which means that agents will be periodically executed. There are also oneshot behaviours,for agents that have the need to be executed only once and there are also Finite State Machine(FSM) behaviours already implemented in the framework.
There is a life cycle manager that monitors agents to create or destroy them once theyare created or finished. Jade also creates a way to schedule behaviours of agents, deliveringspecific actions at a desired time, which is performed by the behaviours’ scheduler.
Jade also provides the infra-structure to enable agent communication. Agents communi-cate with each other through FIPA-ACL messages that are already implemented natively intheir structure. It even enables Jade agents to communicate with other agents that are notimplemented using Jade framework, but which are also compliant with FIPA standards.
Fig. 1 High-level Jadearchitecture
Dynamically reconfigurable hardware/software mobile agents
Mobile agents based on Jade platform are being used in several fields as, for instance,object localization [40], wireless sensors with medical applications [16], electric power dis-tribution systems [13] and even cooperative robot tasks [5]. Jade was chosen as a MASframework because it already integrates a high-level agent communication compliant toFIPA protocol, and also because it eases the system’s usability.
3 Architecture overview
From a high level point of view, the proposed architecture can be divided into three parts,as presented in Fig. 2. At the top lies the Application Layer, where agents related to appli-cation domains are identified. These agents are implemented in the same way as softwareagents using the JADE framework, that means, agents are able to migrate among distributedcomputing nodes, agents are able to discover other agents using the Jade directory services,etc.
As already discussed previously in the paper, JADE was selected because it is a MASframework that is based on the Java language which nicely supports concepts such as en-capsulation, autonomy, also allowing agents migration. JADE has been extensively used byseveral research groups and in important applied research projects [34].
The Reconfigurable Hardware Layer at the bottom represents the environment wherehardware agents are located. Using FPGA and taking advantage from partial reconfigurationtechniques, each hardware agent might be individually changed or reconfigured, whilst otheragents remain working. Similarly to software agents, these hardware agents are also able tomigrate among the distributed nodes, using the underlying agent framework services. Eachhardware agent can also initiate a communication with any agent in the network, in the sameway as any software agent would be able to do. In order to enable this, a so called AgentWrapper Layer (AWL) is introduced.
It has to be noticed that for each hardware agent there must be a correspondent proxysoftware agent. This proxy agent provides a front end to the hardware agent, guaranteeingthe transparency to other agents on the system. The connection between proxy agent andhardware agent is provided by the AWL. Also, it promotes the necessary abstraction ofthe hardware agents, providing mechanisms for data encapsulation and data exchangingbetween hardware and software.
Fig. 2 Layered architecture usedinside a sensor node
D. Cemin et al.
Furthermore, FPGA reconfiguration services are provided by the AWL, and supported bythe underlying Operating Service (OS) services. An agent is capable of configuring itself buteventually this reconfiguration can be triggered by another agent or even by an event externalto the system. In the last case, of course, the agent must be able to make a decision for areconfiguration. A discussion on algorithms for defining the convenient time for startinga hardware/software reconfiguration is beyond of the scope of this paper, which basicallydescribes the developed infra-structure that allows the reconfiguration.
Clearly, by using this layered approach, the multi-agent system framework can be easilychanged. Since key mechanisms and services rely on the AWL, the Application Layer is notrestricted to JADE and could be easily translated to other MAS frameworks.
4 Reconfigurable hardware layer
Using an FPGA with partial reconfiguration capabilities enables the existence of a numberof hardware agents inside the same node (execution platform), which can even be changedduring run-time. Figure 3 depicts partially the hardware platform used, where the three mainparts are showed: External Memory, External Processor and FPGA. The last one is divideditself in two parts: Static portion (where no dynamic reconfiguration occurs), and the recon-figurable part, which is divided in Reconfigurable Modules, each one being able to com-port one hardware agent. The underlying execution platform is not the focus of the currentresearch, and therefore the proposed approach relies on concepts already used in severalproposals in the literature.
This platform connects the processor to the FPGA by means of a Peripheral Local Bus(PLB). One of the static components is the Internal Configuration Access Port (ICAP),which is managed by the Processor for FPGA partial reconfiguration. The AdministrationBlock is responsible for coordinating the Local Bus Macros, used to handle the communica-tion between a Reconfigurable Module and the PLB bus. The FPGA manufacturer providesspecific tools for partial reconfiguration system designs in [48]. The reconfiguration man-agement using ICAP interface is implemented in services provided by the AWL, which willbe explained later on in following sections.
Hardware agent’s behavior is implemented directly in hardware and the wrapper layer isresponsible for copying data between hardware agents and the Jade structure located in the
Fig. 3 Reconfigurable hardware architecture
Dynamically reconfigurable hardware/software mobile agents
Fig. 4 A modified sensor nodewith an FPGA
Proxy Agent. This layer wraps hardware agent’s data to a format that fits into the Jade struc-ture. Therefore, each hardware agent interface must obey a standard in its implementation.
For each hardware agent created, specific memory mapped addresses ranges must beimplemented for specific purposes of wrapper services. A memory region of 512 bytes is re-served for agent communication, and another region with an arbitrary size (specific to eachhardware agent) is currently the standard implemented. The agent communication mem-ory area is also used to verify status and provide handshaking mechanisms during FPGAhardware reconfiguration.
4.1 Reconfigurable agent
Although both hardware and software agents can be reconfigured, their mobility might belimited by the target execution platform. Moreover, the underlying framework responsibleof migration management must be able to handle both kinds of agents.
So, the focus of this work is to enable the coexistence of hardware and software agents ona network, allowing their mobility among nodes and, thereby, the possibility of an agent todynamically change is execution nature (from hardware to software, or vice-versa). A typicalsensor node has a processor, memory, a transceiver, a power unit and a sensing unit. One ofthe purposes of this work is to support agents running in hardware, and to do so, an FPGAis being added to the structure. Figure 4 shows this architecture.
A reconfigurable agent is defined in this work as being an agent capable of migratingbetween software and hardware execution environments, and possibly changing its execu-tion environment. The agent carries the state of the tasks that it executes, so it is able torestart the application from the point where it stopped in both software and hardware. In thearchitecture proposed in this work the location where an agent is executed is abstracted bycreating a layer between hardware agents and software agents that performs the necessarymapping.
The analysis of the platform costs, as for instance the overhead to reconfigure agent ashardware or software or even the costs to move an agent between the network nodes can beused to classify the costs that other reconfigurable agents will have, though this is still notthe main focus of this work.
5 The Agent Wrapper
When a hardware agent is created, it is necessary to keep a proxy agent in software thatwill encapsulate the data that is coming from a hardware agent, and it will allow a trans-parent communication with other software agents. For instance, when an external agent is
D. Cemin et al.
Fig. 5 Data flow inside the wrapper
communicating with a hardware agent, it will do so by using by using services from theMAS framework (in our case from the JADE framework). Hence, the proxy agent acts asan interface to map the data to/from hardware agents. This interface is called wrapper be-cause it concentrates the incoming requests from software agents to access the hardware,and encapsulates the data coming from hardware into a structure that is known by proxyagents.
The Wrapper Layer comprises several layers, mostly at software, but also at hardwarelevel. First, kernel level support is created to access the FPGA, with read/write functionsas well as functions that enable the dynamic FPGA programming directly from the mainprocessor. Second, an user level application that accesses the kernel level is responsiblefor creating mechanisms that guarantee the mutual exclusion from different agents tryingto access one device simultaneously. This is extremely necessary to avoid synchronizationproblems due to concurrent access to shared data. Finally, by using this structure, the map-ping between the agents’ information and the hardware is provided.
The wrapper communicates with both hardware and proxy agents, in this case imple-mented in Java and running on the JADE runtime system. When an agent is requesting aninformation from a memory mapped device for example, it sends a request command to thewrapper that accesses the hardware and answers the request with the desired information.
Figure 5 depicts how data from hardware agents and java agents are exchanged. When-ever hardware agents are communicating with other external agents that can be runningeither in software or even in hardware, they will use the structure already created by Jadeframework, or another MAS equivalent technology. Communication between proxy agentsand Wrapper is performed by using common UDP sockets, which is shown by the signal 1,and this contributes to the system’s portability, once UDP stack is found in several embed-ded operating systems. The communication inside the wrapper, between user and kernelspaces, is performed by using common read/write system calls. Finally, signal 3 representsthe communication using the actual bus between Processor and FPGA.
Agents are able to exchange information by using communication services provided bythe MAS framework. Once a message arrives in a node that has a hardware agent, thismessage is received by the proxy agent (coming from the MAS framework). Then, the agentcontent is copied into the memory mapped region pre-allocated to hardware agents. Thewrapper deals with the communication between the hardware and software environmentsand enables the hardware agents, together with the proxy agent, to be seen as a normalsoftware agent to the outside.
Dynamically reconfigurable hardware/software mobile agents
Fig. 6 Wrapper services for asoftware agent
Fig. 7 Wrapper services for ahardware agent
5.1 Wrapper services
In summary, services provided by Wrapper to a software agent are presented in Fig. 6.They include local services for sensor and actuator access. Additionally, services to manageFPGA and access to hardware agents are also provided. Services for FPGA access willalways respond a special error signal when a FPGA is not present into the target platform.
From a hardware point of view, most wrapper services are related to the communicationwith the proxy agent Fig. 7 shows the summary of these services. From this picture it can beseen that hardware agents can also initiate a communication, which will be first treated bythe wrapper. In the sequence, the wrapper will hand it over to the proxy agent, which willactually do the communication using the MAS framework services.
In order to ensure transparency among all agents, the procedure described in the previ-ous paragraph is followed even in case the target agent is a hardware agent residing in thesame node (that means, even communication among hardware agents located in the samenode have to be handled by their proxy agents). This assures that communication is homo-geneously handled, regardless the type and location of the communication agents.
6 Mobile agents
As it was already explained in Sect. 2.1, mobile agents are intelligent processes that are ca-pable of migrating between devices carrying their code and data. Jade is used to develop theMAS because it has support to be executed in embedded platforms as well as standard desk-tops and it is FIPA-ACL compliant, which is a standard communication protocol. Thesecharacteristics make easier to build experimental tests to prove the transparency commu-nication of the proposed reconfigurable agent with standard software agents, allocated in
D. Cemin et al.
Fig. 8 Internal architecture of a generic JADE agent appended with Agent Wrapper services (in dashed box)
different nodes. Figure 8 shows different parts of Jade agent architecture (with the exceptionof Agent Wrapper dashed box included by this proposal).
6.1 System modularity
Reconfigurable agents need to communicate with any agent in a transparent manner, andthey have to be able to migrate and to use all other services that a normal software agentmight have. Therefore, this work concentrates in providing an extra component (the wrap-per layer, explained in earlier sessions) to glue the hardware agent to their correspondingsoftware agents. So, all services already provided by the MAS framework can also be usedby the hardware agents. This glue can be seen in the Fig. 8 as an extra box (Agent Wrapper).By these means, it is also possible to make the system less dependent on the MAS frame-work used. So, other frameworks offering equivalent services, as for example MobileC [21],could also be used.
Hardware agents are self-contained and they are basically a implementation of a soft-ware agent’s behavior. They communicate directly with the wrapper, which is the interfacebetween proxy software agents and hardware agents. Wrapper is also MAS framework in-dependent, as long as it is running over any Linux operating system that contains the basicnetwork stack for communication. Software agents are currently running under Jade archi-tecture, which implements the communication between several agents in the MAS.
6.2 Agent migration internals
Jade provides methods that support the migration of agents among network nodes. Thesemethods implement the protocol that copy an agent and its context in a node, transfer thisinformation to the target node, stop the agent in the original node and restart it on the targetnode. Sometimes it is necessary to perform some actions before and right after moving anagent, and Jade provides atomic methods that will perform these actions, that means, thereis no context switching on the Jade scheduler when these methods are being executed. Themethods that perform these actions before and after moving are called beforeMove() andafterMove() respectively. They are inherited when one instantiates an object derived fromthe Agent class, and it is possible to overload them to fulfill specific agent’s needs.
The idea of migrating an agent is well defined in the FIPA-ACL specification. The agentmigrates itself by being encapsulated into a message and carried out through the ACL pro-tocol, which means that the agent will be traveling as a content of a message between two
Dynamically reconfigurable hardware/software mobile agents
different containers, and the Jade implementation will take care of starting the agent in thenew location.
A similar procedure is taken in our approach to reconfigure an agent from software tohardware or vice-versa: it is considered that such an action is similar to a migration fromone node to another, but in this case this is a “migration” from software to hardware. Letus consider the reconfiguration of a software agent to become a hardware agent: in thiscase the afterMove() function calls the wrapper service to manage the FPGA configuration(which is actually managed by the underlying Operating Service) and unpack the data tothe local FPGA hardware agent memory. If the migration occurs in other way around, thebeforeMove() must call wrapper services to collect data from FPGA hardware agent memoryand pack it to be sent via the proxy agent.
6.3 Examples of hardware agents communication
In other to better illustrate the interaction of different types of agents, Fig. 9 presents anexample of two platforms, each one comprising several agents. A pure software agent stillmight use (but not necessarily) the wrapper services as, for instance, to reconfigure itself toa hardware agent. A hardware agent, as discussed previously, always needs a proxy agent asfront end to communicate and use MAS framework services.
A sequence diagram of the communication between two hardware agents can be seen inthe Fig. 10. A hardware agent initiates the communication by sending a signal to the wrap-per, which will call the MAS where the desired target agent is located in the network. Thisprocess is done using Jade’s primitives. Once the desired agent is discovered, the communi-cation is established and they can communicate.
The sequence diagram depicted in Fig. 11 shows the process of migrating an agent be-tween two different platforms. A software agent might, by some reason, start the request ofmoving a hardware agent to other platform. Once the process starts, the hardware agent firstis packed and reconfigured as a software agent. Then the migration process takes advantageof the support provided by the MAS framework to move to the target node. After beingmoved, the agent notifies the start node that the process was completed and then reconfigureitself to a hardware agent. After moving, the agent informs the origin node that the processwas completed and then it is unpacked into a hardware architecture.
The sequence diagram in Fig. 12 depicts the process of migrating a software agent tohardware. First, the software agent looks in its own architecture for its hardware version,and, when it can’t find any hardware version, it starts looking in the network. There is alocal table in each agent saying which kind of hardware agents are located on each node, so,once the agent finds a node, it starts the migration process. From this point, the migration is
Fig. 9 Simple MAS comprising two platforms and different types of agents
D. Cemin et al.
Fig. 10 Two hardware agents communicating
Fig. 11 Hardware agent migrating to hardware
very similar to what is done when migrating from hardware, so the agent copies itself intohardware and starts working from the hardware architecture. The migration from hardwareto software is similar, but when the agent arrives in the destination node, it is not moved tohardware, and it starts running from the software architecture.
7 Experimental testbed and results
A testbed was setup in order to evaluate and prove the concepts here presented. The ex-periment was conducted with three execution platforms. One Personal Computer (PC) wasused as a node capable to execute only software agents. In addition, two other identical ex-ecution platforms were used, with the following characteristics: (i) a Freescale processor
Dynamically reconfigurable hardware/software mobile agents
Fig. 12 Software agent migrating to hardware
MPC8314E PowerPC coupled with a Virtex6 LXT130 FPGA from Xilinx; (ii) an externalmemory of 1 GByte of size; (iii) and supported by an embedded Linux operating system.The frequency clocks of the PowerPC and FPGA were 400 MHz and 100 MHz, respectively.These two last platforms are able to execute both software or hardware agents, or a combi-nation of them. On a PC the standard JADE framework was used as MAS framework, andon embedded platforms, the embedded JADE framework version was used.
The experiment was instrumented in order to enable the timing measurements related tothe migration and execution costs. Xilinx tools, such as Chipscope, were used to measurethe overhead of migrating agents to the FPGA as well as to measure the timing behaviorinternal to the FPGA.
Two different agents, both having a hardware and software version, were designed andimplemented in order to evaluate different aspects: a Floating Point Unit (FPU) Agent anda Kalman Filter Agent. These agents were selected because both are frequently used in au-tomation applications: floating point calculations are usually required in navigation tasks forAGVs or UAVs and Kalman Filters are commonly applied in sensor acquisition or controlalgorithms. It was defined that each agent should be able to migrate among nodes as well asto self-reconfigure to hardware or software.
Prior to the experiments execution, the memory overhead of each component was mea-sured and the required memory is depicted in Table 1. The largest amount of memory is usedby the Jade architecture, which can be explained by the use of the virtual machine, whichnormally consumes a considerable amount of memory. In our system, Jade uses approxi-mately 3 % of the system’s memory, while the wrapper uses approximately 0.2 %.
7.1 FPU agent
A system was deployed into two simple agents communicating with each other. In this test,an agent called Requester randomly generates two floating point numbers and send themto another agent, the FPU, asking the FPU to perform a given operation (also sent as aparameter). The FPU agent then calculates the operation and returns the result.
D. Cemin et al.
Table 1 Occupied memory fromdifferent parts of the system Parameter Size
Jade 34.1 MB
Proxy Agent 4.3 MB
Wrapper 184.5 kB
Fig. 13 Requester and FPU agents exchanging data
The FPU agent implements IEEE-754 standard, which defines how floating point num-bers with single and double precision are described [18]. Depicted in Fig. 13, this simplearchitecture with a software agent in a platform called Platform A generates numbers andrequests a result to the FPU agent, which resides at Platform B.
The Requester agent generates two numbers based on math class from java. The oper-ation is incremented from 0 to 3, where 0 means addition, 1 subtraction, 2 multiplicationand 3 division. The information is sent through the network to the FPU agent, which parsesthis information and maps to the hardware the necessary part of the message. The hardwareimplementation from this standard can be found at [24], and it is an open-source code. Two64-bit numbers are mapped into the hardware module as well as the 3 bit operation sig-nal. The FPU implemented in the hardware processes information and returns results. Thewrapper takes the result and encapsulates it in a ACL message and answers to the Requester.
FPU agents are capable of migrating between different nodes perceiving the environmentwhere they are. The first activity when a migration agent arrives on a target node, is theverification of FPGA availability. If so, the wrapper request the FPGA reconfiguration withthe proper bitstream. Currently, the bitstream needs to be stored inside the target node systemmemory. In the future, there will be services that will request a bitstream server, located onthe network.
Once the FPGA is programmed with the FPU hardware agent, the proxy agent copiesthe data that is arriving on the node to the memory region that describes the agent. Allthe information exchanged between the FPGA and the external agents is handled by thewrapper, which is responsible for parsing the data from hardware into jade structures andvice-versa.
Table 2 shows the message size exchanged between Requester and FPU agent (soft-ware or hardware). The values are considerably larger than the amount of bytes mentionedearlier for the operations. The Requester agent sends two 64 bits numbers plus a single byteto describe the desired operation, and so it should cost just 17 bytes from data. However,the results have showed that it is using 560 Bytes to send this information from Platform A
Dynamically reconfigurable hardware/software mobile agents
Table 2 Data size exchangedbetween Requester and FPUagents
Parameter Value
Requester output data 560 Bytes
FPU output data 448 Bytes
Table 3 Time to write/read asingle byte from/to hardwareagent by wrapper services
Parameter Value
Write average time 4.4 µs
Read average time 4.2 µs
Table 4 Measured time of eachelementary operation with a100 MHz clock signal
Operation Clock cycles Time
Addition 26 260 ns
Subtraction 27 270 ns
Multiplication 30 300 ns
Division 77 770 ns
to Platform B. This happens because of the Jade’s message encapsulation, which creates adata overhead that is used to create the FIPA-ACL messages. The same happens to the FPUoutput data message, which should be only 8 bytes long, and generates a message size of448 Bytes.
7.1.1 FPU hardware agent details
Table 3 shows the time costs to write/read a single byte between the proxy agent and thehardware agent, using wrapper services. According to this table a single byte read or writeoperation costs about 4 µs to be executed, and then the necessary time to map all data intohardware is approximately 70 µs, which corresponds to writing two 64-bit registers plus asingle byte. Reading data out consumes almost 35 µs, because it is necessary to read onlyone 64-bit register.
The read and write times are measured in the user space application inside the wrapper.These time measurements reflect how much time is necessary, from the application point ofview, to read and write data in the memory map reserved space. They are both approximately4 µs long, which is relatively high, because this copy can be a bottleneck of the system. Fromthe point of view of programmable logic, the read and write cycles are smaller than 1 µs,according to the simulations that has been done for the platform. So, the time difference oftime is attributed to the overhead inserted by the kernel module that accesses the Localbusand even the Localbus overhead as well.
Chipscope was used to measure the cycles of the four elementary operations provided bythe FPU module. These values are presented in Table 4. The worst case was to perform afloating point division, which required up to 77 clock cycles.
7.1.2 FPU software agent details
Table 5 presents the timing to execute each operation of a FPU software agent. Since this isa pure software version, timing values already embraces the parameters passing to the agent
D. Cemin et al.
Table 5 FPU software agentOperation Time
Addition 1.3 µs
Subtraction 1.4 µs
Multiplication 50.3 µs
Division 340 µs
Table 6 Migration timeMigration case Time
Software to Software 5.3 ms
Software to Hardware 5.6 ms
Software to Hardware + partial FPGA* 2.0 s
Software to Hardware + full FPGA 20.4 s
operation. It has to be noticed that the FPU software agent does not use the CPU floatingpoint unit. Instead, all operations are implemented in JADE agent (Java routines) for a bettercomparison with the hardware agent.
7.1.3 FPU agent migration
Reconfiguration and migration costs in different directions were measured, and the valuesare presented in Table 6. First, it was measured the cost of migrating an agent between twodifferent nodes when both source and destination are software agents. Second, the sameagent is migrating between the same nodes, and now, instead of being a software agent, it isrunning in hardware. In this case, the FPGA is already configured and this result representsthe amount of time of copying data between software and hardware platforms. Finally, twomore cases were tested: Migrating the agent between software and hardware in differentnodes and configuring the FPGA on the destination end. The timing for partial FPGA re-configuration corresponds to preliminary tests, since the partial reconfiguration design wasnot totally finished. The full configuration of the FPGA is the one that is taking more timeto finish, since the FPGA is fully reconfigured.
The FPU mobile agent size was also measured, and it costs 2582 Bytes as message size.This value was the same, regardless of migration case. This was expected to be so, sinceJADE framework does not know about the existence of different agent types. Moreover, inthe FPU agent case study, the FPGA configuration bitstream was considered to be locallystored on target node.
7.2 Kalman filter agent
As a second example, a Kalman filter agent was selected. A Kalman filter represents a morecomplex operation, which is widely used in distributed control and sensor network applica-tions, to name a few, as an ideal estimator.
To give sequence to the experiments, it will be described an example using a Kalmanfilter with the objective of validating the proposed architecture. The main idea of this studyis to leverage the obtained results from the FPU agent as a way to estimate the computationalcost of implementing a Kalman filter agent in hardware. This study is important because it
Dynamically reconfigurable hardware/software mobile agents
provides a classification of agent types (not the focus of this work) that are more suitable tobe executed in hardware based on the migration’s overhead.
Kalman filter is an algorithm that can estimate the variables of a given process and alsopredict the system’s state. From the point of view of state-space modelling, Kalman fil-ter is an ideal estimator. It has been used for example in UAV’s navigation and controlsystems [50], sensor’s fusion and localization [1] and even to estimate a battery’s state ofcharge [9].
Kk = APkCT(CPkC
T + Sz
)−1(1)
x̂k+1 = (Ax̂k + Buk) + Kk(yk+1 − Cx̂k) (2)
Pk+1 = APkAT + Sw − APkC
T S−1z CPkA
T (3)
Kalman filter can be described with Eqs. (1), (2) and (3). Equation (1) shows the Kalmangain and the result from these operations are used inside Eq. (2), which calculates the nextstate based on the current state. The filtering process is based on the covariance error throughthe measured states, which is described by Eq. (3).
Kalman Filter is an iterative algorithm and interested readers are referred to [19] for amore detailed description. A second order system, which describes the movement of an ob-ject with constant acceleration [39], has been analyzed. This system is described by Eq. (4).
xk+1 =[
1 T
0 1
]xk +
[T 2/2
T
]uk + wk
yk = [1 0
]xk + zk
(4)
Table 7 shows the necessary operations to be implemented using the designed Kalmanfilter for this case study. It is considered the worst case with no pipelines and/or paral-lelization whatsoever. From the results showed in Table 7, it is possible to estimate theduration of a single step in Kalman filter in the designed system. This time is estimated as30 × 260 ns + 1 × 270 ns + 40 × 300 ns + 2 × 770 ns = 21510 ns.
The overhead associated is given by the cost of reading and writing data in the memorymapped region. In each loop iteration the Kalman filter agent has to read two 64-bit registers.Using the results provided by Table 3 and knowing that is necessary to read 16 bytes, theestimated time can be calculated as 16 × 4.2 µs = 67.2 µs.
For the Kalman filter agent here evaluated, each loop was repeated 10,000 times and theresults were compared between hardware and software versions. Table 8 presents the valuesmeasured for this run for a hardware and software version of the Kalman agent. The valuemeasured for the software agent represents the CPU loaded by 100 %. The message sizerepresents the amount of data exchanged between the Requester Agent and Kalman Agent.
D. Cemin et al.
Table 8 Measurements forKalman agent Agent type Message size Execution time
Software 5452 Bytes 12.74 s
Hardware 5452 Bytes 5 ms
Table 9 Kalman agentmigration costs Migration case Message size Time cost
Software to Software 3634 Bytes 8.35 ms
Software to Hardware 3634 Bytes 9.18 ms
Software to Hardware + full FPGA 340 KBytes 21.2 s
7.2.1 Kalman agent migration details
Different migration cases were measured for the Kalman agent, and they are presented inTable 9. The first migration case represents the simple migration of a Kalman softwareagent from one platform to another. The second case is the migration of a software agentfrom node and its reconfiguration as a hardware agent in a target node. The third migrationcase considers that the bitstream is also packed as data during the migration and the FPGA isfully programmed in the target node. For Kalman case study, the partial reconfiguration wasnot finished, but a careful simulation point out that this would require around 3.0 seconds.
8 Related work
Agilla [11] is a mobile agent application to detect fire in remote locations. It runs on top of aTinyOS [22], a small footprint operating system with large usage in sensor networks. Agillasupports the migration of mobile agents across computational nodes and it differentiatesbetween two types of migration: when the agent migrates carrying its code and data this isconsidered as a hard migration and when an agent does not carry its execution data duringmigration—that means it is not able to restart application from the point just before themigration time—this is considered as soft migration.
Regarding reconfigurable hardware used in dynamic scenarios, prominent reconfigurableprocessing platforms are worth to be mentioned. Shafique et al. proposes a dynamicallyadapting extensible processor, at instruction level, in [2]. A so called Rotating InstructionSet Processing Platform (RISPP) is composed by a pre-running analysis phase and a run-time adaptation phase. In [36, 37] it is shown that RISPP is energy efficient.
Lysecky et al. introduces the WRAP processor in [25, 26], which is based on a proposedwrap process. In this approach, a binary (processor machine code) is (transparently to theuser) recompiled, applied a hardware/software partition, and the hardware extracted at RTLevel is synthesized (mapping, placing and routing) to an FPGA. All these processes aremade dynamically. It means, a profiling is used to support the hardware/software partitiondecision and, additionally, on-board lean algorithms are needed for hardware synthesis.
These approaches focus on low level bit manipulations, either at instruction level proces-sor or at processor binaries. So, they differ from the proposal here presented since it focuseson agent level, which are written in JAVA to allow code mobility on different executionplatforms.
Dynamically reconfigurable hardware/software mobile agents
In a distributed computing scenario, in [17] Hsieh proposes the usage of reconfigurableplatforms for augmented Wireless Sensor Network (WSN), as Wireless Multimedia SensorNetworks (WMSN), for instance. However, it focuses on the strategies for task allocationon the nodes of such network based on biologically inspired algorithms.
Kearney and Jasiunas in [20] presented a swarm of UAVs, where each node is augmentedwith an FPGA. The proposal enables the migration of tasks among the nodes, profiting fromflexibility, computation performance, and being thereby energy efficient. The applicationscenario is a UAV based swarm for area surveillance. The approach, however, allows onlytasks migration, not agents, and without enabling the reconfiguration from hardware to soft-ware, or vice-versa.
Considering MAS and reconfigurable hardware, there are works that leverage the use ofFPGAs and their reprogrammability. Yan Meng [29, 30] proposes an FPGA based agent ar-chitecture with a proprietary agent communication. It uses a Belief-Desire-Intention (BDI)model to its agents that describes tasks performed in the system. In its work, partial recon-figurability to change the agent is used to keep the system’s real-time constraints to dynam-ically adapt a changing system, as well as to change some tasks at running time based onsystem’s dynamic requirements.
Dante et al. [41, 42] proposes a platform called Hardware Embedded Reactive Platform(HERA) to support the design of hardware agents, in order to integrate sensors into a WSN,using possible different radio technologies. Thereby it enables a dynamic changing of agentto personalize their reactions.
George Mois et al. [31] propose the use of hardware agents in the testing and repairingof distributed systems. They reconfigure FPGAs dynamically when a problem occurs andagents are used to communicate and carry FPGA bitstreams.
In order to allow the development of hardware and software agents, a general archi-tectural framework was introduced in [32]. This framework extends the concept of a Multi-Agent System to support reconfigurable systems, in this case, using an FPGA. Later on [33],this work was extended concerned hardware agents reconfigurability, as well as architecturalaspects. Agents communication among different types, both hardware and software, are dis-cussed.
One of the first work that introduces the hardware agent with migration capability (codemobility aspect) was introduced by Benso et al. in [3]. In its approach a MAS is chosen toconfer self-healing properties to a system. However, the presented system considers onlyhardware agents, with a property architecture and MAS framework. So, interoperability wasnot a present characteristic in that system. Later on, in [6] a more comprehensive solutionwas proposed with no possibility to dynamically reconfigure a hardware agent into a soft-ware agent, or vice-versa.
As it can be observed from the analysis of the state of the art presented in the previousparagraphs, there is no solution, at agent level, that enables an agent to freely migrate amongnetwork nodes, and thereby configuring itself to match the underlying execution platform.The work here presented uses mobile agents as well as reconfigurability concepts. Jade isa case study framework used to move code and data in the network and also to providea reliable agent communication infrastructure. Regardless of the agents’ nature (hardwareor software), the communication infrastructure used is the same, enabling the transparencyamong agents. So, the Jade agent is extended to enable the usage of reconfigurable hardware.
Moreover, the proposed extension enables the dynamic exchange of a software agent toa hardware agent, or vice-versa. The agents migration, regardless its nature, is supported aswell. Even though Jade framework is used for this work, the concepts can be easily exportedto another multi-agent system platform.
D. Cemin et al.
Hence, to the best of our knowledge, no comprehensive solution for a dynamically re-configurable hardware/software mobile agent, assuring thereby transparency under recon-figuration and integration with other agents, has been proposed yet.
9 Conclusions
An architecture and the correspondent strategies to enable the existence of dynamically re-configurable hardware/software agents was presented and implemented. The results demon-strate the feasibility of this proposal, supporting both software and hardware agents commu-nicating transparently. The proposed approach also allows dynamic agent migration alongwith its reconfiguration, that means, a software agent can leave a node and migrate to an-other node on which it will become a hardware agent. Even in this case, the other agentsin the system can continue communicating with the new hardware agent in the same wayas they communicated with the original software agent, because both migration as well asreconfiguration occur in a transparent manner to the other agents in the system.
The results have demonstrated that there is an overhead associated to each migrationthat corresponds to the data mapping between software and hardware. This aspect has tobe taken into account when designing hardware agents that needs to exchange considerableamount of data. This could also be solved by proposing novel architectures for coupling thereconfigurable hardware to the main processor.
FPGA programming is also another aspect that needs special care. Currently, the adoptedstrategy relies on programming the FPGA on the fly. So, the programming time increases thetotal agent migration time. Even for partial reconfiguration, this time might be considerablehigh. A solution that is currently investigated is to proceed with FPGA configuration beforenode migration.
Another aspect is the bitstream location and storage. For the case studies here presented,two alternatives were adopted: bitstream was sent along with the agent; or it was previouslystored on the target node.
This paper has not focused on energy aspects, although authors recognize that energy andpower consumption are important metrics for embedded systems. Hardware agents usuallypresent a better performance than software agents running on GPUs on top of MAS middle-ware. The idea of allowing hardware agents to self-reconfigure their hardware environments,for instance using voltage/frequency scaling techniques is an interesting future work.
References
1. Anjum M, Park J, Hwang W, il Kwon H, Kim HJ, Lee C, Kim SK, il Danr CD (2010) Sensor data fusionusing unscented Kalman filter for accurate localization of mobile robots. In: International conference oncontrol automation and systems (ICCAS), pp 947–952
3. Benso A, Cilardo A, Mazzocca N, Miclea L, Prinetto P, Szilard E (2005) Reconfigurable systems self-healing using mobile hardware agents. In: Proceedings of IEEE international test conference, ITC 2005,pp 9–476
4. Bussmann S, Jennings NR, Wooldridge MJ (2004) Multiagent systems for manufacturing control: a de-sign methodology. Springer series on agent technology. Springer, Berlin, New York, Paris
5. Cervera E (2008) Practical multi-robot applications with player and JADE. In: ICARCV 10th interna-tional conference on control, automation, robotics and vision, pp 2004–2009
6. Cilardo A, Coppolino L, Mazzocca N (2007) A framework for the design of distributed reconfigurableembedded systems. In: 20th international conference on architecture of computing systems (ARCS), pp1–8
Dynamically reconfigurable hardware/software mobile agents
7. de Freitas EP (2011) Cooperative context-aware setup and performance of surveillance missions usingstatic and mobile wireless sensor networks. PhD thesis (doctor of philosophy), Halmstad University,Halmstad, Sweden
8. de Freitas EP, Heimfarth T, Pereira CE, Ferreira AM, Wagner FR, Larsson T (2011) Multi-agent supportin a middleware for mission-driven heterogeneous sensor networks. Comput J 54(3):406–420
9. Di Domenico D, Fiengo G, Stefanopoulou A (2008) Lithium-ion battery state of charge estimation witha Kalman filter based on a electrochemical model. In: IEEE international conference on control applica-tions, CCA, pp 702–707
10. Fok CL, Roman GC, Lu C (2005) Rapid development and flexible deployment of adaptive wirelesssensor network applications. In: Proceedings of the 25th IEEE international conference on distributedcomputing systems, ICDCS ’05. IEEE Computer Society, Washington, pp 653–662
11. Fok CL, Roman GC, Lu C (2009) Agilla: a mobile agent middleware for self-adaptive wireless sensornetworks. ACM Trans Auton Adapt Syst 4:16:1–16:26
12. Garcia P, Compton K, Schulte M, Blem E, Fu W (2006) An overview of reconfigurable hardware inembedded systems. EURASIP J Embed Syst 2006:1–19
13. Gomez-Gualdron J, Velez-Reyes M, Collazo L (2007) Self-reconfigurable electric power distributionsystem using multi-agent systems. In: Electric ship technologies symposium—ESTS. IEEE Press, NewYork, pp 180–187
14. Götz M, Rettberg A, Pereira CE (2006) Run-time reconfigurable real-time operating system for hybridexecution platforms. In: Proceedings of the 12th IFAC symposium on information control problems inmanufacturing (INCOM), Saint-Etienne, France, vol I—Inform. Elsevier, Oxford, pp 81–86
15. Götz M, Dittmann F, Xie T (2009) Dynamic relocation of hybrid tasks: strategies and methodologies.Microprocess Microsyst 33(1):81–90
16. Herbert J, O’Donoghue J, Ling G, Fei K, Fok CL (2006) Mobile agent architecture integration for awireless sensor medical application. In: IEEE/WIC/ACM international conference on web intelligenceand intelligent agent technology workshops, WI-IAT workshops, pp 235–238
17. Hsieh CM, Wang Z, Henkel J (2013) DANCE: distributed application-aware node configuration enginein shared reconfigurable sensor networks. In: Design, automation test in Europe conference exhibition(DATE), pp 839–842
18. IEEE: IEEE 754 standard (2012). http://grouper.ieee.org/groups/754/. Access in March 201219. Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82(Series
D):35–4520. Kearney DA, Jasiunas M (2007) Using simulated partial dynamic run-time reconfiguration to share em-
bedded FPGA compute and power resources across a swarm of unpiloted airborne vehicles. EURASIP JEmbed Syst 2007
21. Laboratory (2012) U.D.I.E.: mobilec project website. http://www.mobilec.org/. Access in March 201222. Levis P, Madden S, Polastre J, Szewczyk R, Whitehouse K, Woo A, Gay D, Hill J, Welsh M, Brewer E,
Culler D (2005) TinyOS: an operating system for sensor networks ambient intelligence. In: Weber W,Rabaey JM, Aarts E (eds) Ambient intelligence. Springer, Berlin/Heidelberg, pp 115–148. Chap 7
23. Luck M, McBurney P, Preist C (2003) Agent technology: enabling next generation computing(a roadmap for agent based computing). Agentlink. ISBN 0854 327886
24. Lundgren D (2012) Opencores project web site. http://opencores.org/project,fpu_double. Access inMarch 2012
25. Lysecky R, Vahid F (2009) Design and implementation of a microblaze-based warp processor. ACMTrans Embed Comput Syst 8(3):22:1–22:22
26. Lysecky R, Stitt G, Vahid F (2006) Warp processors. ACM Trans Des Autom Electron Syst 11:659–68127. Marík V, Lazansky J (2007) Industrial applications of agent technologies. Control Eng Pract
15(11):1364–138028. Marik V, McFarlane D (2005) Industrial adoption of agent-based technologies. IEEE Intell Syst
20(1):27–3529. Meng Y (2005) An agent-based reconfigurable system-on-chip architecture for real-time systems. In:
Second international conference on embedded software and systems (ICESS’05). IEEE Press, New York,pp 166–173
30. Meng Y (2006) An agent-based mobile robot system using configurable SOC technique. In: Proceedings2006 IEEE international conference on robotics and automation, ICRA, pp 3368–3373
31. Mois G, Stefan I, Enyedi S, Miclea L (2010) Reconfiguration and hardware agents in testing and repairof distributed systems. In: Design test symposium (EWDTS), pp 195–198
32. Naji H, Wells B (2002) On incorporating multi-agents in combined hardware/software based reconfig-urable systems—a general architectural framework. In: Proceedings of the thirty-fourth southeasternsymposium on system theory (Cat. no. 02EX540). IEEE Press, New York, pp 344–348
33. Naji HR, Wells B, Etzkorn L (2004) Creating an adaptive embedded system by applying multi-agenttechniques to reconfigurable hardware. Future Gener Comput Syst 20(6):1055–1081
34. Onori M, Lohse N, Barata J, Hanisch C (2012) The IDEAS project: plug & produce at shop-floor level.In: Assembly automation, vol 32. Emerald, Bingley, pp 124–134
35. Quaritsch M, Kruggl K, Wischounig-Strucl D, Bhattacharya S, Shah M, Rinner B (2010) Net-worked UAVs as aerial sensor network for disaster management applications. E&I, Elektrotech Inftech127(3):56–63
36. Shafique M, Bauer L, Henkel J (2009) REMiS: run-time energy minimization scheme in a reconfig-urable processor with dynamic power-gated instruction set. In: IEEE/ACM international conference oncomputer-aided design—digest of technical papers, ICCAD 2009, pp 55–62
37. Shafique M, Bauer L, Henkel J (2010) Selective instruction set muting for energy-aware adaptive pro-cessors. In: IEEE/ACM international conference on computer-aided design (ICCAD), pp 353–360
38. Shen W, Norrie D (1999) Agent-based systems for intelligent manufacturing: a state-of-the-art survey.Knowl Inf Syst 1(2):129–156
39. Simon D (2001) Kalman filtering. In: Embedded systems programming, pp 72–7940. Srivastava S, Nandi G (2010) Localization of mobile robots in a network using mobile agents. In: Inter-
national conference on computer and communication technology (ICCCT), pp 415–42041. Tapia DI, Alonso RS, Rodríguez S, Paz JFD, González A, Juan M (2010) Embedding reactive hardware
agents into heterogeneous sensor networks. In: 2010 13th conference on information fusion (FUSION),Edinburgh. IEEE Press, New York, pp 1–8
42. Tapia D, Alonso R, García O, Corchado J (2011) Highlights in practical applications of agents andmultiagent systems. In: Pérez JB, Corchado JM, Moreno MN, Julián V, Mathieu P, Canada-Bago J,Ortega A, Caballero AF (eds) Highlights in practical applications of agents and multiagent systems—advances in intelligent and soft computing. Advances in intelligent and soft computing, vol 89. Springer,Berlin, pp 249–256. Chap HERA: Hard
43. TILAB: Jade website (2012). http://jade.tilab.com. Accessed in March 201244. Van Dyke Parunak H (1999) In: Weiss G (ed) Multiagent systems, chap. industrial and practical applica-
tions of DAI. MIT Press, Cambridge, pp 377–42145. Walder H, Platzner M (2004) A runtime environment for reconfigurable hardware operating systems. In:
Proceedings of the 14th international conference on field-programmable logic and applications (FPL),pp 831–835
46. Wigley G, Kearney D (2001) The development of an operating system for reconfigurable computing. In:Proceedings of the 9th IEEE symposium on field-programmable custom computing machines (FCCM),pp 249–250
47. Xilinx: Partial reconfiguration user guide (2011)48. Xilinx: Partial reconfiguration user guide. Chapter 4—PlanAhead support (2011)49. Yoneki E, Bacon J (2005) A survey of wireless sensor network technologies: research trends and mid-
dleware’s role. Tech. rep, University of Cambridge, Computer Laboratory50. Zhou QL, Zhang Y, Qu YH, Rabbath CA (2010) Dead reckoning and Kalman filter design for trajectory
tracking of a quadrotor UAV. In: IEEE/ASME international conference on mechatronics and embeddedsystems and applications (MESA), pp 119–124
