1

Modeling and Analysis of Energy Harvesting andSmart Grid-Powered Wireless Communication

Networks: A Contemporary SurveyShuyan Hu, Xiaojing Chen, Wei Ni, Xin Wang, and Ekram Hossain

Abstract—The advancements in smart power grid and theadvocation of “green communications” have inspired the wire-less communication networks to harness energy from ambientenvironments and operate in an energy-efficient manner foreconomic and ecological benefits. This article presents a con-temporary review of recent breakthroughs on the utilization,redistribution, trading and planning of energy harvested infuture wireless networks interoperating with smart grids. Thisarticle starts with classical models of renewable energy harvestingtechnologies. We embark on constrained operation and opti-mization of different energy harvesting wireless systems, suchas point-to-point, multipoint-to-point, multipoint-to-multipoint,multi-hop, and multi-cell systems. We also review wireless powerand information transfer technologies which provide a specialimplementation of energy harvesting wireless communications.A significant part of the article is devoted to the redistributionof redundant (unused) energy harvested within cellular networks,the energy planning under dynamic pricing when smart grids arein place, and two-way energy trading between cellular networksand smart grids. Applications of different optimization tools,such as convex optimization, Lagrangian dual-based method,subgradient method, and Lyapunov-based online optimization,are compared. This article also collates the potential applicationsof energy harvesting techniques in emerging (or upcoming)5G/B5G communication systems. It is revealed that an effectiveredistribution and two-way trading of energy can significantlyreduce the electricity bills of wireless service providers anddecrease the consumption of brown energy. A list of interestingresearch directions are provided, requiring further investigation.

Index Terms—5G/B5G communication networks, energy har-

Work in this paper was supported by the National Natural Science Founda-tion of China under Grant 61671154, and the Innovation Program of ShanghaiMunicipal Science and Technology Commission under Grant 17510710400.The work of W. Ni was supported by the Fudan University Key Laboratory(State Key Lab of ASIC and System) Senior Visiting Scholarship. The workof E. Hossain was supported in part by an ENGAGE Grant (EGP 533553-18) from the Natural Sciences and Engineering Research Council of Canada(NSERC). The work of S. Hu and X. Chen was done when they were with theDepartment of Communication Science and Engineering, Fudan University.

S. Hu is with the State Key Laboratory of ASIC and System, the Schoolof Information Science and Technology, Fudan University, Shanghai 200433,China (e-mail: syhu14@fudan.edu.cn).

X. Chen is with the Shanghai Institute for Advanced Communicationand Data Science, Shanghai University, Shanghai 200444, China (e-mail:jodiechen@shu.edu.cn).

W. Ni is with the Commonwealth Scientific and Industrial Re-search Organization (CSIRO), Sydney, NSW 2122, Australia (e-mail:wei.ni@data61.csiro.au).

X. Wang is with the State Key Laboratory of ASIC and System, theShanghai Institute for Advanced Communication and Data Science, theDepartment of Communication Science and Engineering, Fudan University,Shanghai 200433, China (e-mail: xwang11@fudan.edu.cn).

E. Hossain is with the Department of Electrical and Computer Engi-neering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada (e-mail:ekram.hossain@umanitoba.ca).

The first two authors contributed equally to this work.

vesting, smart grid, energy redistribution and trading, optimiza-tion techniques.

I. INTRODUCTION

Future mobile cellular communication networks are envis-aged to be rolled out with a dramatically increasing numberof cells and continuously reducing cell size, due to explosivemobile traffic [1]. The traffic volume in the emerging fifth-generation (5G) systems and future systems beyond 5G (B5G)is estimated to be tens of Exabytes per month, expecting thecapacity of 5G/B5G networks to be 1000 times higher thanthat of current cellular networks [2], [3]. The thousand-foldincrease of system capacity must be achieved with a similaror even lower power level than today’s [4], [5]. Increasingthe network energy efficiency (EE) has been pursued by theGreenTouch consortium [6], [7]. Huawei has also deployedsolar-powered base stations (BSs) in Bangladesh [8]. Ericsonand Nokia Siemens Networks have designed green BSs withrenewable power supplies, such as wind turbines and solar pan-els, to reduce the consumption of fuel generated electricity [9],[10]. Energy-efficient techniques, such as BS switching [11],offline power allocation [12], and online data scheduling [13],[14] have been developed to reduce power consumption orincrease network capacity.

Along with the development of cellular networks, powergrid is also undergoing a radical revolution. Rapidly emergingsmart grids, enabled with smart meters, are expected to providenew intelligent functionalities, e.g., decentralized power pro-duction/generation, bidirectional (also known as “two-way”)energy trading, energy redistribution, and request manage-ment/coordination [15]. Cellular networks, as integrating com-ponents (or elements) of smart grids, can support effectiveenergy utilization and redistribution, and price negotiation byinteroperating with smart grids [16], [17].

Smart grid and “green communications” have spawnedextensive studies recently. Several magazine articles [1], [14],[16], [18]–[20] and survey papers [3], [21]–[29] reviewenergy-efficient and energy harvesting (EH) powered commu-nication networks from a range of different angles. However,none of the existing reviews captures comprehensively theconstrained wireless operations powered by renewable energysources (RES) and their underlaying optimization method-ologies, and the interoperability between wireless networksand smart grids (as done in this article). Energy-efficienttechniques for 5G networks are summarized in [3] and [21],

arX

iv:1

912.

1320

3v1

[ee

ss.S

Y]

31

Dec

201

9

2

EH and smart grid-powered wireless

networks

Types and models for RES (Sec. II)

Smart grid-powered wireless networks

(Sec. VIII)

Wireless power and information transfer

(Sec. VII)

5G/B5G applications of EH and smart grid-powered wireless networks (Sec. XI)

Energy redistribution

Energy planning under dynamic pricing (Sec. IX)

Two-way energy trading and cooperation (Sec. X)

EH-based wireless networks

Point-to-point link (Sec. III)

Multipoint-to-point system (Sec. IV)

Multipoint-to-multipoint system (Sec. V)

Multi-hop link (Sec. VI)

Lessons learnt and future directions

(Sec. XII)

Fig. 1. The main structure of this article.

from the aspects of network deployment, energy/spectrum ef-ficiency, and delay/bandwidth versus power. Featuring energy-harvesting wireless networks, beamforming techniques arereviewed in [22]. Energy scheduling, optimization, and ap-plication are presented in [23]. Circuit design, hardware im-plementation, EH techniques, and communication protocolsare reviewed in [24]–[27] with specific emphases on radiofrequency networks [24], sensor networks [25], [26], and jointinformation and power transfer systems [27]. Smart grid andits merits in improving the operations of wireless networksand software-defined networks are reported in [28] and [29],respectively.

This article takes a new and unique angle of the inter-operability of wireless communication networks and smartgrids. Recent breakthroughs on the utilization, redistribution,trading and planning of energy harvested in future wirelesscommunication networks interoperating with smart grids arereviewed. We start with state-of-the-art models of renewableEH technologies. Then, we embark on constrained wireless op-erations subject to non-persistent (renewable) energy harvestedfrom ambient environments. A range of energy-harvestingwireless communication systems, such as single point-to-point,multipoint-to-point, multipoint-to-multipoint, multi-hop, andmulti-cell systems. We also go through a special yet widelystudied class of EH techniques, where radio signals also playthe role to deliver energy and joint optimizations of wirelesspower and information delivery are carried out.

Performance metrics of energy-aware/constrained wirelessoperations include energy and operational cost minimization,and utility maximization, by considering power allocation,transmit beamforming, traffic load, and users’ quality-of-service (QoS). Many classical techniques and methodologiesare applied, such as convex optimization, Lagrangian dual-based method, game theory, dynamic programming (DP),(stochastic) subgradient method (SGM), Lyapunov-based on-line optimization, and so forth. Different system models andoptimization criteria allow us to characterize, quantify, de-

sign and compare different operating strategies of wirelessnetworks from different perspectives, when practical designparameters arise.

A substantial part of this article is devoted to the latesttechniques on the redistribution of redundant energy withincellular networks, energy planning under dynamic pricing,and bidirectional energy trading of cellular networks, as wellas smart grids. Empowered by renewable energy sources(RES), wireless power transfer (WPT), and/or smart grid,actions of energy management such as the harvesting of RES,and the redistribution and (predictive) trading of redundantenergy can be performed in the wireless networks to achievegreen and energy-efficient operations. As will be revealed, theuse of RES can significantly reduce the electricity bills ofwireless service providers and decrease the consumption ofbrown energy, i.e. coal and oil. The article also discusses thepotential applications of EH in future 5G/B5G networks, andemerging technologies such as mobile edge computing (MEC),deep learning, ultra reliable and low latency communication(URLLC), non-orthogonal multiple access (NOMA), and soon. As will be discussed, the energy management from thedemand side can have a profound impact on future wirelessnetworks.

By taking the new angle of the interoperability of wirelesscommunication networks and smart grids (as compared tothe existing surveys [3], [21]–[29]), we organize the rest ofthis article in the following way. Different types of RES andtheir popular mathematical models are colated in Section II,which are salient for design optimization of different typesof RES-powered wireless links and networks in Sections IIIto VI. In Section VII, the joint wireless power and infor-mation transfer system is reviewed to provide a differentperspective of how energy and information signals interact inwireless communication networks. From Sections VIII to X,we discuss the roles of wireless networks as energy consumer,harvester, and generator in the context of smart grid, andthus the interoperability between wireless networks and smartpower grids. Specifically, we investigate the redistribution ofredundant energy in wireless networks, energy planning underdynamic pricing policy of the smart grid, and bidirectionalenergy trading of EH wireless networks and smart grids.In Section XI, exciting 5G applications of EH, RES, andsmart grid-powered wireless devices are discussed. In SectionXII, potential research opportunities are summarized, followedby concluding remarks in Section XIII. Fig. 1 provides thediagram to show the organization of this survey.

II. RENEWABLE ENERGY SOURCES (RES)

A. Types of Renewable Energy

EH (or scavenging) is a series of actions to collect and trans-form environmental renewable energy into electrical energy.Unlike coal and oil, renewable energy can be regenerated witha wide range of different methods. There are various types ofRES, such as solar energy [30], [31], wind energy [32]–[34],electromagnetic (EM) radiation energy [23], [24], [35], [36],thermoelectric energy [37]–[39], and biomass energy [40].

3

1) Solar Energy: One of the most favored environmentalRES is solar energy which has been extensively applied todifferent scenarios and applications [30], [31], [41], [42].Sunlight radiation is transformed into electric power viaphotovoltaic cells, and then serves as the energy supply forself-supportable devices. Although a potentially inexhaustibleamount of energy can be obtained in this way, the energyusable to a device could vary drastically even within a shortduration in reality. Furthermore, the EH level can be swayed bymultiple factors, for instance, time of the day, solar elevationangle, weather conditions, environmental prerequisites, andcharacteristics of photovoltaic cells. These factors render thesolar energy to be uncontrollable, unpredictable and nondis-patchable. Typically, the amount of solar energy is in the orderof 100 mW/cm2 [23].

2) Wind Energy: Wind energy can be extracted by captur-ing the motion of the wind by a wind turbine. The rotor speedoutput is used to perform maximal power point tracking. Therotor frequency data is sent to a frequency-to-voltage (FV)converter which produces an appropriate voltage signal. Windenergy can be also obtained by utilizing the movement of ananemometer lever to activate an alternator and then using apulsed buck-boost converter to transform the movement toelectric power [25]. Typically, when the rotor diameter is 1m and the wind speed is 8 m/s, the amount of energy that canbe harvested is around 85 W [18].

3) Electromagnetic (EM) Radiation Energy: Harvestingenergy from EM radiation has provided convenient power sup-plies for networks. Featuring short-distance or long-distancescenarios, the types of EM power supplies can be catego-rized into two groups: near-field and far-field EM radiationenergy. In the near-field scenarios, EM induction and magneticresonance approaches [43], [44] often produce electric powerwithin the range of a wavelength, and hence, the power transferefficiency can exceed 80% in the near-field scenarios [35].In the far-field scenarios up to several kilometers, the EMradiation, propagating in the fashion of radio frequency (RF)or microwave signals, is received by antennas and transformedto electricity by rectifier circuits [24], [36]. The RF/microwavesignals could come from beamforming signals sent by agiven transmitter or environmental EM radiations from thevicinity [45]. Although the power density at the receivingend is related to the energy level of practical suppliers andthe EM wave transmission distance, this type of RES can bereadily utilized, managed, and forecasted, irrespective of time,location, and weather condition.

4) Thermoelectric Energy: The thermoelectric effect can beutilized for EH. In particular, a voltage signal can be generatedbetween two conductors made of different materials whentheir intersections are placed under different temperatures.In practice, such a temperature gradient can take place inhuman bodies or machine operations. The power densities ofthermoelectric generations depend typically on the propertiesand temperature difference of materials. Their values arecomparatively low and lie in the extent from 10 µW/cm2 to1 mW/cm2 [23].

5) Biomass Energy: Since the aforementioned methodsare not applicable for underwater EH, bacterial metabolic

activities have been exploited by microbial fuel cells (MFCs)to generate electricity directly from broken down substratum.Natural water contains abundant varieties of microorganismsand nutrients which are ideal for underwater EH by MFCs.The amount of the energy is generally 153 mW/cm2 [18].

B. Mathematical Models for Renewable Energy

1) Uncertainty Sets: The a-priori knowledge of the stochas-tic RES amount Eti is hardly available. Yet, they can bepotentially inferred and estimated from historical data. Suchestimations are generally bounded by uncertainty sets whichcharacterize the ranges of the forecasted RES amounts. Toaccount for the temporally correlated RES amounts, twouncertainty sets Ei(i = 1, 2, ...) are proposed from the prospectof computational malleability.

The first model is given by a polyhedral set [46]:

Epi :=

{ei | Eti ≤ Eti ≤ E

t

i,

Emini,s ≤

∑t∈Ti,s

Eti ≤ Emaxi,s , T =

S⋃s=1

Ti,s

}(1)

where Et

i (or Eti ) is the maximum (or minimum) value of Eti ;and the operating period T is divided into non-overlapping,adjacent, and much smaller regions Ti,s, s = 1, . . . , S. Each ofthe smaller regions can include several time slots. The overallamount of energy harvested at BS i over the s-th smallerregion is constrained by Emin

i,s and Emaxi,s .

The second model amounts to an uncertainty solution regionwith an ellipsoidal shape [47]

Eei :=

{ei = ei + ςi | ς ′iΣ−1ςi ≤ 1

}(2)

where ei := [E1i , . . . , E

Ti ]′ denotes the nominal EH amount

at the i-th BS, and provides the predictable energy level; inother words, the expected (or mean) energy level. ςi is a vectorcollecting the forecast errors. The given matrix Σ � 0 depictsthe outline of the ellipsoid Ee

i , and thus decides the accuracyof the prediction.

In an attempt to account for random realizations of RESproductions, an uncertainty set can be in any form. Based onthe first- and second-order statistics of the stochastic amounts,polyhedral or ellipsoidal sets are the most popular resorts [48].These two kinds of sets can be empirically obtained fromhistorical statistics, and often used to help solve the relevantoptimization problems. Therefore, they can be implemented toEH systems as well.

2) Stationary Process: In some cases, the stochastic RESamounts are considered as an independently and identicallydistributed (i.i.d.) process. Weibull and Beta distributions arefairly accurate in portraying the traits of wind speed oscil-lation and solar power alteration, respectively [49]–[51]. Therandom RES amount can also be approximated by a Gaussiangeneration process [52].

4

3) Markov Chain: Different from the i.i.d. process, theMarkov process implies that the probability distribution of then-th random variable is a function of that of the previousrandom variable in the process. For RES-powered wirelesssensor networks, the Markov chains are proposed to modelthe EH process under the assumption that the time is slotted,a 2N -state model can be designed to represent the EH state(“active” or “inactive”) with a probability and the residualenergy in the battery [53]. A state transition takes place whenenergy is harvested. Under the same assumption of slottedtime, a discrete-time Markov chain is leveraged to quantify theenergy charged into a battery [54]. Assuming that energy canbe replenished through EH and/or replacement of the battery,a continuous-time Markov chain is designed to model the stateof the battery in [55]. The state transitions are described asthe different rates of EH and battery replacement, each ofwhich follows an independent Poisson process. A stationaryMarkov process is used to model solar EH behavior, whilea non-stationary one is more effective in capturing vibrationsintroduced by other types of RES [56], [57].

III. ENERGY HARVESTING-BASED COMMUNICATIONOVER POINT-TO-POINT WIRELESS LINK

There are many recent research works on powering datatransmission over point-to-point radio link, where the transmit-ters are typically sensors deployed remotely with no access topower grid and harvesting RES from ambient environmentsall year around. The sensors can be used to monitor tem-perature [58], rainfall [59], bush fire [60], and wildlife [61].The key techniques and methodologies applied are convexprogramming, Lagrange multiplier method, and dynamic pro-gramming (DP), as summarized in Fig. 2. Convex problemscan be solved by off-the-shelf general-purpose convex solvers.However, general-purpose convex programming solvers re-quire high-order multiplications and many iterations, leadingto a high-order polynomial complexity and slow convergence[62]. Also, the general-purpose solvers cannot unveil the un-derlying structure of the optimal data transmission policy. Tothis end, the Lagrange multiplier method (typically in couplingwith the celebrated Karush-Kuhn-Tucker (KKT) optimalityconditions [62]) has been widely applied in the literature forsimpler and more insightful solutions [62]. By leveraging theLagrange multiplier method along with the KKT optimalityconditions to convex problems, one can find the global op-timum of the problems subject to the inequality constraints.DP generally simplifies a sophisticated optimization problemby decoupling it to be several subproblems and solving thesubproblems recursively [63]. However, DP is subjected tothe curse of dimensionality.

The arrival processes of data traffic modeled by existingworks can be categorized to two types: heavy data arrivalscenario [64] and moderate data arrival scenario [13]. Theformer type assumes that the data to be transmitted arrivebefore the start of the transmissions [64]; in other words, thereare always data available in the transmit buffer. The latter type,under a more general assumption, considers that the data arriveduring the course of transmission [13].

A. Heavy Data Arrival

By assuming a saturated traffic condition, the works in[64]–[68], [71] focus on the impact of EH on the optimaltransmit power. The works aim to design a transmissionpolicy specifying the transmit power over the interval [0, T ]to maximize the overall throughput. Given a total amount ofdata to send, the minimization of the completion time is alsoinvestigated [64], [65].

1) Idle Circuit Power: A visualization method is used in[65] to unveil the optimal transmission policy. Let Et denotethe amount of energy scavenged at time t. Let Et denote thecumulative curve of harvested energy, i.e., the total amountof energy harvested by time t (or Et =

∑tEt). Similarly,

let Pt denote the cumulative curve of transmitted energy, i.e.,Pt =

∑t Pt. To determine a transmission schedule policy is

to specify the non-decreasing and continuous function Pt.An inherent constraint is energy causality, i.e., no energy

can be utilized before it is harvested [19], [20]. This indicatesthat the curve of transmitted energy Pt should lie underthe curve of harvested energy Et all the time. With finitebattery capacity Emax, the optimal transmission policy shouldprevent the battery from being overcharged, i.e., the gapbetween the curves of transmitted energy and harvested energy(i.e. Et − Pt) should not exceed the battery capacity. Thisconstraint is referred to as no-energy-overflow [20]. Last butnot the least, to deliver the maximum overall throughput orthe minimum response time during the scheduling procedure,all the available energy is expected to be consumed by thedeadline. In other words, the curve of transmitted energy Ptneeds to meet the curve of harvested energy Et at time instantT . Collectively, these three constraints reveal that the optimalcurve of transmitted energy Pt starts from the origin, endsat point (T, Et), and lies between Et and Et − Emax; seeFig. 3. Here, the intersection of Pt and Et means that thetransmitter runs out of energy (or the battery is empty), andthe intersection of Pt and Et −Emax denotes that the batteryis full at the instant.

With the convexity of the transmit power P (r) in regardsto the transmit rate r, a use of Jensen’s inequality [62] canprove that employing a constant power can maximize thetotal volume of data delivered before its due time. In otherwords, the optimal strategy can be obtained by keeping aconsistent power subject to the energy feasibility constraints.It is subsequently unveiled in [65] by analyzing the originalproblem of interest and its constraints that the optimaltransmission schedule which maximizes the throughput obeysthe following two rules:

Rule 1: The transmit power only changes at the instantswhen the battery is completely drained or fully charged. [65].

Rule 2: The transmit power increases only at the instantswhen the battery is completely drained, and decreases onlyat the instants when the battery is fully charged [65].

Following Rules 1 and 2, the optimal transmission policyin [65] is shown to be the shortest path between the originand end points in Fig. 3. It can be obtained by tautening a

5

Convexoptimization [64]–[66],[68],[71],[76],[78] [13],[14],[69],[70],[73],[74],[76]

Convexoptimization [125]–[127],[129]

Lagrangemultiplier [64],[68],[71],[76],[78] [13],[14],[69],[73],[74],[76]

Lagrangemultiplier [127],[129]

Waterfillingmethod [125],[130]

[73],[74]Waterfillingmethod [64],[68],[71],[78]

[72],[76],[77]Dynamicprogramming [64],[68],[76]

Nestedoptimization [67]

Heuristicalgorithm [67] [13],[74]

Stringtauteningmethod [13],[14],[69],[70],[73]

[127]Visualizationmethod

[72]Q-learning

Lyapunov optimization [77]Onlinealgorithm

Linearprogramming [75]

Stochasticdominance [128]

Auctionmethod [130]

Heavydataarrival Moderatedataarrival

Singlepointtopoint

Multi-hoplink

[65],[66]

Convexoptimization [85]–[87],[91],[131],[132]

Lagrangemultiplier [85]–[87],[91],[131]

Waterfillingmethod [85],[86]

DynamicProgramming [131]

Game-theorybasedmethod [88],[89]

[91]

[91],[92]

Multipointtopoint

Fig. 2. A summary of optimization methods for resource allocation in EH-based point-to-point, multipoint-to-point, and multi-hop wireless links.

𝐸" 𝐸# 𝐸$ 𝐸% 𝐸&

𝑇

𝑇

Harvestedenergycurve

Batteryoverflowconstraintcurve

Shortestpath

Fig. 3. Optimal transmission policy is the shortest path in the energyfeasibility region [20].

string between the harvested energy curve Et and the batteryoverflow constraint curve Et−Emax. Such a way of producingthe optimal transmission policy is therefore called the “String

Tautening” method [13], [14]. The optimal transmission policyin [65] is developed over time-invariant channels.

For practical time-varying channels, Ozel et al. [64] proposea directional water-filling algorithm under energy causalityand battery overflow constraints. By formulating a convexproblem maximizing the total throughput, the Lagrange multi-plier method and the KKT optimality conditions are exploitedto achieve the optimal solution. It can be concluded fromFig. 2 that the majority of the water-filling methods in existingworks are derived through convex formulation and Lagrangemultipier method. By applying the Lagrange multiplier methodto convex problems, one can find the globally optimal solutionof the problems subject to the equality constraints, withoutexplicit parameterization in terms of the constraints. TheKKT conditions generalize the Lagrange multiplier method forsolving inequality constrained optimization problems. In [65],the KKT optimality conditions are used to deal with theinequality constraints accounting for the causality of EH andbattery overcharging.

Rules 1 and 2 still hold after replacing “transmit power”with “water-level” [65]. Here the water-levels are defined as

6

TABLE IA SUMMARY OF OPTIMIZATION PROBLEMS FOR EH POINT-TO-POINT LINKS

Settings

ObjectivesThroughput maximization EE maximization Transmission completion time minimization

Time-invariant channel [65]–[68] [13], [14], [67], [69] [65], [70]

Time-varying channel [64], [68], [71], [72] [73]–[77] [64]

Discrete energy arrivals [64], [65], [68], [71], [72] [13], [14], [69], [73]–[77] [64], [65], [70]

Continuous energy arrivals [66], [67] [67] –

Finite battery size [64]–[66], [68], [71], [72] [14], [69], [74], [76] [64], [65]

Strict Deadline [65], [72] [13], [14], [69], [73] [65]

Circuit power consumption [66], [67], [71] [13], [14], [67] –

Offline Algorithm [64]–[68], [71], [72] [13], [14], [67], [69], [73]–[76] [64], [65], [70]

Online Algorithm [64], [67], [68], [72] [13], [67], [74]–[77] [64]

𝑃#

2𝐻

1/𝜙#1/𝜙$

𝑃$

𝑇𝑇/2

𝑃.

𝐻

1/𝜙#1/𝜙$

𝑇𝑇/2

𝐻

𝑃.

(a) (b)

Fig. 4. Power allocation scheme over time-varying channel with φ1 > φ2.(a) 2H units of energy are harvested at instant 0; (b) H units of energy areharvested at instants 0 and T/2 [19].

wt := Pt + 1/φt, t = 1, 2, · · · , where φt are the channelpower gains of slotted time-varying channels. Consider firsta simple case consisting of two slots with duration T , whereφ1 > φ2. The transmit powers can be optimally arranged usinga well-known water-filling algorithm, as shown in Fig. 4(a).The blue regions denote the energy allocated to the transmittersduring each slot, and the height of a region, Pt, is the transmitpower for that slot. The celebrated water-filling algorithmalways assigns higher transmit powers to the channels withstronger channel gains. For a general EH process, applyingthe water-filling algorithm is not straightforward. Assume thatenergy of amounts H is harvested at time instants 0 and T/2,respectively. Different from the case in Fig. 4(a), no morethan H units of energy can be allocated to the first slot dueto the energy causality. The optimal allocation under energycausality is called directional water-filling [65]; see Fig. 4(b).The approach is generalized to the broadcast channels in [78].

2) Non-Idle Circuit Power: In [64] and [65], the circuitpower of e.g., converters, filters and mixers, is assumed tobe negligible. However, in short-range communications, thecircuit power is non-negligible and must be captured in theanalysis [66], [67], [71]. When the circuit power consumption

is negligible, the transmitter keeps active. Nevertheless, whenthe consumed circuit power is comparable to the transmitpower, the optimal transmission policy must include the“sleep” periods, during which the transmitter is off. Whena non-negligible circuit power is considered, the total powerconsumed at the transmitter, denoted by Ptotal, can be writtenas

Ptotal =

{P (r)ϑc

+ ρ, if P (r) > 0,

β, if P (r) = 0.(3)

Here, P (r) is the transmit power which is in general a convexfunction of transmit rate r; ρ and β denote the circuit powerconsumption when the transmitter is on and off, respectively;and ϑc is the transmit efficiency of the transmitter. In mostexisting works, it is generally assumed that ρ > 0 and β = 0,since typically ρ � β; and ϑc = 1, as ϑc is a scaling factor[66], [67], [71].

Considering the above non-ideal circuit power model, a so-called optimal “on-off” transmission policy is achieved in [66]by adjusting the ideal-case optimal transmission scheme in[65]. A minimum power level p∗ can be derived under thenon-negligible circuit power ρ. If the optimal transmit powerin [65] is lower than p∗, then the transmitter sends data withthe power p∗, and turns off once the harvested energy is usedup [66].

In [71], the optimal power allocation over time-varyingchannels is shown to be the solution to a convex optimiza-tion problem, and is portrayed as a directional glue-pouringalgorithm integrating the glue-pouring recently developed in[79] and the aforementioned directional water-filling algorithm[64]. The minimum amount of power is allocated to eachepoch, depending on the channel state and the non-negligiblepower consumption of the transmitter circuitry. The amountof harvested energy decides whether energy is “poured” intopart of an epoch at the minimum power level, or allocated tothe entire epoch with a higher power level.

In [67], the tradeoff between EE and spectrum efficiency(SE) is considered under non-ideal circuit power. An EE-maximizing power level Pee is obtained by maximizing theamount of data that can be transmitted with a unit of energy.

7

Specifically, Pee is defined as

Pee := arg maxP≥0

R(P )

P + ρ, (4)

where Pee can be efficiently solved through bisectional search,as R(P )

P+ρ is quasi-concave. Xu and Zhang [67] show an optimalsolution with a two-phase scheme. The first stage of theapproach is an “on-off” transmission with the EE-maximizingpower allocated to all on-periods, and the second stage is tocontinuously transmit with a non-decreasing SE-maximizingpower. The optimal offline algorithm is later extended to thescenario of multiple additive white Gaussian noise (AWGN)channels, where the above original power allocation problemwith multi-dimensional vectors is solved by equivalently solv-ing a problem with only a single one-dimensional scalar opti-mization variable through nested optimization techniques [67].

By taking the battery storage loss into consideration, a spe-cial pattern of the optimal power is uncovered by combiningthe Lagrange multiplier method and DP [68]. In specific, asthe history of the battery status has a non-negligible influenceon the current status, a DP based technique (or method) isproposed to locate the slot for zero battery level in a backwardinduction manner with an affordable complexity.

B. Moderate Data Arrival

1) Non-Stationary data and energy arrival processes:Section II-A assumes there are always data available in thebuffer for transmission; in a more general scenario, data andenergy arrivals can be bursty over time. The design goal ofthe optimal schedule can be to minimize the total transmissiontime [70] or the total consumed energy [13], [14], [69], [73].Let curves At and Dt be the total amount of data arrived anddelivered by time t, respectively. Based on data causality, curveDt is always below At. The optimal transmission schemeobtained for a heavy data arrival scenario is no longer feasibleas there might not be enough data in the buffer. Moreover,packets can have different deadlines [13], [14], [69], [73].Let curve Dmin

t be the total amount of data that must betransmitted by time t. Then Dt should always lie aboveDmint to guarantee the deadline requirement (or QoS). The

optimal transmission policy must jointly take into account theconstraints in the data domain and those in the energy domain.

To better illustrate the impact of intermittent dataarrivals on EH-based data scheduling, we consider first thescenario where energy is available at the beginning of thetransmission [80]–[82]. Then, a transmission strategy canbe found by equivalently specifying a non-decreasing andcontinuous function Dt over time. In [80], a calculus methodis developed to determine the optimal Dt to minimize thetransmission energy consumption for delay-sensitive packetsover time-invariant channels. It is shown that the value ofDt can be readily optimized by simply tautening a stringbetween the data arrival curve At and deadline curve Dmin

t .It is concluded in [80] that the optimal transmission strategyin a moderate data arrival scenario (with unlimited energy)obeys the following two rules:

𝑇

Waterarrivalcurve

Deadlinecurve

Waterdeparturecurve

Wateramount

Waterlevel

𝑇

Waterlevel

Power

Noiselevel

Fig. 5. (Top) Water arrival curve, deadline curve and optimal water departurestrategy; (Bottom) Water-filling algorithm admitting multiple water levels [81].

Rule 3: The transmit rate shall only vary when the databuffer is empty (i.e., there are no undelivered data), or whenall the deadline-approaching data is transmitted [80].

Rule 4: The transmit rate increases only at the instantswhen the data buffer is empty, and decreases only atthe instants when all the deadline-approaching data istransmitted [80].

Rules 3 and 4 imply that the visualization method usedto construct the optimal transmission policy in [80] hassimilar procedures as in [65], [66], which follow Rules 1and 2. The generalization of Rules 1 and 2 to [80] can beachieved by simply substituting the energy related curvesEt, Pt and Et − Emax in [65], [66] with the correspondingdata related curves At, Dt and Dmin

t . By incorporating thecalculus method [80] into the classic water-filling technique,reference [81] generalizes the optimal rate schedule to time-varying wireless channels. By formulating a convex problemand applying the KKT optimality conditions, an interestingmulti-level water-filling pattern is revealed in the optimalschedule with the minimum power consumption in [81], whichcan be visualized to be the shortest path between the “water”arrival and the corresponding minimum “water” departurecurves, corresponding to the data arrival and deadline curves,respectively, as shown in Fig. 5.

Taking non-ideal circuit power into account, the studyin [82] generalizes the approach developed in [81], and pro-poses an energy-efficient “clipped string-tautening” algorithm.It is revealed in [82] that the transmitter can take the followingthree schemes under the optimal data schedule: i) remainingoff over the entire slot; ii) transmitting at the energy-efficiency(EE)-maximizing rate for part of the slot; or iii) transmittingat a rate larger than the EE-maximizing rate over the entireslot.

We proceed to consider that both energy and data arrivalsare bursty over time. A so-called “dynamic string tautening”

8

technique is put forth to create most energy-savior offlinetransmit policies even in the presence of a non-ideal transmittercircuit with a considerable power consumption and a limitationof a finite battery capacity [69], or without the limitation [13].The transmit rate rt and the “on” duration lt per epoch t arethe variables to be optimized. Neither of rtlt and P (rt)lt isconvex or concave over (rt, lt) in the problem of interest in[13]. Yet, the problem can be transformed into a convex one byapplying variable substitution with Φt := rtlt. For any convexP (rt), P (Φt

lt)lt is known to be its perspective, and is convex

in (Φt, lt).By leveraging convex formulation and its optimality condi-

tions, Rules 1–4 can be met simultaneously, and guide thegeneration of the optimal strategy by recursively tauteninga data departure string in a feasible solution region. Thecomplexity of tautening a string in such a way is low. Itis revealed in [13] that the current string tautening behaviordepends on the past one, as the past data schedule affectsthe available energy left in the battery. Instead of tauteningtwo strings (which are the transmitted energy curve followingRules 1 and 2, and the corresponding transmitted data curvefollowing Rules 3 and 4), only the transmitted data curve needsto be optimized since the energy related constraints curves canbe translated to the data domain [13].

The algorithm developed in [13] is generalized to the onlinescenario, where the transmission policy is generated in realtime. It is also extended to time-varying channels in [73].In [14], appropriate models of EH transmitters, with meticu-lous considerations on the EH rate, deadline requirements, andbattery size, are investigated to balance the QoS guarantee andthe energy consumption based on “dynamic string tautening”algorithms. Targeting at minimizing the total completion timeof transmitting all arrived data, the optimal data schedule isobtained by a recursive visualized scheme which jointly checksthe conditions of the data and energy departure curves [70] inaccordance with Rules 1-4.

Considering joint EH and constant grid power supply, differ-ent resource allocation problems are formulated to minimizethe total power consumption [74], [75], or the grid powerconsumption [76], [77]. Through convex formulation and KKToptimality conditions, a two-stage water-filling policy is pro-posed in a heavily loaded traffic condition with the harvestedenergy distributed in the first stage and the grid energy supplydistributed in the second stage [74]. For a moderate trafficarrival case, a multi-stage water-filling strategy is developedto reversely allocate the energy from the last frame to the firstframe. This policy can only be generated offline. Ahmed etal. [76] develop a stochastic DP algorithm for online powerallocation. To bypass the high complexity of DP, a suboptimalonline scheme is developed through convex formulation for agood performance-complexity tradeoff [76].

2) Stationary data and energy arrival processes: The cel-ebrated Lyapunov optimization framework has been appliedto obtain online dynamic resource allocation with i.i.d. dataand energy arrivals, under data queue stability constraint [77].The Lyapunov optimization technique provides a very effectivetool to stabilize and optimize networks of queues. Let L(t)denote the Lyapunov function, which is a non-negative metric

to measure queue lengths. The growth of L(t) indicates thatthe queueing system becomes increasingly unstable. Considera system of I queues with lengths of (Qt1, Q

t2, . . . , Q

tI). The

arrival process of each queue is assumed to be stationary(stochastic). The Lyapunov function can be formulated asL(t) = 1

2

∑Ii=1(Qti)

2. Then, 4L(t) = L(t+1)−L(t) definesa Lyapunov drift, which is the difference of the Lyapunovfunctions at two adjacent slots. A feasible way to preserve thesystem stability is to minimize the Lyapunov drift at every slotand prevent the queue lengths from growing [83].

The term 4L(t) + V p(t) is defined as the Lyapunov drift-plus-penalty, where p(t) stands for a penalty function, and Vspecifies a positive weight. By minimizing the upper boundof 4L(t) + V p(t) per slot, one can stochastically minimizethe time-average penalty with asymptotic optimality whilepreserving the system stability. In [77], the data queue in thetransmit buffer and the energy queue in the battery are mappedto Qt

i = {Qtd,i, Qte,i}, while the total power consumption spec-ifies the penalty function p(t). As a consequence, minimizingthe maximum value of this Lyapunov drift-plus-penalty perslot leads to the asymptotic minimality of the time-average en-ergy cost, while stabilizing all the data and energy buffers. Byexploiting a continuous-time approximation, DP, and sample-path approach, an analytic framework is introduced to studythe power-delay tradeoff relationship in the small delay regime[77].

In [72], the EH communication system is assumed to bea finite Markov decision process (FMDP) [84], where theenergy and data arrivals are Markov processes. A reinforce-ment learning-based approach, Q-learning, is developed for thetransmitter operation of the EH communication system. Forany given FMDP, Q-learning can select the optimal actionsand maximize the expected total reward for each and allconsecutive steps. In [72], the transmitter is designed tolearn the optimal transmission strategy gradually by takingexploratory actions (i.e. dropping or transmitting the incomingpackets) and maximizing the expected sum rewards (i.e. thetotal throughput). It is shown that the proposed approachasymptotically converges to the global optimum, as the learn-ing time increases.

IV. ENERGY HARVESTING-BASED MULTIPOINT-TO-POINTWIRELESS SYSTEM

Another wireless communication architecture extensivelystudied in the context of RES/EH is multipoint-to-point net-work architecture, where there are multiple EH devices (orsensors) and a single sink node; see Fig. 6. The sensors need tobe scheduled to send their data to the sink. Energy/EH-awarescheduling is a key differentiating factor of the multipoint-to-point networks to the point-to-point networks (discussed inSection III) [85]. In the case of centralized scheduling, the sinkcollects information, such as queue backlog and energy avail-ability, and selects accordingly the devices to transmit [85],[86]. In the case of distributed scheduling, random accessprotocols are developed to trigger the transmission of a devicebased on its own queue and battery states [87]–[92].

9

Sink node

Sensor 1

Sensor 2

Sensor 3

… …

Energy

Data

… …

Energy

Data

… …

Energy

Data

Fig. 6. Diagram of an EH-based multi-point-to-point wireless communicationsystem.

A. Centralized Scheduling

Yang and Ulukus [85] extend their previous works onpower allocation for EH point-to-point links [64], [70] orEH broadcast channels [78] to a multiple access communi-cation system. By capturing the interference between the twousers and following the aforementioned Rules 1 and 2, theauthors of [85] generalize the well-known backward water-filling algorithm to specify the maximum data departure regionwith a given deadline. An offline transmission strategy isthen obtained by decomposing the transmission completiontime minimization problem into several convex subproblems,according to the region sequence at energy arrival instants.

By taking the maximum per-slot energy consumption intoaccount, an iterative dynamic water-filling approach is de-signed in [86] to provide the optimal energy schedule forEH multi-access channels and maximize the throughput of thechannels. It is shown that, in practice, the convergence can bereached within only a few iterations.

The optimal resource allocation is specifically designed inmulti-input multi-output (MIMO) systems powered by smartgrid in [93], [94] by maximizing the weighted or expectedsum-rate. Relying on an uplink-downlink duality derivedinformation-theoretically [95], the downlink MIMO broadcastchannel capacity region in the downlink can be equivalentlycalculated as the union of the uplink multi-access channelscapacity regions when the uplink and downlink have to meetthe same sum-power constraint. To derive the optimal powerallocation strategy in an offline fashion, the Lagrangian dualbased subgradient method is leveraged in [93], [94] by apply-ing a nested optimization process. The downlink covariancematrices can be derived from their uplink counterparts usingthe uplink-downlink duality.

B. Distributed Scheduling

Baknina and Ulukus [87] extend the results of [85] to theonline scenario where the arrivals or availability of ambientenergy are typically unpredictable. A distributed fractionalpower (DFP) policy is proposed and proven to be near-optimal. By assuming that the energy collected by the users issynchronized Bernoulli process, it is shown that the optimally

allocated power decreases until reaching the end of the renewaltime, and has a pentagon time-average throughput region. It isalso shown in [87] that the correlation of the energy sourcesis detrimental to the achievable throughput, as the throughputis much larger under asynchronous Bernoulli energy arrivalsthan it is under synchronized ones.

Michelusi and Zorzi [88] investigate a multi-access policymaximizing the utility of the EH wireless sensor networks(WSNs), where different EH sensors transmit packets to afusion center by randomly accessing a shared wireless channel.Each packet has a random utility value. A distributed randomaccess protocol is designed based on a game theoretic frame-work, where all sensors perform the same strategy. The workin [88] is extended to a multi-channel case with delay-sensitivedata transmissions [89]. Considering a similar scenario asthe one in [88], Iannello et al. [90] design the mediumaccess control (MAC) by balancing the tradeoff between timeefficiency and transmission probability.

A recent work [91] considers a EH-based two-user cooper-ative multiple access channel, where the two users performdata cooperation by cooperatively establishing and sendingcommon messages, and perform energy cooperation by wire-less energy transfer. By leveraging the Lagrange dual basedmethod, the offline energy allocation and transfer policymaximizing the departure region are jointly optimized. Theexact stability region is characterized in [92] where twoEH nodes randomly access the same receiver. Relying onLoynes’ theorem [96], the analysis of the stability regiontakes into account the effect of limited energy availability,finite battery capacity, and data reception capability. Loynes’theorem indicates that, if the inbound rate is smaller on averagethan the outbound counterpart, and the incoming and outgoingprocesses are ergodic, then the queueing system is stable [96].

To solve the problem in [93], [94] over an infinite schedulingperiod by an online algorithm, Wang et al. [97] rely on thestochastic subgradient method to obtain resource schedules“on-the-fly” by suppressing (decoupling) the time-coupling be-tween the variables and constraints. The random variables aresupposed to be i.i.d.. Using stochastically estimated Lagrangemultipliers, this method updates the subgradients with theironline approximations based on the instantaneous decisionvariables per time slot. It is proven in [97] that asymptoticallyoptimal and feasible solutions can be achieved in no need ofany prior knowledge of underlying randomnesses. The opti-mality gap diminishes when the iteration stepsize approacheszero.

When achieving the optimality, the BS purchases moreelectricity to transmit its data and charge its battery from thesmart energy grid if the grid offers a low energy price, and usesthe energy stored in battery if the grid asks for a high price.The BS can even sell some of its surplus energy back into thegrid, hence offsetting part of its energy bill. Compared with atraditional grid-powered BS, the proposed system in [97] canachieve higher sum rates under the same cost budget.

10

Smart power grid

Two-way energy trading

Mobile users

BS2

BS1

Wind turbines

Smart meter

BatteriesRF chain

RF chain

Solar panels

Smart meter

Batteries

RF chain

RF chain

Central controller

Grid deployed

communication/

control links

Wireless

signals

Fig. 7. A typical smart grid-powered CoMP system with two BSs.

V. ENERGY HARVESTING-BASEDMULTIPOINT-TO-MULTIPOINT WIRELESS

COMMUNICATION SYSTEM

Coordinated multi-point transmission and reception (CoMP)was initiated in the Third Generation Partnership Project(3GPP), as one of the key technologies in the long-termevolution-advanced (LTE-A) standard, in order to achieveinterference management and mitigation. A CoMP system canbe viewed as a group of geographically collocated transmitantennas which coordinately serve several multi-antenna endusers [98]. Thus, a CoMP system can be referred to asa multipoint-to-multipoint system. For coordinated transmis-sion in the downlink, the data sent by several transmissionpoints are coordinated to enhance the acquired intensity ofthe expected information at the user equipment (UE) or todecrease the inter-cell interference. For coordinated receptionin the uplink, it is ensured that the uplink data from the UEcan be steadily detected by the network with limited uplinkinterferences and the existence of multiple reception points.

As there are a large number of users and service demands,it costs great energy to realize transmit beamforming, inter-ference alignment, user scheduling, and backhaul signalingin multipoint-to-multipoint systems. The integration of REScan reduce the electricity consumption from traditional powersources and facilitate economic and ecological operations ofsuch systems in the age of “green communications”.

Many recent works have minimized the energy consump-tion and operational cost of RES-powered multipoint-to-multipoint systems. Classic techniques and methodologiesapplied in such problems include the Lagrangian dual-basedmethod, (stochastic) subgradient method (SGM) [99]–[102],

cutting-plane method (CPM) [103]–[105], and proximal bun-dle method (PBM) [106]–[108]. For a minimization problem,its Lagrangian dual function is always concave, even if theprimal problem is not convex [62]. The optimal dual solu-tions can be readily obtained by using off-the-shelf convexoptimization tools, such as SDP, SOCP, and interior pointmethod [62]. To effectively solve non-differentiable convexdual problems, SGM and CPM are commonly used, as theydeal with subgradients, rather than gradients, of the objectivefunctions. Although CPM converges faster (i.e. with feweriterations) than SGM, it is computationally more demanding(per iteration), and does not allow for a distributed imple-mentation [109]. The stochastic SGM is applied when therandom variables are i.i.d. and traditional online solvers, suchas DP, are intractable since the optimization variables areclosely coupled over time. The optimal primal solutions can berecovered from the dual solutions with no duality gap when theprimal problem is strictly convex. Extra care must be taken forsuch operations if the original problem is not strictly convex.A sophisticated method to deal with this situation is the PBM,which is to approach the epigraph of a function through usingthe intersection of several halfspaces.

A. Minimization of Total Energy Consumption

A CoMP downlink energized by a smart power grid isstudied in [110] where the BSs have on-spot RES and carryout bidirectional energy trading in real-time with the smartgrid. Yet, the BSs may not be equipped with energy storagecapability. The optimal solution for minimizing the overallenergy expenditure is obtained by an approach utilizing convexoptimization and uplink-downlink duality [111]. In particular,

11

let k ∈ K = [1, . . . ,K] denote the user index, and wk thebeamforming vector associated with user k. A phase rotationis performed for each wk, since the phase rotation doesnot change the signal-to-interference-plus-noise ratio (SINR)constraints, i.e., SINRk({wk}) = SINRk({wke

jφk}), whereφk is an arbitrary phase. Without loss of optimality, {wk} ischosen in such a way that hHk wk is real, and hHk wk ≥ 0 in[110], to convert the original nonconvex SINR constraints:

|hHk wk|2∑l 6=k(|hHk wl|2) + σ2

k

≥ γk, ∀k ∈ K (5)

into convex (second-order conic) constraints [110, eq. (14)]:√∑l∈K

|hHk wl|2 + σ2k ≤

√1 +

1

γkhHk wk, ∀k ∈ K, (6)

where hk stands for the channel vector of any user k.Xu and Zhang [110] exploit the uplink-downlink duality

to convert the multiple-input single-output broadcast channel(MISO-BC) to a dual single-input multiple-output multipleaccess channel (SIMO-MAC) by taking conjugate transposeof the channel vectors, under the same SINR constraints{γk}. Hence, the optimal transmit beamformers {w∗k} can beobtained by first deriving the uplink dual problem solutionvia an iterative function evaluation procedure [112], and thenmapping the solution to the original problem. Other variablesof the problem are decided via an ellipsoid method [113].

As the optimal solver incurs a high computational complex-ity, Xu and Zhang [110] also propose a suboptimal solutionof a comparatively lower complexity, by implementing zero-forcing (ZF) beamforming methods at the BSs. The transmitbeamforming vectors are produced to cancel any mutualinterference between different users, i.e., hHk wl = 0, wherel, k ∈ K and l 6= k are the indexes to two different users.As typically required, ZF beamforming is only implementableunder the condition that there are fewer users than the transmitantennas across the BSs.

B. Minimization of Operational Cost

Due to an unbalanced supply of RES and energy require-ments across geographically dispersed BSs, and a considerableprice gap of the BSs purchasing (or selling) electricity from(or towards) the smart energy grid, it is cost-effective forthese BSs to collectively plan their transactional dealingswith the smart energy grid, as well as electricity usage forCoMP-enabled communications. A framework is developedin [114] to capture finite storage, bidirectional electricitytrading and dynamic pricing of the smart grid into CoMPdownlink communication systems with imperfect channel stateinformation (CSI) at the transmitter. The system model ispictorially illustrated in Fig. 7, where each BS has RES devices(e.g. solar-electric converters and/or hydroelectric generators),local energy storage devices (e.g. batteries), and a smart meterto collect energy information and coordinate bidirectionalelectricity trading activities by interacting towards the smartpower grid. The BSs in the CoMP cluster jointly serve themobile users. In such a system, a central controller is necessary

to collect the information across the entire network, andcorrespondingly coordinate the activities of the BSs.

The authors of [114] develop the worst-case energy schedul-ing and transmit beamforming schemes to minimize thesystem-wide transaction expenditure, while guaranteeing userQoS in a CoMP-enabled downlink network. Let P tb,i standfor the power transferred into, or taken out of, the batteriesat slot t for BS i. In the case of P tb,i > 0, the BS chargesthe battery. If P tb,i < 0, the battery is being discharged. P tg,idenotes the total power usage for BS i at time slot t (includingthe transmit power with respect to beamforming vectors andconstant circuits consumption). With the auxiliary variableP ti = P tg,i+P

tb,i and the energy transaction prices αt (buying)

and βt (selling), the worst-case transaction expenditure of BS iacross the entire scheduling period can be formulated as [115]

G({P ti }) := maxEi∈Ei

T∑t=1

(αt[P ti −Eti ]+−βt[Eti −P ti ]+

), (7)

where Eti is the harvested energy of BS i at time t, [P ti −Eti ]+is the shortfall of the energy to be procured from the smartpower grid, [Eti − P ti ]+ is the redundant energy to be sold tothe grid, and [a]+ := max{a, 0}.

Applying the semidefinite relaxation (SDR) technique [116]and the S-procedure [117], the energy management and trans-mit beamforming optimization is constructed as a convexproblem. Due to the non-differentiability of G({P ti }), thisnonsmooth convex problem is intractable to general solvers. Itsglobal optimal solution can be acquired offline by employingthe Lagrangian dual based subgradient iteration [102], [109],[118]–[120], together with a proximal bundle method [106],[107]. Ahead-of-time resource planning can be realized, pro-vided that the energy and data arrivals are obtained beforehand.

C. Minimization of Long-term Average Cost

The dynamic energy management of a CoMP system is con-sidered in an infinite scheduling horizon in [121]. The targetis to minimize the time-averaged overall expenditure over aninfinite time horizon by determining the power allocation vari-ables, i.e. min{P t

i ,Ptg,i,C

ti} limT→∞

1T

∑T−1t=0

∑Ii=1G(P ti ).

The battery level relations (as one of the constraints) Ct+1i =

Cti + P ti − P tg,i, ∀i, t couple the optimization variables overtime. This makes the original problem hardly malleable fortraditional solvers such as DP.

Assuming that the RES amounts and energy transactionprices {et, αt, βt} are i.i.d. at every individual slot, the authorsof [121] use the Lagrange dual based stochastic subgradientapproach to tackle this problem. In essence, this methodupdates the Lagrange variables by their stochastic estimatesat each time slot, which can reduce the otherwise high com-putational complexity. Based on the stochastic iterations, theauthors then construct a virtual energy queue Qti for each BSi. The virtual queue obeys the same dynamic equation as inbattery level: Qt+1

i = Qti+Pti (Qt)−P tg,i(Qt), where P ti (Qt)

and P tg,i(Qt) are procured by tackling the dual problem

with stochastic estimates of the Lagrange variables. Differentfrom real queues, the value of Qti can be negative, which is

12

important to the procedures of the virtual-queue based onlinecontrol (VQOC) approach at the central scheduler to obtainthe asymptotically optimal solutions. Depending on the state-of-the-art stochastic optimization methodologies [122], [123],the solution can be proven to be asymptotically optimal overa long term.

A two-scale online resource management problem is in-vestigated for RES-integrated CoMP-enabled communica-tions in [124]. By considering the dynamics of CSI, RES,beforehand/real-time electricity prices, and battery shortcom-ings, a stochastic optimization task is built up to minimizethe average electricity transactional expenditure over a sub-stantially long time frame, and also satisfy the QoS of theusers. The authors of [124] reformulate the primal probleminto a malleable structure by replacing the time-coupled queuedynamics with a time-averaged constraint, i.e. Ct+1

i = ϑbCti +

P tb,i, Cmin ≤ Cti ≤ Cmax, ∀i are replaced by(1−ϑb)Cmin ≤Pb,i ≤ (1 − ϑb)Cmax, ∀i, where ϑb ∈ (0, 1] is the storageefficiency. It is supposed that the battery capacity Cmax is finiteand the minimum energy level of the battery is Cmin. By doingso, the variables {Cti} are suppressed in the original problem,and other optimization variables are “decoupled” over time.The reformulated problem is a relaxed version of the originalproblem.

A two-scale online optimization method is then proposedto create ruling policies in real-time in [124] by minimizingthe Lyapunov drift-plus-penalty 4Lt + V pt, where 4Lt =Lt+1 − Lt is the Lyapunov drift, pt is a penalty function(in the paper pt is the time-average energy consumption (orutilization) of the BSs), and V is a preset positive weightingcoefficient of the penalty. It is analytically validated in [124]that asymptotically close-to-optimal resource allocation canbe established for the above-mentioned original problem withproper settings, without any prior statistical knowledge of theunderlying stochastic processes.

VI. ENERGY HARVESTING-BASED MULTI-HOP WIRELESSCOMMUNICATION LINK

A. Information Cooperation Between NodesCooperation between nodes is an effective way to enlarge

system throughput and improve diversity for wireless com-munication systems. Luo et al. [125] consider a dual-hopnetwork with an EH-powered source node and a half-duplexrelay node powered by persistent power supply, and investigatejoint time and power management. An important insight isthat directional water-filling (DWF) developed for single-hopcommunication systems [64] may be generalized to this two-hop scenario [125]. Specifically, based on DWF, a performanceupper bound is first found for any given energy arrival processat the source. The properties of the optimal solutions are thenderived, which indicate that: i) The source (relay) node shouldemploy the same transmit power during a given DWF interval;and ii) The data buffer at the relay and the energy buffer at thesource are emptied at the end of DWF intervals. The resultantrelaxed EH profile is later modified to optimize the time andpower allocation, as shown in Fig. 8.

The “on-off” strategy during each interval is strongly anal-ogy to the problem of sum-rate maximization in the presence

𝐸(𝑡)

𝑇

TheDWFInterval

AtemporarysolutionfortheoriginalEHprofile

TheDWFEHprofile

𝐸(𝑡)

𝑇TheoptimalsolutionfortheoriginalEHprofile

Fig. 8. The solution to the relaxed EH profile in [125] is slightly modifiedto yield the solution for the original, non-relaxed power allocation problemof [125]. The slope of the solution is the optimal transmit power at everyinstant.

of battery leakage [126] or with non-ideal circuit powerconsumption [66], [67], [71]. However, the “on-off” structurein [125] results from a half-duplex constraint. The structure isattributable to the objective of the optimization problem andthe constraining factor of battery leakage in [126] and non-negligible circuit energy consumption in [66], [67], [71].

Huang et al. [127] consider EH source and relay nodesperforming decode-and-forward (DF) relay, with the a-prioriknowledge of the EH profile. In the case of transmitting delay-constrained (DC) traffic, a new power allocation strategy isdeveloped to maximize the throughput by using the KKToptimality conditions. It is shown in [127] that the searchalgorithm over the two-dimensional EH profiles of the sourceand relay is an extension of the earlier algorithm developedover the one-dimensional EH profiles in [70]. In the caseof transmitting non-delay-constrained (NDC) traffic, basedon a separation principle [127], the original problem can besolved by two stages: the transmit power of the source is firstjointly optimized by ignoring the requirement of the relay,according to which the transmit power of the relay is thenoptimized. It is noted in [127] that, in practical EH scenarios,“energy diversity” exists due to the independent EH processesat the source and the relay. By relaxing the decoding delay,the proposed transmission policy for NDC traffic can exploit“energy diversity” in cooperative communication, and resultin a much larger system capacity than the DC counterpart.

Pappas et al. [128] study the non-negligible effect of EH ona two-hop network with a collision-prone wireless channel,e.g. IEEE 802.11 WiFi, where an EH-powered source nodeand an intermediate relay node both receive external traffic.A cooperation of the source and the relay is performed at theprotocol (network) level, where the relay takes responsibilityof data transmission and can decode the transmissions of thesource. By deriving the sufficient and necessary conditions forthe stability of the traffic queues, tight inner and outer boundsof the stability region are obtained with a given transmissionprobability.

B. Information and Energy Cooperations Between NodesUnlike [125], [127] and [128] where energy cooperation

between nodes is largely overlooked, Gurak et al. [129]

13

Source Destination Relay

Source

Source Destination

Destination

Energy Information

Fig. 9. Cooperative communication using EH relay.

develop an iterative algorithm which jointly optimizes powercontrol, data routing and energy transfer for a EH com-munication network. In [129], all nodes can harvest energyfrom ambient environments, and transfer part of the harvestedenergy to neighboring nodes by energy cooperation. Based onthe Lagrangian function of the convex energy managementproblem, the necessary conditions are established for theoptimal solution. The proposed algorithm is shown to convergetowards a Pareto-optimal equilibrium. When there is no energycooperation, it is shown in [129] that, with fixed data flows, ahigher transmit power needs to be assigned towards the linkseither suffering from stronger noises or admitting higher dataflows. It is also revealed in [129] that, with multiple attempts ofEH per node, the optimal power consumption of the outgoinglinks per slot is equal to that of point-to-point transmission.

In [130], the power allocation strategies and the resultantoutage probabilities are studied in a cooperative communi-cation system, where multiple source-destination pairs areconnected by one single EH relay, as shown in Fig. 9. Byexploiting simultaneous wireless information and power trans-fer (SWIPT) technologies, the source nodes transmit signalsand energize relaying retransmissions. Assuming that CSI isavailable at the transmitter, a centralized strategy is proposedbased on the concept of sequential water-filling. Specifically,the strategy always serves users with better channel conditionsfirst. Such an opportunistic scheme can minimize the outageprobability of the system. A distributed auction based strategyis developed to balance between the overall system perfor-mance and the computational complexity, where the relayupdates the power allocation scheme at each iteration afterthe destination nodes submit their bids to the relay.

VII. WIRELESS POWER AND INFORMATION TRANSFER

So far, energy is harvested from ambient environments (e.g.solar power and wind power) and treated often as constraintsin the optimization of wireless networks. The approach arisingfrom wireless power transfer (WPT) is able to provide aconvenient and flexible way for EH that can be performed

anywhere, at anytime, under any weather condition, and forany desirable amount. Practically, WPT can be realized bymultiple different technologies, such as induction, magneticresonating, and electromagnetic (EM) radiation.

As the low-power Internet-of-Things (IoT) devices such assensors and tags proliferate in the 5G/B5G wireless networks,how to power end users with green energy has become acritical issue for system designs. WPT has arisen as a newand promising technique to offer on-spot and on-demandenergy replenishment to wireless networks. Since radio signalscan transfer power and information simultaneously, study onSWIPT has been pursued for wireless communication systems.

A. SWIPT-Enabled Single-Output SystemsEarlier designs on SWIPT systems focus on single-input

single-output (SISO) settings. In particular, a SISO wirelesslink is considered in [131] where the receiver is short of per-sistent power sources and has to restock energy through WPTfrom the signals sent by the transmitter. The single-antennareceiver cannot decipher information and collect energy inde-pendently from the same communication signal. A dynamicpower splitting (PS) scheme is developed to separate thecommunication signal into two fluids with adaptable powersfor information deciphering and EH, respectively, accordingto the CSI known at the receiver. The PS factors (PSFs) ofthe receiver and transmit power are collectively optimized in[131] to achieve the largest ergodic capacity, satisfying thedemand for EH amount. Given the special structure of the non-convex optimization problem, the Lagrange duality approachis employed to procure the globally optimal solution.

A Pareto-optimal algorithm is proposed in [132] to opti-mally pick a terminal device and distribute its power bud-get across the orthogonal frequency division multiplexing(OFDM) subcarriers under an SISO setting. The maximum EEis attained in both of the forward and reverse link directionsin [132], with known PSFs and uplink/downlink operatingtime. The sum rate in the uplink is maximized in [133] byoptimizing the durations of both the uplink and the downlink,where the terminals are attended one after the other in bothof the uplink and downlink. Combined power allocation andtime switching (TS) policy is developed in [134] for an SISONOMA system. The EE of the network is maximized whilemeeting the demands on transmit power budget, data rate,and EH amount per user. A two-layer approach with theDinkelbach method [135] is put forth to solve the problem.

Some existing works are particularly interested in the down-link of MISO SWIPT networks, and give no considerationto the uplink. Given an access point (AP) installing multipletransmit antennas and the PSFs of multiple users each operat-ing a single receive antenna, transmit beamforming techniquesare investigated in [136] and [137] in attempts to minimizethe downlink transmit power of the multi-antenna AP, whilesatisfying some given requirements of the transmit rate andEH amount. SDR is applied to convert the original task intoa convex program, and suboptimal solutions are obtainedby leveraging ZF [136] and SINR maximization [137]. In-terference alignment is adopted in [138] to subdue interfer-ence among information transmission and achieve the totally

14

minimized transmit power in the downlink. The semidefiniteprogramming (SDP) technique is applied in [139]–[141] toextend [137] and [138] to enhance robust beamforming underimperfect CSI. The EE of an SWIPT system is maximizedin [142] by considering a persistent circuit power at the AP,as well as the terminal stations. Considerable circuit poweris accounted only at the transmit node in [143], withoutaddressing the receiver side.

Harvest-then-transmit policies have been applied to mul-tiuser MISO systems, where only the power is deliveredthrough the downlink stream (as contrary to the transfer ofboth power and information in SWIPT). In this scenario, powersplitting at users would not be called for. Combined timesplitting and beamforming design are considered in [144] tomaximize the sum throughput. In the scenario of perfect CSIat the transmit node, a semi-closed-form solution is developedby utilizing the strict concavity of the problem. Taking intoaccount Gaussian CSI errors, a robust approach is developedto maximize the sum throughput given a channel outageprobability requirement. The downlink energy beamformersand the information transmit power and beamformers in theuplink are collectively adjusted to achieve the maximumsystem throughput in [145], where all users can send dataat the same time. A spectral radius minimization problemis constructed and tackled in [145] by leveraging the non-negative matrix theory. Generalized eigenvectors are applied tofind the optimal beamformers and link operating time in [146].The combined uplink and downlink sum rate is maximized in[147], without considering QoS in the downlink.

Non-linear EH model is considered in [148] and [149]for a multiuser MISO system with SWIPT. In [148], issueson power-efficient, user-fairness, and channel non-reciprocityare addressed by a multi-objective optimization problem. Tosecure the primary system, an artificial-noise-aided collabora-tive jamming policy is developed in [149]. Transmit power isminimized under secrecy rate and EH constraints. Algorithmsbased on SDR or a cost function are developed for theproblem.

B. SWIPT-Enabled MIMO System

Multi-antenna beamforming can potentially enhance theimplementation and operation of SWIPT. An MIMO wirelessbroadcast network with three nodes is studied in [150], where,apart from a source node, one of the nodes gathers RF energyand another node receives information. All of the nodes canhave multiple antennas. Two interesting cases are investigatedin [150]. In the first case, the above-mentioned two nodesare far apart. They have distinctive MIMO channels from thesource. In the other case, the two nodes separately receivingenergy and information are co-located and therefore they haveidentical MIMO channels from the source. In the first case,the optimal transmit policy is derived to optimally trade offinformation data rate for energy delivery, as can be portrayedby a so-termed rate-energy (R-E) region. Fig. 10 [150, Fig. 4]plots this R-E region, where M,NEH and NID are the numberof antennas at the source node, the node harvesting RF energy,and the node receiving information data, respectively, Q is the

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

(REH

,Qmax

)

(Rmax

,QID

)

Rate (Mbps)

Ene

rgy

(mW

)

Optimal transmit covarianceTime sharing

Fig. 10. The R-E region of a MIMO BC with two separate receiving devicesfor energy and information, where M = NEH = NID = 4 [150, Fig. 4].

0 1 2 3 4 5 6 7 8 90

50

100

150

200

250

300

350

Rate (bits/channel use)

Ene

rgy

unit

Outer boundAntenna switchingTS with fixed powerTS with flexible power (P

peak = +∞)

TS with flexible power (Ppeak

= 2P)

Uniform PS (0 ≤ ρ ≤ 1)

Fig. 11. The R-E region of a 2-by-2 MIMO with a single receiver for boththe energy and information, where H = [1 0.8; 0.8 1] [150, Fig. 8].

total harvested RF power, and the transmit power is P = 1W.In the second case, the attainable R-E region is demonstrated,which is potentially limited in practice due to the incapabilityof the EH node to decode information directly. Restrained bythis shortcoming, two practical operating policies are devel-oped for the second scenario, referred to as time switching(TS) and power splitting (PS). The attainable R-E regions ofthe policies are characterized in Fig. 11 (i.e., [150, Fig. 8]),where P = 100.

Multiuser collaborative MIMO SWIPT networks are investi-gated in [151] where non-negligible circuit power consumptionis included. In specific, the authors of [151] aim to maximizethe overall data rate of all the active users in the system uplinkwhile providing satisfactory QoS to the downlink service ofthe users. The task is then transformed into a convex program.The beamformers and operating time, and the PSFs of everyuser are optimally designed with a joint consideration of boththe uplink and downlink. SDP and golden search are usedfor the optimal design. Additionally, an optimally selectedsubset of users are activated in [151], which is achieved byfirst activating the users at the beginning and then turning

15

off those contributing negatively to the growth of the sumrate. Numerical results indicate that the approach developed in[151] incurs a lower computational complexity at a marginalpenalty of sum rate, in comparison with the conventionalcombinatorial integer programming alternative.

The authors of [152] develop a malleable system structureto portray the performance of wireless power and informationtransfer enabled by an mmWave cellular network, wheredevices can align their beams to that of the BS, or whereno such beam alignment is undertaken. For the two operat-ing scenarios, the authors investigate the function of severaldevice-dependent parameters on the system performance. Theauthors find out that the total (power and information) cov-erage probability can be enhanced by optimizing the PSF tooptimally allocate the received signal between the EH and theinformation decoding segments. To utilize multiple antennasat the receiver with the least power consumption, they developa simple switch-based receiver framework for SWIPT.

Many existing designs of SWIPT systems are based onGaussian inputs, which may result in significant performancedeterioration when applied to the situation with finite-alphabetinputs. Authors of [153] design the precoder for such situation,in the presence of real-time CSI of an MIMO channel. Theyconstruct the precoder design task as an optimization problemto maximize the reciprocal information from the transmitterto the receivers, constrained by the transmit power and arequirement on EH amount. Since the problem is NP-hard,the global optimum cannot be obtained in a polynomial time.Utilizing its structure, they relax the problem to an SDPproblem. By applying the SDR technique, they develop ageneral solver for both co-located and separated receiversto realize a near-optimal beamforming precoder. In a specialscenario where multiple receivers are co-located, the authorsdemonstrate that the optimal design of the beamformer ex-hibits strong concavity in regards to the transmit power. Aspecialized algorithm is developed particularly for this specialscenario, and it demonstrates in [153] nearly identical effectyet with a substantially lower complexity, as compared to theSDR solution designed for the general scenario.

In [154], the authors investigate MIMO wireless communi-cation networks constrained by EH amounts. The studied EHnetwork includes one transmitter, one receiver, and multipleEH nodes. The EH nodes can convert their captured electro-magnetic waves into power to extend the system operatingduration. When the transmitter sends data to the receiver,it should also optimize the beamforming/precoder matrix tocharge the EH nodes in the meantime. Additionally, theamount of the charged energy should exceed a given threshold.Under the EH constraints, both the minimum mean-square-error (MMSE) and reciprocal messages are considered asoptimization metrics to contrive the beamformers at the trans-mitter. In order to make the developed approaches adaptablefor real-world implementation with reasonable overhead, theauthors of [154] also generalize the beamforming policies withpartial CSI.

VIII. SMART GRID-POWERED WIRELESS NETWORKS

The traditional power grid is shifting to a “smart” onewith many state-of-the-art new functionalities, such as smartmetering, RES, demand-side management, dynamic pricing,and two-way energy trading. Powered by such an electricitygrid with one or more emerging techniques, cellular networkscan have more choices in energy types and can design energy-efficient operation schemes. For example, there can be re-dundant energy at some BSs, which can be redistributed topower other BSs and their services. Fig. 12 depicts a typicalframework of the smart-grid powered cellular networks, whereeach BS serves several mobile users and is equipped with RESdevices (such as solar-electric converters and/or hydroelectricgenerators) and batteries. The BSs can carry out redistributionof unused and/or redundant energy to enhance the EE acrossthe entire systems.

Featuring (part of) the system model in Fig. 12, many recentworks focus on enhancing system performances by minimizingenergy consumption and operational cost [16], [155]–[161],or maximizing system’s efficiency and operator’s utility [52],[162]–[164]. State-of-the-art techniques and methodologies,such as game theory, DP [165]–[167], and stochastic gradientdescent (SGD) [168]–[171], can be leveraged. The objectivesand the solving techniques of the optimization problems arecategorized and summarized in Table II. Game theory isapplied when competition or cooperation exists in the sys-tem consisting of BSs and electricity retailers. DP usuallydecomposes a sophisticated problem into a set of much easiersubproblems with recurrence expressions. This relationshipis known as Bellman optimality equation. SGD is usuallyapplied to achieve the asymptotically minimum time-averageexpenditure of a network in no need of any prior knowledge ofsystem’s randomness (such as the EH amounts and electricityprices). The real gradient value of the objective is replacedwith the gradient from the training set, which helps to decouplethe optimization and constraints over time. It is proven that theoptimality loss of the objective can asymptotically diminish byreducing the stepsize of SGD [172]. The application scenarios,merits and disadvantages of some popular algorithms aresummarized in Table III. Conditions and auxiliary methodsapplied to solve the problems are summarized in Table IV.

In Sections VIII, IX, and X, we will review the literature onsmart grid-powered cellular networks with different function-alities. We first focus on the RES powered systems in SectionVIII and discuss the works on energy harvesting and sharing.Then we steer the survey to smart-grid powered systems inSection IX with energy purchasing based on dynamic prices.Finally, we review the works on two-way energy trading inSection X.

A. Utility Maximization

Ramamonjison et al. [164] study the resource allocationproblem for a two-tier wireless system, where smart grid-powered BSs can share renewable energy and battery storagethrough the aggregator. The authors aim to maximize thesystem EE while meeting the demand of an average sum-rateat each cell. They first design an extended convex-concave

16

Solar panels Wind turbines Batteries

BS BS BS

Mobile users

Aggregator

Smart power grid

Fig. 12. A smart grid-powered wireless cellular network [114], where each BS serves several mobile users and is equipped with RES devices and batteries.The BSs can carry out bidirectional energy transactions with the smart grid, and share energy with other BSs through the aggregator.

procedure to tackle the non-convexity issue in the problem, andthen take a well-known Dinkelbach method [135] to addressthe resultant subproblems. The offline algorithms developedin [164] have a polynomial-time complexity as the inneroptimization problems can be resolved in a polynomial timeby standard convex solvers like the interior point method [62].

The authors of [188] and [189] design multi-antenna beam-former for EH transmitters according to finite-alphabet inputsand the statistical knowledge of the CSI at the transmittersin real-time. They aim to maximize the sum of the averagereciprocal messages within a channel frame without violatingthe causality of the EH process. This results in a 2N2

t -dimensional stochastic DP problem. The objective of theproblem exhibits non-concavity in which Nt is the number oftransmit antennas. The authors of [188] and [189] prove theequivalence of the multi-dimensional stochastic DP problem toa one-dimensional problem to select the transmit power level.Dealing with the one-dimensional alternative can alleviatethe computations without penalizing the optimality. The one-dimensional alternative is first interpreted as a discrete-battery-state discrete-power-choice task and solved by backward re-cursion [190]; and then translated to a continuous-battery-statecontinuous-power-choice task and tackled by approximatingthe one-dimensional objective over a continuous feasible so-lution region.

B. User-Engaged Energy-Efficient Communication Scheme

A resource allocation task is studied in [179] for energy-collaboration enabled two-tier heterogeneous networks (Het-Nets) with NOMA, including a macro BS and several pico

BSs. User association and power control are optimally de-signed to maximize the efficiency in the energy utilization ofthe whole system under QoS constraints. To achieve this, adecentralized technique is first proposed to draw the optimaluser association given a transmit power. Then, user associationand power control are jointly and optimally specified. Thisallows for far higher EE than other alternative policies, suchas those developed in [191], [192].

A network is proposed in [193] where mobile end userscan share energy with each other at their encounter, henceminimizing the probabilities of inadequate energy for theirconsumption. Optimization of the corresponding network con-tains two major stages. The first stage shares energy optimallyamongst the mobile end users who agree to share (in otherwords, a couple of matched users). A stochastic optimizationproblem is constructed to acquire this optimal scheme, byconsidering the mobility patterns and the energy availabilityof the users. The second stage of the developed network is todesign a steadfast user-matching policy. Each individual of theusers finds a peer as its partner to share energy. The schemedrawn in the earlier stage can be utilized in such a way thata pair of well matched users are most unlikely to undergo anenergy outage.

Energy-aware traffic offloading schemes are studiedin [178], where small BSs (SBSs) are powered by conventionalelectricity grid and/or RES. User associations, on/off modes ofSBSs, and power control are collectively optimized based onthe statistical data of energy and traffic arrival. In a single SBSscenario, the closed-form expression for energy saving gain isderived, which facilitates the calculation of the SBS activationand traffic offloading strategy. A two-phase energy-aware

17

TABLE IIOPTIMIZATION OBJECTIVES AND METHODS FOR SMART GRID POWERED COMMUNICATION SYSTEMS

Optimizationmethods

Optimizationobjectives Total energy

consumptionOperational

costUtility ofoperators

Energy/cost

efficiencyQoS

Weighted/expectedsum rate

Long-termaverage cost

Online greedy/heuristicalgorithm

[155], [173]

Stochastic sub-gradientmethod

[174] [97] [121]

Lyapunov-based onlinealgorithm

[1], [124],[175]–[177]

Stochastic or deterministicADMM approach

[159], [160] [163]

Lagrange dual based(sub-gradient) method

[114], [178] [110] [179] [93], [94]

Multi-stage or nestedoptimization

[74], [156],[161], [173]

[174] [180]–[182] [94]

Ellipsoid method [110]

Evolutionary algorithm [162]

Iterative algorithm [158], [173] [52], [183] [164]

Dynamic programming [156] [157]

Game-theory based method [174] [181], [182] [180],[183],[184]

TABLE IIIANALYSIS OF ALGORITHMS

Algorithms Application scenarios Merits Disadvantages

Greedy algorithm [155],[173], [185]

A wide range of problems Obtaining local optimum in afast and efficient way

Seldom achieving globaloptimum

Deterministic ADMM [163] Distributed computation Fast convergence Many samples needed periteration to deal with

stochasticity

Stochastic ADMM [159],[160], [186]

Stochastic and distributedcomputation

One sample per iteration Converging with oscillations

Lagrange dual-basedsub-gradient method [93],[94], [110], [114], [179]

Non-differentiable functions Converging to globaloptimum for (pseudo)convex

functions under certainconditions

Only locally optimal fornon-(pseudo)convexfunctions; iterative

computation

Stochastic sub-gradientmethod [97], [121], [174]

Time-coupling sub-gradients;sum-minimization problemswhere certain parameters are

to be estimated

Smoother convergence;time-decoupling

Near-optimal; iterativecomputation

Lyapunov-based onlinealgorithm [1], [124],

[175]–[177]

Time-coupling long-termscheduling horizon; uncertain

models

Time-decoupling; ensuringsystem stability and system

penalty minimization

Near-optimal; iterativecomputation

traffic offloading approach is further developed in a multiple-SBS scenario, taking into account multiple characteristics ofSBSs with a range of energy sources.

A user association problem is first formulated in [173] viaconvex optimization in the space dimension. Total energyconsumption is minimized by allocating the traffic amongdifferent BSs dynamically in a given time slot. Then, RESallocation is optimized over different time slots for each BSto minimize the usage of the energy from the grid. To tacklethis optimization task, a low-complexity offline approach withinfinite battery capacity is designed by assuming non-causal

RES amounts and data traffic statistics. The offline algorithmcan achieve the optimality in this scenario, and play the roleof a performance upper bound for evaluating practical onlineapproaches. Heuristic online approaches with finite batterycapacity are further developed which are reliant only on causalRES and traffic information.

C. Minimization of Energy Consumption

Chia et al. [155] present a new model to describe energycooperation among BSs powered by a smart grid (includingconventional electricity grid and RES), limited energy storage,

18

TABLE IVCONDITIONS AND AUXILIARY METHODS

Conditions and auxiliary methods References

S-lemma [114]

Semi-definite relaxation technique [114]

Time-sharing condition [187]

KKT optimality condition [178], [179]

Uplink-downlink duality [110]

Zero-forcing beamforming [110], [177]

Game theory (Nash bargaining, Stackelberg game ) [180]–[184]

and connection by resistive power lines to share surplus elec-tricity. They aim to minimize the expectation of the electricalenergy supplied by the conventional grid and utilized by theBSs, i.e.

EEti

[ 1

T

T∑t=1

(wt1 + wt2)], (8)

where Eti is the EH amount captured by BS i at time t, wtistands for the energy procured from the traditional electricitygrid, and T is the total scheduling time horizon. If therenewable energy profile (REP) and energy demand profile(RDP) of all the BSs are deterministic or known in advance(Case 1), the optimal energy collaboration scheme for BSs canbe awarded straightforwardly by tackling a relatively simplelinear programming problem, since the objective function in(8) is simplified linearly to be

∑Tt=1(wt1 + wt2), and all the

constraints of (8) are linear in the first place. If the REP andRDP are two stochastic processes and only causally knownat the BSs (Case 2), Chia et al. propose a greedy onlinealgorithm by taking a quick picture of the aforementionedlinear optimization problems (Case 1) with T = 1.

Let rα and rβ be the energy loss coefficients which describethe ratio of loss during charging the battery and transferringenergy between BSs, respectively. Chia et al. [155] analyze theoptimal structural properties of the greedy online algorithmunder certain conditions. For example, if rβ = 0 or rβ = 1,the greedy method can preserve optimality for any feasibleenergy profiles. If rβ ≥ rα, then energy transfer is optimal.That is, if Et1 ≥ 0 ≥ Et2 at time slot t, then sending ∆ =min{|Et2|/rβ , Et1} units of electricity from BS 1 to BS 2 tomake up for the shortage of Et2 is part of an optimal scheme.If Et1 ≥ 0,∀t and rβ ≥ rα, the greedy policy is optimal.According to the symmetry between Et1 and Et2, the sameresult holds if Et2 ≥ 0,∀t and rβ ≥ rα.

Chia et al. [155] also provide insightful analysis and conclu-sions: i) for the optimal scheme, no electrical energy should bebought to increase the battery levels from the grid; ii) a systembeginning with highly charged batteries can have a low optimalcost; and iii) it is more cost-saving to save energy locally ateach BS than to transfer and save the energy at the other BS.On the other hand, it is reasonable to assume partial knowledgeof REP, which consists of a deterministic waveform with smallrandom noises added at each time slot as the prediction errors.Inspired by the aforementioned two cases, the authors of [155]then propose a compound approach which can utilize offline

statistics about the REP and operate in an online fashion. Theyuse the non-real-time offline approach to draw the adequatepolicy (or schedule) for the deterministic part of the REP, andthen apply the greedy heuristic to recoup any gaps pertainingto non-negligible noise effects.

The average grid power consumption is minimized in [156]for RES-powered BSs under users’ QoS (blocking probabil-ity) constraints. The task is converted into an unconstrainedoptimization problem to minimize the weighted sum of thegrid energy usage and blocking probability. A two-phase DPapproach is developed by leveraging statistical data for trafficload and RES. The BSs’ on/off modes are optimally decided inthe first phase. The active BSs’ resource blocks are assignediteratively in the second phase. Compared with the optimalcollective BSs’ on/off modes and active resource blocks allo-cation approach, the proposed approach significantly decreasesthe computational complexity and can realize the optimaloperation when the traffic obeys a uniform distribution.

The grid energy expenditure is minimized for smart grid-powered cellular networks in [158]. The task is formulated asan NP-complete mixed-integer nonlinear program. For central-ized systems, a cost-aware approach is designed to tackle theload distribution problem and the energy configuration prob-lem in an alternating manner. The centralized algorithm re-quires a low computational complexity, and rapidly convergesto the near-optimal solutions. For distributed networks, a three-stage decentralized ruling scheme is proposed in [158] wherethe BSs and mobile terminals can independently calibrate theirindividual policies only based on limited knowledge locallyaccessible to them. System expenditure can be greatly reducedin both types of networks.

D. Lyapunov-based Online Optimization

The grid energy consumption (GEC) is minimized in [176]for a smart grid-powered queueing orthogonal frequency-division multiple-access (OFDMA) system with battery leak-age. By considering the temporal variation of the network,the GEC minimization task is formulated to collaborativelydesign the admission regulating, power assignment, subcarrierdistribution, and communication duration. The random RESamounts are simplified to be an i.i.d. process. By takingadvantage of the state-of-the-art Lyapunov optimization, anefficient online approach is developed, termed as leakage-aware dynamic resource allocation strategy. This strategy

19

tracks present system states which consist of CSI and batterylevel with no need for the a-priori knowledge on the states.Furthermore, it is proven that the minimum GEC value can bereached asymptotically with the proposed approach.

The network service cost is introduced in [175] as a per-formance metric to account for both the grid power usageand attainable QoS. A computationally inexpensive onlineapproach is proposed to minimize the average system serviceexpenditure over a long time by conducting joint BS associa-tion and power control (BAPC), referred to as the Lyapunovoptimization based BAPC (LBAPC) algorithm. A merit of theapproach is that the decisions rely solely on the instantaneousstate knowledge with no need for any a-priori knowledge onthe distribution statistics of CSI and EH processes. To decidethe system activity, only a deterministic problem needs to besolved at every time slot. A simple inner-outer optimizationapproach is developed to provide effective solutions to thedeterministic problem. It is further proven that the LBAPCapproach is asymptotically optimal, as the control parameterV (with unit J2/cost) →∞.

IX. ENERGY PLANNING UNDER DYNAMIC PRICING

Dynamic pricing is a vital mechanism of smart power grids.It can help shift some load in peak time to off-peak time,and thus balance the power consumption in these two periodsfor the power grid. Cooperative BSs can plan their energypurchasing, storing, and sharing jointly according to the pricesto reduce the system-wide operational expenditure.

A. Operational Cost Minimization

In [157], the on-grid energy expenditure is minimized in alarge-scale green cellular system by collectively designing theoptimal BS on/off strategy and electricity procuring scheme.The green cellular network often experiences fluctuationsbrought by RES, energy prices, wireless data traffic, BScoordination, and traffic offloads. Hence, it is usually NP-hardto obtain the optimal scheme that minimizes the electricity ex-penditure over a long-term meanwhile guaranteeing the users’QoS. For a dynamic system design, stochastic geometry (Geo)can be applied to account for large-scale wireless network.DP can be used to develop adaptive on/off schedule of theBS and the electricity procurement. By integrating these twoaspects, a new Geo-DP design is shown to guarantee that theoptimal probability of the BS being activated (or staying “on”)can just suffice the QoS requested by the users. Nonetheless,the typical curse of dimensionality of DP [165] prevents theoptimal electricity procurement scheme from scaling up tolarge-scale wireless networks. Therefore, a suboptimal energyprocurement scheme with a low complexity is put forth, whereon-grid electricity is procured in abundance with an adequateprice only if the present battery reading and the expected futureRES level are both low. This suboptimal policy can realize anear-optimal performance.

B. Utility Maximization

The authors of [162] aim to maximize the profit of cellularoperators meanwhile minimizing the CO2 emissions in green

cellular networks and satisfying the desired QoS. The cellularnetwork is powered by the smart grid where electricity retailerssell renewable energy to the BSs. Energy from different retail-ers may have various types, and thus have distinctive pricesand pollutant levels. The BSs’ sleeping policy and electricitypurchase policy are decided via several algorithms, such as aniterative algorithm and an evolutionary algorithm (includingthe genetic algorithm and the particle swarm optimizationmethod). For the iterative algorithm, it is assumed that all BSsare switched on at the beginning. Then, at each precludingstage, one BS is shut down at a time. For the i-th precludedBS, the corresponding optimal utility function is calculatedand compared with the previous maximum utility to determinewhether precluding BSs is possible or not. The algorithmterminates when there are no more BSs to be precluded.

The utility of the mobile operators is maximized in [52],which collaboratively make decisions on energy procurementand BS sleeping policy based on profits, network demands,retailers’ capacity, and CO2 emissions. Energy harvesting,dynamic pricing, and energy sharing are implemented in theframework to help reduce energy consumption of the network.Three utility metrics are introduced to measure the level offairness in optimization, including the weighted sum, themax-min criterion, and proportional fairness (PF). The sumutility metric measures and quantifies the overall profit of thenetwork. This metric promotes operators with higher revenues,while depriving the possibility for low-revenue operators topurchase the cheapest energy. The max-min metric maximizesthe minimum profit across the network, thus improves fairnessto the system. The PF metric maximizes the geometric mean ofthe profits, which can efficiently avoid very low profit since aclose-to-zero profit would cause the entire objective function todiminish. Given the BSs’ on-off state, the energy procurementproblem exhibits convexity and can be readily tackled by theLagrangian method. Given the optimal energy procurement,the BS on-off switching problem is non-convex. A determinis-tic iterative approach is first developed to establish the optimalon-off policy for the BS in a centralized fashion. Later, adecentralized algorithm is proposed with faster convergenceand lower complexity, yet penalizing performance gain.

C. Game Theory for Energy Procurement

Liberalization of the electric power market has been advo-cated in many countries and regions [180], where electricityretailers can set their prices and compete for best interest. Toachieve energy-efficient green communication, it is necessaryto consider data traffic, dynamic electricity price, the pollutantlevel incurred by brown energy consumption, and the robust-ness of the smart grid when shaping green wireless cellularmobile systems. Every BS is expected to choose its most cost-effective electricity retailers at any moment for economical andecological profits [180].

Aligned with this goal, the smart-grid powered cellularnetwork system is constructed as a two-level Stackelberggame in [180], [184]. At the cellular network level, the activeBSs can choose the electricity retailers and the correspondingpurchasing amounts, aiming to achieve the lowest service

20

blocking probability with the least possible expenditures.At the smart grid level, the retailers can decide on theirelectricity prices such that they can acquire as much extraprofit as possible, competing to get selected by the BSs atthe same time. Based on the Lagrange dual method, theexistence and uniqueness of the Stackelberg equilibrium areproven in [180] for the proposed Stackelberg game; while aniterative algorithm with low complexity is applied in [184] todraw the optimal solutions. It is shown that the smart gridhas substantial influence on green wireless cellular systems,and the developed strategy can greatly diminish the energyexpenditure and CO2 emissions in these systems.

The same mechanisms developed in [180] and [184] areextended to a three-level Stackelberg game for cognitiveHetNets in [181], [182], where the three parities in the gameare smart grid retailers, macro-cell BSs (MBSs), and femto-cell BSs (FBSs). A homogeneous Bertrand game is designedto model the price decisions made by the retailers. Then,a backward induction approach is leveraged to analyze thedesigned game. It is observed that both the FBSs and theMBSs intend to choose the electricity retailer with the cheapestprice. The electricity prices are set according to the Nashequilibrium at the smart grid level. In this case, no retailercan increase its individual net profit by selecting a differentprice, given the prices provided by the other retailers.

The RES is utilized to power millimeter-wave (mmWave)backhaul networks in [183], where wireless operators purchaseelectricity from several renewable power suppliers to supportmobile terminals. A lead time-dependent pricing strategy isdeveloped, which enables a wireless operator to control thelatency of traffic and service deliveries over the backhaul linksand coordinate the suppliers to decide on how much RES to bestored at each supplier. The task is constructed as a standardStackelberg game between the network operator and severalrenewable power suppliers. Different from [180]–[182], [184],the wireless operator acts as the leader in this game, whodetermines the pricing mechanism for the power suppliers,while the suppliers play the followers who decide their selfenergy storage policies according to any given pricing scheme.

In a centralized network, the operator and the renewablepower suppliers jointly maximize the system profit [183],while in a decentralized one, the operator and the renewablepower suppliers maximize their individual profit. Effectivedecentralized methods are developed in [183] to acquire theoptimal pricing strategy of the wireless operator, and the Paretoequilibrium storage policies for the suppliers, respectively.Both the algorithms start by treating each renewable powersupplier as a separate group. For the operator’s game, thealgorithm merges two neighboring groups in each iteration.For the suppliers’ game, the algorithm decides whether or notto merge two groups by comparing their supportable trafficloads in each iteration. Both algorithms stop when no moremergers happen. It is also shown in [183] that the proposeddistributed policy allows a wireless operator to generate ahigher profit than a centralized scheme.

Traditional electricity market is shifting to a “smart” one,offering several electricity purchase policies and their com-binations for mobile network operators (MNOs) of cellular

networks. It can be possible for the MNOs to prepay forelectricity day-ahead at a relatively economical price and alsopurchase electricity based on a real-time demand at a relativelyhigh and less economical price, respectively [194]. To reducethe power expenditure, it can be very important for MNOsto comprehensively organize their day-ahead and real-timeelectricity purchases according to their dynamic traffic loadover time. The BSs of the MNOs can offload their traffic andservices, so as to switch the most number of lightly-loadedBSs into the sleep state for system-wide energy-efficiency. Tothis end, the authors of [174] take two different MNOs co-located in a region for an example. The two MNOs interplayin both electricity procurement and traffic load balancing foroperational expenditure reduction. The two MNOs can bothintend to minimize their own electricity expenditure, two-phase stochastic programming is adopted to draw the optimalpolicy for electricity group purchasing with load balance.

In the case where the two aforementioned MNOs are fromtwo competitive organizations and are only interested in mini-mizing their own electricity expenditures, the authors of [174]develop a repeated Nash bargaining scheme, where the MNOsbargain and split electricity expenditures under electricitygroup purchasing and load sharing. At the first stage, thetwo MNOs settle the deals on the day-ahead electricity grouppurchase, as well as how to split the collective electricitycommitments between them, by considering the potentialreal-time collaboration benefits. At the second stage, undergiven day-ahead electricity commitment and the correspondingelectricity group buying, the two MNOs negotiate in real-timeat each slot t about the real-time electricity group purchasing,and how to allocate the collective electricity trading amounts,the wireless loads, and the inter-MNO payment. This Nashbargaining scheme can realize Pareto-optimality and, in turn,the effective electricity expenditure decreases for both MNOs.

X. TWO-WAY ENERGY TRADING AND COOPERATION

The electricity bills of cellular operators continue risingdue to explosive demands for wireless services in the 5Gand beyond communication networks. The BSs are seekingnew approaches to diminish the energy expenditure withthe integration of RES. The bidirectional electricity tradingcapability of smart power grids allows the cellular BSs to selltheir redundant renewable energy to the grid for profit, whichis a straightforward way to save the operational costs.

A. Energy-efficient Schemes

With the spatial variations of user density, data traffic, andthe amount of RES, it is more efficient for the BSs to col-laborate and jointly schedule their energy, wireless resources,and/or downlink users. An energy-efficient framework is pro-posed in [16] where different BSs share or trade electricitywith the assistance of an aggregator in a smart power grid,and/or share radio resources and offload traffic to cut offthe operating expenditure. The following three approaches arepresented to achieve this goal.

The first approach is energy collaboration on the side ofpower supply. The BSs use a bidirectional flow to merchandise

21

redundant electricity with the smart grid or share renewableenergy with each other through the aggregator. The two BSsin the same group schedule the energy to be injected to,or drawn from the grid simultaneously, since their energysurplus and deficit can be matched. The energy demands forcommunications are specified and analyzed.

The second approach is communication collaboration on thedemand side. The BSs perform cost-oriented wireless commu-nication collaboration to share resources and reschedule trafficload both spacially and temporally, by considering a presetamount of the energy (RES and/or traditional). Three differentcost-oriented schemes are typically studied for various timescales to achieve such cooperation.

The first cost-oriented scheme is traffic offloading [195],where the BSs in short of RES can transfer their users toadjacent BSs with ample RES (even if they bear heaviertraffic), to reduce the total amount of electricity purchasedfrom the smart power grid in an attempt to save the operatingexpenditure. Traffic offloading can be executed on a basis ofseveral seconds. The second cost-oriented scheme is spectrumsharing [196], where BS1 shares a portion of its available spec-trum with BS2. Under the same users’ QoS constraints, BS2can reduce its transmit power procured from the grid, whileBS1 consumes more RES for transmission. Hence, the totalcost is diminished. This can be implemented at a time scale ofminutes. The last cost-oriented scheme to achieve cooperationis CoMP [197], which helps pair the BSs’ transmit powerwith their EH amounts by accommodating the BSs’ transmitsignals. In particular, the BSs with larger RES amounts areexpected to use higher transmit powers to provide strongerwireless signals to the users. CoMP runs at a symbol- or frame-level on a typical basis of microseconds to milliseconds. Thiscan be much complicated but save more energy budget, ascompared to the two earlier schemes developed in [195] and[196].

The third approach to achieving bidirectional energy tradingand sharing is joint cooperation of energy and communicationfrom both the sides of demand and supply. The BSs collaborateto reduce their operating expenditure (i.e. electricity bills) tothe greatest extent. A small cell network (SCN) with EHcapabilities is studied in [18], from the perspectives of outageprobability, system performance, grid power consumption, andcell association. Analysis and simulation in [18] reveal thatthe outage probability declines with the density of the BSsdeployed in the SCN (denoted as λBS), and that the gridpower consumption PG also decreases with λBS . To reducePG or enhance system performance, it is more efficient toincrease λBS than it is to increase the EH rate. When λBSis extremely small or large, the battery capacity has littleimpact on PG or system performance. As for cell associationschemes, the distance-based policy suffers performance loss asit overlooks the spatial variation of available energy at differentBSs. Therefore, conventional cell association strategies cannotbe directly adopted in EH-SCNs. It is demonstrated in [18]that the SNR-based scheme outperforms the distance-basedscheme, as it utilizes information on both the distance and thecurrent energy state. However, it still undergoes performancedegradation as it makes decisions only according to the

present system state and overlooks the coupling in differenttransmission blocks among different users.

B. Minimization of Operational Cost

A transmit beamforming scheme is designed in [159], [160]for a coordinated multicell system, where the transmission ispowered by a smart power grid. Operating in a distributedfashion, the scheme uses the state-of-the-art stochastic alter-nating direction method of multipliers (ADMM) [186]. Theconditional-value-at-risk (CVaR) cost [198], [199] is intro-duced to reduce the risk of extremely high cost of the system.The long-term SINR is considered to guarantee users’ QoSbased on the stable downlink channel covariance matricesRijk, where i, j ∈ [1, . . . , I] are the BS indexes. It is proventhat when rank(Rijk) = 1 or when the number of BSs I ≤ 2,the optimal beamforming matrices {W∗

ik} of the SDR problemis always rank-one, which means that the relaxation is tight,and the solutions {W∗

ik} are globally optimal. Simulationresults verify the tightness of the SDR in general cases.

The problem is solved offline in a distributed fashion viathe stochastic ADMM without the a priori knowledge ofthe EH amounts and electricity prices {si}. The BSs updatetheir primal and dual variables by visiting one sample of thehistorical realizations of {si} at each iteration. The historicalrealizations are stored in the data base of each BS i. The BSsdo not have to communicate their inter-BS interference powerswith each other in this way, which greatly reduces signalingand backhaul overheads.

C. Utility Maximization

The idea of cost efficiency (CE) is applied in [163] tocalculate the total data rate transmitted at the cost of adollar for micro-grid (MG)-powered BSs. The MG has smartgrid features such as RES, bidirectional electricity trading,and dynamic price adjustment. CE is maximized by jointlyscheduling the electricity generation in the MG and optimizingthe transmit power of the BSs. The CE maximization taskincludes constraints accounting for the strong multivariatecoupling over time. To address this formulated fractionaloptimization problem, the Dinkelbach method is first applied.Then a low-complexity method exploiting the ADMM ap-proach is developed. Several auxiliary variables are introducedto split variables into two sets, so that the constraints capturingthe different coupling are separable between the two subsetsof all the optimization variables. Consequently, the approachdeveloped in [163] only accommodates simple updates atevery step, and thereby permits decentralized executions. Theapproach is ensured to converge to the global optimum oforiginal maximization of the CE.

Farooq et al. [161] propose a mixed energy redistributionframework for mobile cellular networks, which combinesphysical power lines and energy transaction between the BSsfacilitated by the smart grid. According to the average valueor the full statistics of the availability of the RES, algorithmsare proposed to optimally deploy physical power lines betweenthe BSs. Taking into account the battery volumes and dynamicelectricity pricing, a framework is developed to schedule

22

energy, and determine the optimal amounts of energy and RESto be obtained and shared among the BSs, respectively. Threecases are studied, in which the RES amount is unknown, fullyknown, and partially known, in advance.

D. Lyapunov-based Online Optimization

Based on [124], two-way trading and multi-timescale plan-ning of energy in 5G networks are presented in [1], where im-plementation challenges are discussed, and the use of stochas-tic control theory is studied. The Lyapunov control theory isevaluated for potential applications. The concrete usefulness ofthe theory can be validated by numerical simulation results inpractical scenarios without knowing the future CSI, transactionprices and EH amounts.

The long-term energy cost minimization problem is studiedin [177] for a smart grid powered communication network.Facilitated by the two-way trading mechanism in the network,a harvest-use-trade strategy is selected to deal with the short-comings of the batteries to enhance usage efficiency of theRES. With the help of the Lyapunov optimization techniqueand the Lyapunov drift-plus-penalty function, the stochasticoptimization problem is reformulated into a joint minimizationproblem of energy and packet rate. Two suboptimal algorithmsare proposed to tackle the NP-complete problem by consid-ering the CSI and the packet detection failure, featuring thetechniques of successive approximation beamforming (SABF)and zero-forcing beamforming (ZFBF) [200].

XI. APPLICATIONS OF ENERGY HARVESTING AND SMARTGRID-POWERED WIRELESS COMMUNICATIONS TO

5G/B5GThe importance of energy harvesting and smart grid-

powered wireless communications has also been found inmany emerging applications in the 5G and beyond wirelessnetworks, such as mobile edge computing, machine learning,NOMA, URLLC, and so forth.

A. Mobile Edge Computing (MEC)

Many recent and emerging developments of the IoT haveopened up possibilities for a wide range of new mobile ap-plications, requiring low-latency communication and intensivecomputation over massive mobile devices. The emerging MEChas offered a new paradigm to offload computing workloadfrom mobile users. This can enhance computation capabilityfor mobile users by leveraging remote execution, and sig-nificantly reduce communication latency by having serversplaced in the proximity of mobile users. By integrating thenetwork function virtualization (NFV) techniques, MEC pro-vides flexibility on the resource scheduling and service deploy-ment [201], [202]. Initial investigations on EH powered MECsystems have been conducted in [203]–[211], focusing on EH-based MEC servers [203], [204] and EH mobile users [205]–[210]. In [203], [210], the system operator learns online thetask amounts to be offloaded from the MEC server to thecentral cloud and the CPU frequency of the MEC server, basedon the states of core network congestion and RES. Consider-ing inter-cell interference in dense small cells, a distributed

three-stage strategy is proposed in [204] to jointly optimizetask offloading, channel allocation, and computing resourceallocation by decomposing the original problem. Consideringpractical stochastic system environments, algorithms basedon Lyapunov optimization techniques [205]–[207] and deeplearning [208], [209] have been used to produce dynamic taskoffloading and resource allocation strategies.

B. Deep (Reinforcement) Learning

Deep (reinforcement) learning has been widely employedin image processing and natural language processing [212],[213]. It has also been increasingly used to solve challenging(in many cases, non-linear non-convex) problems in wirelesscommunication systems, e.g., Polar decoding [214], [215] andmassive MIMO channel estimation [216], [217]. Deep neuralnetworks (DNNs) are able to solve sophisticated non-convexproblems without explicit mathematical formulations [218]–[220]. Several recent works have investigated the EH-basedwireless communication systems where deep learning is uti-lized as an optimization method [221]–[223].

C. Non-Orthogonal Multiple Access (NOMA)

NOMA is a useful method to improve the spectral effi-ciency of wireless communication systems. NOMA transmit-ters adopts superposition coding (SC) to stack the signals des-tined for multiple destinations or users. Successive interferencecancellation (SIC) is carried out at each of the destinationsor users to extract the signals of interest and suppress thoseintended for the others. NOMA is able to connect multiple ac-tive users by using a single piece of time-frequency resources[224], [225], and outperform its orthogonal counterpart interms of spectral efficiency (SE) and flexibility.

EH-based NOMA is attracting increasing interests [226]–[228]. In [226], a NOMA-based relaying network is analyzedwith a EH powered relay node to enhance system SE and userfairness. The outage performance of the network is examined,in which the transmit antenna selection is applied at the BSand maximal ratio combining (MRC) is applied at the endusers. Closed-form formulations are established for the outageprobability.

Min and Meng [227] study energy-efficient resource man-agement for NOMA-based wireless powered sensor networksby constructing a EE maximization task regarding the EH timeand transmit powers. A particle-swarm-optimization-based ap-proach is derived with fast convergence by analyzing thestructure of the optimization problem. In [228], NOMA is firstleveraged with beamforming to support several users in eachbeamforming vector. SWIPT is applied to NOMA systemsin [229]–[231]. It is proven in [229]–[231] that integratingSWIPT not only facilitates cooperation among users, but alsoalleviates the level of self-interference stemming from signalleakage from the output to the input. Furthermore, the task ofthe total sum rate maximization is constructed and tackled viaa two-step convex programming based process. The outageprobabilities of both weak and strong users are establishedwith closed-form solutions deduced.

23

Some existing works have studied the power allocation onthe multicast-unicast transmission [232], spectral efficiencyand confidentiality [233] of NOMA, as well as comparisons ofsystem capacity between the emerging MIMO-NOMA and theexisting MIMO-OMA [234]. Transmit antenna selection at theBS is studied in [229]. The weighted sum-rate is maximizedfor an SWIPT enabled cooperative NOMA system in [230],by optimizing the power arrangement of the source and the PScoefficient at the users. A new collaborative SWIPT NOMAnetwork is developed in [231], where the NOMA users nearthe source can serve as EH relays to assist NOMA usersfar away. Closed-form outage probability and sum throughputvalidate that the application of SWIPT does not compromisethe diversity gain, as compared with conventional NOMA. Itis confirmed in [231] that the opportunistic choice of nodelocations for picking users can realize a low outage probabilityand yield better throughput than random picking schemes.

D. Ultra-Reliable Low Latency Communications (URLLC)

5G/B5G mobile cellular networks are expected to supportURLLC which is expected to offer substantially short pro-cessing and transmission delays (< 1 ms), meanwhile securingstrong reliability (i.e. about 99.999% successful delivery rate).Resource allocation are studied to enable URLLC in anOFDMA downlink in [235], [236], by minimizing the requiredsystem bandwidth or maximizing the system sum rate underQoS requirements. An optimization problem is constructed in[237] to find the payload allocation weights that maximizethe reliability at targeted latency values. A risk-aware machinelearning approach is proposed for URLLC traffic managementin [238], by minimizing the risk of loss. An SWIPT-enabledURLLC network is studied in [239], where the credibilityof the system is maximized by choosing the optimal SWIPTparameters based on both the PS and TS protocols.

XII. LESSONS LEARNT AND FUTURE RESEARCHDIRECTIONS

This article reviews the RES/smart-grid powered wirelesscommunication systems, including single point-to-point link,multi-hop link, multipoint-to-multipoint system, multi-point-to-point system, and multi-cell system. In this section, we drawuseful insights from the literature and explore promising futureresearch directions.

A. Point-to-Point Links

As reported in Section III, one of the most popular tech-niques is the Lagrange multiplier method which is appliedto convex programming, as shown in Fig. 2. The method hasbeen utilized to unveil the underlying optimal structure of datatransmission policies. Several main challenges are worth futureresearch.

1) Learn-and-Adapt Algorithms for EH Systems: Most ex-isting works assume non-causal information about the EH,data arrivals and channel states in the typical offline opti-mization framework, which is generally impractical. In theonline scenario, the existing studies, such as [13], [67] and

[74], assume that there exists no a-priori knowledge on thesesystem variations, and develop heuristic schemes. A potentialonline optimization technique is to employ learn-and-adaptalgorithms [240], [241] which are expected to learn onlinethe EH, data arrival processes and channel states, and adaptthe transmission strategy accordingly. In [72], Q-learning isconsidered to learn the optimal transmission policy with theEH, data arrivals and channel states modeled as Markov (de-cision) processes. Exploring other learn-and-adapt algorithmswith affordable complexity is an interesting research direction.

2) Multi-User Networks and Energy Cooperation: Froma networking point-of-view, data transmission of multipleusers/nodes (such as multi-hop and multiple-access networks)is of potential interest. The computational complexity ofspecifying the optimal data schedules increases generally withthe user number. Even in a two-hop link, the data scheduleof the source can affect the data arrivals at the relay, thuscoupling strongly the transmission policy over the system[19]. Additionally, the causal information over different usersis hard to obtain in practice. Optimal schedules based onlyon current information need to be further investigated. Somebasic multi-user scenarios have been studied in the literature,as summarized in Sections IV and V.

B. Multipoint-to-Point Systems

Apart from the data scheduling and energy managementtechniques for EH-powered multi-access networks reviewed inSection IV, potential future research directions are as follows.

1) Power Allocation for EH-powered NOMA: The powerallocation policy for NOMA determines the interference can-cellation capability of the receivers, and directly affects thethroughput and user fairness of NOMA [242]. For EH-powered NOMA, power constraints need to be involved inthe development of power allocation schemes. The optimalscheme can be obtained by searching through the entirelegitimate solutions (satisfying the power constraints), whichwould lead to an excessively high complexity. Low-complexityand dynamic power allocations constitute a promising researchtopic for future work.

2) Channel Estimation and EH Prediction: Most works onmulti-access networks assume perfect CSI and non-causal in-formation about EH, which is hardly possible in practice. Thedesign of practical channel estimators for NOMA is studied in[243] and [244], and optimal approaches have been proposedto reduce the channel estimation errors. However, the increaseof the user number in 5G/B5G systems is expected to resultin severe inter-user interference and, in turn, severe channelestimation errors. Moreover, few works have considered EHprediction for multi-access systems. To this end, advancedchannel estimation and EH prediction methods are requiredfor multi-access systems.

C. Multipoint-to-Multipoint Systems

Existing studies on RES powered CoMP systems typicallyamount to minimize the total energy or cost. Lagrangiandual-based methods and Lyapunov optimization methods aretwo popular classes of techniques used. The optimality can

24

be often achieved for convex problems, while asymptoticoptimality is typically possible for online optimizations. Somefurther research effort could be helpful to further advance thefollowing aspects.

1) Implementation of Uplink CoMP Techniques: Most ex-isting works study the joint transmission and energy man-agement in RES powered downlink CoMP systems. How-ever, in practical network implementations, CoMP also sup-ports accentuations to uplink reference information, powerregulating, and signaling for coordinated uplink reception(CoUR) [98]. An important feature of CoUR is to decouplethe multiple points which send downlink signals from thosewhich receive uplink information. The implementation ofCoUR includes coordination and exchange of informationamong reception points, computation of the receiving com-biner, coordinated designation, and exchange of received data.The energy consumption of end users is not negligible andshould be considered while implementing these techniques andoptimizing CoUR networks. By exploiting energy harvesting,utilization, redistribution, and management policies at theuser side, energy-efficient operations of the CoMP networkbecome promising. New challenges of optimizing the design ofCoUR include low latency, imperfect CSI, increased signalingoverhead, EH and coordination at the user side, SE, EE andcomputational complexities.

2) Dynamic CoMP Clustering: Dynamic CoMP clusteringcan be increasingly complex with growing signaling overhead.Nonetheless, it is responsive to network changes, as comparedto static CoMP scenarios. Inter-cluster interference can beminimized and the cluster size of users can be optimizedadaptively and dynamically. To optimize SE, EE, load balanc-ing, QoS, and backhaul scheduling for dynamic clustering inRES-powered CoMP systems, methodologies such as greedyalgorithms [185], game theoretic approaches [245], and multi-stage optimizations [246] have been utilized. However, thoseexisting dynamic CoMP clustering approaches may lack scal-ability and can suffer from a high complexity of schedulingand precoding designs. It is a challenge to provide fullydynamic clustering at low complexities and costs when thereare increased scheduling and signaling overhead. To this end,machine learning (ML) algorithms [247], [248] can potentiallyhelp design the ahead-of-time and online resource allocationschemes of systems with the integration of unpredictable andnon-dispatchable RES. ML and Big Data techniques [249]can facilitate dynamic energy sharing, traffic scheduling, andCoMP clustering in a network by considering user loca-tions, traffic demands, latency, imperfect CSI, EH amount,and electricity prices; and therefore deserve comprehensiveinvestigations.

D. Multi-Hop Wireless Links

As shown in Section VI, an interesting aspect of multi-hopnetworks is that network nodes can benefit from informationand energy cooperation when they can share information andenergy with each other. This opens up the following potentialresearch opportunities in this context.

1) Opportunistic Routing: Opportunistic routing canachieve reliable data transmission for disconnected and sparsemulti-hop networks, and provide flexibility and easy adapta-tion to high system dynamics [250], e.g. time-varying chan-nels, bursty data arrivals, and intermittent EH. The routingtechniques utilize the broadcast nature of radio. Relay nodesare opportunistically chosen to forward packets. This proce-dure continues until all packets are successfully delivered.Opportunistic routing also supports a high amount of datatransfers. Existing studies on opportunistic routing and EHare limited.

2) Impact of Mobility on EH-Based Multi-Hop Routing:Proper mobility models can estimate the movement of nodeswith regards to position, direction, and velocity. This can bereadily captured in emerging D2D communications [250]. Byobserving the movement pattern of nodes, several works haveproposed proper mobility models and studied the impact ofmobility on EH for one-hop links [251], [252]. Yet, effectiveresource allocation schemes that can integrate the character-istics of EH-based multi-hop routing are still missing. Theimpact of mobility on the routing decisions for multi-hopnetworks is to be investigated.

3) Network Coding-Aware Routing: Network coding-aware(NC-aware) routing techniques use omni-directional antennasto utilize the broadcast nature of wireless channels. Thistechnique shows many performance metrics over conventionalrouting, including improved system capacity and reliability,and reduced delay and energy consumption [250]. It is provenin [253] that NC is effective for achieving the maximumdata flow in D2D networks. Therefore, NC-aware routing canbe potentially used in multi-hop networks for performanceimprovement. How to exploit RES and integrate the EHfeature of multi-hop networks in NC-aware routing is an openproblem.

E. SWIPT SystemsThe application and integration of the SWIPT technique in

SISO, MISO, MIMO, relay, and mmWave systems rely onpower allocation (transmitter) and power splitting (receiver)schemes, as unveiled in Section VII. The following topics onSWIPT enabled systems are worth in-depth studies.

1) Security of SWIPT: Increasing the transmit power canhave double-sided effects on SWIPT systems. On one hand,the desired power transferred from a source to a legitimatedestination can be enhanced. On the other hand, the undesiredrisk of information stealth by an eavesdropper can be esca-lated, leading to a security concern in SWIPT systems. Howto enhance the signal intensity on the legitimate recipient sidewhile reducing the signal intensity on the eavesdropping sidecan be an exciting research direction to pursue.

2) SWIPT for CoMP Systems: Currently, CoMP systemsare categorized into two groups: joint transmission (JT) inwhich an end user is served by several BSs and its datais shared globally, versus coordinated beamforming (CB)where an end user is supported only by a single BS and itsinformation is owned locally. From the practicality perspective,CB-CoMP is preferred over JT-CoMP since it requires muchless signal communication overhead.

25

It is useful to examine the merits and challenges of in-corporating SWIPT techniques into CoMP, where full-scalecooperations can reduce the total transmit power. However,an enormous backhaul capacity would be needed to integrateCoMP with SWIPT if all BSs and end users are involvedin the energy transfer and information sharing. Interferencemanagement and mitigation also needs to be dealt with insuch a system.

3) Robust Designs of SWIPT Systems: Information trans-mission and EH process can have dynamic and time-varyingcharacteristics, rendering difficulties for the transmitters toobtain the accurate CSI ahead of time. Offline ahead-of-timetransmission policies with imprecise CSI may cause a largeoutage rate of the system. It is a necessary and challengingtask to design robust beamforming schemes to cope with thedynamics of SWIPT systems.

F. Energy Trading and Planning

As discussed in Sections VIII, IX, and X, the most popularoptimization metrics of smart grid-powered wireless commu-nications networks are energy consumption and operationalcost, followed by operators’ utility and QoS. The Lagrangedual based method, multi-stage optimization, and game the-ory are among the most widely used techniques for offlinescheduling, while SGD and Lyapunov-based algorithms aretypically applied to online optimization problems, as collatedin Table II. The Lagrange dual based technique provides globaloptimality for convex problems, while the Lyapunov-basedalgorithms are asymptotically optimal for online optimizationsunder i.i.d. environments. The following are some interestingresearch directions to be further pursued.

1) Energy Harvesting at End Users: Many existing worksfocus on EH-based BSs and the energy consumption in thedownlink. Mobile end users also need to consume energy forinformation reception, signal decoding, and communicationin the uplink. To fully realize self-sustained communicationsystems, new frameworks need to be in place to integratethe EH capabilities for the end users in system design. Insuch a system, it is important to carry out grouping amongend users for energy sharing, power allocation, interferencealignment, QoS enhancement, and efficiency improvement.The information exchange and computational complexity inthe system can increase dramatically with the growth ofscheduling variables and dimensions. Distributed optimizationtechniques can help alleviate the complexity of the networkand protect private information of each user, and deserveresearch effort.

2) Application of Machine Learning: ML has been oneof the most active research fields due to its great success inmany domains, such as pattern recognition and computationallearning theory in artificial intelligence. It develops algorithmsto learn from the past and make predictions in complicatedscenarios [248]. ML can be widely applied to model andanalyze technical problems of 5G/B5G networks. In particular,5G/B5G smart end users are expected to access differentspectral bands autonomously with the aid of learning schemesof spectral efficiency. Future Internet service providers are

expected to control their transmit power and adjust theirtransmit protocols with the help of QoS learning. Futuresmart grid-powered BSs are expected to schedule and allocatetheir power according to the demand side response from theusers, and carry out online load sharing and traffic offloadingbased on dynamic user locations, QoS requirements, and EHamounts.

Typical supervised learning methodologies depend on pre-existing models and labels that can support the extrapolationof unknown parameters [254], [255]. They are applicable tochannel estimation, data analytics, spectrum assignment, andEH amount prediction. They can also be applied to extrapolatelocations and behaviors of mobile end users, which can helpenhance users’ QoS. Unsupervised learning depends upon theinput statistics in a heuristic fashion [256], [257]. It can beleveraged for dynamic BS or cell clustering in collaborativesystems, the association of APs in ubiquitously availing WiFienvironments, and load balancing in HetNets. Reinforcementlearning depends on a dynamic iterative learning and decision-making process [258], [259]. It may potentially be utilized tomodel the EH process as a Markov decision process withouthistorical data. It can be applied to infer the decisions made byusers under unknown network conditions. It can also facilitatethe solution for the game theoretic problems between smartgrid, BSs, and users. Right now, uses of these learning tech-niques into the EH powered communication network designare almost an open topic.

XIII. CONCLUSION

We have provided a contemporary and comprehensive sur-vey on recent breakthroughs on the utilization, redistribution,trading and planning of energy harvested in future wirelesscommunication networks connected to smart grids. We havereviewed a wide range of energy harvesting-based wire-less architectures such as point-to-point, multipoint-to-point,multipoint-to-multipoint, multi-hop, and multi-cell architec-tures, with an emphasis on their energy-efficient operations.We have also gone through the SWIPT technologies as anextension of energy harvesting wireless networks. A significantpart of the article has been devoted to the redistributionredundant energy within wireless networks, predictive plan-ning and purchase of energy from smart grids, and two-way trading of energy between the wireless networks andsmart grids. By redistributing and trading redundant energy,wireless service providers can reduce their electricity bills andthe consumption of brown energy. Future research topics oneach of these different aspects of energy harvesting wirelessnetworks and their interoperability with smart grids have alsobeen discussed.

REFERENCES

[1] X. Chen, W. Ni, T. Chen, I. B. Collings, X. Wang, and G. B. Giannakis,“Real-time energy trading and future planning for fifth-generationwireless communications,” IEEE Wireless Commun., vol. 24, no. 4,pp. 24–30, Aug. 2017.

[2] “The 1000x data challenge,” Qualcomm, Tech. Rep., Nov. 2013.[Online]. Available: http://www.qualcomm.com/1000x

26

[3] S. Buzzi, I. Chih-Lin, T. E. Klein, H. V. Poor, C. Yang, and A. Zap-pone, “A survey of energy-efficient techniques for 5G networks andchallenges ahead,” IEEE J. Sel. Areas Commun., vol. 34, no. 4, pp.697–709, Apr. 2016.

[4] “NGMN alliance 5G white paper,” 2015. [Online]. Available:https://www.ngmn.org/5g-white-paper/5g-white-paper.html

[5] J. G. Andrews, S. Buzzi, C. Wan, S. V. Hanly, A. Lozano, A. C. K.Soong, and J. C. Zhang, “What will 5G be?” IEEE J. Sel. AreasCommun., vol. 32, no. 6, pp. 1065–1082, Jun. 2014.

[6] The GreenTouch Project, 2016. [Online]. Available: http://www.greentouch.org

[7] GreenTouch Foundation, “Reducing the net energy consumption incommunications networks by up to 98% by 2020,” Tech. Rep., 2015.[Online]. Available: http://www.greentouch.org

[8] Huawei, “Mobile networks go green.” [On-line]. Available: http://www.huawei.com/en/about-huawei/publications/communicate/hw-082734.htm

[9] “Sustainable energy use in mobile communications,” Ericsson Inc.,Stockholm, Sweden, Tech. paper, Aug. 2017.

[10] E-Plus, “Nokia siemens networks build germany’s firstoff-grid base station,” Nokia Siemens Networks BuildGermany’s First Off-Grid Base Station. [Online]. Available:http://networks.nokia.com/newsevents/press-room/press-releases/e-plus-nokia-siemens-networks-buildgermany-s-first-off-grid-base-station

[11] S. Bi, Z. Fang, X. Yuan, and X. Wang, “Joint base station activationand coordinated downlink beamforming for HetNets: Efficient optimaland suboptimal algorithms,” IEEE Trans. Veh. Tech., vol. 68, no. 4, pp.3702–3712, Apr. 2019.

[12] H. A. Inan and A. Ozgur, “Online power control for the energyharvesting multiple access channel,” in Proc. WiOpt, Tempe, AZ, USA,2016, pp. 1–6.

[13] X. Chen, W. Ni, X. Wang, and Y. Sun, “Optimal quality-of-servicescheduling for energy-harvesting powered wireless communications,”IEEE Trans. Wireless Commun., vol. 15, no. 5, pp. 3269–3280, 2016.

[14] ——, “Provisioning quality-of-service to energy harvesting wirelesscommunications,” IEEE Commun. Mag., vol. 53, no. 4, pp. 102–109,2015.

[15] X. Fang, S. Misra, G. Xue, and D. Yang, “Smart grid–the new andimproved power grid: A survey,” IEEE Commun. Surveys Tuts., vol. 14,no. 4, pp. 944–980, 2012.

[16] J. Xu, L. Duan, and R. Zhang, “Cost-aware green cellular networkswith energy and communication cooperation,” IEEE Commun. Mag.,vol. 53, no. 5, pp. 257–263, Nov. 2015.

[17] X. Lu, D. Niyato, and P. Wang, “Power management for wirelessbase station in smart grid environment: Modeling and optimization,”invited chapter in Green Radio Communication Networks, pp. 209–235,Cambridge University Press, 2012.

[18] Y. Mao, Y. Luo, J. Zhang, and K. B. Letaief, “Energy harvesting smallcell networks: feasibility, deployment, and operation,” IEEE Commun.Mag., vol. 53, no. 6, pp. 94–101, 2015.

[19] D. Gunduz, K. Stamatiou, N. Michelusi, and M. Zorzi, “Designingintelligent energy harvesting communication systems,” IEEE Commun.Mag., vol. 52, no. 1, pp. 210–216, 2014.

[20] O. Ozel, K. Tutuncuoglu, S. Ulukus, and A. Yener, “Fundamental limitsof energy harvesting communications,” IEEE Commun. Mag., vol. 53,no. 4, pp. 126–132, 2015.

[21] S. Zhang, Q. Wu, S. Xu, and G. Y. Li, “Fundamental green tradeoffs:Progresses, challenges, and impacts on 5G networks,” IEEE Commun.Surveys Tuts., vol. 19, no. 1, pp. 33–56, 2017.

[22] Y. Alsaba, S. K. A. Rahim, and C. Y. Leow, “Beamforming in wirelessenergy harvesting communications systems: A survey,” IEEE Commun.Surveys Tuts., vol. 20, no. 2, pp. 1329–1360, 2018.

[23] M. L. Ku, L. Wei, C. Yan, and K. J. R. Liu, “Advances in energyharvesting communications: Past, present, and future challenges,” IEEECommun. Surveys Tuts., vol. 18, no. 2, pp. 1384–1412, 2017.

[24] X. Lu, P. Wang, D. Niyato, I. K. Dong, and Z. Han, “Wireless networkswith RF energy harvesting: A contemporary survey,” IEEE Commun.Surveys Tuts., vol. 17, no. 2, pp. 757–789, 2017.

[25] S. Sudevalayam and P. Kulkarni, “Energy harvesting sensor nodes:Survey and implications,” IEEE Commun. Surveys Tuts., vol. 13, no. 3,pp. 443–461, 2011.

[26] R. V. Prasad, S. Devasenapathy, V. S. Rao, and J. Vazifehdan, “Reincar-nation in the ambiance: Devices and networks with energy harvesting,”IEEE Commun. Surveys Tuts., vol. 16, no. 1, pp. 195–213, 2014.

[27] T. D. Perera, D. N. K. Jayakody, S. K. Sharma, S. Chatzinotas, andJ. Li, “Simultaneous wireless information and power transfer (swipt):

Recent advances and future challenges,” IEEE Commun. Surveys Tuts.,vol. 20, no. 1, pp. 264–302, 2018.

[28] M. Erol-Kantarci and H. T. Mouftah, “Energy-efficient informationand communication infrastructures in the smart grid: A survey oninteractions and open issues,” IEEE Commun. Surveys Tuts., vol. 17,no. 1, pp. 179–197, 2015.

[29] M. H. Rehmani, A. Davy, B. Jennings, and C. Assi, “Software definednetworks based smart grid communication: A comprehensive survey,”IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2637–2670, 2019.

[30] J. Taneja, J. Jeong, and D. Culler, “Design, modeling, and capacityplanning for micro-solar power sensor networks,” in InternationalConference on Information Processing in Sensor Networks, 2008.

[31] V. Raghunathan, A. Kansal, J. Hsu, J. Friedman, and M. B. Srivastava,“Design considerations for solar energy harvesting wireless embeddedsystems,” in International Symposium on Information Processing inSensor Networks, 2005.

[32] C. Park and P. H. Chou, “AmbiMax: Autonomous energy harvestingplatform for multi-supply wireless sensor nodes,” in 3rd Annual IEEECommunications Society on Sensor and Ad Hoc Communications andNetworks, vol. 1, Sep. 2006, pp. 168–177.

[33] M. A. Weimer, T. S. Paing, and R. A. Zane, “Remote area windenergy harvesting for low-power autonomous sensors,” in IEEE PowerElectronics Specialists Conference, Jun. 2006, pp. 1–5.

[34] P. Cuffe, P. Smith, and A. Keane, “Transmission system impact of windenergy harvesting networks,” IEEE Trans. Sustain. Energy, vol. 3, no. 4,pp. 643–651, 2012.

[35] S. Kim, R. Vyas, J. Bito, K. Niotaki, A. Collado, A. Georgiadis, andM. M. Tentzeris, “Ambient rf energy-harvesting technologies for self-sustainable standalone wireless sensor platforms,” Proceedings of theIEEE, vol. 102, no. 11, pp. 1649–1666, Nov. 2014.

[36] C. R. Valenta and G. D. Durgin, “Harvesting wireless power: Surveyof energy-harvester conversion efficiency in far-field, wireless powertransfer systems,” IEEE Microwave Magazine, vol. 15, no. 4, pp. 108–120, Jun. 2014.

[37] X. Lu and S. H. Yang, “Thermal energy harvesting for WSNs,” in IEEEInternational Conference on Systems Man & Cybernetics, 2010.

[38] V. Leonov, “Thermoelectric energy harvesting of human body heat forwearable sensors,” IEEE Sensors J., vol. 13, no. 6, pp. 2284–2291, Jun.2013.

[39] ——, “Thermoelectric energy harvesting of human body heat forwearable sensors,” IEEE Sensors J., vol. 13, no. 6, pp. 2284–2291,2013.

[40] G. Huang, R. Umaz, U. Karra, B. Li, and W. Lei, “A biomass-basedmarine sediment energy harvesting system,” in IEEE InternationalSymposium on Low Power Electronics & Design, 2013.

[41] X. Jiang, J. Polastre, and D. Culler, “Perpetual environmentally pow-ered sensor networks,” in International Symposium on InformationProcessing in Sensor Networks, 2005.

[42] P. Stanley-Marbell and D. Marculescu, “An 0.9 × 1.2”, low power,energy-harvesting system with custom multi-channel communicationinterface,” in Design, Automation & Test in Europe Conference &Exhibition, 2007.

[43] M. Pinuela, D. Yates, P. Mitcheson, and S. Lucyszyn, “Maximizingthe link efficiency of resonant inductive coupling for wireless powertransfer,” in International Workshop on Wireless Energy Transport andHarvesting, Eindhoven, the Netherlands, 2011.

[44] K. Andre, K. Aristeidis, M. Robert, J. D. Joannopoulos, F. Peter,and S. Marin, “Wireless power transfer via strongly coupled magneticresonances,” Science, vol. 317, no. 5834, pp. 83–86, 2007.

[45] Z. Fang, X. Yuan, and X. Wang, “Distributed energy beamforming forsimultaneous wireless information and power transfer in the two-wayrelay channel,” IEEE Signal Process. Lett., vol. 22, no. 6, pp. 656–660,2015.

[46] Y. Zhang, N. Gatsis, and G. B. Giannakis, “Robust energy managementfor microgrids with high-penetration renewables,” IEEE Trans. Sustain.Energy, vol. 4, no. 4, pp. 944–953, Oct. 2013.

[47] Y. Chen and A. D. Domınguez-Garcıa, “A method to study the effectof renewable resource variability on power system dynamics,” IEEETrans. Power Syst., vol. 27, no. 4, pp. 1978–1989, Nov. 2012.

[48] D. Bertsimas, D. B. Brown, and C. Caramanis, “Theory and applica-tions of robust optimization,” SIAM Review, vol. 53, no. 3, pp. 464–501,Aug. 2011.

[49] T. Yeh and L. Wang, “A study on generator capacity for wind turbinesunder various tower heights and rated wind speed using Weibulldistribution,” IEEE Trans. Energy Conversion, vol. 23, no. 2, pp. 592–602, Jun. 2008.

27

[50] S. Wen, H. Lan, Q. Fu, D. Yu, and L. Zhang, “Economic allocationfor energy storage system considering wind power distribution,” IEEETrans. Power Syst., vol. 30, no. 2, pp. 644–652, Mar. 2015.

[51] A. Abdulkarim, S. M. Abdelkader, and D. J. Morrow, “Statisticalanalyses of wind and solar energy resources for the development ofhybrid microgrid,” in 2nd Intl. Congress on Energy Efficiency andEnergy Related Materials (ENEFM2014). Springer, 2015, pp. 9–14.

[52] H. Ghazzai and A. Kadri, “Joint demand-side management in smartgrid for green collaborative mobile operators under dynamic pricingand fairness setup,” IEEE Trans. Green Commun. & Netw., vol. 1,no. 1, pp. 74–88, Mar. 2017.

[53] A. Seyedi and B. Sikdar, “Modeling and analysis of energy harvestingnodes in body sensor networks,” in International Summer School &Symposium on Medical Devices & Biosensors, 2008.

[54] B. Medepally, N. B. Mehta, and C. R. Murthy, “Implications of energyprofile and storage on energy harvesting sensor link performance,” inIEEE Conference on Global Telecommunications, 2009.

[55] L. Jing, R. Yates, and L. Greenstein, “A generic model for optimizingsingle-hop transmission policy of replenishable sensors,” IEEE Trans.Wireless Commun., vol. 8, no. 2, pp. 547–551, 2009.

[56] C. K. Ho, P. D. Khoa, and C. M. Pang, “Markovian models forharvested energy in wireless communications,” in IEEE InternationalConference on Communication Systems, 2010.

[57] P. Lee, A. E. Zhi, M. Han, and H. P. Tan, “Empirical modeling of asolar-powered energy harvesting wireless sensor node for time-slottedoperation,” in Wireless Communications & Networking Conference,2011.

[58] L. Hou, S. Tan, Z. Zhang, and N. W. Bergmann, “Thermal energyharvesting WSNs node for temperature monitoring in IIoT,” IEEEAccess, vol. 6, pp. 35 243–35 249, 2018.

[59] P. Kar, A. Roy, S. Misra, and M. S. Obaidat, “On the effects ofcommunication range shrinkage of sensor nodes in mobile wirelesssensor networks due to adverse environmental conditions,” IEEE Syst.J., vol. 12, no. 3, pp. 2048–2055, 2018.

[60] A. Somov, A. Baranov, D. Spirjakin, and R. Passerone, “Circuit designand power consumption analysis of wireless gas sensor nodes: One-sensor versus two-sensor approach,” IEEE Sens. J., vol. 14, no. 6, pp.2056–2063, 2014.

[61] L. Karim and N. Nasser, “Reliable location-aware routing protocol formobile wireless sensor network,” IET Communications, vol. 6, no. 14,pp. 2149–2158, 2012.

[62] S. Boyd and L. Vandenberghe, Convex Optimization. New York, NY,USA: Cambridge Univ. Press, 2004.

[63] D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed.Athena Scientific, 2006.

[64] O. Ozel, K. Tutuncuoglu, J. Yang, S. Ulukus, and A. Yener, “Transmis-sion with energy harvesting nodes in fading wireless channels: Optimalpolicies,” IEEE J. Sel. Areas Commun., vol. 29, no. 8, pp. 1732–1743,2011.

[65] K. Tutuncuoglu and A. Yener, “Optimum transmission policies for bat-tery limited energy harvesting nodes,” IEEE Trans. Wireless Commun.,vol. 11, no. 3, pp. 1180–1189, 2012.

[66] Q. Bai, J. Li, and J. A. Nossek, “Throughput maximizing transmissionstrategy of energy harvesting nodes,” in Third International Workshopon Cross Layer Design, 2011, pp. 1–5.

[67] J. Xu and R. Zhang, “Throughput optimal policies for energy harvestingwireless transmitters with non-ideal circuit power,” IEEE J. Sel. AreasCommun., vol. 32, no. 2, pp. 322–332, 2012.

[68] F. Yuan, Q. Zhang, S. Jin, and H. Zhu, “Optimal harvest-use-storestrategy for energy harvesting wireless systems,” IEEE Trans. WirelessCommun., vol. 14, no. 2, pp. 698–710, Feb. 2015.

[69] X. Chen, X. Wang, and X. Zhou, “Energy-harvesting powered trans-missions of delay-limited data packets,” in Proc. GLOBECOM, Austin,TX, Dec. 2014, pp. 2550–2555.

[70] J. Yang and S. Ulukus, “Optimal packet scheduling in an energyharvesting communication system,” IEEE Trans. Commun., vol. 60,no. 1, pp. 220–230, 2012.

[71] O. Orhan, D. Gunduz, and E. Erkip, “Throughput maximization foran energy harvesting communication system with processing cost,” inInformation Theory Workshop, 2012, pp. 84–88.

[72] P. Blasco, D. Gunduz, and M. Dohler, “A learning theoretic approach toenergy harvesting communication system optimization,” IEEE Trans.Wireless Commun., vol. 12, no. 4, pp. 1872–1882, 2013.

[73] X. Chen, X. Wang, and Y. Sun, “Energy-harvesting powered trans-missions of bursty data packets with strict deadlines,” in Proc. ICC,Sydney, Australia, June 2014, pp. 4060–4065.

[74] J. Gong, S. Zhou, and Z. Niu, “Optimal power allocation for energyharvesting and power grid coexisting wireless communication systems,”IEEE Trans. Commun., vol. 61, no. 7, pp. 3040–3049, Jul. 2013.

[75] X. Kang, Y.-K. Chia, C. K. Ho, and S. Sun, “Cost minimization forfading channels with energy harvesting and conventional energy,” IEEETrans. Wireless Commun., vol. 13, no. 8, pp. 4586–4598, Aug. 2014.

[76] I. Ahmed, A. Ikhlef, D. W. K. Ng, and R. Schober, “Power allocationfor an energy harvesting transmitter with hybrid energy sources.” IEEETrans. Wireless Commun., vol. 12, no. 12, pp. 6255–6267, Dec. 2013.

[77] Y. Cui, V. K. Lau, and F. Zhang, “Grid power-delay tradeoff for en-ergy harvesting wireless communication systems with finite renewableenergy storage,” IEEE J. Sel. Areas Commun., vol. 33, no. 8, pp. 1651–1666, Aug. 2015.

[78] O. Ozel, J. Yang, and S. Ulukus, “Optimal broadcast scheduling foran energy harvesting rechargeable transmitter with a finite capacitybattery,” IEEE Trans. Wireless Commun., vol. 11, no. 6, pp. 2193–2203, 2013.

[79] P. Youssef-Massaad, L. Zheng, and M. Medard, “Bursty transmissionand glue pouring: on wireless channels with overhead costs,” IEEETrans. Wireless Commun., vol. 7, no. 12, pp. 5188–5194, 2008.

[80] M. A. Zafer and E. Modiano, “A calculus approach to minimumenergy transmission policies with quality of service guarantees,” inProc. INFOCOM 2005, vol. 1, 2005, pp. 548–559.

[81] X. Wang and Z. Li, “Energy-efficient transmissions of bursty datapackets with strict deadlines over time-varying wireless channels,”IEEE Trans. Wireless Commun., vol. 12, no. 5, pp. 2533–2543, 2013.

[82] Z. Nan, T. Chen, X. Wang, and W. Ni, “Energy-efficient transmissionschedule for delay-limited bursty data arrivals under nonideal circuitpower consumption,” IEEE Trans. Veh. Tech., vol. 65, no. 8, pp. 6588–6600, 2014.

[83] M. J. Neely, “Stochastic network optimization with application tocommunication and queueing systems,” Synthesis Lectures on Com-munication Networks, vol. 3, no. 1, p. 211, 2010.

[84] R. Bellman, “A Markovian decision process,” J. Mathematics & Me-chanics, vol. 6, no. 5, pp. 679–684, 1957.

[85] J. Yang and S. Ulukus, “Optimal packet scheduling in a multipleaccess channel with energy harvesting transmitters,” J. Commun. Netw.,vol. 14, no. 2, pp. 140–150, Apr. 2012.

[86] Z. Wang, V. Aggarwal, and X. Wang, “Iterative dynamic water-fillingfor fading multiple-access channels with energy harvesting,” IEEE J.Sel. Areas Commun., vol. 33, no. 3, pp. 382–395, 2014.

[87] A. Baknina and S. Ulukus, “Energy harvesting multiple access chan-nels: Optimal and near-optimal online policies,” IEEE Trans. Commun.,vol. 66, no. 7, pp. 2904–2917, July 2018.

[88] N. Michelusi and M. Zorzi, “Optimal adaptive random multiaccess inenergy harvesting wireless sensor networks,” IEEE Trans. Commun.,vol. 63, no. 4, pp. 1355–1372, 2015.

[89] J. Zheng, H. Zhang, Y. Cai, R. Li, and A. Anpalagan, “Game-theoreticmulti-channel multi-access in energy harvesting wireless sensor net-works,” IEEE Sensors Journal, vol. 16, no. 11, pp. 4587–4594, 2016.

[90] F. Iannello, O. Simeone, and U. Spagnolini, “Medium access controlprotocols for wireless sensor networks with energy harvesting,” IEEETrans. Commun., vol. 60, no. 5, pp. 1381–1389, 2012.

[91] B. Gurakan, O. Kaya, and S. Ulukus, “Energy and data cooperativemultiple access channel with intermittent data arrivals,” IEEE Trans.Wireless Commun., vol. 17, no. 3, pp. 2016–2028, Mar. 2018.

[92] J. Jeon and A. Ephremides, “On the stability of random multipleaccess with stochastic energy harvesting,” IEEE J. Sel. Areas Commun.,vol. 33, no. 3, pp. 571–584, Mar. 2015.

[93] S. Hu, X. Wang, Y. Zhang, and G. B. Giannakis, “Optimal resourceallocation for smart-grid powered MIMO broadcast channels,” in Intl.Conf. Wireless Commun. & Signal Process., Nanjing, China, Oct. 2015,pp. 1–5.

[94] S. Hu, Y. Zhang, X. Wang, and G. B. Giannakis, “Weighted sum-ratemaximization for MIMO downlink systems powered by renewables,”IEEE Trans. Wireless Commun., vol. 15, no. 8, pp. 5615–5625, 2016.

[95] N. Jindal, S. Vishwanath, and A. Goldsmith, “On the duality ofgaussian multiple-access and broadcast channels,” IEEE Trans. Inf.Theory, vol. 50, no. 5, pp. 768–783, 2004.

[96] R. M. Loynes, “The stability of a queue with non-independent inter-arrival and service times,” in Math. Proc. Cambridge Philos. Soc.,vol. 58, no. 3. Cambridge University Press, 1962, pp. 497–520.

[97] X. Wang, T. Chen, X. Chen, X. Zhou, and G. B. Giannakis, “Dynamicresource allocation for smart-grid powered MIMO downlink transmis-sions,” IEEE J. Sel. Areas Commun., vol. 34, no. 12, pp. 3354–3365,Dec. 2016.

28

[98] J. Lee, Y. Kim, H. Lee, B. L. Ng, D. Mazzarese, J. Liu, W. Xiao, andY. Zhou, “Coordinated multipoint transmission and reception in lte-advanced systems,” IEEE Commun. Mag., vol. 50, no. 11, pp. 44–50,Nov. 2012.

[99] A. Ruszczynski and W. Syski, “On convergence of the stochasticsubgradient method with on-line stepsize rules,” J. Math. Analysis andApplications, vol. 114, no. 2, pp. 512–527, 1986.

[100] Y. Nesterov, “Primal-dual subgradient methods for convex problems,”Math. Program., vol. 120, no. 1, pp. 221–259, 2009.

[101] X. Wang, N. Fan, and P. M. Pardalos, “Stochastic subgradient descentmethod for large-scale robust chance-constrained support vector ma-chines,” Optimization Letters, pp. 1–12, 2017.

[102] K. C. Kiwiel, “Convergence of approximate and incremental subgradi-ent methods for convex optimization,” SIAM J. Optim., vol. 14, no. 3,pp. 807–840, 2004.

[103] Y. Nesterov, “Complexity estimates of some cutting plane methodsbased on the analytic barrier,” Math. Program., vol. 69, no. 1, pp.149–176, Jul. 1995.

[104] J. Goffin and J. Vial, “Convex nondifferentiable optimization: A surveyfocused on the analytic center cutting plane method,” OptimizationMethods and Software, vol. 17, no. 5, pp. 805–867, 2002.

[105] Y. Lee, A. Sidford, and C. Wong, “A faster cutting plane methodand its implications for combinatorial and convex optimization,” inIEEE Annual Symposium on Foundations of Computer Science (FOCS),Berkeley, CA, USA, Oct. 2015, pp. 1049–1065.

[106] K. C. Kiwiel, “Approximation in proximal bundle methods and decom-position of convex programs,” J. Optim. and Appl., vol. 84, no. 3, pp.529–548, Mar. 1995.

[107] S. Feltenmark and K. C. Kiwiel, “Dual application of proximal bundlemethods, including Lagrange relaxation of nonconvex problems,” SIAMJ. Optim., vol. 10, no. 3, pp. 697–721, Feb./Mar. 2000.

[108] K. C. Kiwiel, “A proximal bundle method with approximate subgra-dient linearizations,” SIAM J. Optim., vol. 16, no. 4, pp. 1007–1023,2006.

[109] L. Xiao and S. Boyd, “Simultaneous routing and resource allocationvia dual decomposition,” IEEE Trans. Commun., vol. 52, no. 7, pp.1136–1144, July 2004.

[110] J. Xu and R. Zhang, “Cooperative energy trading in CoMP systemspowered by smart grids,” IEEE Trans. Veh. Tech., vol. 65, no. 4, pp.2142–2153, Apr. 2016.

[111] W. Yu and T. Lan, “Transmitter optimization for the multi-antennadownlink with per-antenna power constraints,” IEEE Trans. SignalProcess., vol. 55, no. 6, pp. 2646–2660, 2007.

[112] A. Wiesel, Y. C. Eldar, and S. Shamai, “Linear precoding via conicoptimization for fixed MIMO receivers,” IEEE Trans. Signal Process.,vol. 54, no. 1, pp. 161–176, 2006.

[113] S. Boyd, “Convex optimization II,” [Online]. Available:http://www.stanford.edu/class/ee364b/lectures.html, Stanford Univ.,Stanford, CA, USA.

[114] X. Wang, Y. Zhang, G. B. Giannakis, and S. Hu, “Robust smart-grid-powered cooperative multipoint systems,” IEEE Trans. WirelessCommun., vol. 14, no. 11, pp. 6188–6199, 2015.

[115] Y. Zhang, N. Gatsis, and G. B. Giannakis, “Robust distributed energymanagement for microgrids with renewables,” in IEEE Third Intl. Conf.on Smart Grid Commun., 2012.

[116] Z.-Q. Luo, W.-K. Ma, A. M.-C. So, Y. Ye, and S. Zhang, “Semidefiniterelaxation of quadratic optimization problems,” IEEE Signal Process.Mag., vol. 27, no. 3, pp. 20–34, 2010.

[117] I. Polik and T. Terlaky, “A survey of the s-lemma,” SIAM Rev., vol. 49,no. 3, pp. 371–418, 2007.

[118] D. P. Bertsekas, Nonlinear Programming. Belmont, MA: AthenaScientific, 1999.

[119] D. P. Bertsekas, A. Nedic, and A. Ozdaglar, Convex Analysis andOptimization. Belmont, MA: Athena Scientific, 2003.

[120] N. Gatsis and G. B. Giannakis, “Residential load control: Distributedscheduling and convergence with lost AMI message,” IEEE Trans.Smart Grid, vol. 3, no. 2, pp. 770–786, Jun. 2012.

[121] X. Wang, Y. Zhang, T. Chen, and G. B. Giannakis, “Dynamic energymanagement for smart-grid-powered coordinated multipoint systems,”IEEE J. Sel. Areas Commun., vol. 34, no. 5, pp. 1348–1359, 2016.

[122] R. Urgaonkar, B. Urgaonkar, M. Neely, and A. Sivasubramaniam,“Optimal power cost management using stored energy in data centers,”in Proc. ACM SIGMETRICS, San Jose, CA, Jun. 2011, pp. 221–232.

[123] S. Lakshminaryana, H. V. Poor, and T. Quek, “Cooperation and storagetrade-offs in power grids with renewable energy resources,” IEEE J.Sel. Areas Commun., vol. 32, no. 7, pp. 1–12, Jul. 2014.

[124] X. Wang, X. Chen, T. Chen, L. Huang, and G. B. Giannakis, “Two-scalestochastic control for integrated multipoint communication systemswith renewables,” IEEE Trans. Smart Grid, vol. 9, no. 3, pp. 1822–1834, May 2018.

[125] Y. Luo, J. Zhang, and K. B. Letaief, “Optimal scheduling and powerallocation for two-hop energy harvesting communication systems,”IEEE Trans. Wireless Commun., vol. 12, no. 9, pp. 4729–4741, Sep.2013.

[126] B. Devillers and D. Gunduz, “A general framework for the optimizationof energy harvesting communication systems with battery imperfec-tions,” J. Commun. Netw., vol. 14, no. 2, pp. 130–139, Apr. 2012.

[127] C. Huang, R. Zhang, and S. Cui, “Throughput maximization for thegaussian relay channel with energy harvesting constraints,” IEEE J.Sel. Areas Commun., vol. 31, no. 8, pp. 1469–1479, Aug. 2013.

[128] N. Pappas, M. Kountouris, J. Jeon, A. Ephremides, and A. Traganitis,“Effect of energy harvesting on stable throughput in cooperative relaysystems,” J. Commun. Netw., vol. 18, no. 2, pp. 261–269, Apr. 2016.

[129] B. Gurakan, O. Ozel, and S. Ulukus, “Optimal energy and data routingin networks with energy cooperation,” IEEE Trans. Wireless Commun.,vol. 15, no. 2, pp. 857–870, Feb. 2016.

[130] Z. Ding, S. M. Perlaza, I. Esnaola, and H. V. Poor, “Power allocationstrategies in energy harvesting wireless cooperative networks,” IEEETrans. Wireless Commun., vol. 13, no. 2, pp. 846–860, 2014.

[131] L. Liu, R. Zhang, and K.-C. Chua, “Wireless information and powertransfer: A dynamic power splitting approach,” IEEE Trans. Commun.,vol. 61, no. 9, pp. 3990–4001, 2013.

[132] C. Xiong, L. Lu, and G. Y. Li, “Energy efficiency tradeoff in downlinkand uplink TDD OFDMA with simultaneous wireless information andpower transfer,” in Communications (ICC), 2014 IEEE InternationalConference on. IEEE, 2014, pp. 5383–5388.

[133] H. Ju and R. Zhang, “Throughput maximization in wireless poweredcommunication networks,” IEEE Trans. Wireless Commun., vol. 13,no. 1, pp. 418–428, 2014.

[134] J. Tang, J. Luo, M. Liu, D. K. C. So, and J. Chambers, “Energyefficiency optimization for NOMA with SWIPT,” IEEE J. Sel. TopicsSignal Process., vol. 13, no. 3, pp. 452–466, Jun. 2019.

[135] C. Isheden, Z. Chong, E. Jorswieck, and G. Fettweis, “Framework forlink-level energy efficiency optimization with informed transmitter,”IEEE Trans. Wireless Commun., vol. 11, no. 8, pp. 2946–2957, 2011.

[136] S. Timotheou, I. Krikidis, G. Zheng, and B. Ottersten, “Beamformingfor MISO interference channels with QoS and RF energy transfer,”IEEE Trans. Wireless Commun., vol. 13, no. 5, pp. 2646–2658, 2014.

[137] Q. Shi, L. Liu, W. Xu, and R. Zhang, “Joint transmit beamformingand receive power splitting for MISO SWIPT systems,” IEEE Trans.Wireless Commun., vol. 13, no. 6, pp. 3269–3280, 2014.

[138] Z. Zong, H. Feng, F. R. Yu, N. Zhao, T. Yang, and B. Hu, “Optimaltransceiver design for SWIPT in K-user MIMO interference channels,”IEEE Trans. Wireless Commun., vol. 15, no. 1, pp. 430–445, 2016.

[139] Q. Li, Q. Zhang, and J. Qin, “Beamforming for information and energycooperation in cognitive non-regenerative two-way relay networks,”IEEE Trans. Wireless Commun., vol. 15, no. 8, pp. 5302–5313, 2016.

[140] F. Wang, T. Peng, Y. Huang, and X. Wang, “Robust transceiver opti-mization for power-splitting based downlink MISO SWIPT systems,”IEEE Signal Process. Letters, vol. 22, no. 9, pp. 1492–1496, 2015.

[141] S. Wang and B. Wang, “Robust secure transmit design in MIMOchannels with simultaneous wireless information and power transfer,”IEEE Signal Process. Letters, vol. 22, no. 11, pp. 2147–2151, 2015.

[142] S. He, Y. Huang, W. Chen, S. Jin, H. Wang, and L. Yang, “Energyefficient coordinated precoding design for a multicell system with RFenergy harvesting,” EURASIP J. Wireless Commun. and Netw., vol.2015, no. 1, p. 67, 2015.

[143] C. Zhai, J. Liu, and L. Zheng, “Cooperative spectrum sharing withwireless energy harvesting in cognitive radio networks,” IEEE Trans.Veh. Tech., vol. 65, no. 7, pp. 5303–5316, 2016.

[144] Q. Sun, G. Zhu, C. Shen, X. Li, and Z. Zhong, “Joint beamformingdesign and time allocation for wireless powered communication net-works,” IEEE Commun. Letters, vol. 18, no. 10, pp. 1783–1786, 2014.

[145] L. Liu, R. Zhang, and K.-C. Chua, “Multi-antenna wireless poweredcommunication with energy beamforming,” IEEE Trans. Commun.,vol. 62, no. 12, pp. 4349–4361, 2014.

[146] D. Hwang, D. I. Kim, and T.-J. Lee, “Throughput maximization formultiuser MIMO wireless powered communication networks,” IEEETrans. Veh. Tech., vol. 65, no. 7, pp. 5743–5748, 2016.

[147] C. Qin, W. Ni, H. Tian, and R. P. Liu, “Joint rate maximizationof downlink and uplink in multiuser MIMO SWIPT systems,” IEEEAccess, vol. 5, pp. 3750–3762, 2017.

29

[148] H.-V. Tran, G. Kaddoum, and K. T. Truong, “Resource allocation inSWIPT networks under a non-linear energy harvesting model: Powerefficiency, user fairness, and channel non-reciprocity,” IEEE Trans. Veh.Tech., vol. 67, no. 9, pp. 8466–8480, Sep. 2018.

[149] F. Zhou, C. Zheng, H. Sun, R. Q. Hu, and L. Hanzo, “Artificial noiseaided secure cognitive beamforming for cooperative MISO-NOMAusing SWIPT,” IEEE J. Sel. Areas Commun., vol. 36, no. 4, pp. 918–931, Apr. 2018.

[150] R. Zhang and C. K. Ho, “MIMO broadcasting for simultaneous wirelessinformation and power transfer,” IEEE Trans. Wireless Commun.,vol. 12, no. 5, pp. 1989–2001, May 2013.

[151] C. Qin, W. Ni, H. Tian, R. Liu, and Y. Guo, “Joint beamforming anduser selection in multiuser collaborative MIMO SWIPT systems withnon-negligible circuit energy consumption,” IEEE Trans. Veh. Tech.,Jun. 2017.

[152] T. A. Khan, A. Alkhateeb, and R. W. Heath, “Millimeter wave energyharvesting,” IEEE Trans. Wireless Commun., vol. 15, no. 9, pp. 6048–6062, Sep. 2016.

[153] X. Zhu, W. Zeng, and C. Xiao, “Precoder design for simultaneouswireless information and power transfer systems with finite-alphabetinputs,” IEEE Trans. Veh. Tech., vol. 66, no. 10, pp. 9085–9097, Oct.2017.

[154] C. Xing, N. Wang, J. Ni, Z. Fei, and J. Kuang, “MIMO beamformingdesigns with partial CSI under energy harvesting constraints,” IEEESignal Process. Lett., vol. 20, no. 4, pp. 363–366, 2013.

[155] Y.-K. Chia, S. Sun, and R. Zhang, “Energy cooperation in cellularnetworks with renewable powered base stations,” IEEE Trans. WirelessCommun., vol. 13, no. 12, pp. 6996–7010, Dec. 2014.

[156] J. Gong, J. S. Thompson, S. Zhou, and Z. Niu, “Base station sleepingand resource allocation in renewable energy powered cellular net-works,” IEEE Trans. Commun., vol. 62, no. 11, pp. 3801–3813, Nov.2014.

[157] Y. Che, L. Duan, and R. Zhang, “Dynamic base station operationin large-scale green cellular networks,” IEEE J. Sel. Areas Commun.,vol. 34, no. 12, pp. 3127–3141, Dec. 2016.

[158] M. Sheng, D. Zhai, X. Wang, Y. Li, Y. Shi, and J. Li, “Intelligentenergy and traffic coordination for green cellular networks with hybridenergy supply,” IEEE Trans. Veh. Tech., vol. 66, no. 2, pp. 1631–1646,Feb. 2017.

[159] S. Hu, C. Xu, and X. Wang, “Distributed multicell coordinated beam-forming for cellular networks powered by renewables,” in Proc. IEEEICCS, Shenzhen, China, Dec. 14-16, 2016 (invited).

[160] S. Hu, C. Xu, X. Wang, Y. Huang, and S. Zhang, “A stochasticADMM approach to distributed coordinated multicell beamforming forrenewables powered wireless cellular networks,” IEEE Tans. Veh. Tech.,Sep. 2018.

[161] M. J. Farooq, H. Ghazzai, A. Kadri, H. ElSawy, and M.-S. Alouini, “Ahybrid energy sharing framework for green cellular networks,” IEEETrans. Commun., vol. 65, no. 2, pp. 918–934, Feb. 2017.

[162] H. Ghazzai, E. Yaacoub, M.-S. Alouini, and A. Abu-Dayya, “Optimizedsmart grid energy procurement for LTE networks using evolutionaryalgorithms,” IEEE Trans. Veh. Tech., vol. 63, no. 9, pp. 4508–4519,Nov. 2014.

[163] L. Zhang, Y. Cai, Q. Shi, G. Yu, and G. Y. Li, “Cost-efficient cellularnetworks powered by micro-grids,” IEEE Trans. Wireless Commun.,vol. 16, no. 9, pp. 6047–6061, Sep. 2017.

[164] R. Ramamonjison and V. K. Bhargava, “Energy allocation and coop-eration for energy-efficient wireless two-tier networks,” IEEE Trans.Wireless Commun., vol. 15, no. 9, pp. 6434–6448, Sep. 2016.

[165] R. A. Howard, “Dynamic programming,” Management Science, vol. 12,no. 5, pp. 317–348, 1966.

[166] H. Sakoe and S. Chiba, “Dynamic programming algorithm optimizationfor spoken word recognition,” IEEE Trans. Acoustics, Speech, andSignal Process., vol. 26, no. 1, pp. 43–49, 1978.

[167] A. G. Barto, S. J. Bradtke, and S. P. Singh, “Learning to act usingreal-time dynamic programming,” Artificial Intelligence, vol. 71, no.1-2, pp. 81–138, 1995.

[168] X. Sun, H. Kashima, T. Matsuzaki, and N. Ueda, “Averaged stochasticgradient descent with feedback: An accurate, robust, and fast trainingmethod,” in IEEE Int. Conf. Data Mining, Sydney, NSW, Australia,Dec. 2010.

[169] R. Johnson and T. Zhang, “Accelerating stochastic gradient descentusing predictive variance reduction,” in Intl. Conf. Neural Info. Process.Syst., 2013.

[170] H. Jiang, G. Huang, P. A. Wilford, and L. Yu, “Constrained and pre-conditioned stochastic gradient method,” IEEE Trans. Signal Process.,vol. 63, no. 10, pp. 2678–2691, May 2015.

[171] J. Konecny, J. Liu, P. Richtarik, and M. Takac, “Mini-batch semi-stochastic gradient descent in the proximal setting,” IEEE J. Sel. TopicsSignal Process., vol. 10, no. 2, pp. 242–255, 2016.

[172] C. Ren, X. Lyu, W. Ni, H. Tian, and R. Liu, “Distributed online learningof fog computing under nonuniform device cardinality,” IEEE IoT J.,vol. 6, no. 1, pp. 1147–1159, Feb. 2019.

[173] D. Liu, Y. Chen, K. K. Chai, T. Zhang, and M. Elkashlan, “Two-dimensional optimization on user association and green energy alloca-tion for HetNets with hybrid energy sources.” IEEE Trans. Commun.,vol. 63, no. 11, pp. 4111–4124, Nov. 2015.

[174] J. Xu, L. Duan, and R. Zhang, “Energy group buying with loadingsharing for green cellular networks,” IEEE J. Sel. Areas Commun.,vol. 34, no. 4, pp. 786–799, Apr. 2016.

[175] Y. Mao, J. Zhang, and K. B. Letaief, “A Lyapunov optimizationapproach for green cellular networks with hybrid energy supplies,”IEEE J. Sel. Areas Commun., vol. 33, no. 12, pp. 2463–2477, Dec.2015.

[176] D. Zhai, M. Sheng, X. Wang, and Y. Li, “Leakage-aware dynamicresource allocation in hybrid energy powered cellular networks.” IEEETrans. Commun., vol. 63, no. 11, pp. 4591–4603, Nov. 2015.

[177] Y. Dong, M. J. Hossain, J. Cheng, and V. C. Leung, “Dynamic cross-layer beamforming in hybrid powered communication systems withharvest-use-trade strategy,” IEEE Trans. Wireless Commun., vol. 16,no. 12, pp. 8011–8025, Dec. 2017.

[178] S. Zhang, N. Zhang, S. Zhou, J. Gong, Z. Niu, and X. Shen, “Energy-aware traffic offloading for green heterogeneous networks,” IEEE J.Sel. Areas Commun., vol. 34, no. 5, pp. 1116–1129, 2016.

[179] B. Xu, Y. Chen, J. R. Carrion, and T. Zhang, “Resource allocationin energy-cooperation enabled two-tier NOMA hetnets toward green5G,” IEEE J. Sel. Areas Commun., vol. 35, no. 12, pp. 2758–2770,Dec. 2017.

[180] S. Bu, F. Yu, Y. Cai, and X. Liu, “When the smart grid meets energy-efficient communications: Green wireless cellular networks poweredby the smart grid,” IEEE Trans. Wireless Commun., vol. 11, no. 8, pp.3014–3024, Aug. 2012.

[181] S. Bu and F. R. Yu, “Dynamic energy-efficient resource allocationin cognitive heterogeneous wireless networks with the smart grid,” inIEEE GLOBECOM, 2012, pp. 3032–3036.

[182] S. Bu, F. R. Yu, and Y. Qian, “Energy-efficient cognitive heterogeneousnetworks powered by the smart grid,” in IEEE INFOCOM, 2013, pp.980–988.

[183] D. Li, W. Saad, and C. S. Hong, “Decentralized renewable energypricing and allocation for millimeter wave cellular backhaul,” IEEE J.Sel. Areas Commun., vol. 34, no. 5, pp. 1140–1159, May 2016.

[184] H. Ghazzai, E. Yaacoub, and M. S. Alouin, “A game theoreticalapproach for cooperative environmentally friendly cellular networkspowered by the smart grid,” in IEEE Online Conf. Green Commun.,Taipei, Taiwan, Nov. 2014, pp. 1–6.

[185] A. R. Barron, A. Cohen, W. Dahmen, and R. A. Devore, “Approxima-tion and learning by greedy algorithms,” Annals of Statistics, vol. 36,no. 1, pp. 64–94, 2008.

[186] H. Ouyang, N. He, L. Tran, and A. Gray, “Stochastic alternatingdirection method of multipliers,” in Intl. Conf. Machine Learn., 2013,pp. 80–88.

[187] D. W. K. Ng, E. S. Lo, and R. Schober, “Energy-efficient resourceallocation in OFDMA systems with hybrid energy harvesting basestation,” IEEE Trans. Wireless Commun., vol. 12, no. 7, pp. 3412–3427, Jul. 2013.

[188] W. Zeng, Y. R. Zheng, and C. Xiao, “Energy harvesting transmitterwith finite-alphabet inputs and statistical CSI: Online precoding bydynamic programming,” in Global Communications Conference, 2014,pp. 4306–4311.

[189] ——, “Online precoding for energy harvesting transmitter with finite-alphabet inputs and statistical CSI,” IEEE Trans. Veh. Tech., vol. 65,no. 7, pp. 5287–5302, Jul. 2016.

[190] D. P. Bertsekas, Dynamic Programming and Optimal Control, VolumnI, 3rd ed. Belmont, MA, USA: Athena Scientific, 2005.

[191] Y. Saito, Y. Kishiyama, A. Benjebbour, T. Nakamura, A. Li, andK. Higuchi, “Non-orthogonal multiple access (NOMA) for cellularfuture radio access,” in IEEE 77th Veh. Tech. Conf. (VTC Spring), Jun.2013, pp. 1–5.

[192] Z. Ding, F. Adachi, and H. V. Poor, “The application of MIMO to non-orthogonal multiple access,” IEEE Trans. Wireless Commun., vol. 15,no. 1, pp. 537–552, Jan. 2016.

[193] D. Niyato, P. Wang, D. I. Kim, W. Saad, and Z. Han, “Mobile energysharing networks: Performance analysis and optimization.” IEEE Trans.Veh. Tech., vol. 65, no. 5, pp. 3519–3535, May 2016.

30

[194] A. B. Philpott and E. Pettersen, “Optimizing demand-side bids in day-ahead electricity markets,” IEEE Trans. Power Syst., vol. 21, no. 2, pp.488–498, 2006.

[195] Z. Niu, Y. Wu, J. Gong, and Z. Yang, “Cell zooming for cost-efficientgreen cellular networks,” IEEE Commun. Mag., vol. 48, no. 11, 2010.

[196] A. Goldsmith, S. A. Jafar, I. Maric, and S. Srinivasa, “Breakingspectrum gridlock with cognitive radios: An information theoreticperspective,” Proceedings of the IEEE, vol. 97, no. 5, pp. 894–914,2009.

[197] D. Gesbert, S. Hanly, H. Huang, S. S. Shitz, O. Simeone, and W. Yu,“Multi-cell mimo cooperative networks: A new look at interference,”IEEE J. Sel. Areas Commun., vol. 28, no. 9, pp. 1380–1408, 2010.

[198] R. T. Rockafellar and S. Uryasev, “Conditional value-at-risk for generalloss distributions,” J. Banking & Finance, vol. 26, no. 7, pp. 1443–1471, 2002.

[199] A. G. Quaranta and A. Zaffaroni, “Robust optimization of conditionalvalue at risk and portfolio selection,” J. Banking Finance, vol. 32,no. 10, pp. 2046–2056, 2008.

[200] A. Wiesel, Y. C. Eldar, and S. Shamai, “Zero-forcing precoding andgeneralized inverses,” IEEE Trans. Signal Process., vol. 56, no. 9, pp.4409–4418, 2008.

[201] X. Chen, W. Ni, T. Chen, I. B. Collings, X. Wang, R. P. Liu, and G. B.Giannakis, “Multi-timescale online optimization of network functionvirtualization for service chaining,” IEEE Trans. Mobile Comput.,vol. 18, no. 12, pp. 1–1, Dec. 2019.

[202] X. Chen, W. Ni, I. Collings, X. Wang, and S. Xu, “Automated functionplacement and online optimization of network functions virtualization,”IEEE Trans. Commun., vol. 67, no. 2, Feb. 2019.

[203] J. Xu and S. Ren, “Online learning for offloading and autoscaling inrenewable-powered mobile edge computing,” in Proc. GLOBECOM.IEEE, Washington, DC, USA, Dec.2016, pp. 1–6.

[204] F. Guo, L. Ma, H. Zhang, H. Ji, and X. Li, “Joint load managementand resource allocation in the energy harvesting powered small cellnetworks with mobile edge computing,” in IEEE INFOCOM WKSHPS.IEEE, Honolulu, HI, USA, 2018, pp. 299–304.

[205] G. Zhang, W. Zhang, Y. Cao, D. Li, and L. Wang, “Energy-delaytradeoff for dynamic offloading in mobile-edge computing system withenergy harvesting devices,” IEEE Trans. Ind. Informat., vol. 14, no. 10,pp. 4642–4655, 2018.

[206] G. Zhang, Y. Chen, Z. Shen, and L. Wang, “Distributed energy man-agement for multi-user mobile-edge computing systems with energyharvesting devices and QoS constraints,” IEEE IoT J., 2018.

[207] Y. Mao, J. Zhang, and K. B. Letaief, “Dynamic computation offloadingfor mobile-edge computing with energy harvesting devices,” IEEE J.Sel. Areas Commun., vol. 34, no. 12, pp. 3590–3605, 2016.

[208] M. Min, L. Xiao, Y. Chen, P. Cheng, D. Wu, and W. Zhuang,“Learning-based computation offloading for IoT devices with energyharvesting,” IEEE Trans. Veh. Tech., vol. 68, no. 2, pp. 1930–1941,2019.

[209] Z. Wei, B. Zhao, J. Su, and X. Lu, “Dynamic edge computationoffloading for internet of things with energy harvesting: A learningmethod,” IEEE IoT J., 2018.

[210] F. Wang, J. Xu, X. Wang, and S. Cui, “Joint offloading and computingoptimization in wireless powered mobile-edge computing systems,”IEEE Trans. Wireless Commun., vol. 17, no. 3, pp. 1784–1797, Mar.2018.

[211] X. Lyu, W. Ni, H. Tian, R. P. Liu, X. Wang, G. B. Giannakis, andA. Paulraj, “Optimal schedule of mobile edge computing for internet ofthings using partial information,” IEEE J. Sel. Areas Commun., vol. 35,no. 11, pp. 2606–2615, 2017.

[212] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classificationwith deep convolutional neural networks,” in Advances in neuralinformation processing systems, 2012, pp. 1097–1105.

[213] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances inneural information processing systems, 2015, pp. 91–99.

[214] T. Gruber, S. Cammerer, J. Hoydis, and S. T. Brink, “On deep learning-based channel decoding,” in Annual Conference on Information Sci-ences and Systems (CISS), 2017.

[215] S. Cammerer, J. Hoydis, and S. T. Brink, “Scaling deep learning-based decoding of polar codes via partitioning,” in IEEE GLOBECOM,Singapore, 4-8 Dec. 2017.

[216] H. He, C. K. Wen, S. Jin, and G. Y. Li, “Deep learning-based channelestimation for beamspace mmWave massive MIMO systems,” IEEEWireless Communications Letters, vol. PP, no. 99, pp. 1–1, 2018.

[217] C. K. Wen, W. T. Shih, and S. Jin, “Deep learning for massive MIMOCSI feedback,” IEEE Wireless Commun. Lett., vol. 7, no. 5, pp. 748–751, 2018.

[218] H. Sun, X. Chen, Q. Shi, M. Hong, X. Fu, and N. D. Sidiropoulos,“Learning to optimize: Training deep neural networks for wirelessresource management,” in IEEE International Workshop on SignalProcessing Advances in Wireless Communications, 2017, pp. 1–6.

[219] W. Lee, M. Kim, and D. H. Cho, “Deep power control: Transmitpower control scheme based on convolutional neural network,” IEEECommun. Lett., vol. 22, no. 6, pp. 1276–1279, Jun. 2018.

[220] J. B. Wang, J. Wang, Y. Wu, J. Y. Wang, H. Zhu, M. Lin, and J. Wang,“A machine learning framework for resource allocation assisted bycloud computing,” IEEE Network, vol. 32, no. 2, pp. 144–151, 2018.

[221] Q. V. Do and I. Koo, “Dynamic bandwidth allocation scheme forwireless networks with energy harvesting using actor-critic deep re-inforcement learning,” in Proc. ICAIIC. IEEE, 2019, pp. 138–142.

[222] H. Kim, H. Yang, Y. Kim, and J. Lee, “Action-bounding for reinforce-ment learning in energy harvesting communication systems,” in Proc.GLOBECOM. IEEE, 2018.

[223] M. Chu, H. Li, X. Liao, and S. Cui, “Reinforcement learning basedmulti-access control with energy harvesting,” in Proc. GLOBECOM.IEEE, 2018.

[224] Z. Chen, Z. Ding, X. Dai, and R. Schober, “Asymptotic performanceanalysis of GSVD-NOMA systems with a large-scale antenna array,”IEEE Trans. Wireless Commun., vol. 18, no. 1, pp. 575–590, Jan. 2019.

[225] X. Ke, G. Liang, and M. Kadoch, “Opportunistic multicast NOMA withsecurity concerns in a 5G massive MIMO system,” IEEE Commun.Mag., vol. 56, no. 3, pp. 91–95, 2018.

[226] Y. Zhang and J. Ge, “Performance analysis for non-orthogonal multipleaccess in energy harvesting relaying networks,” IET Commun., vol. 11,no. 11, pp. 1768–1774, 2017.

[227] S. Min and Z. Meng, “Energy efficiency optimization for wirelesspowered sensor networks with nonorthogonal multiple access,” IEEESensors Lett., vol. 20, no. 1, pp. 1–1, Mar. 2018.

[228] Y. Alsaba, C. Y. Leow, and S. K. A. Rahim, “Full-duplex cooperativenon-orthogonal multiple access with beamforming and energy harvest-ing,” IEEE Access, vol. 6, pp. 19 726–19 738, 2018.

[229] J. Men, W. Han, and J. Ge, “Performance analysis for NOMA energyharvesting relaying networks with transmit antenna selection andmaximal-ratio combining over Nakagami-m fading,” IET Commun.,vol. 10, no. 18, pp. 2687–2693, 2016.

[230] B. V. Nguyen, Q. D. Vu, and K. Kim, “Analysis and optimization forweighted sum rate in energy harvesting cooperative NOMA systems,”IEEE Tans. Veh. Tech., vol. 67, no. 12, pp. 12 379–12 383, Dec. 2018.

[231] Y. Liu, Z. Ding, M. Elkashlan, and H. V. Poor, “Cooperative non-orthogonal multiple access with simultaneous wireless information andpower transfer,” IEEE J. Sel. Areas Commun., vol. 34, no. 4, pp. 938–953, 2016.

[232] Y. Zheng, J. A. Hussein, X. Peng, Z. Ding, and W. Yi, “Power alloca-tion study for non-orthogonal multiple access networks with multicast-unicast transmission,” IEEE Trans. Wireless Commun., vol. 17, no. 6,pp. 3588–3599, Jun. 2018.

[233] Z. Ding, Z. Zhao, M. Peng, and H. Vincent, “On the spectral effi-ciency and security enhancements of noma assisted multicast-unicaststreaming,” IEEE Trans. Commun., vol. 65, no. 7, pp. 3151–3163, Jul.2017.

[234] Z. Ming, A. Yadav, O. A. Dobre, G. I. Tsiropoulos, and H. V.Poor, “Capacity comparison between mimo-noma and mimo-oma withmultiple users in a cluster,” IEEE J. Sel. Areas Commun., vol. 35,no. 10, pp. 2413–2424, Oct. 2017.

[235] A. Anand and G. D. Veciana, “Resource allocation and HARQ opti-mization for URLLC traffic in 5G wireless networks,” IEEE J. Sel.Areas Commun., vol. 36, no. 11, pp. 2411–2421, 2018.

[236] W. Ghanem, V. Jamali, Y. Sun, and R. Schober, “Resource allocationfor multi-user downlink resource allocation for multi-user downlinkURLLC-OFDMA systems,” https://arxiv.org/pdf/1901.05825.pdf, Jan.2019.

[237] J. J. Nielsen, R. Liu, and P. Popovski, “Ultra-reliable low latency com-munication (urllc) using interface diversity,” IEEE Trans. Commun.,vol. 66, no. 3, pp. 1322–1334, Mar. 2018.

[238] A. Azari, M. Ozger, and C. Cavdar, “Risk-aware resource allocationfor URLLC: Challenges and strategies with machine learning,” IEEECommun. Mag., vol. 57, no. 3, pp. 42–48, 2019.

[239] Y. Hu, Y. Zhu, M. C. Gursoy, and A. Schmeink, “SWIPT-enabledrelaying in IoT networks operating with finite blocklength codes,” IEEEJ. Sel. Areas Commun., vol. 37, no. 1, pp. 74–88, 2018.

31

[240] T. Chen, A. Mokhtari, X. Wang, A. Ribeiro, and G. B. Giannakis,“Stochastic averaging for constrained optimization with applicationto online resource allocation,” IEEE Trans. Signal Process., vol. 65,no. 12, pp. 3078–3093, 2017.

[241] T. Chen, S. Barbarossa, X. Wang, G. Giannakis, and Z. Zhang,“Learning and management for internet of things: Accounting foradaptivity and scalability,” Proceedings of the IEEE, vol. 107, no. 4,pp. 778–796, Apr. 2019.

[242] L. Dai, B. Wang, Z. Ding, Z. Wang, and L. Hanzo, “A survey ofnon-orthogonal multiple access for 5G,” IEEE Commun. Surveys Tuts.,vol. 20, no. 3, pp. 2294–2323, 2018.

[243] Y. Tan, J. Zhou, and J. Qin, “Novel channel estimation for non-orthogonal multiple access systems,” IEEE Signal Process. Lett.,vol. 23, no. 12, pp. 1781–1785, 2016.

[244] K. Struminsky, S. Kruglik, D. Vetrov, and I. Oseledets, “A newapproach for sparse bayesian channel estimation in SCMA uplinksystems,” in Proc. WCSP, 2016.

[245] L. Song, D. Niyato, H. Zhu, and E. Hossain, “Game-theoretic resourceallocation methods for device-to-device (D2D) communication,” IEEEWireless Commun., vol. 21, no. 3, pp. 136–144, 2014.

[246] M. V. F. Pereira and L. M. V. G. Pinto, “Multistage stochasticoptimization applied to energy planning,” Mathe. Program., vol. 52,no. 1-3, pp. 359–375, 1991.

[247] A. Blum, Online Algorithms in Machine Learning. Springer,Berlin Heidelberg, 1998, pp. 306–325. [Online]. Available: https://doi.org/10.1007/BFb0029575

[248] C. Jiang, H. Zhang, Y. Ren, Z. Han, K. C. Chen, and L. Hanzo,“Machine learning paradigms for next-generation wireless networks,”IEEE Wireless Commun., vol. 24, no. 2, pp. 98–105, Apr. 2017.

[249] P. Gupta, A. Sharma, and R. Jindal, “Scalable machine-learningalgorithms for big data analytics: A comprehensive review,” Wiley

Interdisciplinary Reviews Data Mining & Knowledge Discovery, vol. 6,no. 6, pp. 194–214, 2016.

[250] F. S. Shaikh and R. Wismuller, “Routing in multi-hop cellular device-to-device (D2D) networks: A survey,” IEEE Commun. Surveys Tuts.,vol. 20, no. 4, pp. 2622–2657, 2018.

[251] D. Niyato and W. Ping, “Delay-limited communications of mobile nodewith wireless energy harvesting: Performance analysis and optimiza-tion,” IEEE Trans. Veh. Tech., vol. 63, no. 4, pp. 1870–1885, 2014.

[252] A. H. Coarasa, P. Nintanavongsa, S. Sanyal, and K. R. Chowdhury,“Impact of mobile transmitter sources on radio frequency wirelessenergy harvesting,” in Proc. ICNC, 2013.

[253] A. Douik, S. Sorour, T. Y. Al-Naffouri, and S. Alouini, “Instantlydecodable network coding: From centralized to device-to-device com-munications,” IEEE Commun. Surveys Tuts., vol. 19, no. 2, pp. 1201–1224, 2017.

[254] G. Carneiro, A. B. Chan, P. J. Moreno, and N. Vasconcelos, “Super-vised learning of semantic classes for image annotation and retrieval,”IEEE Trans. Pattern Analysis & Mach. Intelligence, vol. 29, no. 3, pp.394–410, 2007.

[255] E. Alpaydm, Introduction to Machine Learning, 3rd ed. Cambridge,Massachusetts: The MIT Press, 2014.

[256] T. Hofmann, “Unsupervised learning by probabilistic latent semanticanalysis,” Machine Learning, vol. 42, no. 1-2, pp. 177–196, 2001.

[257] J. C. Niebles, H. Wang, and F. F. Li, “Unsupervised learning of humanaction categories using spatial-temporal words,” Intl. J. ComputerVision, vol. 79, no. 3, pp. 299–318, 2008.

[258] L. P. Kaelbling, M. L. Littman, and A. P. Moore, “Reinforcementlearning: A survey,” J. Artificial Intelligence Research, vol. 4, no. 1,pp. 237–285, 1996.

[259] R. S. Sutton and A. G. Barto, “Reinforcement learning: An introduc-tion,” IEEE Trans. Neural Netw., vol. 9, no. 5, pp. 1054–1054, 1998.