Cognitive Radio Technologies 147 Mill Ridge Rd, Ste 212 Lynchburg, VA 24502 Web: Ph: (540) 230-6012...

Cognitive Radio Technologies147 Mill Ridge Rd, Ste 212 Lynchburg, VA 24502

Web: www.crtwireless.comPh: (540) 230-6012Email: [email protected]

Cognitive Radio TechnologiesCognitive Radio Technologies

Applications of Game Theory to CR



MaterialMaterial

• A little about CRT

• A little game theory

• GT + CR Networks

• GT, CRN and CJ



MaterialMaterial




• GT, CRN and CJ



CCognitiveognitive RRadioadio TTechnologiesechnologies

Founded in 2007 by Dr. James Neel and Professor Jeff Reed to commercialize cognitive radio research out of Virginia Tech•6 employees / contractors•07 Sales = 64k, 08 Sales = 127k•09 Sales = 394k, 10 (contracts) = 890k

Business Details

• Partner with established companies to spin in cognitive radio research

• Navy SBIR 08-099 => L3-Nova • Air Force SBIR 083-160 => GDC4S

• Contract research and consulting related to cognitive radio and software radio

• DARPA, DTI, CERDEC, Global Electronics• Position for entry in emerging wireless markets

• Cognitive Zigbee

Business Model



Selected ProjectsSelected Projects

• Prototype SDR for software controllable antenna

• Fundamental limits to SDR performance

• Rapid estimation of SDR resources

• Distributed spectrum management for WNW

• White Space Networking

• Cognitive gateway with ad-hoc extensions

CR Projects SDR Projects

6 MHz Unused(6 MHz)6 MHz

f

Incumbent or other CR user(except microphone user)

TV incumbent user Microphone userFractional useof TV channel

GuardBand

Other CR user or non-microphone incumbent (regulations permitting)

6 MHz Unused(6 MHz)6 MHz

f

Incumbent or other CR user(except microphone user)

TV incumbent user Microphone userFractional useof TV channel

GuardBand

Other CR user or non-microphone incumbent (regulations permitting)



CRT’s Value PropositionCRT’s Value Proposition

• Designing and analyzing systems to work with interactions of complex intelligent agents in distributed processes

• Frequent collaboration with DoD contractors and universities– GDC4S, L3, VT, USF,

UNLV

• Carry a concept from fundamental research to prototype

– Matlab, OPNET, HW-in-the loop sims, prototypes on varying SDRs

– Across and within layers 1-3



MaterialMaterial




• GT, CRN and CJ



CRs don’t just react, CRs don’t just react, they interactthey interact

OutsideWorld

• Outside world is determined by the interaction of numerous cognitive radios

• What makes sense for a link, may not work for a net



WhiteFi Channel AdaptationWhiteFi Channel Adaptation• Access nodes choose tuple (center frequency,

bandwidth)– Uses 5 MHz bandwidth– 20 MHz in 5 UHF channels

• Managed by access nodes with measurements from clients

• Unspecified form of hysteresis• Chosen to maximize (N = num clients, c = channel)

, , ,AP nn

MCham F W N MCham F W MCham F W

,

,5n n

c F W

WMCham F W p c

MHz

1max 1 ,

1n

n c nc

p c AB

n

cB Number AP in c

ncA Airtime Utilization



WhiteFi Channel Adaptation WhiteFi Channel Adaptation Unstable (Not published)Unstable (Not published)

• Consistent with paper assumption that AP much more active than clients

• Infinite Loop!– 4,5,1,3,2,6,4,…

No interference, very high download

1max 1 ,

1n

n c nc

p c AB

, , ,AP nn

MCham F W N MCham F W MCham F W

Chan. (0,0,0) (0,0,1) (0,1,0) (0,1,1) (1,0,0) (1,0,1) (1,1,0) (1,1,1)

Utility (2.49, 2.49, 2.49) (2.99,2.5,3) (2.5,3,2.99) (3,2.99,2.5) (3,2.99,2.5) (2.5,3,2.99) (2.99,2.5,3) (2.49,2.49,2.49)

Utility Characterization

0 1 2 3 4 5 6 7CRT Proprietary10



Game Theory and CRGame Theory and CR• Collection of models / tools for modeling / analyzing

interactive decision problems• Traditional focus on

– Fixed points: Nash Equilibria– Performance: Pareto efficiency, fairness

• Stability had to be grafted on– Convergence to a lesser extent



Potential GamesPotential Games

time

V(a

)

• Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player.

• Cognitive radio interpretation:– Every time a cognitive radio

unilaterally adapts in a way that furthers its own goal, some real-valued function increases.



Exact Potential Game FormsExact Potential Game Forms

• Many exact potential games can be recognized by the form of the utility functionNetwork-wide

Objective FunctionCan’t Influence Own Outcome

Only impacted by self

Sum of mini-coordination games



This correlation between selfish and social This correlation between selfish and social benefit yields desirable behaviorbenefit yields desirable behavior

• Convergence– *ALL* sequences of unilateral selfish

adaptations induce monotonically decreasing network interference levels

– For finite waveform sets, completely unsynchronized adaptations form absorbing Markov chains

• Optimality of steady-states– Assuming exhaustive adaptations,

interference minimizers are the only steady-states

• Stability– Sum network interference is a Lyapunov

function in neighborhoods of isolated interference minimizers

– In practice, many minimizers aren’t isolated, so some hysteresis is needed

Figure from Fig 2.6 in I. Akbar, “Statistical Analysis of Wireless Systems Using Markov Models,” PhD Dissertation, Virginia Tech, January 2007



Implications of MonotonicityImplications of Monotonicity

• Monotonicity implies – Existence of steady-states (maximizers of V)– Convergence to maximizers of V for numerous combinations of

decision timings decision rules – all self-interested adaptations

• Does not mean that that we get good performance– Only if V is a function we want to maximize



16

Other Potential Game PropertiesOther Potential Game Properties

• All finite potential games have FIP• All finite games with FIP are potential games

– Very important for ensuring convergence of distributed cognitive radio networks

• -V is a is a Lyapunov function for isolated maximizers

• Stable NE solvable by maximizers of V• Linear combination of exact potential games is

an exact potential game• Maximizer of potential game need not maximize

your objective function– Cognitive Radios’ Dilemma is a potential game



MaterialMaterial




• GT, CRN and CJ



Interference Reducing Interference Reducing Networks (Dissertation)Networks (Dissertation)• Concept

– Cognitive radio network is a potential game with a potential function that is negation of observed network interference

• Definition– A network of cognitive radios where each adaptation

decreases the sum of each radio’s observed interference is an IRN

• Implementation:– Design DFS algorithms such that network is a potential game

with -V

ii N

I

time

(

)



Bilateral Symmetric Bilateral Symmetric Interference (Dissertation)Interference (Dissertation)• Two cognitive radios, j,kN, exhibit bilateral

symmetric interference if

Source: http://radio.weblogs.com/0120124/Graphics/geese2.jpg

What’s good for the goose, isgood for the gander…

, ,jk j j k kj k k jg p g p ,j j k k k – waveform of radio k• pk - the transmission power of

radio k’s waveform• gkj - link gain from the

transmission source of radio k’s signal to the point where radio j measures its interference,

• - the fraction of radio k’s signal that radio j cannot exclude via processing (perhaps via filtering, despreading, or MUD techniques).

,k j



CRT Proprietary20

An IRN 802.11 DFS AlgorithmAn IRN 802.11 DFS Algorithm(Milcom06)(Milcom06)

• Suppose each access node measures the received signal power and frequency of the RTS/CTS (or BSSID) messages sent by observable access nodes in the network

– Ignore client interference • Assumed out-of-channel

interference is negligible and RTS/CTS transmitted at same power

jkkkjkjjjk ffpgffpg ,,

\

,i i ki k i kk N i

u f I f g p f f

1

,0

i ki k

i k

f ff f

f f

Listen onChannel LC

RTS/CTSenergy detected? Measure power

of access node in message, p

Note address of access node, a

Update interference

tableTime for decision?

Apply decision criteria for new

operating channel, OCUse 802.11h

to signal change in OC to clients

yn

Pick channel tolisten on, LC

y

n

Start

J. Neel, J. Reed, “Performance of Distributed Dynamic Frequency Selection Schemes for Interference Reducing Networks,” Milcom 2006, Washington DC, October 23-25, 2006



CRT Proprietary21

Round-robin

Statistics (Milcom / Statistics (Milcom / Dissertation)Dissertation)• 30 cognitive access nodes in European UNII

bands• Choose channel with lowest interference• Random timing• n=3• Random initial channels• Randomly distributed positions over 1 km2

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

Number of Access Nodes

Red

uctio

n in

Net

Int

erfe

renc

e (d

B)

Asynchronous Legacy Devices

Reduction in Net Interference

Reduction in Net Interference



Asymmetry Extensions Asymmetry Extensions (SDRF07)(SDRF07)

• Symmetry not always there naturally– Power control– Prioritization– Beamforming– Ad-hoc nets

• Symmetry can be induced by manipulating observation processes– Network optimization correlates

with desired metric, but may not be desired metric

• Some practical considerations ignored in pubs.

??



CRT Proprietary23

From Phase I Navy SBIRFrom Phase I Navy SBIR(“Published” at JSTeF 09)(“Published” at JSTeF 09)

• Sources considered–Dynamic multipath environments

(mobile fading)–Hostile users

Tone Jamming (Mobile)

~30 dB

Shifted Left(jammers)

Greater slope

(mobility

)

Tone Jamming(Mobile)

Hopping Jamming (Mobile)

Smaller to zero gain

Hopping Jamming(Mobile)

What if the environment is “unstable”?

Observed InterferenceLevels

Aggregate NetworkInterference

Clusters’Frequencies

0 5 10 15 20

0 5 10 15 20

0 5 10 15 20seconds

• Constraints: –Irregularly timed observations without collaboration or centralization–Preserve performance and responsiveness, minimal complexity–No “special” new measurements

• Performance Impact• Stability impact



Load-Sensitive Routing Load-Sensitive Routing (Not published yet)(Not published yet)

• Traditional stability issues when load-sensitive– Interactions intractable for

ARPANET• Generalized congestion

game– Stable and load-sensitive

• Ignores information distribution

– Each edge is EPG• Action is contribution of traffic

– Cost = 0 if not using edge– Path cost is sum of edge

costsCRT Proprietary



Multi-Layer, Multiple CR Process Multi-Layer, Multiple CR Process Integration (Not published yet)Integration (Not published yet)• Enables stable,

desirable operation of CRN with multiple different distributed processes– Spectrum, routing, DSA

• Orthogonal– No interaction

• Edgewise

– Effectively MSI game + Generalized Congestion

• Network wide (products)

,\

, ,m

e p e f ef e ff E E

c s r t t I s s

CRT Proprietary



Improving CoexistenceImproving Coexistence(SDRF TVWS Workshop 09) (SDRF TVWS Workshop 09)

• Modfifies BSI to induce “affinity” for classes of radios

• Leverages database for locations & classes of radios

• Assumed two step-coexistence process– Distributed sort of fractious networks into

different channels (frequency deconfliction)• Can sort themselves out without direct

coordination– Coordinated coexistence of compatible

networks within channels (transmission time deconfliction ala .22 or .16h)

• Limit frequency deconfliction to when it’s absolutely necessary

• Limits trunking gains– Can account for tethered radios

without revealing location / IDs• Weighted fairness needs mechanism for

broadcasting weights if weights are situationally dependent

Channel

# U

sers

2 171819 27 33 380

1

2

3

4

5

6

7

Channel

# U

sers

2 171819 27 33 380

1

2

3

4

5

6

Initial

Final

1 1

1 1

1

1

A

CRT Proprietary



MaterialMaterial




• GT, CRN and CJ



Malicious != Mischievous Malicious != Mischievous (From dissertation)(From dissertation)• Popular “solution” to

mischievous nodes (selfish nodes that damage network) is to “punish” nodes– Also implies a way to

“brainwash” learning nodes

• Imperfect information can obfuscate punishment from mischievous behavior and produce catastrophic cascades

• Even with perfect information, malicious node may be masochistic

From Fig 6 in [MacKenzie_01]

From [Srivastava_06]



Malicious CRs can blend in Malicious CRs can blend in (Not published yet)(Not published yet)

• Normal CR– Given available adaptations and

knowledge about network state – Maximize system (own)

performance

• Malicious CR– Given available adaptations and

knowledge about network state– Minimize system performance • Adapt at inopportune times

• Simply minimize performance

• Ensure marginally stable network goes unstable

• Plus learning exploits– And spoofing

• And information corruption

5 malicious, 30 normal

35 normal

Average interference levels for nodes 6-35

35 dB

CRT Proprietary



Detecting malicious behavior from Detecting malicious behavior from Nash equilibria (Not published yet, but Nash equilibria (Not published yet, but not that useful )not that useful )

• With non-deterministic decision processes, difficult to say whether instantaneous adaptations are “ok”

• Assume we know radios are trying to maximize specific goals

• We can identify the expected operating points– Assuming CRs adhere to

specified goals • But:–Predictions depend on

environment–Doesn’t help identify the CJ–Convergence to / existence of

NE not generally guaranteed

CR1

CR

2

Allowable region

Predicted operating point

Detected operating point

CRT Proprietary



Malicious User Detection with Malicious User Detection with Potential Games (Not published yet)Potential Games (Not published yet)• Implement as monitoring

system that evaluates potential (emergent) function– Frequently sum of

performance levels– Complexity is in the

transmission / connectivity– No single node / cluster

knows / can evaluate emergent function

• But a malicious CR will lie– E.g., Claim massive gains to

offset others’ losses• With BSI, a malicious node

can’t tell a credible lie!– Other relationships exist– Need to be WPG / EPG for

linear relationships

V

CRT Proprietary



Illustration of using emergent Illustration of using emergent property to detect malicious CRproperty to detect malicious CR

0 50 100 150 200 250 300

40

60

80

100

120

140

Cha

nnel

0 50 100 150 200 250 300-90

-80

-70

-60

-50

-40

I i(f)

(dB

m)

0 50 100 150 200 250 300-80

-70

-60

-50

-40

-30

iteration

(f

) (d

Bm

)

0 50 100 150 200 250 300

40

60

80

100

120

140

Cha

nnel

0 50 100 150 200 250 300-90

-80

-70

-60

-50

I i(f)

(dB

m)

0 50 100 150 200 250 300-75

-70

-65

-60

-55

-50

iteration

(f

) (d

Bm

)

Potential Function

Radio Utilities

Adaptations

Policy restricted channels

Malicious User Detected

No Malicious UserPotential Function

Adaptations

Radio Utilities

CRT Proprietary



CR/CJ Interaction CR/CJ Interaction (in seedling proposal to Bruce)(in seedling proposal to Bruce)

• Cognitive radios will be faced with cognitive attackers– Any fixed mitigation strategy to eventually be learned and defeated

• Partial Solution:– Model as multi-armed bandit problem (classic machine learning model)– Use regret learning to achieve near-optimal performance (given presence

of intelligent adversary)

• Issues:– Bandit (cognitive attacker) strategy nor is solution space constant– Starting from untrained state may be too long to track assailant

• Proposed Solution to Issues:– Seed the routines by learning and classifying what attacks are underway

• Effectively adds case based reasoning and attack recognition / learning

• State at end of seedling:– Proof of concept simulation and analysis

• Anticipated Benefits:– Reduce the period of time from when new attacks emerge to when

defenses are implemented

• Intermediate tasks:– Learn to detect when attacked

• Important to differentiate poor performance from attack

– Learn to classify attacks – Generalize multi-armed bandit solutions to changing solution space– Define how to integrate together CBR and bandit solutions into CR– Characterize “reaction” times (for adversarial OODA loop) Akin to trying to build a HMM,

while an opponent is changing the states

CJ is influencing outside world to intentionally confuse and harm CR

CRT Proprietary



CRT’s Value PropositionCRT’s Value Proposition

• Designing and analyzing systems to work with interactions of complex intelligent agents in distributed processes

• Frequent collaboration with DoD contractors and universities– GDC4S, L3, VT, USF,

UNLV

• Carry a concept from fundamental research to prototype

– Matlab, OPNET, HW-in-the loop sims, prototypes on varying SDRs

– Across and within layers 1-3



ExtrasExtras



CRT Proprietary36

Solved issues with game Solved issues with game theory (JSTeF 09)theory (JSTeF 09)

1. All self-interested adaptations

2. Based only on observations of own performance

3. Decrease aggregate network interference

• Scalable resource utilization• No synchronization required

• No information exchange overhead• More responsive network

• Self-stable• Converges to local-optima

• For example, for a collection of 802.11 clusters independently choosing operating frequencies

Observed Interference

Levels

Aggregate Network

Interference

Clusters’ Frequencies

0 5 10 15 20

0 5 10 15 20

0 5 10 15 20seconds

25-30 dB

25-30 dB



CRT Proprietary37

Gain the performance without Gain the performance without the overhead or complexitythe overhead or complexity(SDRF07)(SDRF07)

• CRT’s distributed algorithms performance equivalent to “omniscient” centralized local search algorithm– Large capacity gains and

interference reduction– Without the overhead,

complexity, or the single-point of failure

– With much better scaling• O(node density)

– Using generally available measurements, e.g., RSS, node ID, time stamps (later)

Support 16 x more links

Reduce interference by 25-30 dB

Network Density

Col

lisio

n P

roba

bilit

yS

tead

y-st

ate

inte

rfer

ence

Network Density



Date post:	28-Dec-2015
Category:	Documents
Upload:	brittney-miller
View:	213 times
Download:	0 times

Cognitive Radio Technologies 147 Mill Ridge Rd, Ste 212 Lynchburg, VA 24502 Web: Ph: (540) 230-6012...

Documents