Reliable and Real-time Communication in Industrial Wireless Mesh Networks

Song Han, Xiuming Zhu, Al MokUniversity of Texas at Austin

Deji Chen, Mark NixonEmerson Process Management

Outline• Introduction

• Network Management Techniques– Reliable graph routing– Schedule construction and channel management

• Performance Evaluation

• Implementation and Deployment

• Future Work

2

Introduction

• WirelessHART network– Secure and TDMA-based wireless mesh networking technology– Centralized network architecture– Stringent timing and reliability requirements

3

• Previous work– Full-blown WirelessHART stack (RTAS’08)– Compliance test suite (RTAS’09)– Data quality maintenance techniques in wireless network

(RTSS’05, TC’08, RTSS’09)

Introduction (cont.)

• Goals– Achieve reliable graph routing in WirelessHART network– Achieve real-time communication by deterministic link and

channel assignment– Evaluate their performance in industrial environments

4

• Challenge– The complexity of network management is pushed to the

centralized manager but engineering decisions can have large performance impact.

Reliable Graph Routing

• Reliable Broadcast Graph (GB)– GB is a graph connecting Gateway (GW) downward to all DEVs– Broadcasts common configuration and control messages– Each DEV has at least two parents in GB

5

Reliable Graph Routing (Cont.)• Reliable Uplink Graph (GU)

– GU is a graph connecting all DEVs upward to the Gateway– DEVs propagate periodic process data– Each DEV has at least two children in GU

– Both GB and GU have no fewer than 2 Access Points

6

Reliable Graph Routing (Cont.)• Reliable Downlink Graph (Gv)

– The graph from the Gateway to DEV v– Transmit unicast messages from the GW and NM to v– Each intermediate DEV has at least two children in Gv

– There exists at least one directed cycle in Gv

7

To avoid infinite forwarding loop:1) Only one cycle of length 2 in Gv

2) Each DEV on the cycle has direct edges to v

Constructing GB

8

G

A A

12

4

5

3

• Drop the links with low Receive Signal Strength (RSS) in the original network topology G

S

Constructing GB

9

G

A A

12

4

5

3

• Drop the links with low RSS in the original network topology G

• Maintain a set of explored node S, initially S = {G, APs}

S

Constructing GB

10

G

A A

12

4

5

3

• Drop the links with low RSS in the original network topology G

• Maintain a set of explored node S, initially S = {G, APs}

• Grow S according to

S = {G, Aps, 1}

S

Constructing GB

11

G

A A

12

4

5

3

• Drop the links with low RSS in the original network topology G

• Maintain a set of explored node S, initially S = {G, APs}

• Grow S according to

S = {G, Aps, 1, 2}

S

Constructing GB

12

G

A A

12

4

5

3

• Drop the links with low RSS in the original network topology G

• Maintain a set of explored node S, initially S = {G, APs}

• Grow S according to

S = {G, Aps, 1, 2, 4}

S

Constructing GB

13

G

A A

12

4

5

3

• Drop the links with low RSS in the original network topology G

• Maintain a set of explored node S, initially S = {G, APs}

• Grow S according to

S = {G, Aps, 1, 2, 4, 5}

S

Constructing GB

14

G

A A

12

4

5

3

• Drop the links with low RSS in the original network topology G

• Maintain a set of explored node S, initially S = {G, APs}

• Grow S according to

S = {G, Aps, 1, 2, 4, 5, 3}

Construct Gv

15

• More complicated than GB and GU:– Only involves part of the nodes in G– The existence of cycle– Restrictions: One cycle (length 2) between the parents of

destination node v

• Standard Reliable Downlink Graph– Construct a completely new graph from GW to DEV v– Configuration in intermediate nodes cannot be reused– High configuration cost and poor scalability

Sequential Reliable Downlink Routing (SRDR)

• Key Principles– Each node only keep a small local graph– Local graphs are reusable building blocks for constructing

reliable downlink graph for multiple destinations

16

Low configuration cost

High Scalability

High Reliability

An example of SRDR

17

G

A1 A2

21 3

4 5

(a) Original network topology(b) Downlink graph: g2Sequential route for Dev 2: g2

G

A1 A2

21 3

4 5

(c) Downlink graph: g3Sequential route for Dev 3: g3

G

A1 A2

21 3

4 5

(d) Downlink graph: g1Sequential route for Dev 1: g2, g1

G

A1 A2

21 3

4 5

(e) Downlink graph: g4Sequential route for Dev 4: g2, g1, g4

G

A1 A2

21 3

4 5

(f) Downlink graph: g5Sequential route for Dev 5: g2, g5

G

A1 A2

21 3

4 5

(b)

Avoid node failure at DEV2

Local graph

SRDR vs. Standard Downlink Graph

18

(b) Downlink graph: g5Sequential route for Dev 5: g2, g5

G

A1 A2

21 3

4 5

(c) Standard downlink graph for Dev 5

G

A1 A2

21 3

4 5

(a) Downlink graph for Dev 2

G

A1 A2

21 3

4 5

Configure cost is reduced by 3 links

More significant improvement in large scale networks

Sequential Reliable Downlink Routing (SRDR) Extensions

• Network layer header extension:

19

SRDR Extensions

• Routing module extension:– retrieve the earliest graph ID in the graph list and route the

packet on this graph

– If current node is the sink of the graph, remove this graph ID and route the packet on the next earliest graph.

– If routing is failed, remove this graph ID and try the next earliest graph ID if it has the corresponding edges.

20

Optimization on SRDR

21

• In SRDR, routing is performed strictly according to the sequence in the ordered graph list.

• SRDR-OPT– Observation: each node can keep graph info to multiple

destination. – Have chance to take the “shortcut”– Principle: Search the ordered graph list backward and route

the packet on the first graph ID that is stored in its table

An example of the SRDR-OPT

22

S 2 4

1 3

g2 g3 g4

S 2

1

g2

2 4

3

g4

2

1 3

g3

Communication Schedule and Channel Management

23

• Key Principles:

– Spread out the channel usage in the network

– Apply Fastest Sample Rate First policy (FSRF)

– Allocate the links iteratively from Src to Dest

– Split traffic (bandwidth) among all successors

Schedule Construction (An Example)

24

G

A B

1 2 3

1 sec 2 sec 1 sec

Ch Offset

Slot

16

0

1

2

100 200 300

.

.

.

400

Dev 1

AP A

Dev 2

Global Channel-Time Slot Matrix

Device Schedule

Schedule Construction (An Example)

25

G

A B

1 2 3

1 sec 2 sec 1 sec

Ch Offset

Slot

16

0

1

2

100 200 300

.

.

.

400

Dev 2

AP A

AP B

Schedule Construction (An Example)

26

G

A B

1 2 3

1 sec 2 sec 1 sec

Ch Offset

Slot

16

0

1

2

100 200 300

.

.

.

400

Dev 3

AP B

Dev 2

Schedule Construction (An Example)

27

G

A B

1 2 3

1 sec 2 sec 1 sec

Ch Offset

Slot

16

0

1

2

100 200 300

.

.

.

400

Dev 2

AP A

AP B

Schedule Construction (An Example)

28

G

A B

1 2 3

1 sec 2 sec 1 sec

Dev 2

AP A

AP B

Ch Offset

Slot

16

0

1

2

100 200 300

.

.

.

400

Channel offset will be converted into practical channel number in the runtime

Performance Evaluation

29

Configuration overhead in broadcast graphs Reachability in broadcast graphs

Recovery overhead to regain reliability Reachability in downlink graph

Performance Evaluation

30

Configuration overhead in downlink graphs Average latency vs. Network size

Success ratio vs. Sample rate Network utilization vs. Sample rate

Reusable local graph makes the difference Shortcut makes the difference

WirelessHART Prototype System

31

Major Components in the prototype :

• Network Manager• Gateway• Host Application• Access Point• Device• Sniffer

PC Side

Embedded Side

Overall Design of the System

32

Overview of the Network Manager

33

Network Topology

Device Configuration

Routing Graphs

Global ScheduleDevice ScheduleDevice Bandwidth

Overview of the System

34

• 10 devices and 1 AP in the system

• Devices publish data to GW with different sampling rates (1sec – 8sec)

• Retry happens but no packet loss is detected

Deployment (Work-in-progress)

35

UT Pickle Research Center Petroleum Engineering Department

Network Manager, Gateway and Access Point

Future Work

36

• A general and adjustable framework– Applications have different requirements on timing, security, …– Building an adjustable MAC state machine

• Collaborative wireless system– Wifi, WirelessHART, ZigBee,…– Competition -> Collaboration