©2003 Semandex Networks, Inc.

Semantic Multicast

Maximilian Ott

Semandex Networks, Inc.WINLAB, Rutgers U.

XML Routing

URL Routing

IP Routing

Creating Networks That Know™

Semantic Multicast is the next evolution

Address

Content

Name

Information Networks

<Product Return> <Customer> <Name> <Address> <Product> <Type> <Model> <Reason>

Customer Service

Sales Rep.

Product Manager

All <Product Return>

My <Customers>AND within 10 miles

My <Product.Model>AND <Reason> == Fault

XML Schemas Everywhere

Tailoring Information to the User’s Needs

XML profiles filter information within the network, meeting need-to-know, bandwidth and device constraints

Scalable Real-time Information Delivery

Interest Profile User

Content Provider

XMLDescriptor

ContentRouter B

ContentRouter A

Distributing Profiles

B

A

U1

U2

4

2

1

3a

3b

P_AB

Aggregating Profiles

SX

SemanticRouter

B

SXSemantic

RouterA

RP1

RP2

RPB

RPB = RP1 v RP2

Example: Interest Descriptor

1: <ri:RI ..>

4: <ri:And>

5: <d:Locale>

6: <d:Position>

7: <ri:distance within=’10’ units=‘mi’>

8: latitude=‘40’ longitude=‘-74.4’/>

9: </d:Position>

10: </d:Locale>

11: <d:Genre>

12: <ri:Or>

13: <ri:string-match>Jazz</ri:string-match>

14: <ri:string-match>Rock</ri:string-match>

15: </ri:Or>

16: </d:Genre>

17: </ri:And>

18: </ri:RI>

5: <d:Locale>6: <d:Position>7: <ri:distance within=’10’ units=‘mi’>8: latitude=’40’ longitude=‘-74.4’/> 9: </d:Position>10: </d:Locale>

Finding Information & Keeping it Current

Netlink supports distributed search with real-time content updates from relevant sources

Spotter2

MD

LiveDocumen

t

Repository

3

?1

Need a map

Deploying Networks That Know™

An overlay hierarchy of network servers provides scalable information connectivity

Internet

Extranet

Intranet

Why not IP Multicast?

Need to map multi-dimensional information space into 1-D address range

“Red Mustang, built 1972-75, < $4K”

Requires “carving-up” the information space

– Wide channels => receive lots of junk

– Narrow channels => publish on multiple channels, potentially receive multiple times

MC address does NOT contain semantic meaning

– Cable channel syndrome

Nobody is using it, anyway

Can’t work – Too many cycle per packet

CPU cycles/packet

Packet/sec

Conventional Wisdom

Why not here?

3Ghz CPUs and

counting

©2003 Semandex Networks, Inc.

Performance

Test Environment for Full Routing

Producer

SemSock

XMLParser

SemNativeIn

SemDatagram PacketsSemDatagram

PacketsSemDatagram SemDatagram

SemNativeOut

Filter

Routing Plane

SemSock

Consumer

Router Configuration

Processor 1.2GHz Pentium III

Memory 512MB

Disk 20GB ATA-100

Network Intel 100Mbps Ethernet

Operating System Red-Hat 7.3 distributionLinux 2.4.18-3 kernel

Baseline Security LSM 2.4

Additional Security NSA Secure Linux (SELinux)release 2002/07/03.13

Application Environment GCC 3.0.4 and GCJ

Encryption Code Semandex sxcryptoAES-128 and SHA-1

System Performance

Window Size vs. Throughput

46.86

47.3357

46 45.53

44.5

45

45.5

46

46.5

47

47.5

WindowSize

Throughput (Mbps)

24 n n

32 n n

64 n n

128 n n

Note: Ethernet port needs to handle TWICE the traffic!

What about Security?

No end-to-end secrets shared between producers and consumers

Provides link-by-link validation

Full encryption is too expensive

– 0.00581 bytes/us/MHz (99.9% linear)

– 26.6 Mbps on target platform (single fan-out)

SHA-1 Signature

– Signature only needs to be checked / 257Mbps

Encrypt signature + sequence counter

Security needs CPU cycles

Encrypting small (32byte) headers with a shared-secret key prevents malicious agents from inserting packets into the data stream

Signing packets with SHA-1 signature prevents unauthorized changes to the content of the packet (e.g. to change the Content Descriptor)

Overall impact is a drop in throughput from 47Mbps to 37 Mbps

Throughput vs. WindowSize ( w/ encryption + signature)

36.98637.1528

36.366

34.625

33

33.5

34

34.5

35

35.5

36

36.5

37

37.5

1

window size

Th

rou

gh

pu

t (M

bp

s)

24 y y

32 y y

64 y y

128 y y

Throughput vs. WindowSize( w/ encryption)

39.99840.0265

40.16

39.8718

39.7

39.75

39.8

39.85

39.9

39.95

40

40.05

40.1

40.15

40.2

Window Size

Th

rou

gh

pu

t in

Mb

ps

24 y n

32 y n

64 y n

128 y n

Nice numbers, but does it scale?

Processing is per packet with no inter-packet state

Fully parallelizable

Scales with CPU speed?– Investigate individual aspects of the routing procedure

– Use sample XML data to generate raw data

– Characterize throughput by fitting lines, or curves to the data

– Combine results into a single formula

– Verify formula using production data

Putting it all together

InputTime = 0.1 PSize2 + 8.9 PSize + 45.2

ParseTime = 0.8 CDesc2 + 49.1 CDesc

MatchTime= 23.16 CDesc NCsmrs

OutputTime = 26.5 PSize (0.55 + 0.4 NCsmrs2)

Does it work out in the wild?

Netlink XML Information Appliance

NIMA

Adaptors: USAMaps (Vector/Raster Maps)USA_IPL (Imagery)

USAMidb

USPACOM

GCCS TRACKS Adaptor: OTH_G

UNITED KINGDOM

Adaptor: GRBMaps (Raster Maps)

HANSCOM

Adapter: MIDB Database

MIDB

AUSTRALIA

Adapter: AusCOINS

DAHLGREN, VA

Adapter:AusCOINS

COINS

CANADA

Adaptor: CANMaps (Vector/Raster Maps)

Other Application Scenarios

Knowledge Sharing

Extensions to IM

Media distribution to mobile device (pro-active caching)

Sensor networks (focus on information aggregates)

Can’t be all Sugar & Spice

Current routing algorithm requires spanning tree

– Not very robust

– Has self-discovery and self-healing, but with delay

Centralized Management

– Is a requirement for current customer set

– Need to be extended to autonomous regions

Tunnel protocol is based on UDP

– Nobody likes UDP

– Works really bad on bad channel

Information Food Chains

Sensor

Interpreted data feeds back into the same system, but using a different schema

Information Foodchains

Autonomous Living

Autonomous Entity

ConsumesInformation& Request

ProducesInformation & Requests

“Global”Information

Space

Observes / Affects

Environment

Conclusion

Routing based on content descriptor

Symmetric use: distribution and discovery

Similar scaling properties as IP

Network is generic; specialization on the edge

Fully parallelizable, relatively linear

Works, in use, makes money

Needs more applications, more robust topologies

©2003 Semandex Networks, Inc.

Semantic Multicast

Max Ott

max@semandex.net

http://www.semandex.net/whitepaper/semantic_multicast.pdf