Research on Key Research on Key Technology in Grid Technology in Grid Delivery Network Delivery Network

Zhihui LvZhihui LvNetwork & Information Engineer CenNetwork & Information Engineer Center of Fudan university, ShangHai, Cter of Fudan university, ShangHai, C

hinahina 2003.8.282003.8.28

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

Clients

1.Content Delivery Network (CDN)

Host Server

MindSpring

PSINetSprint

Gloobix

QWest

@Home

UUnet

Huge Requests

Server Crash

Slow Response

Clients

Clients

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

1.Content Delivery Problems

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

1.(1)Use Client Cache/Client Side Cache Server

Host Server

MindSpring

PSINetSprint

Gloobix

@Home

UUnet

Fewer Requests

Clients

Clients

Clients

ClientCache

ClientSideCacheServer

QWest

Fast Response

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

1.(2)Use Mirror Sites

Host Server

MindSpring

PSINetSprint

Gloobix

QWest

@Home

UUnet

Fewer Requests

Server

Fast Response

Clients

Clients

Clients

Mirror Site

Mirror Site

Need improvement by guiding the selection of mirror servers with server load/network bandwidth measurement

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

1.(3)Edge Network Cache Servers

Host Server

MindSpring

PSINetSprint

Gloobix

QWest

@Home

UUnet

Fewer Requests

Server

Fast Response

Clients

ClientsClients

ClientCache

Mirror Site

Mirror SiteEdgeNetworkCacheServer

CacheServer

CacheServer

CacheServer

CacheServer

ClientSideCacheServer

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

1.Akamai CDN Solution --Edge Delivery Service

Date # of Edge Servers

# of Networks

# of Countries

11/2000 6000 335 54

6/2001 9700 650 56

5/2003 12900 1000 66

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

2.Drawback of CDN

Cache Location Problem: Where to put cache servers?How many are needed?When/where/how to push/delivery the content?How about dynamic content?

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

2.Drawback of CDN

Although CDN is an effective means of multimedia information access and delivery, there are several barriers to making CDN a more common service: Big cost Bad scalability Complicated replication integrity.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

3. Emergence of Grid Delivery

The combination of the grid technology and CDN technology has produced a novel technology——grid delivery. Grid delivery is an emerging technology that enables the distribution of rich media throughout a bandwidth-constrained network.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

3. Emergence of Grid Delivery

GGG （ Great Global Grid ）

HP IBM WebSphereMicrosoft.NetSun ONE

Computing and Data Grid

TeraGridIPGGIGASCI GridData Grid

Information and Knowledge Grid

Semantic WebKnowledgeManagementOntologyInformationPlatform

Business Grid

CDN

RTEC Real-Time Enterprise Computing

Web Service

Other Grid Models

P2PParasitic Computing

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

3. Emergence of Grid Delivery

Grid delivery promises to be a viable solution for distributing on-demand streaming content and other large files directly to end-user systems. Grid delivery has its origins in the world of Internet-based content delivery. The objective of grid delivery is to create a network of systems for enabling content delivery to a single recipient.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

4. Infrastructure of Grid Delivery

Grid Delivery

Server

Grid Delivery Server

Network ManagerNetworkProtector

NetworkPublisher

Grid Delivery Technology

XML Web Services Connector

NetworkAnalyzer

(name might change)

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

4. Infrastructure of Grid Delivery Grid Delivery Technology creates a network of shared resources where any PC or server can deliver content on demand.

• Bandwidth Harvesting

• XML Delivery Security

• Dynamic Optimization

• Grid Delivery Servers

• Delivery Managers

Grid Deliver

y Server

Grid Delivery Server

NetworkAnalyzer

(name might change)

NetworkProtector

NetworkPublisher

Grid Delivery Technology

XML Web Services Connector

Network Manager

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.1Time Shifted Delivery5.1Time Shifted Delivery

Time Shifted DeliveryTime Shifted Delivery

0%0%

100%100%

0%0%

100%100%

Smoothing via Scheduling and Bandwidth Rate LimitingSmoothing via Scheduling and Bandwidth Rate Limiting

Time Shifted DeliveryTime Shifted Delivery

PeakPeak

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology

5.2 Adaptive Rate Multiserving (ARM)5.2 Adaptive Rate Multiserving (ARM)

Adaptive Rate Multiserving (ARM)Adaptive Rate Multiserving (ARM)

OriginServer

Typical Serving Model

User

Adaptive Rate Multiserving Model

OriginServer

1

OriginServer

2

OriginServer

3

User

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.3 Outer Edge Caching5.3 Outer Edge Caching

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.3 Outer Edge Caching 5.3 Outer Edge Caching

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.3 Outer Edge Caching5.3 Outer Edge Caching

LANInternetOrigin Server

Origin Server

USER1

USER2

USER3

USER1Grid

Server

Grid Server

GridServer

Grid Delivery --Outer Edge Relay CachingGrid Delivery-- Outer Edge On-Demand Caching

Grid Server

USER2

2(3) 2(4)

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security

A secure, managed delivery system must satisfy the following security requirements :A secure, managed delivery system must satisfy the following security requirements :

(1)Access Control - The delivery system must provide an access control mechanism that controls access to the content in the network and also provides controls over who is authorized to publish content to the network. This system must work with an enterprise's existing security infrastructure and be integrated with a central directory system to facilitate administration and maintenance of users and groups.

(2)Network Security - The delivery system must ensure that files are not delivered to unauthorized users, and that attackers cannot infiltrate the system itself.

(1)Access Control - The delivery system must provide an access control mechanism that controls access to the content in the network and also provides controls over who is authorized to publish content to the network. This system must work with an enterprise's existing security infrastructure and be integrated with a central directory system to facilitate administration and maintenance of users and groups.

(2)Network Security - The delivery system must ensure that files are not delivered to unauthorized users, and that attackers cannot infiltrate the system itself.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security

A secure, managed delivery system must satisfy the following security requirements :A secure, managed delivery system must satisfy the following security requirements :

(3)Desktop Security - The delivery system must provide a level of control over content use and redistribution that is appropriate to the nature of the content and the identity of the intended recipient.

(4)Enterprise-Ready Standards and Support - The delivery system must support all enterprise standards and be easily deployable within an enterprise. The system must also offer a high level of reliability and scalability to handle large numbers of users across geographic boundaries and network topologies.

(3)Desktop Security - The delivery system must provide a level of control over content use and redistribution that is appropriate to the nature of the content and the identity of the intended recipient.

(4)Enterprise-Ready Standards and Support - The delivery system must support all enterprise standards and be easily deployable within an enterprise. The system must also offer a high level of reliability and scalability to handle large numbers of users across geographic boundaries and network topologies.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security

XML Delivery Security InfrastructureXML Delivery Security Infrastructure

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security

The security measures in this infrastructure are as following:

  （ 1 ） Access Control:

 User Authentication and Authorization;

  Secure Publishing;

 Content Access Rules;

Enterprise Security Infrastructure Integration;

Centralized Deployment and Administrator.

The security measures in this infrastructure are as following:

  （ 1 ） Access Control:

 User Authentication and Authorization;

  Secure Publishing;

 Content Access Rules;

Enterprise Security Infrastructure Integration;

Centralized Deployment and Administrator.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security

The security measures in this infrastructure are as following:

（ 2 ） Network Security:

 Strong Encryption;

XML-based Secure Distributed Delivery Protocol;

 Point-to-Point Security;

  Unique Private Keys;

  Secured Node;

 Content Integrity.

The security measures in this infrastructure are as following:

（ 2 ） Network Security:

 Strong Encryption;

XML-based Secure Distributed Delivery Protocol;

 Point-to-Point Security;

  Unique Private Keys;

  Secured Node;

 Content Integrity.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security

The security measures in this infrastructure are as following:

（ 3 ） Desktop Security and Usage Control:

 Copy and Re-Distribution Prevention;

Encrypted File Controls; Point-to-Point Security;

  The 3rd Party Digital Rights Management Support.

The security measures in this infrastructure are as following:

（ 3 ） Desktop Security and Usage Control:

 Copy and Re-Distribution Prevention;

Encrypted File Controls; Point-to-Point Security;

  The 3rd Party Digital Rights Management Support.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

6.Delivery Grid dramatically increases performance while efficiently utilizing centralized bandwidth

7.30

0.47

0 2 4 6 8

KontikiGrid

CentralServer

Avg. Delivery Speed (Mbps)

Testing Notes:

-Central Server bandwidth speed is 0.47 Mbps or approx. 1/3 use of a dedicated T1

-Delivery speed calculated by ratio of file size to time of delivery for 20MB file

-Load % is ratio of average bytes delivered by Central Server to average bytes delivered by Grid Server.

15x times faster than central server

Central Server Load

Central Server 100%

Grid Server 30%

Central server bandwidth load reduced by 30%

Delivery Performance Use of Internet or WAN Bandwidth

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

6.Average Time to Deliver 20MB file Grid over 15x times faster than central server

- Central Server bandwidth speed is 0.47 Mbps or 1/3 of T1 speed (1.4Mbps)

21.91

342.86

0 100 200 300 400

Kontiki Grid

Central Server

Delivery Time (sec)

Typical Delivery

Grid Delivery

0 100 200 300 400

342

22

Avg.Delivery Time(sec)

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

7. The Challenges and Future Prospect

(1)Rich media grid delivery is a novel model for content delivery initially focused on the needs of the enterprise.

It is a flexible system that utilizes the network and computing resources already present in the enterprise much more efficiently, extremely important in this age of tight IT budgets and staffing.

But the challenge is how to apply Grid Delivery to Internet scale.

(1)Rich media grid delivery is a novel model for content delivery initially focused on the needs of the enterprise.

It is a flexible system that utilizes the network and computing resources already present in the enterprise much more efficiently, extremely important in this age of tight IT budgets and staffing.

But the challenge is how to apply Grid Delivery to Internet scale.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

7. The Challenges and Future Prospect

(2) As content becomes more digital, locating and managing the most atomic elements of that content for potentially millions of simultaneous users will be an exponentially difficult challenge to address.

Structured content such as XML-defined contexts combined with a strict adherence to the practices associated with many commercial content management systems will help alleviate some of this burden.

But the sheer volume and diversity of digital content types—text, image, audio, and video, each with their own variety of file formats—will demand a more flexible solution with massive computational capacity. This is one of the problems that Grid Delivery had to solve in the recent future.

(2) As content becomes more digital, locating and managing the most atomic elements of that content for potentially millions of simultaneous users will be an exponentially difficult challenge to address.

Structured content such as XML-defined contexts combined with a strict adherence to the practices associated with many commercial content management systems will help alleviate some of this burden.

But the sheer volume and diversity of digital content types—text, image, audio, and video, each with their own variety of file formats—will demand a more flexible solution with massive computational capacity. This is one of the problems that Grid Delivery had to solve in the recent future.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

7. The Challenges and Future Prospect

(3) Grid Delivery promises to be a viable solution for distributing on-demand streaming content and other large files directly to end-user systems.

Broadcasting live, concurrent, real-time streams using a grid delivery network is possible, but has not yet proven to be scalable. Therefore, researching and developing real-time grid streaming delivery technology is a great challenge.

(4) Grid Delivery technologies offer an advantage for storing and retrieving large datasets efficiently, but that it remains a question whether the Grid is beneficial for storing and managing large numbers of files that vary greatly in size.

(3) Grid Delivery promises to be a viable solution for distributing on-demand streaming content and other large files directly to end-user systems.

Broadcasting live, concurrent, real-time streams using a grid delivery network is possible, but has not yet proven to be scalable. Therefore, researching and developing real-time grid streaming delivery technology is a great challenge.

(4) Grid Delivery technologies offer an advantage for storing and retrieving large datasets efficiently, but that it remains a question whether the Grid is beneficial for storing and managing large numbers of files that vary greatly in size.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

7. The Challenges and Future Prospect

(5) the next Globus Toolkit upgrade (3.0)—to be rolled out over the course of 2003—will incorporate Open Grid Services Architecture (OGSA), providing standardized discovery, management, and monitoring facilities for coordinating multiple Web services and provisioning their associated resources.

While Globus Toolkit 3 marks a milestone, there is still a long way to go before the ability to access distributed computation and resource is as straightforward as access to the Web.

(5) the next Globus Toolkit upgrade (3.0)—to be rolled out over the course of 2003—will incorporate Open Grid Services Architecture (OGSA), providing standardized discovery, management, and monitoring facilities for coordinating multiple Web services and provisioning their associated resources.

While Globus Toolkit 3 marks a milestone, there is still a long way to go before the ability to access distributed computation and resource is as straightforward as access to the Web.

Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China

Thank You For Your Question!!!

Thank You For Your Question!!!