Testing… Testing… 1, 2, 3.x...

Performance Testing of Pi on NT

George Krc

Mead Paper

What are we trying to do?

O verview

P i P ro file S ys tem(P i 3 on N T)

P i P ro file S ys tem(P i 3 on N T)

P i P ro file S ys tem(P I 3 on N T)

M a in P i S ys tem(P i 2 on V M S )

PI 3 Performance Testing

What are we trying to find out?– Is Pi 3 on NT suitable for gauging data

applications?– How does DEC Alpha compare with Intel

hardware running Windows NT?– How big a box do we need?– How much network bandwidth will be used?

Test Configurations

Configuration #1: “Prioris”– Digital Prioris 6200 MX Server

• Dual Pentium Pro 200MHz Processors• 128mB RAM• 10gb Hard Disk• Fast-wide SCSI controller• PCI 10/100mb Ethernet Carrd

Test Configurations

Configuration #2: “Alpha”– Digital AlphaStation 600 5/266

• Single Alpha 266MHz Processor• 128mB RAM• 10gb Hard Disk• Fast-wide SCSI controller• PCI 10/100mb Ethernet Carrd

Test Configurations

Configuration #3: “Dell”– Dell PowerEdge 6100 Server

• Four Pentium Pro 200MHz Processors• 256mB RAM• 10gb Hard Disk• Fast-wide SCSI controller• PCI 10/100mb Ethernet Carrd

Test Conditions

Pi Interface on Separate NT API node Duplicate data sent to test systems Gauging systems scans every 30 seconds Data transmission for 15 seconds Typical data rates in 30 seconds

– 6633 snapshot events– 6236 archive events– 26 archive posts

More Test Conditions

Pi version used was 3.1, Build 2.81 CPU usage for four Pi modules (PISnapSS,

PiNetMgr, PiArchss, PiBaseSS) Statistics gathered with NT Perf. Monitor Data gathered over 500 seconds intervals Total CPU usage measured across multiple

processors

Test #1

Compare the CPU Utilization of the Alpha and Prioris Servers under normal data gathering conditions

CPU UtilizationPi Data Gathering

5.73

2.08

0.01

3.29

13.64

9.47

1.6

0

2.15

6.99

0

2

4

6

8

10

12

14

Alpha Prioris

PisnapssPinetmgrPibasessPiarchssTotal

Test #1 Inferences

Prioris outperformed the Alpha by about 2:1 on equal processor speed basis

Testing in production environments can be erratic

Test #2

Compare the CPU Utilization of the Alpha and Prioris servers and Profile client response time under two conditions:– One Sensor, 250 Scans, 480 Boxes, 30 second

wait– Three Sensors, 250 Scans, 480 Boxes, 60 second

wait

CPU UtilizationWith Client Profile Access

6.1412.18

0.03

13.18

36.83

6.07

24.8

0.06

22.22

59.38

10.71

1.580.01

9.19

17.3

10.66

26.5

0.01

31.6635.57

0

10

20

30

40

50

60

Alpha - 1Sensor

Alpha - 3Sensors

Prioris - 1Sensor

Prioris - 3Sensors

PiSnapSSPiNetMgrPiBaseSSPiArchSSTotal

CPU UtilizationWith Client Profile Access

13.64

36.82

59.38

6.99

17.3

35.57

0

10

20

30

40

50

60

Alpha Prioris

Normal1 Sensor3 Sensors

Response TimeClient Profile Access

16.5

11.5

0

2

4

6

8

10

12

14

16

18

Alpha Prioris

3 Sensors

Test #2 Inferences

A single client increases Alpha utilization by a factor of 2:1

Multiple profile continues to increase utilization, but by less than a factor of 2:1

CPU utilization advantages on the Prioris give better user response times

The Alpha could support 8 continuous profiles, the Prioris about 15

Test #3

Compare the despooling capabilities of the Alpha and Prioris servers

Despooling TestOne Hour of Data (2300 records)

360

95

480

45

050

100

150200

250300

350400

450500

Alpha Prioris

Time (seconds)CPU usage

Test #3 Inferences

The Alpha despools significantly faster than the Prioris

The Prioris is limited by I/O throughput

Test #4

Compare the CPU Utilization of the Dell and Prioris Servers under normal data gathering conditions

CPU UtilizationPi Data Gathering

2.97

1.59

0

2.662.29

5.32

1.58

0

2.79

6.42

0

1

2

3

4

5

6

7

Dell Prioris

PisnapssPinetmgrPibasessPiarchssTotal

Test #4 Inferences

The Dell matches the Prioris on a per-processor basis

The Dell maintains a 2:1 performance ratio The application scales well to a quad

processor system The Dell gives much better performance of

the snapshot subsystem (extra memory???)

Test #5

Compare the CPU Utilization of the Dell and Prioris servers and Profile client response time under two conditions:– One Sensor, 250 Scans, 480 Boxes, 30 second

wait– Three Sensors, 250 Scans, 480 Boxes, 60 second

wait

CPU UtilizationWith Client Profile Access

3.11

17.29

0

10.268.42

3.9

25.6

0

27.26

13.18

3.71

17.92

0.01

16.98

22.06

0

5

10

15

20

25

30

Dell - 1 Sensor Dell - 3 Sensors Prioris - 1Sensor

PiSnapSSPiNetMgrPiBaseSSPiArchSSTotal

Test #5 Inferences

The Dell demonstrates a better than 2:1 performance increase over the Prioris with twice the processors

The CPU utilization reduction does not translate to a lower response time to the client application

The Dell could support up to 60 continuous profile client accesses

Test #6

Determine Network Loading under 4 conditions– No Pi Activity– Data Gathering Only– 60 Second Client Profile Access– 30 Second Client Profile Access

Network Utilization

12.3212.1320.0318

7 81717

86 878990

0

10

20

30

4050

60

70

80

90

Frames ( x10,000)

AverageUtilization

PeakUtilization

Base LineData Gathering60 Sec. Profiles30 Sec. Profiles

Test #6 Inferences

Data gathering does not put a significant loading on the network

Client access puts an extremely high loading on the network

100mb ethernet will be required for the client application

SO WHAT????

– Pi 3 on Windows NT is suitable for gauging data– Both Alpha and Intel are suitable, but the Intel

architecture is more cost-effective– The minimum reasonable configuration is a dual

Pentium Pro, 200 MHz system with 256 mB RAM and a 100mb ethernet card

AND...

– A reasonable limit for the Pi tag count is 10,000 tags

– 10mb ethernet is insufficient for the application– Pi 3.1 Build 2.81 had a few bugs