Virtualization of EPOCH IPS - integ.com · Virtualization of EPOCH IPS April, 2016 Tatsuya Watanabe...

Virtualization of EPOCH IPS

April, 2016 Tatsuya Watanabe / Assistant Manager System Engineering Team SKY Perfect JSAT Corporation (SJC)

0

Agenda

1. Overview of SJC and YSCC

2. Current status of virtualization in SJC

3. Function of VMware and how SJC uses it

4. Points of concern about virtualization

Copyright © 2016 Sky Perfect JSAT Corporation. All Rights Reserved 1

Overview of SJC and YSCC

2

** JSAT owns 5 Ku-band transponders on IS-15 Horizons-1*

(127W) GEO

JCSAT-4A (82E)

JCSAT-1B (150E)

JCSAT-2A (154E)

JCSAT-RA (Backup)

N-SAT-110 (Backup)

JCSAT-5A (132E)

JCSAT-3A (128E)

Horizons-2* (84.8E)

36,000km

Superbird-B2 (162E)

JCSAT-85** （Intelsat-15）

(85E)

Superbird-A3 （158E）

N-STAR c (136E)

Superbird-C2 (144E)

* Horizons-1 & 2 are Jointly owned by JSAT and Intelsat.

JCSAT-110R*** (110E)

JCSAT-4B (124E) 3rd Party Customer

Satellite(143E) *** JCSAT-110R is Jointly owned by JSAT and B-SAT.

3rd Party Customer Satellite

JCSAT-14

JCSAT-16

JCSAT-15

Superbird-B3

Satellite Fleet

3 Copyright © 2016 Sky Perfect JSAT Corporation. All Rights Reserved

http://www.skyperfectv.co.jp/

As of 2014/12

’89 ’90 ’91 ’92 ’93 ’94 ’95 ’96 ’97 ’98 ’99 ’00 ’01 ’02 ’03 ’04 ’05 ’06 ’07 ’08 ’09 ’10 ’11 ’12 ’13 ’14

J C S A T -1

J C S A T -2

S u p e rb ird -B

S u p e rb ird -A

J C S A T -3

J C S A T -R

S u p e rb ird -A 3

J C S A T -1 B

J C S A T -4 A

S u p e rb ird -B 2

N -S T A R a

N -S T A R b

N -S A T -1 1 0

J C S A T -2 A

N -S T A R c

H o riz o n s-1

3 rd P a rty S a t-A

J C S A T -5 A

J C S A T -3 A

H o riz o n s-2

S u p e rb ird -C 2

J C S A T -R A

J C S A T -8 5

J C S A T -1 1 0 R

J C S A T -4 B

3 rd P a rty S a t-B

Lockheed Martin 社製

Boeing 社製

Orbital社製

Boeing 社製


D e -o rb it

D e -o rb it

D e -o rb it

D e -o rb it

D e -o rb it

D e -o rb it

Space System Loral 社製

三菱電機製

D e -o rb it

D e -o rb it


Orbital 社製

D e -o rb it

Boeing社製


Space Systems Loral 社製

Orbital 社製

三菱電機社製

NEC 社製

B o e in g

B o e in g

B o e in g

B o e in g

B o e in g

B o e in g

N T T S S L

N T T S S L

L M

B o e in g

O rb ita l

B o e in g

L M

S S L

S S L

B o e in g

B o e in g

S S L

M e lco

L M

L M

O r b ita l

O r b ita l

N T T

N E C

L M

Boeing

Lockheed Martin

Space Systems Loral

Orbital

Melco

NECL M

Satellite Operations Experience


Yokohama Satellite Control Center （YSCC）

Primary Satellite Control Center and Teleport Facilities

Gunma Satellite Control Station （GSCS）

Sub Ground Station

Ibaraki Network Control Center （SPE）

Backup Satellite Control Center and Teleport

Yamaguchi Network Control Center（SPW）

Sub Ground Station

SJC Ground Stations


Satellite Planning & Operations Group

Satellite Operations Division

Noriko Masuda

Satellite Control Team

Yoji Sato

System Engineering Team

Hiroaki Nagai

Satellite Engineering Team

Azuma Miyata

Orbital Dynamics Team

Hideshi Chazono

Service Operations Division

Hideo Hayashi

Network Operations Team

Teleport Operations Team-1

Teleport Operations Team-2

Satelite Facilities Management &

Administration Div.

Tetsuya Segawa

Team-1

Team-2

Team-3

Ground Systems Procurement & Maintenance

General affairs & administrations in YSCC

General affairs & administrations in SPE

YSCC Organization


Current status of virtualization in SJC

7

Reasons to use virtualization

SJC will launch 4 satellites within 2 years. work on reducing the operational load in several ways One of the way is virtualization of EPOCH IPS

Virtualization refers to the creation of a virtual machine that acts like a real computer with an operating system(#).


Traditional Architecture Virtual Architecture

(#)Wikipedia: https://en.wikipedia.org/wiki/Virtualization

https://en.wikipedia.org/wiki/Virtualization

SJC decided to use VMware as a virtualization software for EPOCH IPS.

VMware has a lot of merits as follows: Cost effective

Resource Optimization, Saving Rack space, Consumed Power Functions to Reduce Operational Load

vMotion, Snapshot, cloning, HA, Backup, Converter, and so on.

Last year, SJC verified following points: Essential functions of VMware work as expected EPOCH IPS on Virtual Server is working just like on a Physical Server

Reasons to use virtualization

9

Bus

Type

Boeing

601 (5 satellites)

A2100

(5 satellites)

StarBus2 (1 satellites)

DS2000 (1 satellites)

SMACS (1 satellites)

New

Satellites (4 satellites)

Operations

software TCR/STA

EPOCH v4

OASYS

EPOCH v4

OASYS

BirdStar

ORAMS SMACS

EPOCH v4

OASYS

Coming Soon

Plan to change to EPOCH v4 & OASYS

The relation between virtualization and increase of EPOCH IPS

EPOCH IPS constitutes an increasingly large portion of the SJC. The more the percentage of EPOCH IPS is increasing, the more we’ll see

the effect of virtualization of EPOCH IPS go up.


Consider the way to change to EPOCH IPS without extra cost (because these are old so it’s difficult to secure a budget for now)

Bus Type / Operations Software

SJC has virtualized only part of EPOCH IPS servers.

SJC will continue to virtualize other servers and to investigate the appropriate method to virtualize workstation.

Servers for existing satellites in YSCC

Servers for upcoming satellites in YSCC

Servers in Sub-site

Workstations in YSCC

Workstations in Sub-site

Already virtualized

Virtualization of workstation is under investigation.


Current situation and virtualization plan in the future

Percentage of EPOCH IPS servers and workstations in each site

Functions of VMware and how SJC uses it

12

Physical Configuration Overview

VMware is Installed on both of 2 dell servers Virtual Machines (VM) are saved to a Shared Disk. One of 2 VMware ESXis loads and executes VM onto its memory

Dell Shared Storage for saving VMs

Dell L2SW

VMware ESXi VMware ESXi

Dell Server

EPOCH SV REDHAT

Virtual Machines (VM)


EPOCH SV REDHAT

NAS for backup VMs

Compass Win Server

Compass Win Server

1

2

3

4

5

6

7

8

9

10

2 Dell servers equipped with 2 (6 core,HT) CPUs and 192GB Memory each

2 Dell Layer-2 Switches (back)

NAS equipped with 4 7,200 RPM SATA HDDs

Dell Shared Storage equipped with 24 15,000 RPM SAS HDDs

Saving Rack Space of Physical Configuration

Lots of VM are consolidated and working on these machines. Hardware failure has a significant impact on VM, so it’s necessary to

make each Machine as a redundant configuration.


Copyright © 2016 Sky Perfect JSAT Corporation. All Rights Reserved

Basic License Type of VMware

VMware ESXi

VMware ESXi

VMware ESXi

So far, SJC is using Essentials Plus License of VMware that is reasonable but has many functional restrictions.

This License enables us to: Use basic functions such as vMotion, Snapshot, clone, HA, and VDP

This License doesn’t enable us to: Scale-out more than 3 servers Use advanced functions such as FT, DRS, Storage vMotion, App-HA,

and so on

VMware ESXi

15

vMotion enables running VM to move into another server with zero downtime

SJC confirmed that EPOCH server can finish its vMotion process without

disconnecting from DSS.

vMotion

①Stop ①Activate

DSS

Telemetry

EPOCH SV REDHAT

EPOCH SV REDHAT




④Back to the previous state

Snapshot

Snapshot preserves the state of the VM at a specific point in time.

When you revert to a snapshot, you discard all changes made to the VM since you took the snapshot.

①Take a snapshot

③become Unstable state

⑤return to Stable state

②Apply Patch

Stable state

Clone

Clone is a copy of an existing virtual machine.

If you want to make a copy of a virtual machine for separate use, create a clone.

If you want to save the current state of the virtual machine, so you can revert to that state in case you make a mistake, take a snapshot.


EPOCH SV

REDHAT

EPOCH SV

REDHAT

VMware ESXi

①Cloning

18

Clone


VMware ESXi #2 VMware ESXi #1

1. Virtual Machine Failover

②Reboot VM on the same host

2. Host Machine Failover


③VMs start

monitor

VMware HA

Failover function when Physical or Virtual Machine fails.

①failure

②Stop all powered-on VMs

monitor

VM VM

VM ①failure

VM VM


VMware Data Protection (VDP)


VDP is an agent-less, image-level VM backup tool

Saving a lot of backups causes unstable state of VDP, so SJC set up

backup schedule of VDP as following Once a day for 7 days, Once a week for 4 weeks, Once a month for 12 months

Dell Shared Storage

Dell L2SW

Dell Server


Backup

VDP

EPOCH SV

REDHAT


vCenter Converter

It’s hard to rebuilt already running EPOCH IPS servers as a VM.

vCenter Converter enables us to convert EPOCH IPS into VM easily.

SJC conducted load tests of the converted EPOCH server on VM, and confirmed it works fine.

SJC has already been operating the converted EPOCH Server as a VM.

VMware ESXi 5.5

EPOCH SV

REDHAT

Control Control

Target Physical Servers

Destination

vCenter Converter

EPOCH SV

REDHAT

EPOCH CL

Windows7 Copy

EPOCH CL

Windows7

Other features of VMs

VM is just a file, so it’s easy to move VM between a production environment and a test environment.

EPOCH SV

REDHAT

EPOCH SV

REDHAT

Test environment

VMware Player

EPOCH SV

REDHAT

VMware Player

VMware ESXi (Production environment)

copy

EPOCH SV

REDHAT

copy


EPOCH SV

REDHAT Cloning

Points of concern about virtualization

23

24

Points of resource allocation

It’s necessary to decide resource allocation size of each VM.

Resources are such as number of CPU cores, Memory size, and HDD size

If the resource allocation size is too much, and the total of all the VM’s

actual resource consumption exceeds physical server’s resource size, all

unprotected VMs become unstable.

On the other hand, if the resource allocation size is too little, it affects the

performance.

To prevent these situations, it’s necessary for important VMs such as

EPOCH server to investigate and to reserve (a function to assure allocated

resource) the minimum requirement of resource size.


Resource allocation size of each EPOCH IPS server in SJC

SJC conducted load tests and observed the resource utilization of each

EPOCH IPS server.

These specs are smaller than on Physical servers, yet they’re working fine.

The appropriate size depends on the use case scenario.

EPOCH server for 1 telemetry

Task Initiator

Compass Server

CPU core 2～ 2～ 2～

Memory 8GB～ 4GB～ 4GB～

HDD 100GB～ 30GB～ 30GB～

Specific resource allocation size of each EPOCH IPS server in SJC (#)

(#) The memory and HDD sizes will greatly differ according to whether or not we use the memory over commit and the thin provisioning functions

(SJC tries not to use these functions)


Failure detecting and resource monitoring

Hardware failure and resource shortage on the Physical Machine have a

significant impact on all virtual machines.

SJC built an NMS server to detect Hardware, VMware Hypervisor,

vCenter process failure and to monitor resource utilization.

SNMP Polling & VMware API

SNMP Polling & SNMP Trap

SNMP Trap

NMS Network Management System

iDrac

VMware ESXi

OS

vCenter

Dell Shared Storage

Dell L2SW

Dell Server

Conclusion

27


There was a lot of hard work involved in our first introduction of

virtualization of EPOCH IPS as follows

to consider hardware specs, redundancy design, and migration plan

to prepare the way of failure detecting and resource monitoring (#)

However, VMware functions provide the benefit of increasing the

efficiency of the system operation.

Advantages and Disadvantages of VMware

(#) Preparing fault isolation methods is indispensable. If something happened to a VM, He that is unfamiliar with virtualization may doubt VMware.


Thank you. Please feel free to contact me;

[email protected]

29

mailto:[email protected]



Appendix

30


SJC is working on virtualizes only servers in main site for now, however it’s necessary to consider multi-site design. For example,

SJC needs sufficient number of client only in case of emergency in other site.

There is some possibility to make unused licenses that can be used for a purpose of replace TCR/STA to EPOCH IPS.

Main Site Other Site (Sub-site)

Server

REDHAT

Total optimization in the next stage

Most part of the clients are deactivate in normal time and activate in case of emergency

Client

Windows

Server

REDHAT

Client

Windows

Copy VM by using VMware or Storage replication

31


Ideal design of HA configuration

It may be possible to monitor the EPOCH server on the VM with higher

class license and custom settings of VMware.

This way is smart because it’s not necessary to prepare EPOCH Server on

the each VMware ESXi, but we have to consider the failover downtime,

reconnection with client / baseband, and get over some hurdles as

mentioned above.

Redhat

VMware ESXi #1

monitor

EPOCH Server Processes

monitor

①failure

②reboot Application

VMware ESXi #2 monitor


VMware HA in SJC

Redhat


monitor monitor

EPOCH Server Process

Not monitor

If the application's process finishes, we want VMware to reboot the

application to recover. However, due to the licensing constraints and

other problems, this is not realized for now.

SJC prepares EPOCH server on the each VMware ESXi for redundancy.

Telemetry


Redhat

monitor

Not monitor

BB

Problem that we have to solve


Redhat


monitor monitor


Not monitor


Redhat

monitor

Not monitor

NMS

monitor

SJC needs the way to confirm whether or not EPOCH IPS server process is

alive without monitoring GUI. As one of the methods, we can use “ps aux

| grep epoch”, but it’s an uncertain way.

Does anyone know any monitoring functions of the EPOCH server like the

“statusEpoch” command?

34

Date post:	11-Mar-2019
Category:	Documents
Upload:	hoangkiet
View:	234 times
Download:	0 times

Virtualization of EPOCH IPS - integ.com · Virtualization of EPOCH IPS April, 2016 Tatsuya Watanabe...

Documents