JPSS CommonGround System

JPSS CGS IDPS Product Generation

January, 2012

Raytheon

David C Smith, JPSS CGS Chief Architect

Kerry Grant, JPSS CGS Chief Scientist

JPSS CommonGround System

Page 2Page 2

JPSS System

mm/dd/yyyy

SDS

NESDISAFWAFNMOCNAVO

C3SegmentC3Segment Field

Terminal SegmentFieldTerminal Segment

GPS

SvalbardPrimary T&CNPP SMD

SvalbardPrimary T&CNPP SMD

TDRSS

LaunchSupportSegment

LaunchSupportSegment MMC at Suitland

Flight Operations Team• Enterprise Management• Mission Management• Satellite Operations• Data Monitoring & Recovery

White Sands ComplexLEO&A Backup T&C

White Sands ComplexLEO&A Backup T&C

A-DCS

SARSAT

HRDField

Terminal

HRDField

Terminal

LRDField

Terminal

LRDField

Terminal

Aurora MMCContingency Operations Team

Data Handling Nodes reside at each Central

15 Globally DistributedReceptor Sites Interconnectedby Commercial Fiber

15 Globally DistributedReceptor Sites Interconnectedby Commercial Fiber

JPSS Stored Mission Data Command and Telemetry

Interface Data Processing SegmentInterface Data Processing SegmentOne full set resides in each of the 4 CentralsOne full set resides in each of the 4 Centrals

Data MgtData MgtInfra

IngestIngest

Proc

ess

Proc

e ss

Data DelData Del

CLASS

TDRSS

NPP Stored Mission Data

Data MgtData MgtInfra

IngestIngest

Proc

ess

Proc

e ss

Data DelData Del

Data MgtData MgtInfra

IngestIngest

Proc

ess

Proc

e ss

Data DelData Del

Data MgtData MgtInfra

IngestIngest

Proc

ess

Proc

e ss

Data DelData Del

Offline SupportOffline Support

NPP (1330) 1330 1730

JPSS & DWSSSatellitesResiduals

SpaceSegment

DQMDQM

JPSS CommonGround System

Page 3Page 3

Remote Sensing System Evolution

mm/dd/yyyy

DMSP(Defense Meteorological

Satellite Program)

EOS (Earth Observing

System)

NPOESS (National Polar-orbiting

Operational Environmental Satellite System)

Sensor data rate: 1.5 MbpsData latency: 100-150 min.

1.7GigaBytesper day (DMSP)6.3GigaBytesper day (POES)

15 Mbps sensor data rateData latency: 100-180 min.Data availability: 98%Ground revisit time: 12 hrs.

2.6TeraBytesper day (EOS)2.4TeraBytesper day (NPP)

20 Mbps sensor data rateData latency: 28 min.Data availability: 99.98%Autonomy capability: 60 daysSelective encryption/deniabilityGround revisit time: 4-6 hrs.

8.1 TeraBytesper day

POES(Polar Orbiting

Operational Environmental Satellites)

NPP(NPOESS

Preparatory Project)

1960 -2010 2000 -2010 2010 –2020+

DMSP(Defense Meteorological

Satellite Program)

EOS (Earth Observing

System)

NPOESS

Sensor data rate: 1.5 MbpsData latency: 100-150 min.

1.7GigaBytesper day (DMSP)6.3GigaBytesper day (POES)

15 Mbps sensor data rateData latency: 100-180 min.Data availability: 98%Ground revisit time: 12 hrs.

2.6TeraBytesper day (EOS)2.4TeraBytesper day (NPP)

20 Mbps sensor data rateData latency: 28 min.Data availability: 99.98%Autonomy capability: 60 daysSelective encryption/deniabilityGround revisit time: 4-6 hrs.

8.1 TeraBytesper day

POES(Polar Orbiting

Operational Environmental Satellites)

NPP(NPOESS

Preparatory Project)

1960 -2010 2000 -2010 2010

DMSP(Defense Meteorological

Satellite Program)

EOS (Earth Observing

System)

JPSS

Sensor data rate: 1.5 MbpsData latency: 100-150 min.

1.7GigaBytesper day (DMSP)6.3GigaBytes per day

(POES)

15 Mbps sensor data rateData latency: 100-180 min.Data availability: 98%Ground revisit time: 12 hrs.

2.6TeraBytesper day (EOS)3.4TeraBytesper day (NPP )

20 Mbps sensor data rateData latency: 28 min.Data availability: 99.98%Autonomy capability: 60 daysSelective encryption/deniabilityGround revisit time: 4-6 hrs.

8.1 TeraBytesper day

POES(Polar Orbiting

Operational Environmental Satellites)

NPP(NPOESS

Preparatory Project)

1960 -2015 2000 -2017 2016 +

The evolution of Government remote sensing systems over the last 40 years. JPSS, the Joint Polar Satellite System, is the next generation low-earth orbiting environmental remote sensing platform. JPSS will play a pivotal role in our nation’s weather forecasting and environmental awareness for the next two decades. * -- current estimates

(Joint Polar Satellite System)

JPSS CommonGround System

Page 4Page 4

Mission data processing is performed by the Interface Data Processing Segment (IPDS)

The IDP segment combines software and hardware flexibility, expandability, and robustness to meet stringent performance requirements levied by the Joint Polar Satellite System Common Ground System (JPSS CGS) requirements.

Sensor application packets are passed to IDPS. The data stream is broken into granules, which are a subsectioning of the data stream into manageable time intervals. The granules of data can be processed in parallel by IDPS thus ensuring processing of high quality products within latency timelines.

Mission Data Processing Capabilities

mm/dd/yyyy

JPSS CommonGround System

Page 5Page 5

Ingest (ING)– The ING Subsystem creates sensor Raw Data Records (RDR) from multiple Sensor Application Packet streams received from C3S. It

separates the incoming streams into granules by sensor. Ingest extracts JPSS Auxiliary data from Stored Mission Data (SMD) and create RDRs or Bus TM and S/C Diary information. It also accepts external Ancillary data sets that are required for EDR processing.

Processing (PRO)– The PRO Subsystem encapsulates all of the data algorithms that must be executed to turn the RDRs into higher level products. First

processing creates sensor specific SDRs or Temperature Data Records (TDRs) from RDRs. These are corrected, calibrated and geolocated sensor data. A complex set of processing chains (see Figure 6) is then used to produce the required 46 EDRs. Some of these are produced as individual products or one algorithm may yield a group of related EDR products.

Data Management (DMS)– The DMS Subsystem provides internal short-term (24 hour requirement) storage of all JPSS data, as well as management of shared memory

(cache), which is a critical component of the IDPS in meeting data product latency.

Infrastructure (INF)– The INF Subsystem provides the Workflow management functions for IDPS. It has total control of process startup, monitoring, shutdown and

re-start upon error conditions. INF also provides common utilities and tools, such as logging, debug, timers, performance monitoring, data availability and accounting, and HW monitoring.

Data Delivery (DDS)– The DDS Subsystem is the single provider of all data between IDPS and the local Central. It converts requested products into Hierarchical

Data Format 5 (HDF5) format with data and metadata aggregation. HDF5 is a self-describing data format with community supplied implementation libraries.

Data Quality Monitoring (DQM)– The DQM Subsystem provides the Data Quality Engineer automated and ad-hoc processing in support of Data Quality Notifications from the

Processing system. The Data Quality Engineer is provided with a tool kit of Geographic Information System based modules that allow the IDPS data to be registered to a geographic grid and analyzed, viewed, and trended. The DQM and the DQE support the program’s larger Calibration and Validation (Cal/Val) activities and supports the troubleshooting of data anomalies.

Mission Data Processing Subsystem Capabilities

mm/dd/yyyy

JPSS CommonGround System

Page 6Page 6

JPSS EDR Latency

mm/dd/yyyy

The JPSS sensor suites on the JPSS vehicles produce data at up to a 20 MB/sec rate. This is an increase over DMSP of more than 13:1. The total xDR (RDR, SDR and EDR) products produced by an instance of IDPS per day have a volume increase over DMSP of 2000:1 In addition to handling these increased rates, latency requirements (defined as time from sensing of phenomena to production and delivering of products) decreases by a factor of 4. A graphical representation of the latency requirements that are imposed on JPSS CGS and projected actual performance is contained in Figure 3.

0:00

10:00

20:00

30:00

40:00

50:00

60:00

70:00

80:00

90:00

100:00

110:00

120:00

130:00

140:00

150:00

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Syst

em L

aten

cy (

mm

:ss)

% of Products in 1 Orbit

NPP System Latency for 44-orbit 4-copy run (orbit 17)

CrIMSS EDR

VIIRS EDR

VIIRS IP

ATMS RDR

CRIS RDR

VIIRS RDR

ATMS SDR

CRIS SDR

VIIRS SDR

ATMS TDR

System Latency Req't

Data Age @ IDPS Arrival

Final product for each chain shown- Most products are delivered earlier

JPSS CommonGround System

Page 7Page 7

JPSS products are generated from a complex network of processing algorithms.

A number of interdependencies between algorithms exist in order to provide the required data quality to the end User.

The following slides illustrate the interdependencies between products and processes within the IDPS needed to produce the NPP-era SDRs, EDRs, and Gridded Intermediate Products (GIP’s).

The execution of this processing flow for each granule of data is managed within the Infrastructure SI by the Workflow Manager (WFM) component through the use of a configurable Data Processing Guide (DPG).

Algorithm Workflow

mm/dd/yyyy

JPSS CommonGround System

Page 8Page 8

VIIRS Algorithm Data and Control Flow

mm/dd/yyyy

COP

VIIRS SDR

LSTSfcTyp

SIQ/Sfc Tmp SIC

VI**

APSMAPSP**

OCCSST

Land AlbNCC

M-Band / GTM

VFM

CCL CBHCTP GCE

VIIRS RDR IST

SIA

I-Band / GTM

SfcRflVCM

PPC

NHF/Ocn Alb

Comb AlbSnow

AOT

RDRs SDRs

SensorCal Tables

Geoloc IPDual-GainBand IP

OBC IP

Geoid

Ephem. &Pointing

AF ARP VCM IP

NCEPData

COP IPIce/Night

Water CTT IP

CTP IP

Para CorrCOP IP

Para CorrCTP IP

Para CorrVCM IP

C Cover IP

C Type IP

C LayerType IP

AggrGeo IP C Base

Height IP CBH

CC/L

CTP

CTH

CTT

CEPS

COP

GTM to DNBSwath Map IP

Imagery

GTM to ModSwath Map IP

GTM to ImgSwath Map IP Ice Conc

IP

Sea SurfTemp

Snow/IceGIP

NDVIGIP

SR/VI/BTGIP

MODISL/W Msk

Land AlbIP

Oc AlbIP

Ice AlbIP

SurfAlb

NAAPS

AMI IP

AOT

Susp Mat

APSAPS

APS IP

AOT IP

Ice QualIP

Surf TempIP

Ice WgtsIP

Ice ReflTempr IP

Prev IceAge GIP

Sea IceCharac

Ice SurfTemp

OCC

Net HeatFlux

SRT30+Nitrate

Depl Temp

CMN GEOCMN GEO

Surf ReflIP

SnowCover

VegIndex

Land AlbGIP

SurfType

Land SurfTemp

Ancillary Data S/C DataProcessesIntermediate

ProductsDeliveredProducts

Process outputProcess output

Process inputProcess input

Process controlProcess control

VIIRSAPs

Ann M/MVI GIP

Gn2GdGn2Gd

Dly SurfRefl GIP

QSTGIP

Gn2GdGn2Gd

DEM

Gn2Gd

JPSS CommonGround System

Page 9Page 9

CrIMSS Algorithm Data and Control Flow

Interface Data Processing Segment ArchitectureFigure 1----

ATMS RDR

CrISRDRs

ATMSCal Tables

Geoid &DEM

Ephem. &Pointing

ATMSAPs

CrISCal Tables

CrISAPs

CrIS RDR

CrIS SDR

CMN GEO

ATMS SDR

CMN GEO

ATMSRDRs

ATMSTDRs

ATMSSDRs

CrISSDRs

ATMS Remap

ATMSSDRs

(remapped)

CrIMSS

QSTGIP

NCEPData

Atm VertTempr Profile

Atm VertPress Profile

Atm VertMoist Profile

JPSS CommonGround System

Page 10Page 10

OMPS Algorithm Data and Control Flow

mm/dd/yyyymm/dd/yyyy

NP RDR

NP EVRDRs

Geoid &DEM

Ephem. &Pointing

OMPS TCCal & EV

APs

OMPS NPCal & EV

APs

TC RDR

NP EV SDR

CMN GEOCMN GEO

TC EV SDR

CMN GEOCMN GEO

TC EVRDRs

TC EVSDRs

NP EVSDRs

Total ColumnEV EDRs

Total ColumnEV IPs

Nadir ProfileEV IPs

OMPS LPCal & EV

APs

LP RDR

TLM RDR

OMPSHKPTLM

LP EVRDRs

TLMRDRs

NP CalRDRs

TC CalRDRs

LP CalRDRs

TC Cal SDR

CMN GEOCMN GEO

NP Cal SDR

CMN GEO

NP Cal SDR

CMN GEOCMN GEO

QST SIC

NP CalTables

NP CalSDRs

TC CalSDRs

TC CalTables

NP IP/EDR

TC EDR

TC IP

VIIRSEDRs/IPs

CrIS/SDRs

CrIMSSEDRs/IPs

JPSS CommonGround System

Page 11Page 11

Interface Data Processing Segment Architecture

mm/dd/yyyy

JPSS CommonGround System

Page 12Page 12

IDPS Hardware Architecture

mm/dd/yyyy

IDPS utilizes a mix of IBM System p servers and Storage Area Network (SAN) / Fibre Channel technology to meet the program’s demanding data product latencies, assure fast and successful delivery of data to Users, provide very high operational availability, and allow for significant expandability to meet any changes to support JPSS objectives.

INFING Hot Spare

Admin

DQM

I&T ING I&T INFI&T DQM

Inte

rnal W

eb

Exte

rnal W

eb

HM

C 1

HM

C 2

RD

BM

S-1

RD

BM

S-2

RD

BM

S-3

RD

BM

S-4

ACI Fiber Channel Switches

PRO1 PRO2 DDS I&T PRO1 I&T PRO2I&T DDS

4Gbps IDPS Fiber Channel Switches

Ethernet Switches

System p

System p

System p

System p

JPSS CommonGround System

Page 13Page 13

JPSS data processing is designed to provide high-quality environmental and meteorological data to the JPSS System Users with very low latency

Leverages highly flexible and expandable, robust hardware to maximize data availability, operational availability, and assured delivery

The SW architecture provides an efficient solution for NPP and provides a scalability to meet future JPSS needs.

SUMMARY

mm/dd/yyyy