LTP Data Analysis: Status
M Hewitson for the LTP DA TeamSWT Meeting
ESTEC, Feb 2nd 2010
+----------------------------------------------------+| || || **** || ** || ------------- || //// / \\\\ || /// / \\\ || | / | || ** | +----+ / +----+ | ** || ***| | |//-------| | |*** || ** | +----+ /+----+ | ** || | / | || \\\ / /// || \\\\ // //// || ------------- || ** || **** || || Welcome to LTPDA Toolbox || || Version: 2.2 || Release: (R2009a) || Date: 23-09-09 || |+----------------------------------------------------+
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Contents
• Toolbox• new features• releases
• Current activities• in labs• algorithms• MDC3• STOC Exercises
• Project status• release and testing schedule
2
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
The Team
3
Adrien Grynagier
Martin Hewitson Miquel Nofrarias Ingo Diepholz Anneke Monskey Heather Audley
Michele Armano Marc Diaz Aguiló
Andrea Manica Andrea Mattioli
Mauro Hueller Luigi Ferraioli
Giuseppe Congedo Fabrizio De Marchi
Boutheina Rais
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
DA Infrastructure
4
MUST (raw data) Converter Repository
Web interface
LTPDA Toolbox
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
LTPDA Repository
5
RepositoryMySQL Server
Apache Web Server Web browser
LTPDA (MATLAB)
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Repository Status
• Version 2.2 just released• fully functional web interface:
• user interface• administration interface
• built-in ‘robot’ for dumping objects to XML store• supports simple update procedures• guided installation through web-browser
• In (heavy) use (10’s of thousands of objects)• AEI
• MDCs, LTPDA Testing, OMS hardware tests• UTN
• MDCs, LTPDA Testing, FE tests• STOC
• LSS testing, LSS and STOC Exercises• Glasgow, Birmingham, Imperial
6
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
DA Infrastructure
7
MUST (raw data) Converter Repository
Web interface
LTPDA Toolbox
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
DA Infrastructure
8
LTPDA Toolbox
GUIs
Algorithms Utilities
Repository Client
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Graphical User Interfaces
9
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Algorithms
10
AO
• Standard data container for• y-data, xy-data, time-series, freq-
series, xyz-data• Algorithms for• spectral estimations• linear and non-linear fitting• filtering• whitening• standard math
MFIR/MIIR/Filterbank
• Implementation of digital filters• Compute response• Create from various models• pole/zero models• statespace models• ....
SSM
• Implementation of statespace models
• Many ‘built-in’ models for LTP• Methods for• assembling systems• setting parameters• discretisation• extracting TFs• time-domain simulations
pzmodel/parfrac/rational
• Implementation of transfer functions• pole/zero models• partial fractions• rational function
smodel• Parametric models of 1 dimension• Parameter substitution• Evaluation to AO• Various symbolic transformations• fourier, ifourier• laplace, ilaplace• diff, ztrans, ....
matrix/collection
• Containers for other LTPDA objects• matrix: e.g. 2D array of AOs• collection: cell-array of objects
• Various methods• matrix: filtering, det, math, conj, ...• collection: useful for repository
interaction
... and much more...
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Repository Client
11
• Interact with LTPDA repository from within MATLAB• query repository to search for objects• submit one or more LTPDA object to repository• retrieve one or more LTPDA object from repository• update data for an object in the repository• move/copy objects between repositories (coming soon)
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Current DA activities
• Toolbox development• structure mostly done • work still needed on algorithms
• maintenance, bug fixing, final delivery• Analysis preparation• mock-data challenges• STOC exercises• technical notes
• Lab hardware testing• OMS EM test campaign in Hannover• Torsion pendulum + FM testing of GRS in
12
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010 13
LTP DA Usage
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
OMS Test Campaign
• Compliment the formal analysis by Astrium• Heavy use of repository• Very realistic treatment of • telemetry packets• analysis routines• ‘real’ data
14
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Data flow
15
RAW DATA FILE(128.3,128.4)
RAW DATA AO
RAW DATA AO
RAW DATA AO
RAW DATA AO
TEST CASEAO
SubmitToRepository CreateTestCase
Definitionfile
Method does:
1. Reading Raw data files (128.3,128.4) in HEX format
2. Split regardless of experiment, by: - If gap > 1s - or if duration > 3600s - or if SID changes 3. Create Raw data AOs and sets
unit,name and description according to definition file.
4. Raw data AOs defined by Metadata: - Exp. title = 'OMS EM Raw data' - reference_id = 't0' - quantity = package ID (e.g. 128.3)
Method checks: 1. Is sampling frequency ok? 2. Do we get the expected number of AOs?
Method does:
1. Create AOs according to test step-by-step 2. Adds definition file information to
AO (files needed for calibration) or to Metadata (experiment id,etc.).
3. Collapse X's to fs =10 if check is ok, if not keeps original X's.
4. TestCase AO defined by: - Exp. title = 'OMS-C-K1' - reference_id = 'S4.1-12' Method history:
Contains history steps up to Raw data AO. No Raw datafile needed to rebuild
% Global fields
TestCase.experiment_title = 'OMS-C-K1';
TestCase.keywords = 'TP-3045,DWS-K1,DWS-C1';
TestCase.additional_authors = '';
% Step 4.1 - 4.12
n = 1;
TestCase(n).experiment_description = 'S4.1-S4.12: wipe piezos [-8V,8V], adjusting piezo manually ';
TestCase(n).analysis_description = 'Splitting by SID';
TestCase(n).quantity = {'128.3', '128.4 (SID9001)'};
TestCase(n).reference_ids = 'S4.1-12';
TestCase(n).description = 'OMS_Param_1p7_1p1_1p0_1p1, OMS-C_COEFF-L1.';
TestCase(n).additional_comments = 'OMS_Param_1p7_1p1_1p0_1p1, OMS-C_COEFF-L1.';
TestCase(n).t1 = [date '08:54:36.883'];
TestCase(n).t2 = [date '13:02:00.883'];
Repository
Definitionfile
HEXconverter % dtype % name % unit % description
addDef(18, @omsHex2single, 'tsdata', 'HORIZ_DWS_F', 'rad', 'LST15683');
addDef(19, @omsHex2single, 'tsdata', 'VERT_DWS_F', 'rad', 'LST15684');
addDef(20, @omsHex2single, 'tsdata', 'HORIZ_DWS_R', 'rad', 'LST15700');
addDef(21, @omsHex2single, 'tsdata', 'VERT_DWS_R', 'rad', 'LST15701');
addDef(22, @omsHex2single, 'tsdata', 'HORIZ_DC_R', 'rad', 'LST15698');
addDef(23, @omsHex2single, 'tsdata', 'VERT_DC_R', 'rad', 'LST15699');
addDef(24, @omsHex2single, 'tsdata', 'CTAST_1', '', 'LST15663');
addDef(25, @omsHex2single, 'tsdata', 'CTAST_12', '', 'LST15680');
addDef(26, @omsHex2single, 'tsdata', 'CTAST_F', '', 'LST15697');
addDef(27, @omsHex2single, 'tsdata', 'CTAST_R', '', 'LST15714');
addDef(28, @omsHex2single, 'tsdata', 'HORIZ_DC_F', 'rad', 'LST15681');
addDef(29, @omsHex2single, 'tsdata', 'VERT_DC_F', 'rad', 'LST15682');
Data flow
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Handling telemetry packets
• Repository contains two RawData AOs for EM campaign• Laser House-keeping [ Pack.ID: 3.25 (1Hz) ]• Telemetry [ Pack.ID: 128.3/128.4 (10Hz)]
• RawData AOs are created from telemetry data• set basic properties (name, units,...)• sampling frequency is not exactly 10Hz due to DMU time
stamping• Individual RawData AOs are created if:• gap > 10 samples• duration > 3600s• SID changes
• RawData AOs are identified in the repository by• Experiment Title (e.g. ‘OMS EM Raw Data’ )• Name (e.g. 'FAST_LOOP_CTRL_OUTPUT32')• Timespan
16
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Example
17
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
New functions to handle telemetry
18
omsSplitFileBySegment Splits by segments. A segment is specified by: 1.) The package ID changes (only TLM 128.4) 2.) ‘Large’ gap 3.) The file exceeds the max number of seconds
omsSplitLA_HK_FileBySegment Idem. for Laser Assembly House-keeping
omsSubmitFilesToDatabase Submits AOs collections to repository
omsSubmitLA_HK FilesToDatabase Idem. for Laser Assembly House-keeping
omsIdentifyGaps Find gaps in the original telemetry package
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Functions for analysis
19
oms_em_retrieve_raw data built-in model: gets AO from repository given timespan
omsGetTelemetry (beta)
Used to get relevant information for the analysis stored in telemetry/telecommand. Eg: signal amplitude, sig. frequency, signal t0,...
analysis_OMS_OPD Full characterisation of OPD control loop: OLG, CLG, delay, fit pole/zero andmoldel
analysis_OMS_Frequency Idem. for Frequency Loop
analysis_OMS_Cs Get phasemeter calibration factors
analysis_OMS_Ks Get interferometer geometric calibration factors
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Extracting the goodness
• Utility scripts can be used • to enhance AO converter• in further test campaigns (LA FM, OMS FM)
• Analysis scripts can be directly converted to LTPDA methods and used in the mission
20
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Example: Frequency Loop analysis
21
Frequency Control loop: all loop variables as DMU variables
G
HSlow
ψR
xs
ε
xo
HFast
OPD_LOOP_CONTRL_OUTPUT32
OPD_LOOP_CONTROL_ERROR32
OPD_LOOP_VAR_SETPOINT32
LONG_PHASE_R32
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Frequency Loop analysis: the data
• LSM10031: 128.4 telemetry (user-defined data package)• SID set to 9030 to get on-line telemetry • Data: (x0,xS,ε,ᴪR) at 10Hz
• LSM10031: 128.4 telemetry (user-defined data package)• SID set to 9031 to get Interferometer Data Log (IDL)• Data: (x0,xS,ε,ᴪR) at 100Hz
• LSC10127: start IDL download tele-command • Data: Start time for IDL data
• LSC10088: set Testing Frequency Offset tele-command• Data: Start time, amplitude, frequency for the sinusoid
• LSC10129: stop Testing Frequency Offset tele-command• Data: Stop time for the sinusoid
22
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Frequency Loop: get AOs from repository
23
date = ['2009-12-02',' '];t1 = [date '09:00:00.100'];t2 = [date '12:00:00.000']; % get 9030 objectsname = 'FAST_LOOP_CTRL_OUTPUT32';
pl = plist(... 'database' ,'oms_em_tests',... 'package ID' ,'128.4 (SID9030)',... 'names' ,name,... 'start time' ,t1,... 'stop time' ,t2);
Out = ao(plist('built-in', 'retrieve_oms_raw_data_by_timespan'),pl);
0 2000 4000 6000 8000 10000−15
−10
−5
0
5
10
15Time origin: 2009−12−02 09:02:48.000
Am
plitu
de [V
]
Time [s]
FAST_LOOP_CTRL_OUTPUT32
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Frequency Loop: get relevant segments
Start and end time from tele-commands are crucial to get the signal data
24
0 1000 2000 3000 4000 5000 6000 7000−10
−8
−6
−4
−2
0
2
4
6
8
10Time origin: 2009−11−26 17:47:30.000
[r
ad]
Time [s]
0 1000 2000 3000 4000 5000 6000 7000−10
−8
−6
−4
−2
0
2
4
6
8
10Time origin: 2009−11−26 17:47:30.000
[r
ad]
Time [s]
Set
Set IDL
Original data Only non-red segments are selected
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Frequency Loop: Open Loop Gain
25
Compute OLG: full characterisation needs both 128.3 and 128.4 data to be used
10−2
10−1
100
101
10−2
100
102
104
am
plit
ude []
Frequency [Hz]
10−2
10−1
100
101
−200
−100
0
100
200
imag
Frequency [Hz]
OLG
OLG (IDL)
f−1
OLG
OLG (IDL)
OLG =DFT( R)
DFT(")
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Frequency Loop: loop delay
26
Delay is obtained as linear fit of IDL data
0 2 4 6 8 10 12 14 16 18 20−2
−1
0
1
2
3
4
5
am
plit
ude []
Frequency [Hz]
unwrap(angle(split(OLG (IDL))))
4.7305*os + −0.29422*xs
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Frequency Loop: model
27
Vector fit applied to get pole/zero model
10−2
10−1
100
101
102
10−4
10−2
100
102
104
am
plit
ude []
Frequency [Hz]
10−2
10−1
100
101
102
−200
−100
0
100
200
Phase [deg]
Frequency [Hz]
OLG
OLG (IDL)
model
OLG
OLG (IDL)
model
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Towards data analysis integration with ASD
• EM test campaign data analysis performed independently by AEI and ASD groups
• Equivalent results so far: • Quantitative test performed in PSD estimate from AEI and
ASD tools. No difference.• Control loop analysis lead to equal delay and loop
parameters (OLG, CLG, Controller,etc.)• LTPDA handling of the repository and analysis tools
proved to be efficient• OMS EM campaign worked as LTPDA set-up test• Ongoing work for common use of LTPDA during OMS FM
28
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
LTPDA at UTN
• Single test mass pendulum facility: • AO converter running• noise estimate almost complete• coherent analysis: next
• 4 test mass pendulum facility, testing Flight Model Replica GRS : • AO converter running• other analysis: mostly “variation on the theme”
• Other LTP activities (magnetics):• Complete chain in place
29
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Torsion pendulum data analysis: preprocessing
30
AO#retrieve#Transducer#gain#scaling#+#rota4on#
Raw#data# Calibrated#data# Preprocessing#data#
AC#
AO#retrieve#EMS#
Time#range#
Calibra4on#matrix#
Sensor/channel#names#
LockDin#gain#
Angle#
Zero#offset#
Transducer#gain#scaling#
Drop#duplicates#+#
Interpolate#missing#
+#Fix#sampling#freq.#
Sampling#freq.#
Drop#duplicates#+#
Interpolate#missing#
+#Fix#sampling#freq.#
Filtering#+##
downsampling#
Filtering#+##
downsampling#
Sampling#freq.#
Filter#
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Torsion pendulum data analysis: Conversion to torque
31
Torque'conversion'
Torque'conversion'
AC'
EMS'
Quality'factor'
Deriva9on'method'
Moment'of'iner9a'
Period'
Torque'conversion'
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Calibration
32
Built&a&bilinear&fit&(mul/linear)&method&
Handling&uncertain/es&
φEMSηEMS
#
$%%
&
'((=
∂φ∂Vx1
∂φ∂Vx2
∂η∂Vx1
∂η∂Vx2
#
$
%%%%%
&
'
(((((
Vx1Vx2
#
$%%
&
'((−
φoffηoff
#
$%%
&
'((=
φACηAC
#
$%%
&
'((
Applied&to&EMS&data&Applied&to&AC&
data&
Calibra/on&
AC&
Sensi/vity&
AC&rota/on&
EMS&voltages&
Sensi/vity&Calibra/on&Rota/on&Offset&
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Disturbance subtraction
• Known disturbances can be subtracted• Low frequency, linear coupling assumed:
• PSD and CPSD• BH92 window, 50% overlap• De-trending: linear• Binning/averaging• Need to make a “binning” before averaging over windows• Instrument noise: built-in models
33
N t( ) = N t( ) − knxn t( )
n∑
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Spectral estimation: example
34
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010 35
STOC Exercises
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
STOC Exercises
• Aim to validate mission time-line using all components• Tight integration and training of data analysis team in a
mission-realistic environment• more realistic data• aim to use fixed releases of tools• to train the ability to ‘work-around’ problems
• give structure to • toolbox/algorithm development• we need tool ‘x’ before the next STOC exercise!
• the validation of TNs through MDCs• model development for an ‘unknown’ system
36
STOC Exercise
First-cut analysis
In-depth analysis
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Filling the time...
37
STOC Exercise N
Data Preparation •define PORs •run simulations •upload data to repo
In-STOC activity •retrieve data •analyse data •Q: was the run successful?
In-depth investigations •follow-up analysis •quantitative results •report
STOC Exercise N+1
Data Preparation •define PORs •run simulations •upload data to repo
In-STOC activity •retrieve data •analyse data •Q: was the run successful?
STOC Exercise N-1
In-depth investigations •follow-up analysis •quantitative results •report
2-3 days
4-5 weeks
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
STOC Components
38
Data Generation
Data Ingestion
Data Conversion
Data Storage
Data Retrieval
Data Analysis
Data Import/Export
Documentation
Mission LPF MUST + DT tool
AO Converter (final version)
AO Repository
(final version)
MATLAB LTPDA client LTPDA (V3)
Repository, mysql cluster + dedicated data moving
tools
STOC Wiki
EX 1 LSSdrag files into folder, fake EOP
flag file
AO Converter (0.1 beta version)
AO Repository
V2.1
MATLAB Client CVS
versionLTPDA CVS
version not tested not tested
EX 2 LSSdrag files into folder, fake EOP
flag file
AO Converter V1.0
AO Repository
V2.1.1MATLAB
Client V2.1.1LTPDA V2.1.1 not tested STOC Wiki
EX 3 LSSdrag files into folder, fake EOP
flag file
AO Converter V1.0
AO Repository
V2.2
MATLAB Client CVS
VersionLTPDA CVS
version not tested STOC Wiki
... ... ... ... ... ... ... ... ...
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
STOC Ex 3 - highlights
• Aim to ‘simulate’ experiments of UTN-TN-3045• system identification along the x-axis
• Sub-aims:• ‘understand’ the configuration of the STOC simulator• ‘understand’ the noise models
39
1/2/10 6:50 AMLogbook 3 - LPF STOC Wiki
Page 1 of 7http://www.rssd.esa.int/wikiSI/index.php?title=Logbook_3&instance=LISAPATHFINDER_STOC
Logbook 3
From LPF STOC Wiki
Contents1 Introduction2 Noise breakdown3 Investigate injected signals4 Investigate signal-only run5 Make transfer function measurements6 Parameter estimation
IntroductionDue to some problems with the ESA repository (database was too large) downloadingfiles was too slow from the stoc_simulator_exports database. Luckily Michele has alocal copy of all the objects. We created a new database on the STOC server called raw3where we will upload the raw data objects again. The analyses below will then start byreading data from that database. Any results produced will be written to the databasecalled exp3.
Once the data is in raw3 we will begin as if that is end-of-pass.
Noise breakdownA number of runs were made with different noise sources switched off. Here we want tolook at the spectra of all of these compared to the full noise run, and see where eachnoise is contributing.
Noise run 1 (experiment tag: noise_24h_exp_20100111_165805)
First we will download objects from the noise run:
o1 == 174o12 == 175
First we'll take a look at the time-series of these runs
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010
Project time-line
40
Nov 2009 Dec 2010STOC Exercises
MDC3
V2.2V3.0
LTPDA Status, SWT Meeting, ESTEC, Feb 2nd 2010 41
End