LTPDA A data analysis framework for LISA PathÞnderhewitson/presentations/presentations...A data...

LTPDAA data analysis framework for LISA PathfinderM Hewitson for the LTP team

LISA BB Meeting, Hannover, January 2007

On the critical path to LISATechnology demonstrator for LISA

drag-free controlfree-falling test-mass acceleration noise

LPF Mission

2

Laser Interferometer Space Antenna

5x106 km




LPF Mission

2


30cm5x106 km




LPF Mission

2


5x106 km




LPF Mission

2


5x106 km




LPF Mission

2


5x106 km




LPF Mission

2


5x106 km

Launch: mid-2010


Measurements

3


Measurements

3

IFO signalsmeasure longitudinal for TM1/SC and TM1/TM2measure TM angles

Capitative sensorsx,y,z for each TM3 angles for each TM

Many environmental monitors, e.g.temperaturetest-mass chargeparticle countersmagnetometers


DA challenges

4


DA challengesVery limited data download rate (15kbps)

4


DA challengesVery limited data download rate (15kbps)Interesting features are on time-scales similar to experiment lengths

each experiment ~24Hlooking at <mHz in 24 hours of data

4


DA challengesVery limited data download rate (15kbps)Interesting features are on time-scales similar to experiment lengths

each experiment ~24Hlooking at <mHz in 24 hours of data

Signals sampled with different clocks at different rates

1/600Hz - a few Hz sample ratesneed to resample and synchronise signals before doing coherence analyses

4


Data analysisNeed the ‘usual’ commissioner’s tool-set

Spectral estimationsDigital filtersModel fittingSimulated signals...

5


Data analysis requirementsUnusual mission

data analysis is part of mission operationsdata analysis software is operational software

6




6

Use commercial software environment




6

Use commercial software environmentMATLAB




6


All data analysis results should be accountable and reproducible




6



Concept of Analysis Objects




6




All data analysis to be done ‘graphically’




6




All data analysis to be done ‘graphically’allow for non-experts in MATLAB




6





Multi-user data access




6





Multi-user data accessclient/server system


Analysis Objects

7


Analysis ObjectsWhat a result isn’t:

7



an image

7



an imagea plot

7



an imagea plota text file full of numbers

7



an imagea plota text file full of numbers

7

Analysis Object

Numerical

Data

Provenance

Processing

history

Additional

meta-data

- creator

- date

- IP address

- Hostname

- Operating System

- software versions

- Name

- ID number

- Comment

- pipeline file(s)

- Name

- Algorithm version

- Parameter list

- Creation date/time

- Input histories

- Name

- Numerical data

vectors

- Creation date/time

- Additional flags


Taking care of history

8

Input

AO

Output

AO(s)

Input

AO

Algorithmic step

input

history

input

history

Algorithm

history

Intelligent algorithms


Taking care of history

8

Input

AO

Output

AO(s)

Input

AO

Algorithmic step

input

history

input

history

Algorithm

history

Intelligent algorithms


Implementation

9

Environment is:object-orientedimplemented as MATLAB toolboxGraphical programming achieved using SIMULINK as a ‘drawing pad’


Toolbox contents

10


Toolbox contents•toolbox comprises

• new classes

• helper functions

• signal processing functions (algorithms)

• documentation

10



• new classes



• documentation

10

Core functionality is ~65000 lines of MATLAB code



• new classes



• documentation

10


GUIs ~6000 lines of MATLAB code



• new classes



• documentation

10


GUIs ~6000 lines of MATLAB code

Documentation is ~62000 lines of html (includes src, some automatically

generated)


Class hierarchy

11

time

timespan

pzmodel

plist

miir mfir specwin

timeformat

pole zero

param

tsdatafsdataxydata cdata

provenance

history

ao User Classes


Class hierarchy

11

time

timespan

pzmodel

plist

miir mfir specwin

timeformat

pole zero

param

tsdatafsdataxydata cdata

provenance

history

ao User Classes


Benefits of classes

12


Benefits of classesMany and various constructors

12


Benefits of classesMany and various constructorsDefine object properties in central location

12

'name' 'data' 'hist'

'provenance' 'description'

'mfile' 'mfilename'

'mdlfile' 'mdlfilename'

'version' 'created'


Benefits of classesMany and various constructorsDefine object properties in central locationOverload operators

can directly operate with AOs

12



'mfile' 'mfilename'


'version' 'created'

% Create cdata AOsa1 = ao(10);a2 = ao('foo.xml'); % Add thema3 = a1+a2; % Subtract constanta4 = a3-10; % Testsif a1 < 100 disp('small ao');end




Overload functionsexamples:

12



'mfile' 'mfilename'


'version' 'created'





Overload functionsexamples:

12

exp mean sqrt abs export median stairs acos fft minus std ao filter mpower string ao2m filtfilt mrdivide submit aosplit find mtimes submit_fastinsert asin ge mux subsasgn atan get ne subsref attachm getAOdata norm sum attachmdl gt phase svd cat hist plot tag char imag plus tan complex index polyfit testAO conj interp power times cos inv rdivide transpose ctranspose iplot real var decimate join resample xml demux le save xmladd det len select xmlparse diag ln set display log simple_plot eig log10 sin eq lt split



'mfile' 'mfilename'


'version' 'created'



Visualising data

13


Visualising data

13

>> figure, iplot(a4)


Visualising data

13

>> figure, iplot(a4)


Inside the functions

14

b = foo(a,pl)

20



14

b = foo(a,pl)

output AOs

20



14

b = foo(a,pl)

output AOs input AOs

20



14

b = foo(a,pl)

output AOs input AOs input parameters

20



14

b = foo(a,pl)Description

in standard MATLAB format so that ‘help fcn’ works as it should

20



14



Unpack input AO(s) and PLIST(s)

20



14



Unpack input AO(s) and PLIST(s)do something to data

20



14



Unpack input AO(s) and PLIST(s)do something to datacreate new output data object (tsdata or fsdata)

20



14



Unpack input AO(s) and PLIST(s)do something to datacreate new output data object (tsdata or fsdata)create new output history

h = history(ALGONAME, VERSION, pl, [a.hist b.hist])

20

History of input AOs



14





Create new output AO(s)

20




14





Create new output AO(s)same for wrappers and new functions

20



Example wrapper

15


Example wrapper

15

function varargout = ltpda_pwelch(varargin)% LTPDA_PWELCH makes power spectral density estimates of the time-series objects%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% DESCRIPTION: LTPDA_PWELCH makes power spectral density estimates of the% time-series objects in the input analysis objects.%% CALL: b = ltpda_pwelch(a1,a2,a3,...,pl)%% INPUTS: b - output analysis objects% aN - input analysis objects% pl - input parameter list%% Makes PSD estimates using ltpda_psd() of each input AO.%% If the last input argument is a parameter list (plist) it is used.% The following parameters are recognised.%% PARAMETERS: 'Win' - a specwin window object [default: Kaiser -200dB psll]% 'Nolap' - segment overlap (number of samples) [default: taken from window function]% 'Nfft' - number of samples in each fft [default: fs of input data]% 'Debug' - debug level for terminal output (0-1)%% VERSION: $Id: ltpda_pwelch.m,v 1.12 2007/07/16 12:52:21 ingo Exp $%% HISTORY: 07-02-2007 M Hewitson% Creation%% The following call returns a parameter list object that contains the% default parameter values:%% >> pl = ltpda_pwelch('Params')%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


Example wrapper

15


Win = find(pl, 'Win'); % Window object if isempty(Win) Win = specwin('Kaiser', Nfft, 200) disp(sprintf('! Using default Window of %s', strrep(Win.name, '_', '\_'))) end plo = append(plo, param('Win', Win)); Nolap = find(pl, 'Nolap'); % Amount to overlap each fft if isempty(Nolap) Nolap = round(Win.rov*Nfft/100); disp(sprintf('! Using default overlap of %d samples', Nolap)) end plo = append(plo, param('Nolap', Nolap));


Example wrapper

15



% Compute PSD using pwelch [pxx, f] = pwelch(d.x, Win.win, Nolap, Nfft, d.fs);


Example wrapper

15



% Compute PSD using pwelch [pxx, f] = pwelch(d.x, Win.win, Nolap, Nfft, d.fs);

% create new output fsdata fs = fsdata(f, pxx, d.fs); fs = set(fs, 'name', sprintf('PSD(%s)', d.name)); fs = set(fs, 'yunits', [d.yunits '/\surdHz']); fs = set(fs, 'enbw', Win.nenbw); % create new output history h = history(ALGONAME, VERSION, plo, a.hist); % make output analysis object b = ao(fs, h); % set name b = set(b, 'name', sprintf('PSD(%s)', a.name)); % add to outputs bs = [bs b];


History structure

16

ao


History structure

16

ao

name

history

version

data


History structure

16

ao

name

history

version

data

name

input histories

version

params


History structure

16

ao

name

history

version

data

name

input histories

version

params

name

input histories

version

params


History structure

16

ao

name

history

version

data

name

input histories

version

params

name

input histories

version

params

name

input histories

version

params


History structure

16

ao

name

history

version

data

name

input histories

version

params

name

input histories

version

params

name

input histories

version

params


History structure

16

ao

name

history

version

data

name

input histories

version

params

name

input histories

version

params

name

input histories

version

params

Entire processing history of every AO is recorded


Reliving history

17

%% Reproduce from history% Write an m-file from AOao2m(a4, 'test.m'); edit 'test.m'


Reliving history

17


function a_out = test % TEST.M % % % written by ao2m / $Id: ao2m.m,v 1.11 2007/11/14 16:30:18 ingo Exp $% % based on analysis object:% name: a3 - 10 / ((Data) + (Data)) - (Data)% provenance: created by hewitson@localhost[192.168.2.104] on MACI/7.5 (R2007b)/0.7 (R2007b) at 2008-01-06% description: % original m-file: % % a7331931 = ao(plist([param('VALS', [10]) ]));a7331814 = ao(plist([param('VALS', [[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 a7331773 = ao(plist([param('VALS', [10]) ]));a7331865 = plus(a7331773, a7331814);a7331960 = minus(a7331865, a7331931);a_out = a7331960; % END


Reliving history

17


function a_out = test % TEST.M % % % written by ao2m / $Id: ao2m.m,v 1.11 2007/11/14 16:30:18 ingo Exp $% % based on analysis object:% name: a3 - 10 / ((Data) + (Data)) - (Data)% provenance: created by hewitson@localhost[192.168.2.104] on MACI/7.5 (R2007b)/0.7 (R2007b) at 2008-01-06% description: % original m-file: % % a7331931 = ao(plist([param('VALS', [10]) ]));a7331814 = ao(plist([param('VALS', [[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 a7331773 = ao(plist([param('VALS', [10]) ]));a7331865 = plus(a7331773, a7331814);a7331960 = minus(a7331865, a7331931);a_out = a7331960; % END

=a4


Visualise history

18


Visualise history

18

>> figure, plot(a4.hist)


Visualise history

18

>> figure, plot(a4.hist)


Saving your objectsAll LTPDA (user) objects can be saved to file (and loaded from)File format is XML

.mat works as well

19

>> p = param('a', 2)---- param 1 ----key: aval: 2-----------------



.mat works as well

19

>> p = param('a', 2)---- param 1 ----key: aval: 2----------------->> save(p, 'param.xml')



.mat works as well

19

>> p = param('a', 2)---- param 1 ----key: aval: 2-----------------

<?xml version="1.0" encoding="utf-8"?><param> <Object> <param> <version>$Id: param.m,v 1.9 2007/08/31 17:40:08 hewitson Exp $</version> <key>a</key> <value>2</value> </param> </Object></param>

>> save(p, 'param.xml')


Signal processing

20

% Load raw datao1 = ao('o1_1.xml');o12 = ao('o12_1.xml'); % Compute ASDs pl = plist('Win', specwin('Kaiser', 10, 150), 'Nfft', o1.data.fs*20000); o1_xx = ltpda_pwelch(o1, pl);o12_xx = ltpda_pwelch(o12, pl); % Save objectssave(o1_xx, 'o1_xx.xml');save(o12_xx, 'o12_xx.xml');


Signal processing

20


Load Data


Signal processing

20


Process Data

Load Data


Signal processing

20


Process Data

Load Data

pl = plist('Win', specwin('Kaiser', 10, 150), 'Nfft', o1.data.fs*20000);


Signal processing

20


Process Data

Load Data


Signal processing

20


Process Data

Load Data

Save Data


More complicated e.g.Frequency-domain calibration of IFO outputs from MDC1

21



21

Cdf = df_controller(Sdf.data.x); %ao('Cdf.xml');Ssus = ao('Ssus.xml');Sw1 = ao('Sw1.xml');Sw3 = ao('Sw3.xml'); % spectrao12xx = ao('o12_xx.xml');o1xx = ao('o1_xx.xml'); % Other valuesw1 = ao('w1.xml');w3 = ao('w3.xml');delta = ao('delta.xml');wd = ao('wd.xml'); % Load reference tracesa11r = ao('../../../ltp_noise_models/A11xx.xml');a12r = ao('../../../ltp_noise_models/A1212xx.xml'); %% Calibratet1 = abs(Sw1 - Cdf); a1xx = o1xx .* t1;a1xx = set(a1xx, 'name', 'ASD(a1)');beta = wd.^2 - delta.*Sw3; t3 = Sw3./Ssus;a12xx = abs(o12xx.*t3 + beta.*o1xx);a12xx = set(a12xx, 'name', 'ASD(a12)'); %% Plot

figureiplot(a1xx , a11r) figureiplot(a1xx ./ a11r)allyscale('lin') % END



21



Load Objects



21



Manipulate Objects



21





21




Looking at the history

22



22



22


Data access

23

LTPDA Repository

Disk

MySQL Server

converter

Datafrom S/C

AO

XML

MATLAB Client

MATLAB Client

MATLAB Client

Centralised data distributionMultiple clients can access the data at the same timeData can be accessed from anywhere

firewall permitting

idhashxml

objsidobj_idobj_typenamecreatedversioniphostnameossubmittedcomment1comment2comment3comment4comment5comment6validatedvdate

objmeta

idobj_iduser_idtransdatedirection

transactionsidnobjsobj_ids

collectionsidfirstnamefamilynameusernameemailtelephoneinstitution

users

idobj_iddata_typedata_idmfilenamemdlfilename

aoidobj_idin_filefs

miir/mfir

idxunitsyunitsfsnsecst0

tsdataidxunitsyunitsfs

fsdataidxunitsyunits

cdataidxunitsyunits

xydata


The LTPDA Repository

24

Share your LTPDA objects with your friendsclient/server systemserver:

mysql database

client:MATLAB LTPDA functions built around Database Toolbox

Can store:filters, parameter lists, AOs, etcjust pass object ref ID to your friends


Interaction in MATLABSince we have a database at the core, we can:

submit objectsquery for objectsretrieve objects

all from within MATLAB

25


Repository GUI - submit

26



26

Ob

ject

s in

Wor

ksp

ace



26

Ob

ject

s in

Wor

ksp

ace

Describe object(s)


Query

27


Query

27


Retrieve

28

Retrieve:object 12

collection 1


Retrieve

28

Retrieve:object 12

collection 1


Graphical programmingusing SIMULINK

29


Analysis diagrams

30

Diagram is executed from main control panelCalls functions in the underlying toolbox


Help!

31


Get your copy now!

32

http://www.lisa.uni-hannover.de/ltpda/index.html

Date post:	09-Mar-2018
Category:	Documents
Upload:	lyhanh
View:	214 times
Download:	1 times

LTPDA A data analysis framework for LISA PathÞnderhewitson/presentations/presentations...A data...

Documents