fluxer Documentation - Read the Docs · ﬂuxer Documentation, ... planarfit Calculate planar ﬁt...

fluxer DocumentationRelease 0.1.0

Sebastian Luque

Feb 27, 2018

Contents

1 Basic usage 3

2 Project configuration 5

3 eddycov package 7

4 underway package 9

5 API documentation 115.1 fluxer API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5.1.1 Subpackages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.1.1.1 fluxer.eddycov package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.1.1.2 fluxer.underway package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.1.2 Submodules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.1.2.1 fluxer.flux_config module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

6 Indices and tables 23

Python Module Index 25

i

ii

fluxer Documentation, Release 0.1.0

Routines and tools to process flux (eddy covariance) data, as collected by CEOS.

Contents 1


2 Contents

CHAPTER 1

Basic usage

The package currently has 2 sub-packages:

• eddycov

• underway

eddycov is the eddy covariance package, and underway is a package for calculation of pCO2 from an underwaysystem. Each package can be imported individually as usual, e.g.:

import fluxer.eddycov as eddycov

Or

import fluxer.underway as underway

3


4 Chapter 1. Basic usage

CHAPTER 2

Project configuration

The easiest way to use the packages is to set up a configuration for any given project. The source for fluxer includesthe default configuration settings required by each sub-package (under the config/ directory). These settings arespecified in a *.cfg file (syntax instructions are given in the default files).

5


6 Chapter 2. Project configuration

CHAPTER 3

eddycov package

The main interface for this package is the function main:

# The main() function takes a configuration file and runs the analysesimport fluxer.eddycov as ecec.main("ec_config.cfg")

and there is also a command-line utility:

get_fluxes ec_config.cfg

7


8 Chapter 3. eddycov package

CHAPTER 4

underway package

This package offers two functions: main and underway_pCO2.

# The main() function takes a configuration file and runs the analysesfrom fluxer import underwayunderway.main("ec_config.cfg")

# The underway_pCO2() function takes an input data file and a *parsed*# configuration file and runs the analysis for itfrom fluxer import underwayfrom fluxer.flux_config import parse_configconfig = parse_config("uw_config.cfg")eddycov.underway_pCO2("YYYYMMDD_100000_20min.csv", config)

It is also possible to perform the analysis from the shell command line:

get_pCO2 ec_config.cfg

9


10 Chapter 4. underway package

CHAPTER 5

API documentation

5.1 fluxer API

CEOS tools for analysis of atmospheric fluxes

Includes utilities for eddy covariance analysis and underway pCO2, as well as corrections for platform motion.

5.1.1 Subpackages

5.1.1.1 fluxer.eddycov package

Processing flow chart

The first step is to correct data for ship motion effects:

11


12 Chapter 5. API documentation


Vector rotation conventions

The coordinates of a vector 𝑖 in coordinate system 𝑘, i.e. �⃗�𝑖,𝑘, can be expressed in or transformed to a differentcoordinate system by multiplication with a rotation matrix 𝑅𝑛 around axis 𝑛. In a right-handed coordinate system,positive rotation of the coordinate system around a coordinate axis is clockwise when looking along rotation axis fromthe origin towards the positive direction. A single rotation around the 𝑥, 𝑦, or 𝑧 axis is performed by pre-multiplyingthe corresponding rotation matrix:

𝑅𝑥,𝜃 =

⎡⎣1 0 00 cos 𝜃 sin 𝜃0 − sin 𝜃 cos 𝜃

⎤⎦ 𝑅𝑦,𝜃 =

⎡⎣cos 𝜃 0 − sin 𝜃0 1 0

sin 𝜃 0 cos 𝜃

⎤⎦ 𝑅𝑧,𝜃 =

⎡⎣ cos 𝜃 sin 𝜃 0− sin 𝜃 cos 𝜃 0

0 0 1

⎤⎦by column vector �⃗�𝑖,𝑘. For instance, rotating vector �⃗�𝑖,𝑘‘s coordinates from coordinate system 0 to coordinate system1 around the vertical 𝑧 axis is defined as:

�⃗�𝑖,1 = 𝑅𝑧,𝜃�⃗�𝑖,0(5.2)

This particular definition is known as “passive” or “alias” rotation. Alternatively, the same rotation can be achieved bypost-multiplying the row vector �⃗�𝑖,𝑘 by the inverse (or transpose, in orthogonal matrices such as these) of the rotationmatrix in equation (5.1):

�⃗�𝑖,1 = �⃗�𝑖,0𝑅ᵀ𝑧,𝜃(5.3)(5.3)

Using 𝑅𝑧,𝜃 instead of the inverted (transposed) matrix results in what is known as an “active” or “alibi” rotation ofthe vector in the opposite direction (i.e. counterclockwise using the convention described above) in the same fixedcoordinate system. Equation (5.3) is computationally more convenient, as vector data are typically acquired and storedin rows, so this is the convention adopted in eddycov . Figure 1 illustrates this operation for a commonly used rotationto eliminate the angular offset of an anemometer relative to the main longitudinal axis of the platform where it wasmounted.

Module contents

Tools for eddy covariance analysis requiring motion correction

main Perform flux analyses, given a configuration filesmooth_angle Smooth angles by decomposing them, applying a boxcar

averageplanarfit Calculate planar fit coefficients for coordinate rotationrotate_coordinates Rotate vector coordinate system or vectors themselves

around an axisrotate_wind3d Transform 3D wind vectors to reference mean streamline

coordinate systemwind3D_correct Correct wind vector measurements from a moving platformdespike_VickersMahrt Vickers and Mahrt (1997) signal despiking procedurewind3D_correct_period Perform wind motion correction on period dataframe

5.1. fluxer API 13


x

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00 y−1.00

−0.75−0.50

−0.250.00

0.250.50

0.751.00

z

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00

x0

y0

z0

x1

θ

y1

θ

z1

~v1,0~v′1,1

~v2,0

~v′2,1

~v3,0~v′3,1

Rotation around z-axis by θ = 36.0◦



fluxer.eddycov.main(config_file)Perform flux analyses, given a configuration file

Parameters config_file (str) – Path to configuration file.

Returns Writes summary file and prints messages from process.

Return type None

fluxer.eddycov.wind3D_correct_period(ec_prep, config, **kwargs)Perform wind motion correction on period dataframe

Parameters

• ec_prep (pandas.DataFrame) – Pandas DataFrame with prepared flux data.

• config (OrderedDict) – Dictionary with parsed configuration file.

• tilt_motion (numpy.array, optional keyword) – Passed to wind3D_correct.

• tilt_anemometer (numpy.array, optional keyword) – Passed towind3D_correct.

Returns wind corrected flux data.

Return type pandas.DataFrame

fluxer.eddycov.smooth_angle(angle, vmagnitude=1, kernel_width=21)Smooth angles by decomposing them, applying a boxcar average

Smoothing is done by using a 1D kernel smoothing filter of a given width.

Parameters

• angle (numpy.ndarray) – The angle(s) in degree units.

• vmagnitude (numpy.ndarray, optional) – The magnitude array associated witheach angle. It can be a scalar that is common to all angle elements.

• kernel_width (int, optional) – The width of the filter kernel.

Returns namedtuple with ndarrays angle and magnitude, in that order.

Return type namedtuple

fluxer.eddycov.planarfit(vectors)Calculate planar fit coefficients for coordinate rotation

See Handbook of Micrometeorology (Lee et al. 2004). Ported from getPlanarFitCoeffs.m from Patric Sturm<[email protected]>.

Parameters vectors (numpy.ndarray) – A 2-D (Nx3) array with x, y, and z vectors, expressedin a right-handed coordinate system. These vectors may correspond to u, v, and w wind speedvectors, or inertial acceleration components.

Returns

PlanarFitCoefs – namedtuple with (index and name in brackets):

numpy.ndarray [0, k_vct] 1-D array (1x3) unit vector parallel to the new z-axis.

numpy.ndarray [1, tilt_coefs] 1-D array (1x3) Tilt coefficients b0, b1, b2.

numpy.float [2, phi] Scalar representing roll angle 𝜑.

numpy.float [3, theta] Scalar representing pitch angle 𝜃.


5.1. fluxer API 15

mailto:[email protected]


fluxer.eddycov.rotate_coordinates(vectors, theta, axis=2, **kwargs)Rotate vector coordinate system or vectors themselves around an axis

A right-handed coordinate system is assumed, where positive rotations are clockwise when looking along therotation axis from the origin towards the positive direction. With the default active=False, each row vector𝑖 in input coordinate system 0 is rotated around the given axis by angle 𝜃, so that it can be expressed in coordinatesystem 1 as follows:

�⃗�𝑖,1 = �⃗�𝑖,0𝑅𝜃

where the input and output coordinate systems are different. If the input and output coordinate systems are thesame, i.e. active=True, the vectors are transformed according to:

�⃗�𝑖,1 = �⃗�𝑖,0𝑅ᵀ𝜃

Parameters

• vectors (array_like) – An nx3 array of vectors with their x, y, z components

• theta (numeric) – The angle (degrees) by which to perform the rotation.

• axis (int) – Axis around which to perform the rotation (x = 0; y = 1; z = 2).

• **kwargs – Optional keywords

active [bool] Whether to return the coordinates of each vector in the rotated coordinatesystem or to rotate the vectors themselves onto the same coordinate system. Default is toperform a passive transformation, i.e. rotation of the coordinate system.

Returns The vector array with the same shape as input with rotated components.

Return type array_like

fluxer.eddycov.rotate_wind3d(wind3D, method=’PF’, **kwargs)Transform 3D wind vectors to reference mean streamline coordinate system

Use double rotation, triple rotation, or planar fit methods (Wilczak et al. 2001; Handbook of Micrometeorology).

This is a general coordinate rotation tool, so can handle inputs such as wind speed and acceleration from inertialmeasurement units.

Parameters

• wind3D (numpy.ndarray) – A 2-D (Nx3) array with x, y, and z vector components,expressed in a right-handed coordinate system. These may represent u, v, and w wind speedvectors, or inertial acceleration.

• method ({"DR", "TR", "PF"}, optional) – One of: “DR”, “TR”, “PF” for dou-ble rotation, triple rotation, or planar fit.

• k_vector (numpy.ndarray, optional) – 1-D array (1x3) unit vector parallel tothe new z-axis, when “method” is “PF” (planar fit). If not supplied, then it is calculated.

Returns

RotatedVectors – namedtuple with (index, name in brackets):

numpy.ndarray [0, rotated] 2-D array (Nx3) Array with rotated vectors

numpy.ndarray [1, phi_theta] 1-D array (1x2) 𝜑 and 𝜃 rotation angles. The former is the esti-mated angle between the vertical unit coordinate vector in the rotated frame and the verticalunit vector in the measured uv plane, while the latter is wind direction in the measured uvplane. Note these are not roll and pitch angles of the measurement coordinate frame relativeto the reference frame.




fluxer.eddycov.wind3D_correct(wind_speed, acceleration, angle_rate, heading, speed, anemome-ter_pos, sample_freq, Tcf, Ta, tilt_motion=array([0., 0.]),tilt_anemometer=array([0., 0.]))

Correct wind vector measurements from a moving platform

This is a port of S. Miller’s motion Matlab function, which implements Miller et al. (2008) approach. Coordi-nate frame is assumed to be right-handed:

• x - positive towards the bow.

• y - positive towards port side.

• z - positive upwards.

Parameters

• wind_speed (numpy.ndarray) – A 2-D (Nx3) array with u, v, and w wind speed (m/s)vectors.

• acceleration (numpy.ndarray) – A 2-D (Nx3) array with x, y, and z acceleration(m/s/s) vectors.

• angle_rate (numpy.ndarray) – A 2-D (Nx3) array with angular rates (radians/s)vectors.

• heading (array_like) – A vector with ship heading measurements (deg) in right-handcoordinate system.

• speed (array_like) – A vector with ship speed (m/s).

• anemometer_pos (array_like) – [x, y, z] position vector of anemometer, relative tomotion sensor (m).

• sample_freq (float) – Sampling frequency (Hz).

• Tcf (float) – Complimentary filter period (s).

• Ta (float) – High-pass filter cutoff for accelerations (s). This can be a scalar if the samecutoff is used for the three components, or a 3 element vector to indicate cutoff period forthe x, y, z components.

• tilt_motion (array_like, optional) – [roll, pitch] vector with mean tilt (radi-ans) relative to a horizontal plane. Roll angle offset is positive is port side up, and pitch ispositive for bow down (see Miller 2008 for info, set to [0 0] if unknown).

• tilt_anemometer (array_like, optional) – [roll, pitch] vector with mean tilt(radians) relative to a horizontal plane. Roll angle offset is positive is port side up, and pitchis positive for bow down (see Miller 2008 for info, set to [0 0] if unknown).

Returns

CorrectedWind3D – namedtuple with (index, name in brackets):

numpy.ndarray [0, uvw_ship] 2-D array (Nx3) with corrected wind vectors in ship-referencedframe with z-axis parallel to gravity.

numpy.ndarray [1, euler_angles] 2-D array (Nx3) with Euler angles.

numpy.ndarray [2, euler_angles_angular_rates] 2-D array (NX3) with Euler angles from ratesensors (unfiltered).

numpy.ndarray [3, euler_angles_accelerations] 2-D array (NX3) with Euler angles from ac-celerometers (unfiltered).

5.1. fluxer API 17


numpy.ndarray [4, euler_angles_slow] 2-D array (NX3) with slow Euler angles (low pass fil-tered).

numpy.ndarray [5, euler_angles_fast] 2-D array (NX3) with fast Euler angles (high pass fil-tered).

numpy.ndarray [6, mount_offset_rotations] 2-D array (3X3) with mounting offset rotationmatrix (see Miller 2008).

numpy.ndarray [7, uvw_earth] 2-D array (NX3) with measured velocity, rotated to the earthframe.

numpy.ndarray [8, uvw_angular] 2-D array (NX3) with velocity induced by angular motion.

numpy.ndarray [9, uvw_linear] 2-D array (NX3) with velocity induced by platform linear mo-tion.

numpy.ndarray [10, ship_enu] 2-D array (NX3) with ship velocity in eastward, northward, upframe. (low pass filtered).

numpy.ndarray [11, uvw_enu] 2-D array (NX3) with corrected wind in eastward, northward,up frame.

numpy.ndarray [12, imu_enu] 2-D (NX3) array with displacement of the motion sensor.


References

Miller,S.D., Hristov,T.S., Edson,J.B., and C.A. Friehe, 2008: Platform Motion Effects on Measurements ofTurbulence and Air-Sea Exchange Over the Open Ocean, J. Atmo. Ocean. Tech. 25(9), 1683-1694, DOI:10.1175/2008JTECHO547.1.

fluxer.eddycov.despike_VickersMahrt(x, width, zscore_thr, nreps, step=None, nrep_thr=None,interp_nan=True)

Vickers and Mahrt (1997) signal despiking procedure

The interpolating function is created by the InterpolatedUnivariateSpline function from the scipy package, anduses a single knot to approximate a simple linear interpolation, so as to keep the original signal as untouched aspossible.

Parameters

• x (numpy.ndarray) – A 1-D signal vectors to be despiked.

• width (int) – Window width.

• step (int, optional) – Step size for sliding windows. Default is one-half windowwidth.

• zscore_thr (float) – The zscore beyond which an observation is considered to be anoutlier. Default is zero.

• nrep_thr (int, optional) – The maximum number of consecutive outliers thatshould occur for a spike to be detected. Default: 3.

• nreps (int, optional) – How many times to run the procedure. Default is zero.

• interp.nan (bool, optional) – Whether missing values should be interpolated. In-terpolated values are computed after despiking. Default is True.



Returns

VickersMahrt – Tuple with (index, name in brackets):

numpy.ndarray [0, x] 1-D array with despiked input.

numpy.int [1, nspikes] Number of spikes detected.

numpy.int [2, ntrends] Number of outlier trends detected.

numpy.int [3, kclass] Number of iterations performed.

Return type tuple

Submodules

parse_ecfile

Parse and flag input eddy covariance file

exception fluxer.eddycov.parse_ecfile.FluxErrorBases: exceptions.Exception

Base class for Exceptions in this module

exception fluxer.eddycov.parse_ecfile.SonicError(message, flags)Bases: fluxer.eddycov.parse_ecfile.FluxError

Critical sonic anemometer Exception

exception fluxer.eddycov.parse_ecfile.NavigationError(message, flags)Bases: fluxer.eddycov.parse_ecfile.FluxError

Critical navigation Exception

exception fluxer.eddycov.parse_ecfile.MeteorologyError(message, flags)Bases: fluxer.eddycov.parse_ecfile.FluxError

Critical meteorology Exception

fluxer.eddycov.parse_ecfile.prepare_period(period_file, config)Parse input period file and set up data for flux analyses

Parameters

• period_file (str) – Path to file with pre-filtered, candidate, flux data.


Returns Tuple of two pandas.DataFrame objects; first represents prepared flux data, and thesecond the flags encountered during tests. The latter is also passed along with exception, ifraised.

Return type tuple

settings

Basic flags and objects required throughout the package

fluxer.eddycov.settings.FLUX_FLAGS = ['failed_prep_flag', 'open_flag', 'closed_flag', 'sonic_flag', 'motion_flag', 'bad_navigation_flag', 'bad_meteorology_flag']Flags used during processing

5.1. fluxer API 19


tilt_windows

Calculate tilt angles in flux files using non-overlapping moving windows

class fluxer.eddycov.tilt_windows.TiltWindows(ec_files=[], win_minutes=0)Well-defined pandas DataFrame

get_tilts_planarfit(config)Compute tilt angles in all windows using planar fit method

Parameters config (OrderedDict) – Dictionary with parsed configuration file.

Returns tilt angles for each window.


plot(fig_file, step=1, title=None)Generate a plot of tilt angles and diagnostics

Parameters

• fig_file (str) – Path to file to write plot to

• step (int, optional) – Step (stride) to subsample the time series by. Default is 1.

• title (str) – Title for plot. Default is None.

5.1.1.2 fluxer.underway package

Module contents

Tools for processing underway pCO2 data

main Perform underway pCO2 analyses, given a configurationfile

underway_pCO2 Perform underway pCO2 computations on period file

fluxer.underway.main(config_file)Perform underway pCO2 analyses, given a configuration file

Parameters config_file (str) – Path to configuration file.

Returns Writes summary file and prints messages from process.

Return type None

fluxer.underway.underway_pCO2(period_file, config)Perform underway pCO2 computations on period file

Parameters

• period_file (str) – Path to file with pre-filtered, candidate, flux data.


Returns object with additional columns as results of computations.




5.1.2 Submodules

5.1.2.1 fluxer.flux_config module

Utility for configuring user variables for flux analyses

fluxer.flux_config.parse_config(cfg_file)Parse configuration file for essential variables during flux analysis.

Parameters cfg_file (str) – Path to configuration file.

Returns Ordered dictionary with variables for each section. A list of input files found, given theinput directory and file pattern, is also appended to the ‘Inputs’ section.

Return type collections.OrderedDict

5.1. fluxer API 21



CHAPTER 6

Indices and tables

• genindex

• modindex

• search

23


24 Chapter 6. Indices and tables

Python Module Index

ffluxer, 11fluxer.eddycov, 13fluxer.eddycov.parse_ecfile, 19fluxer.eddycov.settings, 19fluxer.eddycov.tilt_windows, 20fluxer.flux_config, 21fluxer.underway, 20

25


26 Python Module Index

Index

Ddespike_VickersMahrt() (in module fluxer.eddycov), 18

FFLUX_FLAGS (in module fluxer.eddycov.settings), 19fluxer (module), 11fluxer.eddycov (module), 13fluxer.eddycov.parse_ecfile (module), 19fluxer.eddycov.settings (module), 19fluxer.eddycov.tilt_windows (module), 20fluxer.flux_config (module), 21fluxer.underway (module), 20FluxError, 19

Gget_tilts_planarfit() (fluxer.eddycov.tilt_windows.TiltWindows

method), 20

Mmain() (in module fluxer.eddycov), 15main() (in module fluxer.underway), 20MeteorologyError, 19

NNavigationError, 19

Pparse_config() (in module fluxer.flux_config), 21planarfit() (in module fluxer.eddycov), 15plot() (fluxer.eddycov.tilt_windows.TiltWindows

method), 20prepare_period() (in module fluxer.eddycov.parse_ecfile),

19

Rrotate_coordinates() (in module fluxer.eddycov), 15rotate_wind3d() (in module fluxer.eddycov), 16

Ssmooth_angle() (in module fluxer.eddycov), 15SonicError, 19

TTiltWindows (class in fluxer.eddycov.tilt_windows), 20

Uunderway_pCO2() (in module fluxer.underway), 20

Wwind3D_correct() (in module fluxer.eddycov), 17wind3D_correct_period() (in module fluxer.eddycov), 15

27

Date post:	12-May-2018
Category:	Documents
Upload:	vuongdien
View:	231 times
Download:	2 times

fluxer Documentation - Read the Docs · ﬂuxer Documentation, ... planarfit Calculate planar ﬁt...

Documents