Stoner Package Documentation...Version 0.9 is tested with Python 2.7, 3.5, 3.6 using the standard...

Stoner Package DocumentationRelease 1

Gavin Burnell et al

Dec 24, 2020

Contents

1 Introduction 3

2 Getting this Code 52.1 Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Overview 73.1 Data and Friends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 DataFolder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.3 Image Subpackage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.4 Multiprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Resources 9

5 Contact and Licensing 11

6 Recent Changes 136.1 Current PyPi Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136.2 Development Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136.3 Stable Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

7 User Guide 157.1 The Stoner Python Package User Guide . . . . . . . . . . . . . . . . . . . . . . . . . . 15

8 Package Documentatuion 1398.1 Stoner Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Bibliography 3327

Python Module Index 3329

Index 3331

i

Stoner Package Documentation, Release 1

Contents 1

https://travis-ci.org/stonerlab/Stoner-PythonCodehttps://coveralls.io/github/stonerlab/Stoner-PythonCode?branch=masterhttps://www.codacy.com/app/stonerlab/Stoner-PythonCode?utm_source=github.com&utm_medium=referral&utm_content=stonerlab/Stoner-PythonCode&utm_campaign=Badge_Gradehttps://badge.fury.io/py/Stonerhttps://anaconda.org/phygbu/stonerhttp://stoner-pythoncode.readthedocs.io/en/latest/?badge=latesthttps://zenodo.org/badge/latestdoi/10057055


2 Contents

CHAPTER 1

Introduction

The Stoner Python package is a set of utility classes for writing data analysis code. It was written withinthe Condensed Matter Physics group at the University of Leeds as a shared resource for quickly writingsimple programs to do things like fitting functions to data, extract curve parameters, churn through largenumbers of small text data files and work with certain types of scientific image files.

For a general introduction, users are referred to the Users Guide, which is part of the online documenta-tion along with the API Reference guide. The github repository also contains some example scripts.

3

http://stoner-pythoncode.readthedocs.io/en/latest/http://stoner-pythoncode.readthedocs.io/en/latest/http://www.github.com/stonerlab/Stoner-PythonCode/


4 Chapter 1. Introduction

CHAPTER 2

Getting this Code

The Stoner package requires h5py>=2.7.0, lmfit>=0.9.7, matplotlib>=2.0,numpy>=1.13, Pillow>=4.0,scikit-image>=0.13.0 & scipy>=1.0.0 and also optional depends on filemagic, npTDMS, imreg_dft andnumba.

Ananconda Python (and probably other scientific Python distributions) include nearly all of the depen-dencies, and the remaining dependencies are collected together in the phygbu repositry on anacondacloud. The easiest way to install the Stoner package is, therefore, to install the most recent AnacondaPython distribution.

2.1 Compatibility

Versions 0.9.x (stable branch) are compatible with Python 2.7, 3.5, 3.6 and 3.7. The latest 0.9.6 versionis also compatible with Python 3.8 The development verstion (0.10dev, master branch) is compatriblewith Python 3.6, 3.7 and 3.8 but not 2.7.

Conda packages are prepared for the stable branch and when the development branch enters beta testing.Pip wheels are prepared for selected stable releases only.

5

https://www.youtube.com/watch?v=uZ_yKs11W18


2.2 Installation

After installing the current Anaconda version, open a terminal (Mac/Linux) or Anaconda Prompt (Win-dows) an do:

conda install -c phygbu Stoner

If you are not using Anaconda python, then pip should also work:

pip install Stoner

This will install the Stoner package and any missing dependencies into your current Python environment.Since the package is under fairly constant updates, you might want to follow the development with git.The source code, along with example scripts and some sample data files can be obtained from the githubrepository: https://github.com/stonerlab/Stoner-PythonCode

6 Chapter 2. Getting this Code

https://github.com/stonerlab/Stoner-PythonCode

CHAPTER 3

Overview

The main part of the Stoner package provides two basic top-level classes that describe an individualfile of experimental data and a list (such as a directory tree on disc) of many experimental files. Forour research, a typical single experimental data file is essentially a single 2D table of floating pointnumbers with associated metadata, usually saved in some ASCII text format. This seems to cover mostexperiments in the physical sciences, but it you need a more complex format with more dimensions ofdata, we suggest you look elsewhere.

3.1 Data and Friends

Stoner.Data is the core class for representing individual experimental data sets. It is actually composedof several mixin classes that provide different functionality, with methods to examine and manipulatedata, manage metadata, load and save data files, plot results and carry out various analysis tasks. It hasa large number of sub classes - most of these are in Stoner.formats and are used to handle the loading ofspecific file formats.

3.2 DataFolder

Stoner.DataFolder is a class for assisting with the work of processing lots of files in a common directorystructure. It provides methods to list. filter and group data according to filename patterns or metadataand then to execute a function on each file or group of files and then collect metadata from each file inturn. A key feature of DataFolder is its ability to work with the collated metadata from the individualfiles that are held in the DataFolder. In combination with its ability to walk through a complete heirarchyof groups of Data objects, the handling of the common metadata provides powerful tools for quicklywriting data reduction scripts.

The Stoner.HDF5 module provides some additional classes to manipulate Data and DataFolder objectswithin HDF5 format files. HDF5 is a common chouse for storing data from large scale facilties, althoughproviding a way to handle arbitary HDF5 files is beyond the scope of this package at this time - the formatis much too complex and flexible to make that an easy task. Rather it provides a way to work with large

7


numbers of experimental sets using just a single file which may be less brutal to your computer’s OSthan having directory trees with millions of individual files.

The module also provides some classes to support loading some particular HDF5 flavoured files into aData object.

The Stoner.Zip module provides a similar set of classes to Stoner.HDF5 but working with the ubiqui-tous zip compressed file format.

3.3 Image Subpackage

The Stoner.Image package is a feature of recent versions of the package and provides dedicated classesfor working with image data, and in particular for analysing Magnetic microscopy files such as KerrMicroscope image files or Scanning transmission microscopy files.

It provides an ImageFile class that is functionally similar to Data except that the numeri-cal data is understood to represent image data and additional methods are incorporated tofacilitate processing. The ImageFolder and ImageStack classes provide similar function-ality to DataFolder but with additional methods specific to handling collections of images.ImageStack

uses a 3D numpy array as it’s primary image store which permits faster access (at the expense of a largermemory footprint) than the lazy loading ordered dictionary of ImageFolder

The ImageFile class allows a direct interaction with many of the image routines in scikit-image andthe scipy.ndimage modules - which when coupled with the ability of ImageFolder and ImageStack toiteratively process their ImageFile members can allow scripts to process large numbers of images to bewritten in only a few lines of code.

3.4 Multiprocessing

The DataFolder and ImageFolder/ImageStack classes will optionally make use of the multiprocessingmodule when applying a function to their members for more efficient parallel processing of large num-bers of data files. This is disabled by default due to issues with threading and plotting on posix platformsand slow process start ups on Windows.

8 Chapter 3. Overview

CHAPTER 4

Resources

Included in the github repository are a (small) collection of sample scripts for carrying out variousoperations and some sample data files for testing the loading and processing of data. There is also aUser_Guide as part of this documentation, along with a complete API reference

9

http://www.github.com/stonerlab/Stoner-PythonCode/http://stoner-pythoncode.readthedocs.io/en/latest/UserGuide/ugindex.html


10 Chapter 4. Resources

CHAPTER 5

Contact and Licensing

The lead developer for this code is Dr Gavin Burnell , but many current andformer members of the CM Physics group have contributed code, ideas and bug testing.

The User Guide gives the current list of other contributors to the project.

This code and the sample data are all (C) The University of Leeds 2008-2020 unless otherwise indficatedin the source file. The contents of this package are licensed under the terms of the GNU Public Licensev3

11

http://www.stoner.leeds.ac.uk/people/gbmailto:[email protected]


12 Chapter 5. Contact and Licensing

CHAPTER 6

Recent Changes

6.1 Current PyPi Version

The current version of the package on PyPi will be the stable branch until the development branch entersbeta testing, when we start making beta packages available.

6.2 Development Version

The current development version is hosted in the master branch of the repository and will become version0.10. There is no definitive list of features at this time. Better integration with pandas and xarray areunder consideration as is depricating some of the less optimal parts of the api.

New Features in 0.10-dev include:

• Refactor Stoner.Core.DataFile to move functionality to mixin classes

• Start implementing PEP484 Type hinting

• Support pathlib for paths

• Switch from Tk based dialogs to Qt5 ones

• Refactoring the baseFolder class so that sub-groups are stored in an attribute that is an instanceof a custom dictionary with methods to prune and filter in the virtual tree of sub-folders.

6.2.1 Build Status

Version 0.7 onwards are tested using the Travis-CI services with unit test coverage assessed by Coveralls.

Version 0.9 is tested with Python 2.7, 3.5, 3.6 using the standard unittest module.

The development version - which will be 0.10 is tested using pytest with Python 3.6, Python 3.7 andPython 3.8.

13


6.2.2 Citing the Stoner Package

We maintain a digital object identifier (doi) for this package (linked to on the status bar at the top of thisreadme) and encourage any users to cite this package via that doi.

6.3 Stable Versions

Version 0.9 is the current stable version. This is the last version to support Python 2 and 3

CHAPTER 7

User Guide

7.1 The Stoner Python Package User Guide

7.1.1 Installation of Stoner Package

Introduction

This manual provides a user guide and reference for the Stoner python pacakage. The Stoner pythonpackage provides a set of python classes and functions for reading, manipulating and plotting data ac-quired with the lab equipment in the Condensed Matter Physics Group at the University of Leeds.

Getting the Stoner Package

We recommend the use of Ananconda Python for use with the Stoner package and provide conda pack-ages for Stoner and its non-standard dependencies. To install the Stoner package in an Anaconda Pythonenvironment, simply do:

conda install -c phygbu Stoner

If you are not using Ananconda python, then we also make Python wheels available that may be installedusing pip:

pip install Stoner

The advantage of getting the package this way is that it is installed into your Python path properly.The disadvantage is that you don’t get this user guide and the version may not be the most up to date(although given the fragile and continuously being broken state of the code that may be a good thing !).

Getting the Latest Development Code

15


Note: These instructions are for members of the University of Leeds Condensed Matter Physics Group.External users are recommended to download the source from GitHub

The source code for the Stoner python module is kept on github using the git revision con-trol tool. A nightly development release of the code is available for copying and use in\\stonerlab\data\software\python\PythonCode\.

The Stoner Package currently depends on a number of other modules. These are installed on the labmachines that have Python installed. Primarily these are Numpy, SciPy and Matplotlib. The easiestway to get a Python installation with all the necessary dependencies for the Stoner Package is to installthe Enthought Python Distribution, Canopy*. Installers for Windows, MacOS and Linux are kept in\\stonerlab\data\software\Python

The canonical source of the latest version is the master branch of the github archive. If you have gitinstalled on your machine then:

pip install git+https://github.com/gb119/Stoner-PythonCode.git

should install the current master branch. Otherwise download the zip file from the github site and do:

python setup.py install

to insatall it locally.

Using the Development Version of the Stoner Package

Note: You only need to follow this section if you are working with the version installed from the githubrepository. If you have installed the Stoner Package with the easy_install command given above, thenyou can disregard this section.

The easiest way to use the development version of the Stoner Package is to add the path to the directorycontaining the files from github to your PYTHONPATH environment variable. This can be done onMacs and Linux by doing.

cd export PYTHONPATH=`pwd`:$PYTHONPATH

On a windows machine the easiest way is to create a permanent entry to the folder in the system envi-ronment variables. Go to Control Panel -> System -> Advanced Tab -> click on Environment button andthen add or edit an entry to the system variable PYTHONPATH.

One this has been done, the Stoner module may be loaded from python command line:

import Stoner

or:

from Stoner import *

16 Chapter 7. User Guide


Documentation

These pages provide a user guide to the Stoner package and its various modules and classes. It is not areference to the library but instead aims to explain the various operations that are possible and provideshort examples of use. For the API reference for the library, please see the Module Api Documentation. There is also a single sided cheat sheet that summarises the examples in this user guide inthe github repository.

Warning: The code is still under active development to fix bugs and add features. Generally thingsdon’t get deliberately broken, but accidents happen, so if something stops working, please either fixand commit the code or tell Gavin.

Package Options

The Stoner package supports a set of package level options to control preferences such as whether toshow objects using ‘rich’ representations such as png or html output formats.:

from Stoner import OptionsOptions.short_repr=True

The options object supports reading and setting options through attribute assignment. Deleting an Op-tion atribute resets the Option back to the corresponding default value. dir(Options) will give a list ofall possible package options.

7.1.2 Loading and Examining Data

Loading a data file

The first step in using the Stoner module is to load some data from a measurement.:

from Stoner import Datad=Data('my_data.txt')

In this example we have loaded data from my_data.txt which should be in the current directory

The Stoner.Data class is actually a shorthand for importing the Stoner.Core.Data class which inturn is a superset of the classes in the Stoner package. This includes code to automatically detect theformat of many of the measurement files that we use in our research.

The native file format for the Stoner package is known as the TDI 1.5 format - a tab delimited text filethat stores arbitary metadata and a single 2D data set. It closely matches the DataFile class of theStoner.Core module.

Note: DataFile will also read a related text format where the first column of the first line containsthe string TDI Fromat=Text 1.0 which are produced by some of the LabVIEW rigs used by the DeviceMaterials Group in Cambridge.

7.1. The Stoner Python Package User Guide 17


The Various Flavours of the DataFile Class

To support a variety of different input file formats, the Stoner package provides a slew of subclassesof the base DataFile class. Each subclass typically provides its own version of the DataFile._load() method that understands how to reqad the relevant file.

Base Classes and Generic Formats

DataFile Tagged Data Interchange Format 1.5 – the default format produced by theLabVIEW measurement rigs in the CM Physics group in Leeds

Stoner.formats.generic.CSVFile Reads a generic comma separated value file.The Stoner.FileFormats.CSVFile.load() routine takes four additionalparameters to the constructor and load methods. In addition to the two extra argu-ments used for the BigBlue& variant, a further two parameters specify the delimina-tors for the data and header rows. :py:class:‘Stoner.FileFormats.CSVFile‘ also offersa **save* method to allow data to be saved in a simple deliminated text way (seeSection Saving Data for details).

Stoner.formats.generic.JustNumbersFile This is a subclass of CSVFilededicated for reading a text file that consists purely of rows of numbers with no headeror metadata.

Stoner.FileFormats.SPCFile Loads a Raman scan file (.spc format) produced bythe Rensihaw and Horiba Raman spectrometers. This may also work for other instru-ments that produce spc files, but has not been extensively tested.

Stoner.FileFormats.TDMSFile Loads a file saved in the National InstrumentsTDMS format

Stoner.FileFormats.QDFile Loads data from various Quantum Design instru-ments, cincluding PPMS, MPMS and SQUID VSM.

Stoner.FileFormats.OVFFile OVF files are output by a variety of micomagneticssimulators. The standard was designed for the OOMMF code. This class will handlerectangualr mesh files with text or binary formats, versions 1.0 and 2.0

Classes for Specifc Instruments (Mainly ones owned by the CM Physics Group in Leeds)

Stoner.FileFormats.VSMFile The text files produced by the group’s Oxford In-struments VSM

Stoner.FileFormats.BigBlueFile Datafiles produced by VB Code running onthe Big Blue cryostat. The Stoner.FileFormats.BigBlueFile version of theDataFile.load() and DataFile constructors takes two additional parametersthat specify the row on which the column headers will be found and the row on whichthe data starts.

Stoner.FileFormats.XRDFile Loads a scan file produced by Arkengarthdale - thegroup’s Brucker D8 XRD Machine.

Stoner.FileFormats.MokeFile Loads a file from Leeds Condensed MatterPhysics MOKE system.



Stoner.FileFormats.FmokeFile Loads a file from Dan Allwood’s FocussedMOKE System in Sheffield.

Stoner.FileFormats.LSTemperatureFile Loads and saves data fromLakeshore inc’s .340 Temperature Calibration file format.

Classes for Instruments at Major Facilities

Stoner.FileFormats.BNLFile Loads a SPEC file from Brookhaven (so far onlytested on u4b files but may well work with other synchrotron data). Produces metadataSnumber: Scan number, Stype: Type of scan, Sdatetime: date time stamp for themeasurement, Smotor: z motor position.

Stoner.FileFormats.OpenGDAFile Reads an ascii scan file generated by OpenGDA – a softwaresuite used for synchrotron such as Diamond.

Stoner.FileFormats.RasorFile Simply an alias for OpenGDAFile used for theRASOR instrument on I10 at Diamond.

Stoner.FileFormats.SNSFile Reads the ascii export file format from QuickNXSthe data reduction software used on the BL-4A Beam Line at the SNS in Oak Ridge.

Stoner.FileFormats.MDAASCIIFile This class will read some variants of theoutput of mda2ascii as used at the APS in Argonne.

These classes can be used directly to load data from the appropriate format.:

import Stonerimport stoner.FileFormats as SFFd=Stoner.DataFile()d.load('my_data.txt')v=SFF.VSMFIle()v.load('my_VSM_data.fld')c=SFF.CSVFile('data.csv',1,0,',',',')

Note: The load method, like many of the DataFile methods returns a copy of the Datafile object as wellas modifying the object itself. The advantage of this is that it is then possible to chain several methodsinto one command

Sometimes you won’t know exactly which subclass of DataFile is the one to use. Unfortunately,there is no sure fire way of telling, but DataFile.load() will try to do the best it can and will tryall of the subclasses in memory in turn to see if one will load the file without throwing an error. If thissucceeds then the actual type of file that worked is stored in the metadata of the loaded file.

Warning: The automatic loading assumes that each load routine does sufficient sanity checkingthat it will throw and error if it gets bad data. Whilst one might wish this was always true it relieson whoever writes the load method to make sure of this ! If you want to stop the automatic guessingfrom happening use the auto_load=False keyword in the load() method.

You can also specify a filetype parameter to the DataFile.load() method or directly to theStoner.Data constructor as illustrated below to load a simple text file of un labelled numbers:


http://sourceforge.net/projects/quicknxs/


from Stoner import Datad=Data("numbers.txt",filetype="JustNumbers",column_headers=["z","I","dI"],→˓setas="xye")

If filetype is a string, it can match either the complete name of the subclass to use to load the file, or partof it.

Loading Data from a string or iterable object

In some circumstances you may have a string representation of a DataFile object and want to trans-form this into a proper DataFile object. This might be, for example, from transmitting the data overa network connection or receiving it from another program. In these situations the left shift operator


Two Argument Constructor

If the second argument is a dictionary or a list or other iterables of strings, then the first argumentinterpreted as for the sibgle argument constructors above, and then the second argument is interpreted inthe same way. (Internally this done by calling the single argument constructor function twice, once foreach argument.)

Many Argument Constructor

If there are more than two arguments and they are numpy arrays, then they are used as the columns ofthe new DataFile.

Keyword Arguments in Constructor

After the positional arguments are dealt with, any keywords that match attriobutes of the DataFile areused to set the corresponding attribute.

Examples

# Load a file from disc, set the setas attribute and column headersd=Data("filename.txt",setas="xy", column_headers=["X-Data","Y-Data"])# Create a DataFile from a dictionary:d=Data({"Temperature":temp_array,"Resistance":res_data})# The same, but set metadata tood=Data({"Temperature":temp_array,"Resistance":res_data},{"User":"Fred",→˓"Sample":"X234_a","Field":2.4})# From a pandas DataFramedf=pd.DataFrame(...)d=Data(df)

Examining and Basic Manipulations of Data

Data Structure

Data, Column headers and metadata

Having loaded some data, the next stage might be to take a look at it. Internally, data is represented as a2D numpy masked array of floating point numbers, along with a list of column headers and a dictionarythat keeps the metadata and also keeps track of the expected type of the metadata (ie the meta-metadata).These can be accessed like so:

d.datad.column_headersd.metadata



Masked Data and Why You Care

Masked data arrays differ from normal data arrays in that they include an option to mask or hide indi-vidual data elements. This can be useful to temporarily discount parts of your data when, for example,fitting a curve or calculating a mean value or plotting some data. One could, of course, simply ignorethe masking option and use the data as is, however, masking does have a number of practical uses.

The data mask can be accessed via the DataFile.mask attribute of DataFile:

import numpy.ma as maprint d.maskd.mask=Falsed.mask=ma.nomaskd.mask=numpy.array([[True, True, Fale,...False],...,[False,True,...True]])d.mask=lambda x: x[0]


Marking Columns as Dimensions: the magic setas attribute

Often in a calculation with some data you will be using one column for ‘x’ values and one or more ‘y’columns or indeed having ‘z’ column data and uncertainties in all of these (conventionally we call these‘d’, ‘e’ and ‘f’ columns so that ‘e’ data is the error in the y data). DataFile has a concept of markinga column as containing such data and will then use these by default in many methods when appropriateto have ‘x’ and ‘y’ data.

In addition to identifying columns as ‘x’,’y’, or ‘z’, for data that describes a vector field, you can markthe columns as containing ‘u’, ‘v’, ‘w’ data where (u,v,w) is the vector value at the point (x,y,z). There’sno support at present for uncertainities in (u,v,w) being marked.

Setting Column Types

To set which columns contain ‘x’,’y’ etc data use the DataFile.setas attribute. This attribute cantake a list of single character strings from the set ‘x’,’y’,’z’,’d’,’e’, ‘f’, ‘u’, ‘v’, ‘w’ or ‘.’ where eachelement of the list refers to the columns of data in order. To specify that a column has unmarked datause the ‘.’ string. The string ‘-‘ can also be used - this indicats that the current assignment is to be left asis.

Alternately, you can pass DataFile.setas a string. In the simplest case, the string is just read inthe same way that the list would have been - each character corresponds to one column. However, if thestring contains an integer, then the next non-numeric character will be interpreted that many times, so:

d.setas="3.xy"d.setas="...xy"d.setas=['.','.','.','x','y']

There are still more ways of setting column types with the DataFile.setas attribute:

d.seetas[3]="x"d.setas["x"]=3d.setas("3.xy")d.setas(['.','.','.','x','y'])d.setas(x=3,y=4)

All achieve the same effect of setting the same columns as containing x and y data.

Once you have identified columns for the various types, you also have access to utility attributes toaccess those columns:


https://www.youtube.com/watch?v=LbSIqxTD9Xc


d.setas="3.xye'd.x == d.column(3)d.y == d.column(4)d.e == d.column(5)

Note that if DataFile.setas is not set then attempting to use the quick access column attributes willresult in an exception. Once the DataFile.setas attribute is set, a further set of virtual or derivedcolumn attributes become available.:

d.setas="xyz"d.r # The magnitude of the point x,y,zd.q # The angle in the x-y plane relative to the x axisd.p # The angle relative to the z axisd.setas="xyz..uvw"d.r # the magnitude of (u,v,w)d.q # The angle in the x-y plane of (u,v) relative to +xd.p # The angle of (u,v,w) relatiuve to +z

Where fewer than 3 or 6 dimensions are specified, these virtual columns fallback to working with theappropriate reduced number dimensions.

There are some more convenience ways to set which columns to use as x,y,z etc.:

d.setas={"x":0,"y":"Y Column title"}d.x=0d.y="Y Column title"

d.setas["Temperature"]="y"

In each of these cases, the DataFile will try to work out what you intended to achieve for maximumflexibility and convenience when writing code. However it can be fooled if one of your columns is called‘x’ or ‘y’ !

Reading Column Types

The normal representation of DataFile.setas is as a list, but it also has a string conversion avail-able. You can also find which column has been assigned as ‘x’, ‘y’ etc. by treating the DataFile.setas as a dictionary:

d.column_headers=["One","Two","three","Four"]d.setas="xy.z"

print list(d.setas) #['x','y','.','z']print d.setas #'xy.z'print d.setas['x'] # "One"print d.setas["#x"] # 0

Note that the DataFile.setas attribute supports reading keys that are either the single letter t getthe name of the column or the letter preceded by a # character to get the number of the column.

Alternatively, and equivalently, you can access the column indexes via attributes of DataFile.setas:

d.setas.has_xcol # True d.setas.has_ucol # False d.setas.ycol # [1] d.setas.xcol # 0



Implied Column Assignments

If you do not specify the column types via the setas attributes, then DataFile will try to guess sensiblecolumns assignments based on the number of columns in your data file. These default assignments areonly done at the point at which the DataFile.setas attribute is consulted. The default assignmentsare:

Number of Columns Assignments2 x, y3 x, y, e4 x, d, y, e5 x, y, u, v, w6 x, y, z, u, v, w

Swapping and Rotating Column Assignments

Finally, if the DataFile.setas attribute has been set with x, y (and z) columns then these assing-ments can be swapped around by the invert operator ~. This either swaps x and y with eir associatederrorbars for 2-D datasets, or rotates x to y, y to z and z to x )again with their associated errors bars.:

d.setas="xye"print d.setas>>> ['x','y','e']e=~dprint e.setas>>> ['y','x','d']

Printing the Complete DataFile

If the optional tabulate package is installed, then a pretty formatted representation of the DataFilecan be generated using:

print(repr(d))

This will give something like:

================================ ============= ============ ===========TDI Format 1.5 Temperature Resistance Column 2index 0 1 2================================ ============= ============ ===========Stoner.class{String}= b"'Data'" 291.6 4.7878 0.04687595Measurement Title{String}= b"''" 291.6 4.78736 1.125022Timestamp{String}= b"''" 291.6 4.78788 2.187542User{String}= b"''" 291.6 4.78758 3.250062TDI Format{Double Float}= b'1.5' 291.6 4.78782 4.312583Loaded as{String}= b"'DataFile'" 291.6 4.7878 5.375103

291.6 4.78748 6.453249291.6 4.7878 7.515769291.6 4.78789 8.57829

If more columns exist in the DataFile then the repr method attempts to pick ‘interesting’ columns.Thealgorithm will prioritise showing columns that have been assigned a meaning with the DataFile.



setas attribute. If there are space for further columns, then the last column will be shown and othercolumns that follow from any that are marked in DataFile.setas. If no columns are marked asinteresting, then the first n-2 columns and the last column will be shown.:

==================== ===================== =====================→˓==================== =================== ============= ==============TDI Format 1.5 Magnetic Field (Oe) Moment (emu) M. Std.→˓ Err. (emu) Transport Action .... Map 16index 3 (x) 4 (y)→˓ 5 6 71==================== ===================== =====================→˓==================== =================== ============= ==============Stoner.class{String} 0.990880310535431 -5.83640043200129e-06 8.→˓83351337362955e-09 1.0 nan= b"'Data'" 0.965473115444183 -5.82915649064088e-06 1.→˓23199616933718e-08 1.0 ... nanTitle,{String}= 1.16873073577881 -5.82160118818014e-06 8.→˓92093033864879e-09 1.0 ... nanb'None' 1.13061988353729 -5.8201045325249e-06 1.→˓07142611311115e-08 1.0 ... nanFileopentime{String} 1.33387744426727 -5.83922456812945e-06 1.→˓02539653165381e-08 1.0 ... nan= b"'3540392668.062 1.49902403354645 -5.81961870971478e-06 1.→˓04490646832536e-08 1.0 ... nan

The table header lists the column titles, numerical indices for each column and the assignment in theDataFile.setas attribute.

If the file has more than 256 rowns, then the first 128 rows and last 128 rows will be shown with a rowof . . . to show the split.

Many of the methods in the Stoner package return a copy of the current Stoner.Data object and inipython consoles and jupyter notebooks these will get printed out using the table formats above. Thismay be more than is required, in which case you can set options in the Stoner package to control theoutput format.:

from Stoner import OptionsOptions.short_data_repr = True # or "short_repr" for short representations→˓of all objects.print(repr(d))>>> TDI_Format_RT.txt() of shape (1676, 3) (xy.) and→˓6 items of metadata

Working with columns of data

This is all very well, but often you want to examine a particular column of data or a particular row:

d.column(0)d.column('Temperature')d.column(['Temperature',0])

In the first example, the first column of numeric data will be returned. In the second example, the columnheaders will first be checked for one labeled exactly Temperature and then if no column is found, thecolumn headers will be searched using Temperature as a regular expression. This would then matchTemperature (K) or Sample Temperature. The third example results in a 2 dimensional numpy array



containing two columns in the order that they appear in the list (ie not the order that they are in the datafile). For completeness, the DataFile.column() method also allows one to pass slices to selectcolumns and should do the expected thing.

There are a couple of convenient short cuts. Firstly the floormod operator // is an alias for theDataFile.column() method and secondly for working with cases where the column headers arenot the same as the names of any of the attributes of the DataFile object:

d//"Temperature"d.Temperatured.column('Temperature')

all return the same data.

Whenever the Stoner package needs to refer to a column of data, you cn specify it in a number of ways:-

1) As an integer where the first column on the left is index 0

2) As a string. if the string matches a column header exactly then the index of that column is returned.If the string fails to match any column header it is compiled as a regular expression and thenthat is tried as a match. If multiple columns match then only the first is returned.

3) As a regular expression directly - this is similar to the case above with a string, but allows you to pass a pre-compiled regularexpression in directly with any extra options (like case insensitivity flags) set.

4) As a slice object (ee.g. 0:10:2) this is expanded to a list of integers.

5) As a list of any of the above, in which case the column finding routine is called recursively in turn for each element of the list andthe final result would be to use a list of column indices.

For example:: import re col=re.compile(‘^temp’,re.IGNORECASE) d.column(col)

Working with complete rows of data

Rows don’t have labels, so are accessed directly by number:

d[1]d[1:4]

The second example uses a slice to pull out more than one row. This syntax also supports the full slicesyntax which allows one to, for example, decimate the rows, or directly pull out the last fews rows in thefile.

Special Magic When Working with Subsets of Data

As mentioned above, the data in a DataFile is a sepcial siubclass of numpy’s Masked Array -DataArray. A DataArray understands that columns can have names and can be assigned to holdspecific types of data - x,y,z values etc. In fact, the logic used for the column names and setas attributein a DataFile is actually supplied by the DataArray. Whn you index a DataFile or it’s data, theresulting data remembers it’s column names and assignments and these can be used directly:

r=d[1:4]print r.x,r.y



In addition to the column assignments, DataArray also keeps a track of the row numbers and makesthem available via the i attribute.:

d.data.i # [0,1,2,3...,len(d)]r=d[10]r.i # 10r.column_headers

You can reset the row numbers by assiging a value to the i attribute.

A single column of data also gains a .name attribute that matches its column_header:

c=d[:,0]c.name == c.column_headers[0] #True

Manipulating the metadata

What happens if you use a string and not a number in the above examples ?:

d['User']

in this case, it is assumed that you meant the metadata with key User. To get a list of possible keys inthe metadata, you can do:

d.dir()d.dir('Option\:.*')

In the first case, all of the keys will be returned in a list. In the second, only keys matching the pat-tern will be returned – all keys containing Option:. For compatibility with normal opython semantics:DataFile.keys() is synonymous with DataFile.dir().

If the string you supply to get the metadata item does not exactly match an item of metadata, then it isinterpreted as a regular expression to try and match against all the items of metadata. In this case, ratherthan returning a single item, all of the matching metadata is returned as a dictionary. PAssing a compiledregular epxression as the item name also has the same effect - this is useful if the regular expression youwant to match is also an exact match to one particular metadata name.

We mentioned above that the metadata also keeps a note of the expected type of the data. You can get atthe metadata type for a particular key like this:

d.metadata.type('User')

to get a dictionary of all of the types associated with each key you could do:

dict(zip(d.dir(),d.metadata.type(d.dir())))

but an easier way would be to use the typeHintedDict.types attribute:

d.metadata.types

More on Indexing the data

There are a number o other forms of indexing supported for DataFile objects.:



d[10,0]d[0:10,0]d[10,'Temp']d[0:10,['Voltage','Temp']

The first variant just returns the data in the 11th row, first column (remember indexing starts at 0). Thesecond variant returns the first 10 values in the first column. The third variant demonstrates that columnscan be indexed by string as well as number, and the last variant demonstrates indexing multiplerows andcolumns – in this case the first 10 values of the Voltage and Temp columns.

You might think of the data as being a list of records, where each column is a field in the record. Numpysupports this type of structured record view of data and the DataFile object provides the DataFile.records attribute to d this. This read-only attribute is just providing an alternative view of the samedata.:

d.records

Finally the DataFile.dict_records atrtibute does the same thing, but presetns the data as anarray of dictionaries, where the keys are the column names and each dictionary represents a single row.

Selecting Individual rows and columns of data

Many of the function in the Stoner module index columns by searching the column headings. If onewishes to find the numeric index of a column then the DataFile.find_col() method can be used:

index=d.find_col(1)index=d.find_col('Temperature')index=d.find_col('Temp.*')index=d.find_col('1')index=d.find_col(1:10:2)index=d.find_col(['Temperature',2,'Resistance'])index=d.find_col(re.compile(r"^[A-Z]"))

DataFile.find_col() takes a number of different forms. If the argument is an integer then itreturns (trivially) the same integer, a string argument is first checked to see if it exactly matches one ofthe column headers in which case the number of the matching column heading is returned. If no exactmatch is found then a regular expression search is carried out on the column headings. In both cases,only the first match is returned. If the string still doesn’t match, then the string is checked to see if it canbe cast to an integer, in which case the integer value is used.

The final three examples given above both return a list of indices, firstly using a slice construct - in thiscase the result is trivially the same as the slice itself, and in the second example by passing a list ofcolumn headers to look for. The final example uses a compiled regular expression. Unlike passing astring which contains a regular expression, passing a compiled regular expression will return a list of allcolumns that match. This distinction allows you to use a unique partial string to match just one column- but if you really want all possible columns that would match the pattern, then you can compile theregular expression and pass that instead.

This is the function that is used internally by DataFile.column(), DataFile.search() etcand for this reason the trivial integer and slice forms are implemented to allow these other functions towork with multiple columns.

Sometimes you may want to iterate over all of the rows or columns in a data set. This can be done quiteeasily:



for row in d.rows():print row

for column in d.columns():print column

......

If there is no mask set, then the first example could also have been written more compactly as:

for row in d:print row

......

Note: DataFile.rows() and DataFile.columns() both take an optional parameternot_masked. If this is True then these iterator methods will skip over any rows/columns with masked outdata values. When iterating over the DataFile instance directly the masked rows are skipped over.

Searching, sectioning and filtering the data

Searching

In many cases you do not know which rows in the data file are of interest - in this case you want tosearch the data.:

d.search('Temperature',4.2,accuracy=0.01)d.search('Temperature',4.2,['Temperature','Resistance'])d.search('Temperature',lambda x,y: x>10 and x10 and

x


Filtering

Sometimes you may want not to get the rows of data that you are looking for as a separate array, butmerely mark them for inclusion (or exclusion) from subsequent operations. This is where the maskedarray (see ‘Masked Data and Why You Care) comes into its own. To select which rows of data have beenmasked off, use the DataFile.filter() method.:

d.filter(lambda r:r[0]>5)d.filter(lambda r:r[0]>5,['Temp'])

With just a single argument, the filter method takes a complete row at a time and passes it to the firstargument, expecting to get a boolean response (or list olf booleans equal in length to the number ofcolumns). With a second argument as in the second example, you can sepcify which columns are passedto the filtering function in what order. The second argument must be a list of things which can be usedto index a column (ie strings, integers, regular expressions).

Selecting

A very powerful way to get at just the dat rows you want is to make use of the DataFile.select()method. This offers a simple way to query which rows have columns matching some criteria.:

d.select(Temp=250)d.select(Temp__ge=150)d.select(T1__lt=4,T2__lt=5).select(Res__between=(100,200))

The general form is to provide keyword arguments that are something that can be used to index a column,followed by a double underscore, followed by and operator. Where more than one keyword argumentis supplied, the results of testing each row are logically ORed. The result of chaining together twoseparate calls to select will, however, logically AND the two tests. So, in the examples above, the, firstline will assume an implicit equality test and give only those rows with a column Temp equal to 250.The second line gives an explicit greather than or equal to test for the same column. The third line willselect first those rows that have column T1 less than 4.2 or column T2 less than 5 and then from thoseselect those rows which have a column Res between 100 and 200. The full list of operators is given inStoner.Folders.baseFolder.select().

Sectioning

Another option is to construct a new DataFile object from a section of the data - this is particularlyuseful where the DataFile represents data correspondi ng to a set of (x,y,z) points. For this case the:py:,eth:DataFile.section method can be used:

d.setas="x..y.z."slab=d.section(x=5.2)line=d.section(x=4.7,z=2)thick_slab=d.section(z=(5.0,6.0))arbitary=d.section(r=lambda x,y,z:3*x-2*y+z-4==0)

After the x, y, z data columns are identified, the DataFile.section() method works with ‘x’, ‘y’and ‘z’ keyword arguments which ar then used to search for matching data rows (the arguments to thesekeyword arguments follow the same rules as the DataFile.search() method).



A final way of searching data is to look for the closest row to a given value. For this the eponymousmethod may be used:

r=d.closest(10.3,xcol="Search Col")r=d.closest(10.3)

If the xcol parameter is not supplied, the value from the DataFile.setas attribute is used. Since thereturned row is an instance of the DataArray that has been taken from the original data, it will knowwhat row number it was and will make that available via it’s i attribute.

Find out more about the data

Another question you might want to ask is, what are all the unique values of data in a given column (orset of columns). The Python numpy package has a function to do this and we have a direct pass throughfrom the DataFile object for this:

d.unique('Temp')d.unique(column,return_index=False, return_inverse=False)

The two optional keywords cause the numpy routine to return the indices of the unique and all non-unique values in the array. The column is specified in the same way as the DataFile.column()method does.

Copying Data

One of the characteristics of Python that can confuse those used to other programming languages is thatassignments and argument passing is by reference and not by value. This can lead to unexpected resultsas you can end up modifying variables you were not expecting ! To help with creating genuine copiesof data Python provides the copy module. Whilst this works with DataFile objects, for convenience, theDataFile.clone atribute is provided to make a deep copy of a DataFile object.

Note: This is an attribute not a method, so there are no brackets here !

t=d.clone

Modifying Data

Appending data

The simplest way to modify some data might be to append some columns or rows. The Stoner moduleredefines two standard operators, + and & to have special meanings:

a=Stoner.DataFile('some_new_data.txt')add_rows=d+aadd_columns=d&a

In these example, a is a second DataFile object that contains some data. In the first example, a newDataFile object is created where the contents of a are added as new rows after the data in d. Any



metadata that is in a and not in d are added to the metadata as well. There is a requirement, however,that the column headers of d and a are the same – ie that the two DataFile objects appear to representsimilar data.

In the second example, the data in a is added as new columns after the data from d. In this case, there isa requirement that the two DataFile objects have the same number of rows.

These operators are not limited just to DataFile objects, you can also add numpy arrays to the DataFileobject to append additional data.:

import numpy as npx=np.array([1,2,3])new_data=d+xy=np.array([1,2,3],[11,12,13],[21,22,23],[31,32,33]])new_data=d+yz={"X":1.0,"Y":2.1,"Z":7.5}new_data=d+znew_data=d+[x,y,z]column=d.column[0]new_data=d&column

In the first example above, we add a single row of data to d. This assumes that the number of elementsin the array matches the number of columns in the data file. The second example is similar but this timeappends a 2 dimensional numpy array to the data. The third example demonstrates adding data froma dictioary. In this case the keys of the dictionary are used to determine which column the values areadded to. If their columns that don’t match one of the dictionary keys, then a NaN is inserted. If thereare keys that don’t match columns labels, then new columns are added to the data set, filled with NaN.In the fourth example, each element in the list is added in turn to d. A similar effect would be achievedby doing new_data=d+x+y+z.

The last example appends a numpy array as a column to d. In this case the requirement is that the numpyarray has the same or fewer rows of data as d.

Working with Columns of Data

Changing Individual Columns of Data

The DataFile.data attribute is not simply a 2D numpy array, but a special subclass DataArray,but still can be directly modified like any other numpy array like class might be. If, however, theDataFile.setas attribute has been used to identify columns as containing x,y,z,u,v,w,d,e or f typedata, then the correspondign attributes can be written to as well as read to directly modify the datawithout having to keep track any further of which column(s) is indexed. Thus the following will work:

d.setas="x..y..z"d.x=d.x-5.0*d.y/d.zd.y=d.z**2d.z=np.ones(len(d))

When writing to the column attriobutes you must supply a numpy array with the correct number ofelements (although DataFile will try to spot and correct if the array needs to be transposed first). Ifyou specify more than one column has a particular type then you should supply a 2D array with thecorresponding number of columns of data setting the attribute.

In order to preserve the behaviour that allows you to set the column assingments by setting the attribute



to an index type, the DataFile checks to see if you are setting something that might be a column indexor a numpy array. Thus the following also works:

d.x="Temp" # Set the Temp column to be x datad.x=np.linspace(0,300,len(d)) # And set the column to contain a linear→˓array of values from 0 to 300.

You cannot set the p,q, or r attributes like this as they are calculated on the fly from the cartesian co-ordinates. On the otherhand you can do an efficient conversion to polar co-ordinates with:

d.setas="xyz"(d.x,d.y,d.z)=(d.p,d.q,d.r)

Rearranging Columns of Data

Sometimes it is useful to rearrange columns of data. DataFile offers a couple of methods to help withthis.:

d.swap_column('Resistance','Temperature')d.swap_column('Resistance','Temperature',headers_too=False,setas_too=False)d.swap_column([(0,1),('Temp','Volt'),(2,'Curr')])d.reorder_columns([1,3,'Volt','Temp'])d.reorder_columns([1,3,'Volt','Temp'],header_too=False,setas_too=False)

The DataFile.swap_column() method takes either a either a tuple (or just a pair of arguments)of column names, indices or a list of such tuples and swaps the columns accordingly, whilst theDataFile.reorder_columns() method takes a list of column labels or indices and constructsa new data matrix out of those columns in the new order. The headers_too=False options, as thename suggests, cause the column headers not be rearranged.

Note: The addition of the setas_too keyword to swap the column assignments around as well is new in0.5rc1

Renaming Columns of Data

As a convenience, DataFile also offers a useful method to rename data columns:

d.rename('old_name','new_name')d.rename(0,'new_name')

Alternatively,of course, one could just edit the column_headers attribute.

Inserting Columns of Data

The append columns operator & will only add columns to the end of a dataset. If you want to add acolumn of data in the middle of the data set then you should use the DataFile.add_column()method.:



d.add_column(numpy.array(range(100)),header='Column Header')d.add_column(numpy.array(range(100)),header='Column Header',index=Index)d.add_column(lambda x: x[0]-x[1],header='Column Header',func_args=None)

The first example simply adds a column of data to the end of the dataset and sets the new column headers.The second variant inserts the new column before column Index. Index follows the same rules as for theDataFile.column() method. In the third example, the new column data is generated by applyingthe specified function. The function is passed s dingle row as a 1D numpy array and any of the keyword,argument pairs passed in a dictionary to the optional func_args argument.

The DataFile.add_column() method returns a copy of the DataFile object itself as well as modi-fying the object. This is to allow the metod to be chained up with other methods for more compact codewriting.

Deleting Rows of Data

Removing complete rows of data is achieved using the DataFile.del_rows() method.:

d.del_rows(10)d.del_rows(1:100:2)d.del_rows('X Col',value)d.del_rows('X Col',lambda x,y:x>300)d.del_rows('X Col',(100,200))d.del_rows(;X Col',(100,200),invert=True)

The first variant will delete row 10 from the data set (where the first row will be row 0). You can alsosupply a list or slice (as in the second example) to DataFile.del_rows() to delete multiple rows.

If you do not know in advance which row to delete, then the remiaining variants provide more advancedoptions. The third variant searches for and deletes all rows in which the specified column contains value.The fourth variant selects which ros to delete by calling a user supplied function for each row. The usersupplied function is the same in form and definitition as that used for the DataFile.search()method:

def user_func(x_val,row_as_array):return True or False

The final two variants above, use a tuple to select the data. The final example makes use of the invertkeyword argument to reverse the sense used to selkect tows. In both cases rows are deleted(kept forinvert = True) if the specified column lies between the maximum and minimum values of the tuple. Thetest is done inclusively. Any length two iterable object can be used for specifying the bounds. Fianlly,if you call DataFile.del_rows() with no arguments at all, then it removes all rows where at leastone column of data is masked out.:

d.filter(lambda r:r[0]>50) # Mask all rows where the first column is→˓greater than 50d.del_rows() # Then delete them.

For simple caases where the row to be delted can be expressed as an integer or list of integers, thesubtration operator can be used.:



e=d-2e=d-[1,2,3]d-=-1

The final form looks stranger than it is - it simply deletes the last row of data in place.

Deleting Columns of Data

Deleting whole columns of data can be done by referring to a column by index or column header - theindexing rules are the same as used for the DataFile.column() method.:

d.del_column('Temperature')d.del_column(1)

Again, there is an operator that can be used to achieve the same effect, in this case the modulus operator%.:

e=d%"temperature"e=d%1d%=-1

Sorting Data

Data can be sorted by one or more columns, specifying the columns as a number or string for singlecolumns or a list or tuple of strings or numbers for multiple columns. Currently only ascending sorts aresupported.:

d.sort('Temp')d.sort(['Temp','Gate'])

Saving Data

Only saving data in the TDI format and as comma or tab deliminated formats is supported.

For example:

d.save()d.save(filename)d=Stoner.CSVFile(d)d.save()d.save(filename,'\t')

In the first case, the filename used tosave the data is determined from the filename attribute of theDataFile object. This will have been set when the filewas loaded from disc.

If the filename attribute has not been set eg if the DataFile object was created from scratch, then theDataFile.save() method will cause a dialogue box to be raised so that the user can supply afilename.

In the second variant, the supplied filename is used and the filename attribute is changed to match thisie d.filename will always return the last filename used for a load or save operation.



The third is similar but convert the file to cvs format while the fourth also specifies that the eliminatoris a tab character.

Exporting Data to pandas

The Stoner.Data.to_pandas() method can be used to convert a Stoner.Data object to apandas.DataFrame. The numerical data will be transferred directly, with the DataFrame columns beingset up as a two level index of column headers and column assignments. The Stoner library registers anadditional metadata extension attribute for DataFrames that provides thin sub-class wrapper around thesame regular expression based and type hinting dictionary that is used to store metadata in Stoner.Data.metadata.

The pandas.DataFrame produced by the Stoner.Data.to_pandas() method is reversibly con-vertable back to an identical Stoner.Data object by passing the DataFrame into the constructor ofthe Stoner.Data object.

7.1.3 Plotting Data

Data plotting and visualisation is handled by the Data sub-class of Stoner.Core.Data. The pur-pose of the methods detailed here is to provide quick and convenient ways to plot data rather thanproviding publication ready figures. The Data is included as apart of the Data class:

Quick Plots

The Data class is intended to help you make plots that look reasonably good with as little hassle aspossible. In common with many graph plotting programmes, it has a concept of declaring columns ofdata to be used for ‘x’, ‘y’ axes and for containing error bars. This is done with the Data.setasattribute (see Marking Columns as Dimensions: the magic setas attribute for full details). Once this isdone, the plotting methods will use these to try to make a sensible plot.:

p.setas="xye"p.plot()

Data.plot() is simply a wrapper that insepects the available columns and calls either Data.plot_xy() or Data.plot_xyz() as appropriate. All keyword arguments to Data.plot() arepassed on to the actual plotting method.

Each Data instance maintains it’s own matplotlib.pyplot.Figure in the Data.fig attribute,if no figure is set when the Data.plot() method is called, a new figure will be created.

Types of Plot

Data.plot() will try to make the msot sensible choice of plot depending on which columns you havespecified and the number of axes represetned.

• x-y, x-y+-e, x-y-e-y-e etc. data will default to a 2D scatter plot with error bars and call data.plot)xy()

• x-y-z data will be converted to a grid and plotted as a 3D surface plot and call Data.plot_xyz()



• x-y-u-v (i.e. 2D vectro fields) and x-y-u-v-w (3D vectros on a 2D plane) will be plotted as a colour imagewhere the colour is mapped to hue-saturation-luminescence scale. The hue gives the inplane ange while the luminescene gives the out of plane component of the vector field.These are handdled with Data.plot_xyuv()

• x-y-z-u-v-w data (i.e. 3D vector field on a 3D grid) is represented as a 3D quiver plot with coloured quivers (using the same H-S-Lcolour space mapping as above) assuming mayavi is importable, otherwise using a 3Dquiver plot from matplotlib’s 3D toolkit. Data.plot_xyzuvw() handles the work forthis case.

An alternative plot type for data with errorbars in plotutils.errorfill(). This uses a shadedline as an alternative to the error bars, where the shading of the line varies from intense to transparentthe further one gets from the mean value. For 3D x-y-z plotting there is a Data.contour_xyz()and a Data.image_plot() methods available. These give contonour plots and 2D colour map plotsrespectively.

Plotting 2D data

x-y plots are produced by the Data.plot_xy() method:

p.plot_xy(column_x, column_y)p.plot_xy(column_x, [y1,y2])p.plot_xy(x,y,'ro')p.plot_xy(x,[y1,y2],['ro','b-'])p.plot_xy(x,y,title='My Plot')p.plot_xy(x,y,figure=2)p.plot_xy(x,y,plotter=pyplot.semilogy)

The examples above demonstrate several use cases of the Data.plot_xy() method. The first pa-rameter is always the x column that contains the data, the second is the y-data either as a single columnor list of columns. The third parameter is the style of the plot (lines, points, colours etc) and can eitherbe a list if the y-column data is a list or a single string. Finally additional parameters can be given tospecify a title and to control which figure is used for the plot. All matplotlib keyword parameters can bespecified as additional keyword arguments and are passed through to the relevant plotting function. Thefinal example illustrates a convenient way to produce log-linear and log-log plots. By default, Data.plot_xy() uses the pyplot.plot function to produce linear scaler plots. There are a number of usefulplotter functions that will work like this

• [matplotlib.pyplot.semilogx(),:py:func:matplotlib.pyplot.semilogy] These two plot-ting functions will produce log-linear plots, with semilogx making the x-axes the log one andsemilogy the y-axis.

• [matplotlib.pyplot.loglog()] Like the semi-log plots, this will produce a log-log plot.

• [matplotlib.pyplot.errorbar()] this particularly useful plotting function will draw er-ror bars. The values for the error bars are passed as keyword arguments, xerr or yerr. In standardmatplotlib, these can be numpy arrays or constants. Data.plot_xy() extends this by inter-cepting these arguements and offering some short cuts:

p.plot_xy(x,y,plotter=errorbar,yerr='dResistance',xerr=[5,'dTemp+'])

This is equivalent to doing something like:


https://matplotlib.org/api/_as_gen/matplotlib.pyplot.semilogx.html#matplotlib.pyplot.semilogxhttps://matplotlib.org/api/_as_gen/matplotlib.pyplot.loglog.html#matplotlib.pyplot.logloghttps://matplotlib.org/api/_as_gen/matplotlib.pyplot.errorbar.html#matplotlib.pyplot.errorbar


p.plot_xy(x,y,plotter=errorbar,yerr=p.column('dResistance'),xerr=[p.→˓column(5),p.column('dTemp+')])

If you actually want to pass a constant to the x/yerr keywords you should use a float rather thanan integer.

The X and Y axis label will be set from the column headers.

Plotting 3D Data

A number of the measurement rigs will produce data in the form of rows of $x,y,z$ values. Often it isdesirable to plot these on a surface plot or 3D plot. The Data.plot_xyz() method can be used forthis.

"""3D surface plot example."""import numpy as npimport matplotlib.cm

from Stoner import Data

x, y = np.meshgrid(np.linspace(-2, 2, 100), np.linspace(-2, 2, 100))x = x.ravel()y = y.ravel()z = np.cos(4 * np.pi * np.sqrt(x ** 2 + y ** 2)) * np.exp(

-np.sqrt(x ** 2 + y ** 2))

p = Data()p = p & x & y & zp.column_headers = ["X", "Y", "Z"]p.setas = "xyz"

p.plot_xyz(cmap=matplotlib.cm.jet)p.title = "Surface plot"

"""Plot 3d fdata as a colourmap."""import numpy as np



-np.sqrt(x ** 2 + y ** 2))

p = Data(np.column_stack((x, y, z)), column_headers=["X", "Y", "Z"])p.setas = "xyz"

p.colormap_xyz()p.title = "Colourmap plot"

By default the Data.plot_xyz()will produce a 3D surface plot with the z-axis coded with a rainbowcolour-map (specifically, the matplotlib provided matplotlib.cm.jet colour-map. This can be overriden



Column 0

2 1 01

2 Colu

mn 0

21

01

2

Colu

mn

11e1

5

0

5

Surface plot

5.02.5

0.02.55.0

1e 1



2 1 0 1 2X

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

Y

Colourmap plot

6

4

2

0

2

4

6

8

1e 1



with the cmap keyword parameter. If a simple 2D surface plot is required, then the plotter parametershould be set to a suitable function such as pyplot.pcolor.

Like Data.plot_xy(), a figure parameter can be used to control the figure being used and anyadditional keywords are passed through to the plotting function. The axes labels are set from the corre-sponding column labels.

Another option is a contour plot based on (x,y,z) data points. This can be done with the Data.contour_xyz() method.

"""Plot 3D data on a contour plot."""import numpy as npfrom Stoner import Data


-np.sqrt(x ** 2 + y ** 2))

p = Data()p = p & x & y & zp.column_headers = ["X", "Y", "Z"]p.setas = "xyz"

p.contour_xyz()p.title = "Contour plot"

Both Data.plot_xyz() and Data.contour_xyz() make use of a call to Data.griddata()which is a utility method of the Data – essentially this is just a pass through method to the underlyingscipy.interpolate.griddata* function. The shape of the grid is determined through a combination of thexlim, ylim and shape arguments.:

X,Y,Z=p.griddata(xcol,ycol,zcol,shape=(100,100))X,Y,Z=p.griddata(xcol,ycol,zcol,xlim=(-10,10,100),ylim=(-10,10,100))

If a xlim or ylim arguments are provided and are two tuples, then they set the maximum and minimumvalues of the relevant axis. If they are three tuples, then the third argument is the number of pointsalong that axis and overrides any setting in the shape parameter. If the xlim or ylim parameters are notpresents, then the maximum and minimum values of the relevant axis are used. If no shape informationis provided, the default is to make the shape a square of sidelength given by the square root of the numberof points.

Alternatively, if your data is already in the form of a matrix, you can use the Data.plot_matrix() method.

"""Plot data defined on a matrix."""import numpy as np


x, y = np.meshgrid(np.linspace(-2, 2, 101), np.linspace(-2, 2, 101))z = np.cos(4 * np.pi * np.sqrt(x ** 2 + y ** 2)) * np.exp(

-np.sqrt(x ** 2 + y ** 2))

(continues on next page)



Column 0

21

01

2

Colum

n 0

21

01

21e

17.55.02.5

0.02.55.07.5

Contour plot



(continued from previous page)

p = Data()p = p & np.linspace(-2, 2, 101) & zp.column_headers = ["X"]for i, v in enumerate(np.linspace(-2, 2, 101)):

p.column_headers[i + 1] = str(v)

p.plot_matrix(xlabel="x", ylabel="y", title="Data as Matrix")

x

2 1 01

2

y2

10

12

Z Da

ta

0.5

0.0

0.5

1.0

Data as Matrix

642

024

1e 1

The first example just uses all the default values, in which case the matrix is assumed to run from the2nd column in the file to the last and over all of the rows. The x values for each row are found fromthe contents of the first column, and the y values for each column are found from the column headersinterpreted as a floating pint number. The colourmap defaults to the built in ‘jet’ theme. The x axis labelis set to be the column header for the first column, the y axis label is set either from the meta data item‘ylabel or to ‘Y Data’. Likewise the z axis label is set from the corresponding metadata item or defaultsto ‘Z Data;. In the second form these parameters are all set explicitly. The xvals parameter can be eithera column index (integer or sring) or a list, tuple or numpy array. The yvals parameter can be either arow number (integer) or list,tuple or numpy array. Other parameters (including plotter, figure etc) workas for the Data.plot_xyz() method. The rectang parameter is used to select only part of the dataarray to use as the matrix. It may be 2-tuple in which case it specifies just the origin as (row,column) ora 4-tuple in which case the third and forth elements are the number of rows and columns to include. Ifxvals or yvals specify particular column or rows then the origin of the matrix is moved to be one columnfurther over and one row further down (ie the matrix is to the right and below the columns and rows used



to generate the x and y data values). The final example illustrates how to generate a new 2D surface plotin a new window using default matrix setup.

Plotting 3D Scalar Fields

A 3D scalar field is a some 𝐹 (𝑥, 𝑦, 𝑧) where the function returns a scalar value. One option is toplot this as a volumetric plot where a colour is interpolated over a volume. For this purpose, Data.voxel_plot() is provided. The function will accept x,y and z column indices for the co-ordinates(or take them from the Data.setas attribute) and a fourth, u column index (or the column definedas holding the u data in the Data.setas attribute). Keyword arguments visible or $filled$ indicatewhether a particular voxel will be plotted or not to allow a cross section of the 3D co-ordinate space tobe show.

"""3D surface plot example."""import numpy as npimport matplotlib.cm


x, y, z = np.meshgrid(np.linspace(-2, 2, 21), np.linspace(-2, 2, 21), np.linspace(-2, 2, 21)

)x = x.ravel()y = y.ravel()z = z.ravel()u = np.sin(x * y * z)

p = Data(x, y, z, u, setas="xyzu", column_headers=["X", "Y", "Z"])

p.plot_voxels(cmap=matplotlib.cm.jet, visible=lambda x, y, z: x - y + z <→˓2.0)p.title = "Voxel plot"

Plotting Vector Fields

For these purposes a vector field is a set of points $(x,y,z)$ at which a 3D vector $(u,v,w)$ is defined.Often vector fields are visualised by using quiver plots, where a small arrow points in the direction ofthe vector and whose length is proportional to the magnitude. This can also be suplemented by colourinformation - in one scheme a hue-saturation-luminence space is used, where hue described a direction inthe x-y plane, the luminence describes the vertical component and the saturation, the relative magnitude.This is a common scheme in micro-magnetics, so is supported in the Stoner package. Following thenaming convention above, the Data.plot_xyzuvw() method handles these plots.:

p.plot_xyzuvw(xcol,ycol,zcol,ucol,vcol,wcol)p.plot_xyzuvw(xcol,ycol,zcol,ucol,vcol,wcol,colors="color_data")p.plot_xyzuvw(xcol,ycol,zcol,ucol,vcol,wcol,colors=np.random(len(p))p.plot_xyzuvw(xcol,ycol,zcol,ucol,vcol,wcol,mode='arrow')p.plot_xyzuvw(xcol,ycol,zcol,ucol,vcol,wcol,mode='arrow',colors=True,scale_→˓factor=1.0)p.plot_xyzuvw(xcol,ycol,zcol,ucol,vcol,wcol,plotter=myplotfunc)

The Data.plot_xyzuvw() method uses a default vector field plot function that is based on mayavifrom Enthought. The import is done when the plot is required to speed loading times for the Stoner.



X

21

01

2

Y

21

01

2

Z

2

1

0

1

2

Voxel plot



plot when 2D plotting only is required. If the mayavi package is not available, then matplotlib’s 3Dquiver plot is used as a fall back.

The first example above will result in a plot using flat arroiws coloured according to the vector mag-nitude. The second examnple will instead color them using the specified column from the data. Thethird example demonstrates passing in a separate list of colour data. In both of these cases the relativemagnitude of the colors data is mapped to a colour map (which can be given via a colormap keywordparameter).

The next example demonstrates changing the glyph used for each data point - in this case to a 3D arrow,but “cone” is also a common choice. In the fifth example, the colors keyword is set to True. This signalsthe plotting function to colour the individual pints with the H-S-L colour space as described above. Theother keyword parameter, scale_factor is passed through (as indeed are any other keyword parameters) tothe underlying mayavi.mlab functions. The final example demonstrates the use of the plotter keyword,analogous to the 2D and 3D examples to switch the actual plotting function.

As the mayavi.mlab quiver3d plotting function doesn’t support a title, and axes labels by default, theseare not used by default in this function.

As usual, the default operation should still produce reasonable graphs.

"""Create a 2D vector field plot."""from os import path

from Stoner import Data, __home__

d = Data(path.join(__home__, "..", "sample-data", "OVF1.ovf"))e = (

d.select(Z__between=(10, 11)).select(X__between=(10, 18)).select(Y__between=(5, 13))

)e.figure(figsize=(8, 4))

# 2D vectors on a 2D Fielde.setas = "xy.uv."e.subplot(121)e.plot()e.title = "3D Vector, 2D Field"

# 3D Vector on a 2D Fielde.subplot(122)e.setas = "xy.uvw"e.plot()e.title = "3D Vector, 3D Field"

e.tight_layout()

Very Quick Plotting

For convenience, a Data.plot() method is defined that will try to work out the necessary details. Incombination with the Data.setas attribute it allows very quick plots to be constructed.

"""Simple plot in 2 lines."""from Stoner import Data

(continues on next page)



1.1 1.2 1.3 1.4 1.5 1.6 1.7X (nm) 1e1

0.6

0.7

0.8

0.9

1.0

1.1

1.2Y

(nm

)1e1 3D Vector, 2D Field

1.1 1.2 1.3 1.4 1.5 1.6 1.7X (nm) 1e1

0.6

0.7

0.8

0.9

1.0

1.1

1.2

Y (n

m)

1e1 3D Vector, 3D Field

(continued from previous page)

p = Data("sample.txt", setas="xy")# Quick plotp.plot()

Getting More Control on the Figure

It is useful to be able to get access to the matplotlib figure that is used for each Data instance. TheData.fig attribute can do this, thus allowing plots from multiple Data instances to be combined in asingle figure.:

p1.plot_xy(0,1,'r-')p2.plot_xy(0,1,'bo',figure=p1.fig)

Likewise the Data.axes attribute returns the current axes object of the current figure in use by theData instance. Data.axes will always return a list of matplotlib.axes.Axes instances, onefor each sub-plot on the figure.

There’s a couple of extra methods that just pass through to the pyplot equivalents:

p.draw()p.show()

Setting Axes Labels, Plot Titles and Legends

Data provides some useful attributes for setting specific aspects of the figures. Any get_ and set_method of matplotlib.pyplot.Axes or matplotlib.pyplot.Figure can be read or writ-ten as a Data attribute.

Internally this is implemented by calling the corresponding method on the current axes or fgiure ofthe Data instance. When setting the attribute, lists and tuples will be assumed to contain positional


https://matplotlib.org/api/axes_api.html#matplotlib.axes.Axes


0.0 50.0 100.0 150.0 200.0 250.0 300.0Temperature

5.0

6.0

7.0

8.0

9.0

10.0

11.0

Resis

tanc

e

sample.txt



arguments and dictionaries keyword arguments. If you want to pass a single tuple, list or dictionary,then you should wrap it in a single element tuple.

Particualrly useful attributes include:

• Data.xlabel, Data.ylabel will set the x and y axes labels

• Data.title will set the plot title

• Data.xlim, Data.ylim will accept a tuple to set the x- and y-axes limits.

• Data.labels sets a list of strings that are the preferred name for each column. This is toallow the plotting routines to provide a default name for an axis label that can be differentfrom the name of the column used for indexing purposes. If the label for a column isnot set, then the column header is used instead. If a column header is changed, then theData.labels attribute will be overwritten.

In addition, you can read any function of matplotlib.pyplot, or method of matplotlib.pyplot.Axes, or matplotlib.pyplot.Figure as an attribute of Data. This allows one tocall the methods directly on the Data instance without needing to extract a reference to the current axesor figure.:

p.xlabel="My X Axis"p.set_xlabel("My X Axis")

are both equivalent, but the latter form allows access to the full keyword arguments of the x axis labelcontrol.

Plotting on Second Y (or X) Axes

To plot a second curve using the same x axis, but a different y axis scale, the Data.y2() method isprovided. This produces a second axes object with a common x-axis, but independent y-axis on the rightof the plot.

"""Plot data using two y-axes."""from Stoner import Data

p = Data("sample.txt", setas="xyy")

p.plot_xy(0, 1, "k-")p.y2()p.plot_xy(0, 2, "r-")

There is an equivalent Data.x2() method to create a second set of axes with a common y scale butdifferent x scales.

To work out which set of axes you are current working with the Data.ax attribute can be read or set.:

p.plot_xy(0,1)p.y2()p.plot_xy(0,2)p.ax=0 # Set back to first x,y1 plotp.label="Plot 1"p.ax=1 # Set to the secondp.yabel="Plot 2"


https://matplotlib.org/api/_as_gen/matplotlib.pyplot.html#module-matplotlib.pyplot


0.0 50.0 100.0 150.0 200.0 250.0 300.0Temperature

5.0

6.0

7.0

8.0

9.0

10.0

11.0

Resis

tanc

e

sample.txt

0.0

250.0

500.0

750.0

1000.0

1250.0

1500.0

1750.0

Colu

mn

2

sample.txt



Plot Templates

Frequently one wishes to create many plots that have a similar set of formatting options - for examplein a thesis or to conform to a journal’s specifications. The Stoner.plot.formats in conjunctionswith the Data.template attribute can help here.

A Data.template template is a set of instructions that controls the default settings for many aspectsof a matplotlib figure’s style. In addition the template allows for pyplot formatting commands to beexecuted during the process of plotting a figure.

"""Customising a template for plotting."""from Stoner import Datafrom Stoner.plot.formats import SketchPlot

p = Data("sample.txt", setas="xy", template=SketchPlot)p.plot()

The template can either be set by passing a subclass of Stoner.plot.formats.DefaultPlotStyle or a particular instance of such a subclass. This latter option allows youto override the default plot attribute settings defined by the template class with your own choice - orindeed, to add further style attributes. This can be done by either calling the template and passingit arguments as keywords, or by settiong attributes directly on the template. Because the matplotlibrcParams dictionary has keys that are not valid Python identifiers, periods in the rcParams keys aretranslated as underscores and thus underscores become double underscores. When setting attributes onthe template directly, prefix the translated rcParam key with template_.



"""Simple ploting with a template."""from cycler import cycler

from Stoner import Datafrom Stoner.plot.formats import DefaultPlotStyle

p = Data("sample.txt", setas="xy", template=DefaultPlotStyle())p.template(

axes__prop_cycle=cycler("color", ["r", "g", "b"]))p.plot()p.y += 1.0p.plot()p.y += 1.0p.plot()

0.0 50.0 100.0 150.0 200.0 250.0 300.0Temperature

5.0

6.0

7.0

8.0

9.0

10.0

11.0

12.0

13.0

Resis

tanc

e

sample.txtResistanceResistanceResistance

You can also copy the style template from one plot to the next.

q=Data() q.template=p.template

Matplotlib makes use of stylesheets - essentially separate files of matplotlibrc entries. The templateclasses have a stylename parameter that controls which stylesheet is used. This can be either one ofthe built in styles (see matplotlib.pyplot.styles.available) or refer to a file on disc. Thetemplate will search for the file (named .mplstyle) in.

1. the same directory as the template class file is,

2. a subdirectory called stylelib of the directory where the template class file is or,

3. the stylelib subfolder of the Stoner package directory.

Where a template class is a subclass of the DefaultPlotStyle, the stylesheets are inherited in thesame order as the class heirarchy.



Changing the value of Lpy:attr:DefaultPlotStyle.stylename will force the tempalte to recalculate thestylesheet heirarchy, refidning the paths to the stylesheets. You can also do any in-place modificationsto the template stylesheet list (e.g. appending or extending it).

Further customisation is possible by creating a subclass of DefaultPlotStyle and over-riding the DefaultPlotStyle.customise() method and DefaultPlotStyle.customise_axes() method.

One-off changes in the template can also be done. The DefaultPlotStyle behaves like a dic-tionary - the keys of the dictionary correspond to settings of the matplotlib.RcParams. Read-ing the keys of the DefaultPlotStyle returns the current settings, setting them changes theunderlying matplotlib.RcParams and deleting them resets the settings back to defaults. TheDefaultPlotStyle otherwise behaves like a standard mutable-mapping object, supporting the ex-pected methods such as get, pop, keys, values etc. It is also possible to access the settings as attributesof the template except that they are prefixed with template_ and periods ‘.’ are replced with doubleunderscores ‘__’.

"""Example customising a plot using default style."""import os.path as pathfrom cycler import cycler

from Stoner import Data, __home__from Stoner.plot.formats import DefaultPlotStyle

filename = path.realpath(path.join(__home__, "..", "doc", "samples", "sample.txt")

)

d = Data(filename, setas="xyy", template=DefaultPlotStyle)d.normalise(2, scale=(0.0, 10.0)) # Just to keep the plot sane!

# Change the color and line style cyclesd.template["axes.prop_cycle"] = cycler(color=["Blue", "Purple"]) + cycler(

linestyle=["-", "--"])

# Can also access as an attributed.template.template_lines__linewidth = 2.0

# Set the default figure sized.template["figure.figsize"] = (6, 8)d.template["figure.autolayout"] = True# Make figure (before using subplot method) and select first subplotd.figure()d.subplot(211)# Pkot with our customised defaultsd.plot()d.grid(True, color="green", linestyle="-.")d.title = "Customised Plot settings"# Reset the template to defaults and switch to next subplotd.template.clear()d.subplot(212)# Plot with defaultsd.plot()d.title = "Style Default settings"# Fixup layoutd.figwidth = 7 # Magic pass through attribute access


https://matplotlib.org/api/matplotlib_configuration_api.html#matplotlib.RcParamshttps://matplotlib.org/api/matplotlib_configuration_api.html#matplotlib.RcParams


0.0 50.0 100.0 150.0 200.0 250.0 300.0Temperature

0.0

2.0

4.0

6.0

8.0

10.0

Resis

tanc

e,Co

lum

n 2(

norm

)

Customised Plot settingsResistanceColumn 2(norm)

0.0 50.0 100.0 150.0 200.0 250.0 300.0Temperature

0.0

2.0

4.0

6.0

8.0

10.0

Resis

tanc

e,Co

lum

n 2(

norm

)

Style Default settingsResistanceColumn 2(norm)



The seaborn package offers many options for producing visually appealing plots with a higher level ab-straction of the matplotlib api/ The SeabornPlotStyle template class offers a quick interface to us-ing this. It takes attributes SeabornPlotStyle.stylename, SeabornPlotStyle.contextand SeabornPlotStyle.palette to set the corresponding seaborn settings.

Making Multi-plot Figures

Adding an Inset to a Figure

The Data.inset()method can be used to creagte an inset in the current figure. Subsequent plots willbe to the inset until a new set of axes are chosen. The pyplot.Axes instance of the inset is appendedto the Data.axes attribute.

"""Add an inset to a plot."""from Stoner import Data

p = Data("sample.txt", setas="xy")p.plot()p.inset(loc=1, width="50%", height="50%")p.setas = "x.y"p.plot()p.title = "" # Turn off the inset title

0.0 50.0 100.0 150.0 200.0 250.0 300.0Temperature

5.0

6.0

7.0

8.0

9.0

10.0

11.0

Resis

tanc

e

sample.txt

0.0 100.0

Date post:	05-Sep-2020
Category:	Documents
Upload:	others
View:	11 times
Download:	0 times

Stoner Package Documentation...Version 0.9 is tested with Python 2.7, 3.5, 3.6 using the standard...

Documents