magdataframe

An extension of the Pandas DataFrame class for Full Magnetic Tensor Gradiometry surveys, and associated data handling and visualization tools.

Overview

magdataframe (magdf) is a package centered around the MagDataFrame (MDF), an extension of the Pandas DataFrame class that adds methods and functionality facilitating the treatment and visualization of Full Magnetic Tensor Gradiometry (FMTG) data.

At its core, the MDF is a Pandas DataFrame with additional structure and methods pertaining to FMTG data. The columns of the DataFrame are divided in categories:

  • Positions (pos) : [x, y, z]
  • Time (time) : [t]
  • Magnetic fields (fields) : [Bx, By, Bz]
  • Magnetic field gradients (grads) : [Bxx, Bxy, Bxz, Byx, Byy, Byz, Bzx, Bzy, Bzz]
  • Magnetic field uncertainties (sfields) : [sBx, sBy, sBz]
  • Magnetic field gradient uncertainties (sgrads) : [sBxx, sBxy, ... , sBzz]
  • Extra (extra) : everything else

This classification creates a basis used by the operations and methods developed in the package to enable easy handling of FMTG data. Plotting an unstructured 2D map of FMTG data, which includes the magnetic fields (Bx, By, Bz) and their gradient (Bxx, Bxy, Bxz, Byx, ..., Bzz) is now as simple as:

MDF.slice_to_plot()

Which, without any arguments, will produce an output in the following format:

Basic slice_to_plot() output

All such functions are adapted to work on any type of dataset including any number of FMTG components.

The package also includes:

  • Operations to go from bare magnetic field data on individual sensors to FMTG data (vector field rotation, MDF addition, etc.)
  • Fast Fourier Transform and filtering functions;
  • Import and export of FMTG data to standard formats (CSV, JSON).

Installation

The package is hosted on PyPi and can be installed through pip:

pip install magdataframe

Dependencies

magdataframe has the following dependencies, all of which are installed through the pip command:

  • pandas (as MDF are based on pandas.DataFrame)
  • numpy (most non-trivial mathematical treatment)
  • scipy (data interpolation)
  • matplotlib (all plotting)
  • nfft (FFTs of unstructured data)
  • json (JSON format i/o)
  • datetime (handling of timestamped data)
  • haversine (conversion from GPS coordinates to relative position)

Module Descriptions

The package is divided in the following modules:

  • base_classes : Defines the MDF class with all of its methods.
  • file_handling (io) : Data import and export methods, including:
    • CSV import/export;
    • magsurvey_json import/export;
    • Conversions from GPS coordinates and timestamps to relative position and time.
  • operations (ops) : Basic math operations on MDFs, including:
    • Calculation of addition of MDFs and of gradiometry components;
    • Data interpolation;
    • Data cropping;
    • Fast Fourier Transforms (FFT);
    • Basic filtering.
  • plotting (plt) : Data visualization functions, including:
    • Line plotting of MDF fields and gradients;
    • 2D plotting of MDF fields and gradients;
    • Plotting of the FFT of MDF fields and gradients.

Issues and Feature Requests

Issues should be directed to the package's Github repository. The Issue and Feature Requests project board gives an overview of the features to come as well as those included in the last release.