Welcome to Mesmerize’s documentation!

biorxiv: https://doi.org/10.1101/840488

GitHub: https://github.com/kushalkolar/MESmerize

Questions/Discussion: Gitter room

Contact: kushalkolar@alumni.ubc.ca

Overview

Mesmerize is a platform for the annotation and analysis of neuronal calcium imaging data. It encompasses the entire process of calcium imaging analysis from raw data to semi-final publication figures that are interactive, and aids in the creation of FAIR-functionally linked datasets. It is applicable for a broad range of experiments and is intended to be used by users with and without a programming background.

Welcome Window

Share your analysis pipelines and figures along with your publication

The Viewer

Explore image sequences, and use various modules for pre-processing and signal-extraction. Annotate regions of interest with any relevant information. Map stimuli/behavior periods.

CaImAn modules

Mesmerize contains front-end GUI modules for the CaImAn library. This makes it very easy for users without a programming background to use the library.

CaImAn Elastic Motion Correction

CNMFE

Batch Manager

Computationally intense procedures performed can be organized with the Mesmerize Batch Manager.

Project Organization

Explore project data and create experimental groups.

Data analysis - pyqtgraph programmable flowcharts.

Build your own analysis piplines using flowcharts.

Interactive Plots

Create shareable interactive plots where the analysis history of every datapoint is traceable. Explore information associated with a datapoint, such as the spatial localization of its parent ROI and raw data.

Interactive Heatmaps

Interactive Cross-correlation analysis

Other types of plots: Beeswarm, Violins, KShape, Proportions, Scatter

Installation

We currently support Linux & Mac OS X. On Windows it is tricky to set up an environment that works reliably with compilers for building some dependencies. From our experience it’s much easier to just install Linux and use the Snap.

Linux

The easiest way to get Mesmerize is through the Snap Store. You can also install from the GitHub repo.

Snap

Command line snap installation:

sudo snap install mesmerize

After installation simply run mesmerize in the terminal and the application will launch in ~10-30 seconds. Make sure your PYTHONPATH environment variable is empty otherwise it might conflict with the snap:

export PYTHONPATH=

You can also open an ipython console in the snap environment:

mesmerize.ipython

Note

You might get the following warnings when you launch the snap, this is normal. Just be patient and wait a few minutes:

warnings.warn(‘%s%s’ % (e, warn))

/snap/mesmerize/x1/lib/python3.6/site-packages/matplotlib/font_manager.py:281: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.

‘Matplotlib is building the font cache using fc-list. ‘

Bokeh could not be loaded. Either it is not installed or you are not running within a notebook

Loading, please wait…

Qt: Session management error: Authentication Rejected, reason : None of the authentication protocols specified are supported and host-based authentication failed

Requirements

Make sure you have snapd installed, which is required for running snap apps. Ubuntu 16.04 and later usually come pre-installed with snapd.

You should be able to install snapd through apt for most Debian based distros:

sudo apt update
sudo apt install snapd

Installing snapd on Fedora:

sudo dnf install snapd

To install snapd on other distros please see: https://docs.snapcraft.io/installing-snapd

If you have trouble installing Mesmerize via snap you might need to install core18 first:

sudo snap install core18

Limitations

The snap installation has several limitations, most importantly you will not be able to access arbitrary filesystems. If you need this you will have to install directly from the repo (see From GitHub). If you are able to mount your external filesystem in /media (or wherever your distro places removeable media) then you should be able to access these filesystems if you do the following:

sudo snap connect mesmerize:removable-media

Alternatively you can install the snap in devmode (gives that snap broad access to the system):

sudo snap install mesmerize --devmode

Warning

Analysis graphs do not work in the snap version at the moment.

From GitHub

First, make sure you have compilers & python.

For Debian & Ubuntu based distros

Get build tools and other things:

sudo apt-get install build-essential

For other distros look for the equivalent meta-package that contains gcc, glibc, etc.

If you don’t have python3.6:

sudo apt-get install python3.6

For other distros lookup how to install python3.6 through their package manager.

Install dependencies:

sudo apt-get install qt5-default tcl graphviz git

For other distros these packages probably have the same or similar names.

Create a virtual environment & install Mesmerize

1. Create a virtual environment:

   # Choose a path to house the virtual environment
   python3.6 -m venv /path/to/venv
   

2. Activate the virtual environment:

   source /path/to/venv/bin/activate
   

3. Clone the repo:

   git clone https://github.com/kushalkolar/MESmerize.git
   

4. cd & switch to the snap branch:

   cd MESmerize
   git checkout snap
   

5. Install some build dependencies:

   pip install Cython numpy python-dateutil
   

6. Install remaining dependencies:

   pip install -r requirements.txt
   

7. Build some things:

   python setup.py build_ext -i
   

8. Add to PYTHONPATH environment variable. You will always need to add the path to MESmerize to the PYTHONPATH environment varible before launching.:

   export PYTHONPATH=$PWD:$PYTHONPATH
   

9. Launch:

   python ./mesmerize
   

Mac OSX

This requires Anaconda and will install Mesmerize in an Anaconda environment. Tested on macOS Catalina 10.15.1

Download Anaconda for Python 3: https://www.anaconda.com/distribution/

First make sure you have xcode:

xcode-select --install

This might take a while.

Create an environment & install Mesmerize

1. Create a new environment using python 3.6:

   conda create --name mesmerize python=3.6
   

2. Enter the environment:

   source activate mesmerize
   

3. Install cython, numpy and pandas:

   conda install cython numpy pandas
   

4. Clone the mesmerize repo and enter it:

   git clone https://github.com/kushalkolar/MESmerize.git
   cd MESmerize
   

5. Checkout the snap branch:

   git checkout snap
   

6. Install more dependencies:

   pip install -r requirements.txt
   

7. Install Mesmerize:

   CFLAGS='-stdlib=libc++' python setup.py build_ext -i
   

Launching Mesmerize

1. Export the path to the MESmerize repo directory:

   export PYTHONPATH=<path_to_MESmerize_dir>
   

2. Launch. It may take a few minutes the first time:

   python <path_to_MESmerize_dir>/mesmerize
   

You might get a matplotlib error, if so execute the following which appends the default matplotlib backend-option. Note that this will probably affect matplotlib in all your environments:

echo "backend: qt5" >> ~/.matplotlib/matplotlibrc

Customized

Troubleshooting

Qt version

PyPI

FAQs

ROIs

1. Can I delete an ROI?

   • See ROI Manager guide

2. I don’t want to delete ROIs but I want to mark them for exclusion in further analysis, how can I do this?

   • You can do this by creating an ROI type category. See <link here> Add New ROI Type Later which uses this as an example. You can also create this ROI Type category when you create a New Project, not necessarily when you already have a project as the example uses.

3. Can I tag more than one piece of information to each ROI?

   • Yes, add as many ROI Type categories as you want in the Project Configuration.

   See also

   Project Configuration

4. I already have a Mesmerize project with many Samples in it. Can I add a new ROI Type category?

   • Yes, just add it to your Project Configuration

5. Can some samples in my project have ROIs that originate from CNMF(E) and others that are manually drawn?

   • Yes, but be aware that you must separate the CNMF(E) and manual data in downstream analysis if you use flowchart nodes that only work with CNMF(E) data, such as NormRaw and DetrendDFoF <node_DetrendDFoF>.

CNMFE

1. I have ROIs that clearly encompass multiple cells instead of just one

   • Increase min_coor

   • Might help to reduce gSig as well

2. I have too many bad ROIs around random regions that are clearly noise

   • Increase min_pnr

3. Min_PNR image is completely blue and void of any signals

   • Increase gSig

4. Vmin slider is stuck in Inspect Correlation & PNR GUI.

   • Close and reopen it. This is a matplotlib issue, not something I can fix.

Caiman Motion Correction

1. I have video tearing

   • Try increasing upsample grid

   • It’s possible that the movement is too severe to be motion corrected. When the movement is so severe that the information do not exist, it is impossible to motion correct it.

2. My animal is growing

   • This is growth, not motion. Unfortunately cannot be corrected for. If you have an idea for a technique I can try it out.

3. The output actually has more motion, it has created false motion.

   • Try these things:

     • Reduce Strides & Overlaps by ~25%

     • Reduce max shifts X & Y by ~25%

     • Reduce max deviation from rigid by ~25%

Project Organization

1. Can I modify a sample?

   • Yes. Double click the Sample ID in the Project Browser to open it in a viewer. You can then make any modifications you want and then go to File -> Add to Project and select the “Save Changes (overwrite)” option at the bottom. If you have not changed the image sequence itself you can uncheck “Overwrite image data”.

2. Can I change the SampleID?

   • No this is fundamentally impossible.

   • A work-around is to open that Sample in the viewer (double click it in the project browser), make any modifications if necessary, then go to File -> Add to Project, enter the the information for this sample and a new Animal ID (and Trial ID if wanted), and then select the option “Add to Project Dataframe” at the bottom and click Proceed. This will now add a new Sample to the project with this Sample ID. You can then delete the previous Sample.

3. Can I add a new Custom Column, ROI Column, or Stimulus Column to my project when I already have samples in my project?

   • Yes, just modify your Project Configuration. In the Welcome Window go to Configure -> Project Configuration. Add anything that you want, and then click “Save and Apply”. It’s best to immediately restart Mesmerize whenever you change your project configuration.

   • If you are adding a new Custom Column you can enter a “Dataframe replace value”. This will allow you to set a value for all existing Samples in your project for this new column.

   • If you do not set a Dataframe replace value it will label all existing as “untagged”

Citation guide

Mesmerize provides interfaces to many great tools that were created by other developers. Please cite the papers for the following Viewer Modules and analysis methods that you use in addition to citing Mesmerize. I would also suggest citing numpy, pandas, scipy, sklearn, and matplotlib.

Mesmerize relies heavily on pyqtgraph widgets. Citing pyqtgraph.

Viewer

Module	Cite
CNMF	Giovannucci A., Friedrich J., Gunn P., Kalfon J., Brown, B., Koay S.A., Taxidis J., Najafi F., Gauthier J.L., Zhou P., Baljit, K.S., Tank D.W., Chklovskii D.B., Pnevmatikakis E.A. (2019). CaImAn: An open source tool for scalable Calcium Imaging data Analysis. eLife 8, e38173. https://elifesciences.org/articles/38173 Pnevmatikakis, E.A., Soudry, D., Gao, Y., Machado, T., Merel, J., … & Paninski, L. (2016). Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron 89(2):285-299. http://dx.doi.org/10.1016/j.neuron.2015.11.037 Pnevmatikakis, E.A., Gao, Y., Soudry, D., Pfau, D., Lacefield, C., … & Paninski, L. (2014). A structured matrix factorization framework for large scale calcium imaging data analysis. arXiv preprint arXiv:1409.2903. http://arxiv.org/abs/1409.2903
CNMFE	In addition to the above CNMF papers: Zhou, P., Resendez, S. L., Rodriguez-Romaguera, J., Jimenez, J. C., Neufeld, S. Q., Giovannucci, A., … Paninski, L. (2018). Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. ELife, 7. doi: https://doi.org/10.7554/eLife.28728.001
Caiman Motion Correction	Giovannucci A., Friedrich J., Gunn P., Kalfon J., Brown, B., Koay S.A., Taxidis J., Najafi F., Gauthier J.L., Zhou P., Baljit, K.S., Tank D.W., Chklovskii D.B., Pnevmatikakis E.A. (2019). CaImAn: An open source tool for scalable Calcium Imaging data Analysis. eLife 8, e38173. https://elifesciences.org/articles/38173 Pnevmatikakis, E.A., and Giovannucci A. (2017). NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data. Journal of Neuroscience Methods, 291:83-92. https://doi.org/10.1016/j.jneumeth.2017.07.031

Nodes/Analysis

Node/Method	Cite
k-Shape clustering	Paparrizos, J., & Gravano, L. (2016). k-Shape. ACM SIGMOD Record, 45(1), 69–76. doi: http://dx.doi.org/10.1145/2723372.2737793 Romain Tavenard, Johann Faouzi, Gilles Vandewiele and Felix Divo, Guillaume Androz, Chester Holtz, Marie Payne, Roman Yurchak, Marc Ruβwurm, Kushal Kolar, & Eli Woods. (2017). Tslearn, A Machine Learning Toolkit for Time Series Data. Journal of Machine Learning Research, (118):1−6, 2020. http://jmlr.org/papers/v21/20-091.html
Cross-correlation	Romain Tavenard, Johann Faouzi, Gilles Vandewiele and Felix Divo, Guillaume Androz, Chester Holtz, Marie Payne, Roman Yurchak, Marc Ruβwurm, Kushal Kolar, & Eli Woods. (2017). Tslearn, A Machine Learning Toolkit for Time Series Data. Journal of Machine Learning Research, (118):1−6, 2020. http://jmlr.org/papers/v21/20-091.html
TVDiff Node	Rick Chartrand, “Numerical Differentiation of Noisy, Nonsmooth Data,” ISRN Applied Mathematics, vol. 2011, Article ID 164564, 11 pages, 2011. https://doi.org/10.5402/2011/164564.

Scientific Libraries

Library
numpy	Van Der Walt, S., Colbert, S. C. & Varoquaux, G. The NumPy array: A structure for efficient numerical computation. Comput. Sci. Eng. (2011) doi:10.1109/MCSE.2011.37
pandas	McKinney, W. Data Structures for Statistical Computing in Python. Proc. 9th Python Sci. Conf. (2010)
scipy	Virtanen, P., Gommers, R., Oliphant, T.E. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat Methods (2020). https://doi.org/10.1038/s41592-019-0686-2
sklearn	Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. (2011)
matplotlib	Hunter, J. D. Matplotlib: A 2D graphics environment. Comput. Sci. Eng. (2007)
pyqtgraph	http://www.pyqtgraph.org/

Create a New Project

Biological Questions

Before you create a new Mesmerize Project you must thoroughly think about the biological questions that you are interested in. Here are some thoughts to help you:

• The effects of different types of temporary stimulation? Such as poking or odors?

• Are you interested in neuronal activity during specific behavioral periods?

• Differences in calcium dynamics between different anatomical regions?

• Chemogenetic experiments using transgenes to express DDREADs.

• Non-temporary effects of drugs (for example, if the animal is bathed in drug for longer recordings).

  • For example, if you are inducing seizures with PTZ, where you are interested in the differences between a control recording of 5 minutes and subsequent 5 minute recordings where the animal is bathed in PTZ (or whatever duration you determine is biologically meaningful). You could also be interested in a recovery stage to see what happens to the calcium dynamics when you “perfuse-back” the liquid medium (such as seawater, steinberg’s solution etc.) without the drug.

• Differences in calcium dynamics between different stages during development

• Differences in calcium dynamics between different cell types using GCaMP driven by specific promoters.

---

New Project

To create a new project click New Project in the Welcome Window. You will then be prompted to choose a location and a name for the project. This will create a directory with the chosen name in the location you previously selected.

Project Configuration

After setting a project name you must configure it. This is where your biological questions of interest are important. You can change your project configuration later, but it is most time efficient if you enter all your categories of interest now.

Warning

Restart Mesmerize whenever you change the project configuration.

Note

If you have Samples in your project and you change the project configuration at a later date to add new columns, all existing rows in your project DataFrame are labelled as “untagged” for the new columns.

See also

Add To Project Guide to understand how the project configuration relates to the addition of data samples to your project

Categorical Data Columns

Mesmerize allows you to create three main different types of categorical data columns (for the project DataFrame), and an unlimited number of each type. These categorical data columns allow you to group your data during analysis, and therefore perform comparisons between experimental groups. In essense, these categorical data columns form scaffold with which you can create your experimental groups during analysis.

Note

You can change the project configuration at any point in the future by adding new columns or changing the visible/hidden columns.

ROI Type Columns

Create ROI-bound categories with which you want to group your data. Enter the desired name for the category and click Add. Here are some examples:

• If you are interested in calcium dynamics between different anatomical regions, you create a column named anatomical_region.

• You want to define defined notochord cell identities on a anterior-posterior axis, defined as “cell_1”, “cell_2”, … “cell_n”. You can create an ROI Type Column named notochord_cell_id.

See also

ROI Manager to understand how labels can be tagged onto ROIs using these categories that you have defined in the ROI Type Columns.

Stimulus Type Columns

If you’re interested in mapping temporal information to your traces, such as stimuli or behavioral periods, add a “Stimulus Type column” for each type. This is only for temporary stimulation or behavioral periods that do not span the entire length of the video.

See also

<link here> Stimulus Mapping guide, to understand how stimuli can be labelled.

Custom Columns

Here you can create categories to tag any other piece of useful information to each Sample. i.e. to the entire video recording. For example:

• You are studying seizures, you perform a 5 minute recording in the medium, and then subsequent 5 minute recordings in PTZ. You can create a category called “drug_state”. When you add samples to your project you can tag drug states named “control”, “ptz_1”, “ptz_2”, “ptz_recovery_1” etc.

• This is also what you would use for chemogenetics experiments if you are recording for example without CNO for 5 minutes, and then with CNO for another 5 minutes.

Three different data types can be tagged to a category, standard text, whole numbers, and decimal numbers.

Warning

Data types cannot be changed later. If you are familiar with pandas you can manually change it, and the corresponding value in the project config file.

If you want to tag numerical information, such as the animal’s development stage, it can be useful to set the data type to whole numbers. This allows you to sort your data numerically. For example you may want to compare dynamics of all curves between stage 48 and 72.

If you are interested in dynamics between different cell types for which you are using specific GCaMP promoters, you can create a custom column called promoter or cell_type and select standard text as the data type.

When you add samples to your project from the viewer, you will be prompted to enter information that is directly based on the Custom Columns that you create here.

See also

Add to Project guide

---

Visible / Hidden in Project Browser

You can drag and drop items (column names) between these two lists to set which ones are visible in the Project Browser. This is just to avoid clutter.

See also

Project Browser

Project Browser

Browse, edit and sort the project DataFrame

You can open the Project Browser from the Welcome Window after you have opened a project.

The columns that are visible in the Project Browser Window correspond to the Project Configuration. For each column you will see a list which is a set of unique elements from that column in the project DataFrame.

Functions

Open Sample

Double-click on a Sample in the SampleID column to open it in the Viewer.

In the viewer you can make changes and then save it by going to File -> Add to Project. You will see a “Save changes (overwrite)” option which will overwrite the data for this project Sample with the current data in the viewer work environment. If you have not changed the image sequence data you can uncheck the “Overwrite image data” checkbox, useful if your image sequences are large.

Note

You can make any changes that you want to the Sample. This may include things such as changing or adding new tags to ROIs, changing stimulus maps, tagging a new custom column etc.

Warning

You can never change the AnimalID or TrialID (i.e. SampleID) since these are partially used as unique identifiers. A workaround is described in the FAQ for Project Organization.

Filter

You can sort your Project DataFrame into different groups (such as experimental groups) using text and numerical filters. Type a filter into the text entries that are below the list of elements for a column. You can also click on one or many elements in a column to set those elements as the filters.

If you filter out of the root tab, it will always create a new tab with a name of your choice. If you filter out of any other tab it will apply the filter in-place unless you right click on the “Apply” button and choose “Apply in new tab”

Text filters

Partial match

To filter out a group based on partial text matches just enter the text into the filter text entry below the column(s) of interest and click “Apply”

Since this is filtering out of the root tab, you will be prompted to give a name for a new tab that will be created based on the filter you have entered.

The result is a DataFrame containing all rows where the cell_name contains ‘aten’

If you go to View -> Current dataframe you can see the whole dataframe.

To see how the filter translates to pandas commands go to View -> Current tab filter history

Multiple filters

You can combine filters together by using the | seperator. The | acts as an “or” operator.

The result is all rows where mn, palp, or pr are in the cell_name column.

Note

This can be combined with Modifiers

Filter multiple columns

You can filter multiple columns simultaneously if you are not in the root tab. You can create a new tab that is essentially the same as the root by just keeping the filter entries blank and clicking “Apply”.

Filter out all rows where the cell_name column contains ‘pr’ and promoter column contains ‘pc2’ or ‘cng_ch4’.

Right click on the “Apply” button and choose “Apply all” or “Apply all in new tab”

If you view the pandas filter history (View -> Current tab filter history) you can see that the filters for each column are simply applied sequentially.

The dataframe

Modifiers

You can perform other types of matches, such as exact matches, negations, and exact negations. Enter the filter and then right click on the text entry to see available modifiers and choose the desired modifier.

Modifier	Description
$NOT:	Results in the negation of partial matches
$STR:	Treats the filter as a str, same as Partial Match (see above sub-section)
$STR=:	Exact text match
$STR!=:	Negation of exact text match

Numerical filters

By default the filters in all entires are treated as text. If your column contains numerical data you have additional options for modifiers. The first four modifiers are the same as explained above. The rest are self explanatory.

Editor

You can view and edit the Project DataFrame directly in a GUI using the DataFrame editor.

Warning

Make sure you know what you are doing when you directly modify the Project DataFrame. Changes cannot be undone but you can restore a backup from the project’s dataframe directory. For example, do not modify data under the following columns: CurvePath, ImgInfoPath, ImgPath, ROI_State, any uuid column.

See also

Uses the Spyder object editor

Console

If you are familiar with pandas you can interact with the project DataFrame directly. If you are unfamiliar with pandas it’s very easy to learn.

See also

Pandas documentation

Useful Callables

Callable	Purpose
get_dataframe()	returns dataframe of the current project browser tab
get_root_dataframe()	always returns dataframe of the root tab (entire project DataFrame)
set_root_dataframe()	pass a pandas.DataFrame instance to set it as the project DataFrame

Usage

General usage to modify the project DataFrame would be something like this:

# Get a copy the project DataFrame to modify
df = get_root_dataframe().copy()

# Do stuff to df
...

# Set the project DataFrame with the modified one
set_root_dataframe(df)

Example

Let’s say you have been inconsistent in naming “ATENA” ROI Tags in the “cell_name” column. You can rename all occurances of ‘atena’ to ‘ATENA’

# Get a copy of the project DataFrame
>>> df = get_root_dataframe().copy()

# View all occurances of 'atena'
>>> df.cell_name[df.cell_name == 'atena']
2      atena
3      atena
4      atena
5      atena
6      atena
205    atena
Name: cell_name, dtype: object

# Rename all occurances of 'atena' to 'ATENA'
>>> df.cell_name[df.cell_name == 'atena'] = 'ATENA'

# Check that there are more occurances of 'atena'
>>> df.cell_name[df.cell_name == 'atena']
Series([], Name: cell_name, dtype: object)

# Check that we have renamed the 'atena' occurances to 'ATENA'
# Indices 2-6 and 205 were named 'atena'
>>> df.cell_name
0      untagged
1      untagged
2         ATENA
3         ATENA
4         ATENA
5         ATENA
6         ATENA
7         atenp
...
Name: cell_name, Length: 311, dtype: object

# Check index 205
>>> df.cell_name.iloc[205]
'ATENA'

# Finally set the changed DataFrame as the root (project) DataFrame
>>> set_root_dataframe(df)

Viewer overview

Based on the pyqtgraph ImageView widget.

The Viewer allows you to do the following things:

• Examine your calcium movies

• Use modules to perform things like motion correction, CNMF(E), ROI labeling, and stimulus mapping. See their respective guides for details.

• You can also make modifications to an existing Sample in your project by opening it in the Viewer. See Modify Sample and Overwrite guide.

Layout

To access Viewer modules choose the module you want to run from the Modules menu at the top. All modules, except the Batch Manager, are small floating windows which you can dock into the Viewer by dragging them to an edge of the viewer.

Work Environment

Everything in the viewer is stored in a Work Environment object. The main data attributes of the viewer work environment are outlined below.

See also

ViewerWorkEnv API

Attribute	Description
imgdata	ImgData object containing the Image Sequence and meta data from the imaging source
roi_manager	The back-end ROI Manager that is currently in use
sample_id	SampleID, if opened from a project Sample
stim_maps	Stimulus maps, if any are defined
history_trace	History log, currently used for logging caiman motion correction, CNMF and CNMFE history.
UUID	If opened from a project Sample, it refers to the ImgUUID

You can view everything in the current work environment by going to View -> Work Environment Editor. You cannot edit through this GUI at this time.

Menubar

File

Add to Project

Add the current work environment as a Sample to the project.

Open Work Environment

Deprecated

Save Work Environment

Deprecated

Clear Work Environment

Clear the current work environment. Useful for freeing up RAM.

Edit

Deprecated

Image

Reset Scale

Reset the scale of the image ViewBox

Resize

Resize the image sequence using interpolation.

Crop

Crop the image sequence.

Usage

1. When you click this option a square crop region will appear in the top left corner of the image sequence.

2. You can change its shape using the handle in the bottom right corner.

3. To crop to the selection, in the menubar go to Image -> Crop. To cancel cropping right click in the crop region and click “Remove ROI”.

Measure

Measure the distance (in pixels) between two points in the image sequence.

Usage

1. After clicking this option in the menubar, click on a point in the image sequence. You will not see anything yet.

2. Click on a second point in the image sequence, a line will appear connecting the first and second points that you clicked.

3. You can use the handles at the endpoints of the line to change the line.

4. To measure the distance of the line go to Image -> Measure. A window will pop up displaying the change in x, y, and length of the line in pixels.

Change dtype

Not implemented yet. You can change the dtype through the console.

Projections

View Mean, Max, and Standard Deviation projections of the current image sequence in the work environment.

Modules

Default Viewer Modules. These are explained in more details in the Viewer Modules chapters.

Plugins

Custom viewer modules.

View

Work Envionment Editor

Explore the data in your work environment using a GUI.

Note

This is read only, you cannot edit via this GUI.

For example if you want to see your meta data, double click on “imgdata” and then you can see that “imgdata” has two things, the image sequence (i.e. your video) and the meta data.

If you double click on “meta” above you can see your meta data.

You can view your meta data more quickly using the console.

Open the console by going to View -> Console. You can then call get_meta() to print the meta data dict.

Console

View/hide the viewer console

Help

Open docs

Open these docs

Console

You can interact directly with the work environment using the console.

See also

Viewer Core API, Overview on consoles

Namespace

Reference	Description
vi	Instance of ViewerUtils. Use this to interact with the viewer.
all_modules	List all available modules (includes default and any available plugins/custom modules)
ViewerWorkEnv	Use for creating new instances of ViewerWorkEnv
ImgData	Use for creating new instances of ImgData
get_workEnv()	Get the current viewer work environment (instance of ViewerWorkEnv)
get_image()	Get the current image sequence (returns current ViewerWorkEnv.imgdata.seq)
get_meta()	Get the current meta data
get_module(<name>)	Pass the name of a module as a string. Returns that module if it is available.
get_batch_manager()	Get the batch manager.
update_workEnv()	Update the viewer GUI with the viewer work environment (vi.viewer.workEnv)
clear_workEnv()	Clear the current work envionment, cleanup the GUI and free the RAM

Examples

View meta data

>>> get_meta()

{'origin': 'AwesomeImager', 'version': '4107ff58a0c3d4d5d3c15c3d6a69f8798a20e3de', 'fps': 10.0, 'date': '20190426_152034', 'vmin': 323, 'vmax': 1529, 'orig_meta': {'source': 'AwesomeImager', 'version': '4107ff58a0c3d4d5d3c15c3d6a69f8798a20e3de', 'level_min': 323, 'stims': {}, 'time': '152034', 'date': '20190426', 'framerate': 10.0, 'level_max': 1529}}

View history trace

>>> get_workEnv().history_trace

[{'caiman_motion_correction': {'max_shifts_x': 32, 'max_shifts_y': 32, 'iters_rigid': 1, 'name_rigid': 'Does not matter', 'max_dev': 20, 'strides': 196, 'overlaps': 98, 'upsample': 4, 'name_elas': 'a1_t2', 'output_bit_depth': 'Do not convert', 'bord_px': 5}}, {'cnmfe': {'Input': 'Current Work Environment', 'frate': 10.0, 'gSig': 10, 'bord_px': 5, 'min_corr': 0.9600000000000001, 'min_pnr': 10, 'min_SNR': 1, 'r_values_min': 0.7, 'decay_time': 2, 'rf': 80, 'stride': 40, 'gnb': 8, 'nb_patch': 8, 'k': 8, 'name_corr_pnr': 'a8_t1', 'name_cnmfe': 'a1_t2', 'do_corr_pnr': False, 'do_cnmfe': True}}, {'cnmfe': {'Input': 'Current Work Environment', 'frate': 10.0, 'gSig': 10, 'bord_px': 5, 'min_corr': 0.9600000000000001, 'min_pnr': 14, 'min_SNR': 1, 'r_values_min': 0.7, 'decay_time': 4, 'rf': 80, 'stride': 40, 'gnb': 8, 'nb_patch': 8, 'k': 8, 'name_corr_pnr': '', 'name_cnmfe': 'a1_t2', 'do_corr_pnr': False, 'do_cnmfe': True}}]

Open image

# Get the tiff module
tio = get_module('tiff_io')

# Set the tiff and meta file paths
tiff_path = '/path_to_tiff_file.tiff'
meta_path = '/path_to_meta_file.json'

tio.load_tiff(tiff_path, meta_path, method='asarray')

See also

Tiff IO API, Information on the meta data format

Use the ViewerCore API to open any arbitrary image

import tifffile
import json

# Open an image file in whatever way you require
seq = tifffile.imread('/path_to_tiff_file')

meta = ...

# Create ImgData instance
# Image sequence must be numpy array of shape [x, y, t]
imgdata = ImgData(seq.T, meta)

# Create a work environment instance
work_env = ViewerWorkEnv(imgdata)

# Set this new work environment instance as the viewer work environment
vi.viewer.workEnv = work_end

# Update the viewer GUI with the new work envionment
update_workEnv()

Splice img seq

Extract the image sequence between frame 1000 and 2000. Image sequences are simply numpy arrays.

See also

Numpy array indexing

# Get the current image sequence
seq = get_image()

# Trim the image sequence
trim = seq[:, :, 1000:2000]

# Set the viewer work environment image sequence to the trim one
vi.viewer.workEnv.imgdata.seq = trim

# Update the GUI with the new work environment
update_workEnv()

Running scripts

You can use the Script Editor to run scripts in the Viewer console for automating tasks such as batch creation. It basically allows you to use the viewer console more conveniently with a text editor.

Convert Meta Data

In order for you to use your videos with Mesmerize, each of your videos must have meta data organized in the format below.

The simplest way to use your meta data with Mesmerize is to dump all of it into a single json file with the same name as that of the video it is for. You could also add another elif block to mesmerize.viewer.core.ViewerWorkEnv._organize_meta() .

For example if you have a movie named “xyz.tiff” the json meta data file for that movie would be “xyz.json” for the Tiff input module to automatically recognize it.

Minimal Example

{
         "source":    "our_recording_program",
         "fps":       20.0,
         "date":      "20190425_114405"
}

Fields and data types

These fields and their types are mandatory

• source (str): Name of recording program used, for your own record

• fps (float): Framerate of the video

• date (str): Recording date & time separated by an underscore

If you have other data that you would like to keep for the purpose of custom modules or other analysis you can organize them into a single dict under the “orig_meta” key.

Example with more fields:

{
         "source":    "our_recording_program",
         "version":   0.1.2,
         "fps":       20.0,
         "date":      20190425_114405,
         "orig_meta":
                        {
                                "scanner_info":   [1, 2, 3],
                                "other_stuff:     "val"
                        }
}

Add a Sample to the Project

When you are happy with the ROIs in the viewer for the current CNMF(E) derived or manually created ROIs, you can add this as a Sample to your project.

Each sample in your project contains the following:

• The imaging data from which ROIs were extracted (the video)

• All the ROIs with their spatial location, temporal dynamics, and any tags that you have entered in the ROI Manager.

• Stimulus mappings, if your project is configured for this.

• Meta data (that were associated with the imaging video), the date, video framerate.

• Any further information that you have chosen to add based on your Project Configuration

Note

If your ROIs were obtained through CNMF/CNMFE the following attributes from the final cnm object are stored: cnm.A, cnm.b, cnm.C, cnm.f, cnm.YrA

How to

To add the current viewer work environment (see above) as a sample to your project, go to File -> Add To Project. You will be presented with a window similar to this:

The entries that you are prompted with directly correspond to the custom columns in your Project Configuration.

See also

Project Configuration

Every Sample in a project has a unique SampleID which is the combination of AnimalID + TrialID.

Warning

You can never change the AnimalID or TrialID (i.e. SampleID) since these are partially used as unique identifiers. A workaround is described in the FAQ for Project Organization.

Warning

AnimalID and TrialID are separated by the -_- character combination when stored as a SampleID. Therefore do not use that character combination within your AnimalID or TrialID.

Tiff file module

To open a tiff file go to Modules -> Load Images -> Tiff files.

Note

You can also use this module through the console and scripts. See Tiff Module

To open tiff files just click the “Select file” button and choose your file. You can also drag and drop a tiff file.

When you choose a tiff file it will automatically find the associated .json meta data if it has the same filename.

Finally, select an appropriate load method (see next section) and click “Load into workEnv”

Warning

If the name of the tiff file and .json meta data file are different, you must specify the .json meta data file using the Select meta data button.

Warning

You cannot perform any analysis without the meta data file since you need the sampling rate of the video and it is specified in the meta data file.

Load Method

The options for “Load Method” correspond to the tifffile library method that is used for loading the images.

If you are not sure which method you should use, try all of them and see which one loads your data appropriately. If none of them work, contact us and I may be able to add more methods.

Note

If you have an unsaved work environment open (such as a video with ROIs for example) it will prompt you to confirm that you want to clear the work environment before loading the chosen image into the work environment.

asarray

Should work for most tiff files, fast method

asarray - multi series

Also fast. Use this if it is a multi-page tiff. For example if the tiff file was created by a program that appends each frame to a file as they are being acquired by the camera.

imread

Usually slower, should work for most tiff files.

Batch Manager

Batch process computationally intensive tasks.

See also

Batch Manager API

This is currently used for Caiman Motion Correction, CNMF, and CNMFE.

The Batch Manager can be accessed in the viewer through Modules -> Batch Manager. If you don’t have a batch open you will be prompted with a dialog to open a batch or to select a location for a new batch.

Warning

The full path to the batch directory must not contain spaces or special characters, only a-z, A-Z, 0-9 and underscores.

The Batch Manager processes the batch items in external processes, allowing you to add batch items when that batch is being processed.

Layout

Window title: Name of batch directory

Top: Parent directory of batch directory

Top left: list of batch items and some controls.

Colors	Description
Green	Finished without exceptions
Red	Did not finish, click on the item to see the exceptions in the bottom right information area
Yellow	Currently being processed
Orange	Item aborted by user
Blue	Output data for this item are being moved from the work dir to the batch dir.

Button	Description
Start	Process the batch from the first item.
Start at selection	Process the batch starting from the item that is currently selected in the list.
Delete selection	Delete the item that is currently being selected along with the associated data in the batch dir.
Export shell scripts	Export bash scripts so that the batch items can be run on a computing cluster
Abort	Abort the current batch item
New batch	Create a new batch
Open batch	Open a batch
View Input	Open the input work environment, in the viewer, for the currently selected item
Compress	Not implemented

Use work dir: Check this box to use the work dir that has been set in the System Configuration

Top right: Standard out from the external processes that are processing the batch items.

Bottom left: Parameters for the selected batch item. The first line is the UUID of the batch item.

Bottom right: Output information area for the currently selected item.

Scheduling

You can schedule a batch to run at a later time using the following bash script. Doesn’t work for a snap installation yet.

mesmerize-scheduler

Usage:

Usage: mesmerize-scheduler -b <batch> -i <start item> -t <start time>

    -b      full batch path in quotes, no spaces
    -i      uuid of the batch item to start from, no quotes
    -t      time at which to start the batch, no quotes

    examples of how to specify time:
            23:00  7:30Feb30
            use 24hr time and no spaces

Full usage example:
    mesmerize-scheduler -b "/share/data/temp/kushal/pc2_batch" -i a80d1923-e490-4eb3-ba4f-7e651d4cf938 -t 2:00

Stimulus Mapping

API Reference

Map temporal information such as stimulus or behavioral periods.

Stimulus Mapping Module

Stimulus periods illustrated on the viewer timeline

The tabs that are available in the stimulus mapping module corresponds to the stimulus types in your Project Configuration.

You can add stimulus periods either manually or through a script.

Manual Annotation

1. To add a stimulus manually click the “Add Row” button. This will add an empty row to the current tab page.

2. Enter a name for the stimulus, start time, end time, and pick a color for illustrating the stimulus periods on the Viewer timeline.

3. To remove a stimulus click the “Remove stim” button. Stimulus periods do not have to be added in chronological order.

4. Click “Set all maps” to set the mappings for all stimulus types. You can then choose to illustrate a stimulus on the viewer timeline by selecting it from “Show on timeline”

Import and Export are not implemented yet.

Warning

At the moment, only “frames” are properly supported for the time units.

Script

See also

API Reference

You can also use the Stimulus Mapping module’s API to set the stimulus mappings from a pandas DataFrame.

This example creates a pandas DataFrame from a csv file to set the stimulus mappings. It uses the csv file from the pvc-7 dataset availble on CRCNS: http://dx.doi.org/10.6080/K0C8276G

You can also download the csv here: stimulus_pvc7.csv

import pandas as pd
from mesmerize.plotting.utils import get_colormap

# Load dataframe from CSV
df = pd.read_csv('/share/data/longterm/4/kushal/crcns_datasets/pvc-7/122008_140124_windowmix/stimulus.csv')

# Sort according to time (stimulus "start" frame column)
df.sort_values(by='start').reset_index(drop=True, inplace=True)

# Trim off the periods that are not in the current image sequence
# This is just because this example doesn't use the whole experiment
trim = get_image().shape[2]
df = df[df['start'] <= trim]

# Remove the unused columns
df.drop(columns=['sf', 'tf', 'contrast'])

# Rename the stimulus column of interest to "name"
df.rename(columns={'ori': 'name'}, inplace=True)

# Get the names of the stimulus periods to create a colormap for illustration in the curves plot area
oris = df['name'].unique()
oris.sort()

# Create colormap to visualize the stimuli in the viewer's curve plots area
oris_cmap = get_colormap(oris, 'tab10', output='pyqt', alpha=0.6)

# Create a column with colors that correspond to the orientations column values
df['color'] = df['name'].map(oris_cmap)

# name column must be str type for stimulus mapping module
# the ori data from the original csv is integers so you must change it
df['name'] = df['name'].apply(str)

# Get the stimulus mapping module
smm = get_module('stimulus_mapping')

# Set the ori colormap
smm.maps['ori'].set_data(df)

ROI Manager

API Reference

Manage and annotate ROIs

The ROI Manager has a manual mode, to draw ROIs manually, and a CNMF(E) mode where ROIs can be imported from CNMF(E) outputs.

See also

CNMF Usage and CNMFE Usage.

Note

You cannot combine manual and CNMF(E) ROIs in the same sample.

The ImageJ ROI import uses the read-roi package by Hadrien Mary https://pypi.org/project/read-roi/

Layout

Controls

UI	Description
Add ROI button	Add Polygon ROI (Manual mode) Right click this button to add an elliptical ROI
Show all	Show all ROIs in the viewer
Live plot	Live update of the curve plot with changes (Manual mode)
Plot	Plot the curves (Manual mode)
Import from ImageJ	Import ROIs from an ImageJ ROIs zip file (Manual mode)
Switch to manual …	Switch to Manual mode. Clears CNMF(E) ROIs.
ROIs list	Color-coded list of ROIs. Left click to highlight the ROI in the viewer Right click to show the context menu allowing you to delete the selected ROI
Tags list	List of tags for the selected ROI Correspond to the ROI Type Columns of the Project Configuration
Add Tag to ROI Def…	Set the tag for the current selection in the Tags list
Set ROI Tag	Click to set the tag, or just press return in the text entry above

Keyboard shortcuts.

These only work when the ROI manager is docked within the Viewer and while you are typing in the Add Tag to ROI Definition text entry.

Key	Description
Page Up	Select previous ROI
Page Down	Select next ROI
Right Arrow	Play the video at high speed
Left Arrow	Play the video backwards at high speed
Home	Go to the beginning of the video
End	Go to the end of the video

Manual ROI Mode

When you click the “Add ROI” button to add a Manual Polygon ROI, a new rectangular ROI will be add in the top left corner of the image. You can add new vertices to this polygon by clicking on any of its edges. You can drag the vertices to change the shape of the polygon, and you can drag the entire ROI as well by clicking and dragging within the ROI region. Similarly you can reshape elliptical ROIs.

Hovering over the ROI selects it in the ROI list.

Console

Access the back-end ROI Manager through the viewer console or Script editor to interact with the ROIs.

See also

Back-end ROI Manager APIs, ROIList API, ROI Type APIs

Get the back-end ROI Manager, see ROI Manager APIs

>>> get_workEnv().roi_manager

<mesmerize.viewer.modules.roi_manager_modules.managers.ManagerCNMFE object at 0x7f01b8780668>``

Get the ROI List, see ROIList API

>>> get_workEnv().roi_manager.roi_list

[<mesmerize.viewer.modules.roi_manager_modules.roi_types.CNMFROI object at 0x7f01bc78b278>, <mesmerize.viewer.modules.roi_manager_modules.roi_types.CNMFROI object at 0x7f01bc817630>, <mesmerize.viewer.modules.roi_manager_modules.roi_types.CNMFROI object at 0x7f01bc817668>, <mesmerize.viewer.modules.roi_manager_modules.roi_types.CNMFROI object at 0x7f01bc7c5438>, <mesmerize.viewer.modules.roi_manager_modules.roi_types.CNMFROI object at 0x7f01bc7c5208>]

Work with an ROI object, see ROI Type APIs

# Get the curve data of an ROI
>>> get_workEnv().roi_manager.roi_list[3].curve_data

(array([   0,    1,    2, ..., 2995, 2996, 2997]), array([ -207.00168389,  -161.78229208,  -157.62522988, ..., -1017.73174502,
   -1030.27047731, -1042.26989668]))

# Get the tags of an ROI
>>> get_workEnv().roi_manager.roi_list[2].get_all_tags()

{'anatomical_location': 'tail', 'cell_name': 'dcen', 'morphology': 'untagged'}

# Get a single tag
>>> get_workEnv().roi_manager.roi_list[2].get_tag('cell_name')

'dcen'

Caiman Motion Correction

Perform motion correction using the NoRMCorre implementation in the CaImAn library.

I highly recommend going through the following before using this module

• NoRMCorre paper

  Pnevmatikakis, E. A., & Giovannucci, A. (2017). NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data. Journal of Neuroscience Methods, 291, 83–94.

• The CaImAn demo notebook, the implementation in Mesmerize is basically from the demo

  https://github.com/flatironinstitute/CaImAn/blob/master/demos/notebooks/demo_motion_correction.ipynb

Parameters

Output bit depth: The motion corrected image sequences are of float32 type. You can optionally convert the output to 8 or 16 bit uint types to save disk space. This doesn’t always work from my experience, values might get clipped.

For all other parameters please see the demo notebook mentioned above.

Usage

This module adds a “caiman motion correction” item to the batch. Set the desired parameters (see demo notebook) and then enter a name to add it as an item to the batch. After the batch item is processed, double-click the batch item to open the motion corrected image sequence in the viewer. You can then use this motion corrected image sequence for further analysis, such as CNMF or CNMFE.

See also

This modules uses the Batch Manager.

Note

The parameters used for motion correction are stored in the work environment of the viewer and this log is carried over and saved in the Project Sample as well. To see the parameters that were used for motion correction in the viewer, execute get_workEnv().history_trace in the viewer console and look for the caiman_motion_correction entry.

Script Usage

A script can be used to add caiman motion correction batch items. This is much faster than using the GUI.

See also

Script Editor

Add items

This example shows how to add all tiff files (of image sequences) from a directory as batch items with 3 different variants of parameters.

See also

This example uses the Caiman Motion Correction Module API, ViewerWorkEnv API, and Batch Manager API

# Import glob so we can get all tiff files in a dir
from glob import glob
# Import os to get filenames from paths
import os

# Motion correction params.
# Params are identical to the GUI ones

mc_params = {
            "max_shifts_x":         6,
            "max_shifts_y":         6,
            "iters_rigid":          2,
            "name_rigid":           "Does not matter",
            "max_dev":              3,
            "strides":              196,
            "overlaps":             98,
            "upsample":             4,
            "name_elas":            "will set later per file",
            "output_bit_depth":     "Do not convert",
            "gSig_filt"             (10, 10)
            }

# Path to the dir containing images
files = glob("/full_path_to_raw_images/*.tiff")
# Sort in alphabetical order (should also work for numbers)
files.sort()

# Open each file, crop, and add to batch with 3 diff mot cor params
for i, path in enumerate(files):
    print("Working on file " + str(i + 1) + " / " + str(len(files)))

    # get json file path for the meta data
    meta_path = path[:-5] + ".json"

    # Create a new work environment with this image sequence
    work_env = ViewerWorkEnv.from_tiff(path, "asarray-multi", meta_path)

    # Get caiman motion correction module, hide=False to not show GUI
    mc_module = get_module("caiman_motion_correction", hide=True)

    # Set name for this video file
    name = os.path.basename(path)[:-5]
    mc_params["name_elas"] = name

    # Set the input work environment
    mc_module.set_input_workEnv(work_env)

    # First variant of params
    mc_params["strides"] = 196
    mc_params["overlaps"] = 98
    # Add one variant of params for this video to the batch
    mc_module.set_params(mc_params)
    mc_module.add_to_batch()

    # Try another variant of params
    mc_params["strides"] = 256
    mc_params["overlaps"] = 128
    # Set these params and add to batch
    mc_module.set_params(mc_params)
    mc_module.add_to_batch()

    # Try one more variant of params
    mc_params["strides"] = 296
    mc_params["overlaps"] = 148
    # Set these params and add to batch
    mc_module.set_params(mc_params)
    mc_module.add_to_batch()

# If you want to process the batch after adding the items uncomment the following lines
#bm = get_batch_manager()
#bm.process_batch(clear_viewers=True)

Crop and add items

This example shows how to crop videos prior to adding them as batch items. This is useful if you want to crop-out large unchanging regions of your movides. It uses either simple thresholding or spectral salieny on a standard deviation projection to determine the bounding box for cropping.

See also

This example uses the Caiman Motion Correction Module API, ViewerWorkEnv API, and Batch Manager API

# Import glob so we can get all tiff files in a dir
from glob import glob
# Import os to get filenames from paths
import os

# Just get a shortcut reference to the auto_crop function
auto_crop = image_utils.auto_crop

# Parameters for cropping, these should work for everything
# These worked well for various different constructs
# If you get non-specific cropping (too much black) try "method" as "spectral_saliency" (See below)
crop_params = {
                "projection":       "max+std",
                "method":           "threshold",
                "denoise_params":   (32, 32),
                }

# Spectral saliency is another method
# You can try and play around with the parameters
# If the cropping is insufficient, you can set "projection" to just "max" or "std"
# If you get too much junk blackness around the animal try increasing denoise_params
# or reduce padding. Default padding is 30 (when nothing is specified like above)
crop_params_salient = {
                        "projection":    "max+std",
                        "method":   "spectral_saliency",
                        "denoise_params":   (16, 16),
                        "padding":  40
                        }

# Motion correction params.
# Params are identical to the GUI ones
    mc_params = {
            "max_shifts_x":         6,
            "max_shifts_y":         6,
            "iters_rigid":          2,
            "name_rigid":           "Does not matter",
            "max_dev":              3,
            "strides":              196,
            "overlaps":             98,
            "upsample":             4,
            "name_elas":            "will set later per file",
            "output_bit_depth":     "Do not convert",
            "gSig_filt"             (10, 10)
            }

# Path to the dir containing images
files = glob("/full_path_to_raw_images/*.tiff")
# Sort in alphabetical order (should also work for numbers)
files.sort()

# Open each file, crop, and add to batch with 3 diff mot cor params
for i, path in enumerate(files):
    print("Working on file " + str(i + 1) + " / " + str(len(files)))

    # get json file path for the meta data
    meta_path = path[:-5] + ".json"

    # Create a new work environment with this image sequence
    work_env = ViewerWorkEnv.from_tiff(path, "asarray-multi", meta_path)

    print("Cropping file: " + str(i + 1))

    raw_seq = work_env.imgdata.seq
    # Auto crop the image sequence
    cropped = auto_crop.crop(raw_seq, crop_params)
    # Set work env img seq to the cropped one and update
    work_env.imgdata.seq = cropped

    # Get caiman motion correction module, hide=False to not show GUI
    mc_module = get_module("caiman_motion_correction", hide=True)

    # Set name for this video file
    name = os.path.basename(path)[:-5]
    mc_params["name_elas"] = name

    # Set the input work environment
    mc_module.set_input_workEnv(work_env)

    # First variant of params
    mc_params["strides"] = 196
    mc_params["overlaps"] = 98
    # Add one variant of params for this video to the batch
    mc_module.set_params(mc_params)
    mc_module.add_to_batch()

    # Try another variant of params
    mc_params["strides"] = 256
    mc_params["overlaps"] = 128
    # Set these params and add to batch
    mc_module.set_params(mc_params)
    mc_module.add_to_batch()

    # Try one more variant of params
    mc_params["strides"] = 296
    mc_params["overlaps"] = 148
    # Set these params and add to batch
    mc_module.set_params(mc_params)
    mc_module.add_to_batch()

# If you want to process the batch after adding the items uncomment the following lines
#bm = get_batch_manager()
#bm.process_batch(clear_viewers=True)

CNMF

Perform CNMFE using the implementation provided by the CaImAn library. This modules basically provides a GUI for parameter entry.

I highly recommend going through the following before using this module

• CNMFE builds upon CNMF

  Pnevmatikakis, E. A., Gao, Y., Soudry, D., Pfau, D., Lacefield, C., Poskanzer, K., … Paninski, L. (2014). A structured matrix factorization framework for large scale calcium imaging data analysis, 1–16.

  Pnevmatikakis, E. A., Soudry, D., Gao, Y., Machado, T. A., Merel, J., Pfau, D., … Paninski, L. (2016). Simultaneous Denoising, Deconvolution, and Demixing of Calcium Imaging Data. Neuron, 89(2), 285.

• CaImAn demo notebook, the implementation in Mesmerize is basically from the demo. The second half of the notebook describes CNMF

  https://github.com/flatironinstitute/CaImAn/blob/master/demos/notebooks/demo_pipeline.ipynb

Parameters

Please see the CaImAn demo notebook mentioned above to understand the parameters.

Console

A script can be used to add CNMFE batch items. This is much faster than using the GUI. This example sets the work environment from the output of a batch item. See the Caiman Motion Correction script usage examples for how to load images if you want to add CNMF items from images that are not in a batch.

See also

Script Editor, :ref:`CNMF module API

# CNMF Params that we will use for each item
params =   {'Input': 'Current Work Environment',
            'p': 2,
            'gnb': 1,
            'merge_thresh': 0.25,
            'rf': 70,
            'stride_cnmf': 40,
            'k': 16,
            'gSig': 8,
            'min_SNR': 2.5,
            'rval_thr': 0.8,
            'cnn_thr': 0.8,
            'decay_time': 20,
            'name_cnmf': 'set_later_per_file',
            'refit': True
            }

# Get the batch manager
bm = get_batch_manager()
cnmf_mod = get_module('cnmf')

# Start index if we want to start processing the new items after they have been added
start_ix = bm.df.index.size + 1

for ix, r in bm.df.iterrows():
    # Use output of items 6 - 12
    if ix < 6:
        continue
    if ix > 12:
        break

    # Get the name of the mot cor item
    name = r['name']

    # Set the name for the new cnmf item
    params['name_cnmf'] = name

    # Load the mot cor output
    bm.load_item_output(module='caiman_motion_correction', viewers=viewer, UUID=r['uuid'])

    # Set the CNMF params and add to batch
    cnmf_mod.set_params(params)
    cnmf_mod.add_to_batch_cnmf()

# Cleanup the work environment
vi._clear_workEnv()

# Uncomment the last two lines to start the batch as well
#bm.process_batch(start_ix, clear_viewers=True)

CNMFE

Perform CNMFE using the implementation provided by the CaImAn library.

I highly recommend going through the following before using this module

• The paper on CNMF-E

  Zhou, P., Resendez, S. L., Rodriguez-Romaguera, J., Jimenez, J. C., Neufeld, S. Q., Stuber, G. D., … Paninski, L. (2016). Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. ELife, 1–37.

• CNMFE builds upon CNMF

  Pnevmatikakis, E. A., Gao, Y., Soudry, D., Pfau, D., Lacefield, C., Poskanzer, K., … Paninski, L. (2014). A structured matrix factorization framework for large scale calcium imaging data analysis, 1–16.

  Pnevmatikakis, E. A., Soudry, D., Gao, Y., Machado, T. A., Merel, J., Pfau, D., … Paninski, L. (2016). Simultaneous Denoising, Deconvolution, and Demixing of Calcium Imaging Data. Neuron, 89(2), 285.

• CaImAn CNMF-E demo notebook, the implementation in Mesmerize is basically from the demo

  https://github.com/flatironinstitute/CaImAn/blob/master/demos/notebooks/demo_pipeline_cnmfE.ipynb

Parameters

Please see the CaImAn demo notebook mentioned above to understand the parameters.

Ain: Seed spatial components from another CNMFE item by entering its UUID here.

Usage

This module creates two types of batch items, one where you can inspect the Correlation & PNR images and another that performs CNMFE and extracts components. Here is an outline of typical usage:

• Enter a gSig parameter value and a name for “Inspect Correlation and PNR”, the text entry for “Stop here”. Click “Add to batch”. Run the batch item.

• Double-click the batch item, you will be presented with a GUI to help optimize min_corr and min_pnr. For the correlation image use the vmin slider to optimize the seperation of cells and set the min_corr parameter to this value. Likewise, optimize the value for the PNR until the PNR image mostly contains regions that show real signal and no or few regions that are likely to be just noise and set this vmin value as the min_pnr parameter. You may need to try slightly different variations to optimize the parameters.

• Enter the rest of the parameters and give a name under “Perform CNMF-E”, click “Add to batch” and run the item.

• Double-click the batch item and you will be presented with 3 options. The first option will display the correlation-pnr images and the second option is currently non-functional (matplotlib Qt issue). The last option will import the components extracted by CNMFE into an open Viewer. The components are managed by the ROI Manager.

See also

ROI Manager

See also

This modules uses the Batch Manager.

Note

The parameters used for CNMFE are stored in the work environment of the viewer and this log is carried over and saved in Project Samples as well. To see the parameters that were used for CNMFE in the viewer, execute get_workEnv().history_trace in the viewer console and look for the ‘cnmfe’ entry.

Script Usage

A script can be used to add CNMFE batch items. This is much faster than using the GUI.

See also

Script Editor.

Add Corr PNR items

Add Corr PNR batch items from a batch that contains motion corrected items. This example add 2 variants of parameters (just gSig) for each motion corrected item.

See also

This example uses the Caiman CNMFE module API and Batch Manager API

See also

Caiman Motion Correction script usage examples for how to load images if you want to add Corr PNR items from images that are not in a batch.

# Get the batch manager
bm = get_batch_manager()

# Get the CNMFE module
cnmfe_mod = get_module('cnmfe', hide=True)

# Start index to start processing the new items after they have been added
start_ix = bm.df.index.size + 1

for ix, r in bm.df.iterrows():
        if ix == start_ix:
                break

        # Get the name of the mot cor item
        name = r['name']

        # Load the output of the motion corrected batch item
        # The output will load into the viewer that this script
        # is running in.
        bm.load_item_output(module='caiman_motion_correction', viewers=viewer, UUID=r['uuid'])

        # Get the currently set params
        # You just need the dict with all the correct keys
        # You will just modify the "gSig" and "name_corr_pnr" keys
        params = cnmfe_mod.get_params()

        # Set the gSig and name params
        params['gSig'] = 8
        params['name_corr_pnr'] = name

        # Set the params and add to batch
        cnmfe_mod.set_params(params)
        cnmfe_mod.add_to_batch_corr_pnr()

        # Another variant of params
        # Set the gSig and name params
        params['gSig'] = 10
        params['name_corr_pnr'] = name

        # Set the params and add to batch
        cnmfe_mod.set_params(params)
        cnmfe_mod.add_to_batch_corr_pnr()

# Cleanup the work environment
vi._clear_workEnv()

# Start the batch from the start_ix
bm.process_batch(start_ix, clear_viewers=True)

Script Editor

A simple text editor for writing scripts that can be run in the viewer console

The scripts are simply ran in the viewer console and all output will also be visible in the viewer console.

See also

Viewer Core API

Warning

There is no auto-save function

Flowchart Overview

The flowchart allows you to analyze samples in your project and create plots by arranging analysis nodes. Each node takes an input, performs an operation, and produces an output. For example the Derivative node takes use-specified numerical arrays, computes the derivative of these arrays, and then outputs the result.

The Flowchart is based on the pyqtgraph flowchart widgets

Flowchart Window

Add node: Right click -> Add node -> Choose from selection

Click on a node to highlight the Control Widget

Remove node: Right click -> Remove node

Connecting nodes: Click on a node terminal and drag to another terminal

Save the flowchart layout: Click “Save as…” to save the layout to a new file. You must specify the file extension as “.fc”. If you save this file within the “flowcharts” directory of your project it will show up in the Welcome Window when you open your project.

Note

This does not save the data, use the Save node to save data.

Warning

Due to a weird Qt or pyqtgraph bug certain parameter values (such as those in drop-down menus) can’t be saved. Similarly, parameters values are lost when you save to an existing .fc file. If you’re interested take a look at pyqtgraphCore.WidgetGroup. Anyways you shouldn’t be using the flowchart layout to save this information, that’s what the History Trace in Transmission objects is for.

Load an .fc file: Click the “Load” button.

Reset View button: Reset the view, for example if you zoom out or pan too far.

Transmission

API Reference

Almost every node uses a Transmission object for input and output. A Transmission is basically a DataFrame and a History Trace (analysis log) of the data within the DataFrame.

Transmission DataFrame

The Transmission DataFrame is created from your ProjectDataFrame (or sub-DataFrame) by the node_Load_Proj_DF node. This initial DataFrame will contain the same columns as your Project DataFrame, and a new column named _RAW_CURVE. Each element (row) in the _RAW_CURVE column is a 1-D numerical array representing a single raw curve extracted from an ROI.

A new column named _BLOCK_ is also added which contains the UUID for logging the analysis history of this newly created block of DataFrame rows, known as a data block. This allows you to merge Transmissions (see Merge node) and maintain their independent analysis logs prior to the merge.

Naming conventions for DataFrame columns according to the data types

• numerical data: single leading underscore ( _ ). All caps if produced by a flowchart node.

• categorial data: no leading underscore. All caps if produced by flowhchart node.

• special cases: Peak detection data are placed in a column named peaks_bases where each element is a DataFrame.

• uuid data: has uuid or UUID in the name

Note

_BLOCK_ is an exception, it contains UUIDs not numerical data.

History Trace

The History Trace of a Transmission is a log containing the discrete analysis steps, known as operations, along with their parameters and any other useful information. When a flowchart node performs an operation it stores the output(s) data in the Transmission DataFrame and appends the operation parameters to this log. A seperate log is kept for each data block present in the Transmission DataFrame.

Console

You have direct access to the data within the nodes through the console in the flowchart. To show the console go to View -> Console.

See also

If you are unfamiliar with the console see the overview on Consoles

Call get_nodes() to view a dict of all nodes in the flowchart. You can access the output Transmission in most nodes through the attribute t. You can access the transmission dataframe through t.df.

See also

See the Transmission API for more information. Sources for the nodes at mesmerize/pyqtgraphCore/flowchart/library.

Example, directly accessing DataFrame elements through the flowchart console

Transmission Files

You can save a Transmission files using the Save node and work with the data directly in scripts, jupyter notebooks etc. You can also save them through the flowchart console (and plot consoles) through Transmission.to_hdf5.

Load Transmission

Load a saved Transmission instance using Transmission.from_hdf5

>>> from mesmerize import Transmission
>>> from uuid import UUID

>>> t = Transmission.from_hdf5('/share/data/temp/kushal/data.trn')
<mesmerize.analysis.data_types.Transmission at 0x7f4d42f386a0>

# The DataFrame is always the 'df' attribute
>>> t.df.head()

                                           CurvePath  ... FCLUSTER_LABELS
0  curves/a2-_-1-_-843c2d43-75f3-421a-9fef-483d1e...  ...               8
1  curves/brn3b_a6-_-2-_-21557a64-6868-4ff4-8db1-...  ...               4
2  curves/brn3b_a6-_-2-_-21557a64-6868-4ff4-8db1-...  ...               5
3  curves/brn3b_day1_3-_-2-_-ff3e95df-0e15-495c-9...  ...               8
4  curves/brn3b_day1_3-_-2-_-ff3e95df-0e15-495c-9...  ...               6

[5 rows x 27 columns]

View History Log

Transmissions have a history_trace attribute which is an instance of HistoryTrace.

Use the get_data_block_history and get_operations_list methods to view the history log of a data block.

# To view the history log, first get the block UUID of the dataframe row of which you want the history log

# Block UUIDs are stored in the _BLOCK_ column
>>> bid = t.df.iloc[10]._BLOCK_
>>> bid

'248a6ece-e60e-4a09-845e-188a5199d262'

# Get the history log of this data block
# HistoryTrace.get_operations_list() returns a list of operations, without parameters
# HistoryTrace.get_data_block_history() returns the operations list with the parameters
>>> t.history_trace.get_operations_list(bid)

['spawn_transmission',
 'splice_arrays',
 'normalize',
 'rfft',
 'absolute_value',
 'log_transform',
 'splice_arrays',
 'fcluster']

# View the entire history log with all params
>>> t.history_trace.get_data_block_history(bid)

[{'spawn_transmission': {'sub_dataframe_name': 'neuronal',
'dataframe_filter_history': {'dataframe_filter_history': ['df[~df["promoter"].isin([\'cesa\', \'hnk1\'])]',
    'df[~df["promoter"].isin([\'cesa\', \'hnk1\'])]',
    'df[~df["cell_name"].isin([\'not_a_neuron\', \'non_neuronal\', \'untagged\', \'ependymal\'])]']}}},
{'splice_arrays': {'data_column': '_RAW_CURVE',
'start_ix': 0,
'end_ix': 2990,
'units': 'time'}},
{'normalize': {'data_column': '_SPLICE_ARRAYS', 'units': 'time'}},
{'rfft': {'data_column': '_NORMALIZE',
'frequencies': [0.0,
    0.0033444816053511705,
    0.0033444816053511705,
    0.006688963210702341,
    ...

# Get the parameters for the 'fcluster' operation
>>> fp = t.history_trace.get_operation_params(bid, 'fcluster')

# remove the linkage matrix first so we can view the other params
>>> fp.pop('linkage_matrix');fp

{'threshold': 8.0,
 'criterion': 'maxclust',
 'depth': 1,
 'linkage_params': {'method': 'complete',
 'metric': 'wasserstein',
 'optimal_ordering': True}}

Nodes

Data

These nodes are for performing general data related operations

LoadFile

Source

Loads a save Transmission file. If you have a Project open it will automatically set the project path according to the open project. Otherwise you must specify the project path. You can specify a different project path to the project that is currently open (this is untested, weird things could happen). You should not merge Transmissions originating from different projects.

Note

You can also load a saved Transmission file by dragging & dropping it into the Flowchart area. It will create a LoadFile node with the name of the dropped.

Terminal	Description
Out	Transmission loaded from the selected file.

Parameters	Description
load_trn	Button to choose a .trn file (Transmission) to load
proj_trns	Load transmission file located in the project’s “trns” directory
proj_path	Button to select the Mesmerize project that corresponds to the chosen .trn file.

Note

The purpose of specifying the Project Path when you load a save Transmission file is so that interactive plots and the Datapoint Tracer can find raw data that correspond to datapoints.

LoadProjDF

Source

Load the entire Project DataFrame (root) of the project that is currently open, or a sub-DataFrame that corresponds a tab that you have created in the Project Browser.

Output Data Column (numerical): _RAW_CURVE

Each element in this output column contains a 1-D array representing the trace extracted from an ROI.

Terminal	Description
Out	Transmission created from the Project DataFrame or sub-DataFrame.

Parameters	Description
DF_Name	DataFrame name. List correponds to Project Browser tabs.
Update	Re-create Transmission from corresponding Project Browser tab.
Apply	Process data through this node

Note

The DF_Name options do not update live with the removal or creation of tabs in the Project Browser, you must create a new node to reflect these types of changes.

Save

Source

Save the input Transmission to a file so that the Transmission can be used re-loaded in the Flowchart for later use.

Usage: Connect an input Transmission to this node’s In terminal, click the button to choose a path to save a new file to, and then click the Apply checkbox to save the input Transmission to the chosen file.

Terminal	Description
In	Transmission to be saved to file

Parameters	Description
saveBtn	Button to choose a filepath to save the Transmission to.
Apply	Process data through this node

Note

You must always save a Transmission to a new file (pandas with hdf5 exihibts weird behavior if you overwrite, this is the easiest workaround). If you try to overwrite the file you will be presented with an error saying that the file already exists.

Merge

Source

Merge multiple Transmissions into a single Transmission. The DataFrames of the individual Transmissions are concatenated using pandas.concat and History Traces are also merged. The History Trace of each indidual input Transmission is kept separately.

Warning

At the moment, if you create two separate data streams that originate from the same Transmission and then merge them at a later point, the analysis log (History Trace) of the individual data streams are not maintained. See the information about data blocks in the Transmission.

Terminal	Description
In	Transmissions to be merged
Out	Merged Transmission

ViewTransmission

Source

View the input Transmission object using the spyder Object Editor. For example you can explore the Transmission DataFrame and HistoryTrace.

ViewHistory

Source

View the HistoryTrace of the input Transmission in a nice Tree View GUI.

TextFilter

Source

Include or Exclude Transmission DataFrame rows according to a text filter in a categorical column.

Usage Example: If you want to select all traces that are from photoreceptor cells and you have a categorical column, named cell_types for example, containing cell type labels, choose “cell_type” as the Column parameter and enter “photoreceptor” as the filter parameter, and select Include. If you want to select everything that are not photoreceptors select Exclude.

Note

It is recommended to filter and group your data beforehand using the Project Browser since it allows much more sophisticated filtering.

Terminal	Description
In	Input Transmission
Out	Transmission its DataFrame filtered accoring parameters

Parameters	Description
Column	Categorical column that contains the text filter to apply
filter	Text filter to apply
Include	Include all rows matching the text filter
Exclude	Exclude all rows matching the text filter
Apply	Process data through this node

HistoryTrace output structure: Dict of all the parameters for this node

SpliceArrays

Source

Splice arrays derived in the specified numerical data column and place the spliced output arrays in the output column.

Output Data Column (numerical): _SPLICE_ARRAYS

Terminal	Description
In	Input Transmission
Out	Transmission with arrays from the input column spliced and placed in the output column

Parameters	Description
data_column	Numerical data column containing the arrays to be spliced
indices	The splice indices, “start_index:end_index”
Apply	Process data through this node

DropNa

Source

Drop NaNs and Nones (null) from the Transmission DataFrame. Uses DataFrame.dropna and DataFrame.isna methods.

• If you choose “row” or “column” as axis, entire rows or columns will be dropped if any or all (see params) of the values are NaN/None.

• If you choose to drop NaNs/Nones according to a specific column, it will drop the entire row if that row has a NaN/None value for the chosen column.

Terminal	Description
In	Input Transmission
Out	Transmission NaNs and None’s removed according to the params

Parameters	Description
axis	Choose to rows, columns, or a rows according to a specific column.
how	any: Drop if any value in the row/column is NaN/None all: Drop only if all values in the row/column are Nan/None ignored if “axis” parameter is set to a specific column
Apply	Process data through this node

NormRaw

Source

Scale the raw data such that the min and max values are set to the min and max values derived from the raw spatial regions of the image sequences they originate from. Only for CNMFE data.

The arrays in the _RAW_CURVE column are scaled and the output is placed in a new column named _NORMRAW

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
option	Derive the raw min & max values from one of the following options: top_5: Top 5 brightest pixels top_10: Top 10 brighest pixels top_5p: Top 5% of brightest pixels top_10p: Top 10% of brightest pixels top_25p: Top 25% of brightest pixels full_mean: Full mean of the min and max array
Apply	Process data through this node

Note

If the raw min value is higher than the raw max value the curve will be excluded in the output. You will be presented with a warning box with the number of curves that were excluded due to this.

---

Display

These nodes connect input Transmission(s) to various plots for visualization

The actual Plot Widget instance that these nodes use can be accessed through the plot_widget attribute in the flowchart console.

For example

# Get a heatmap node that is named "Heatmap.0"
>>> hn = get_nodes()['Heatmap.0']

# the plot widget instance
>>> hn.plot_widget

<mesmerize.plotting.widgets.heatmap.widget.HeatmapTracerWidget object at 0x7f26e5d29678>

BeeswarmPlots

Source

Based on pqytgraph Beeswarm plots.

Visualize data points as a pseudoscatter and as corresponding Violin Plots. This is commonly used to visualize peak features and compare different experimental groups.

For information on the plot widget see Beeswarm Plots

Terminal	Description
In	Input Transmission The DataFrame column(s) of interest must have single numerical values, not arrays

Heatmap

Source

Used for visualizing numerical arrays in the form of a heatmap. Also used for visualizing a hieararchical clustering tree (dendrogram) along with a heatmap with row order corresponding to the order leaves of the dendrogram.

For information on the plot widget see Heat Plot

Terminal	Description
In	Input Transmission The arrays in the DataFrame column(s) of interest must be of the same length

Note

Arrays in the DataFrame column(s) of interest must be of the same length. If they are not, you must splice them using the SpliceArrays node.

CrossCorr

Source

Perform Cross-Correlation analysis. For information on the plot widget see CrossCorrelation Plot

Plot

Source

For information on the plot widget see <plot_SimplePlot>

A simple plot.

Terminal	Description
In	Input Transmission

Parameters	Description
data_column	Data column to plot, must contain numerical arrays
Show	Show/hide the plot window
Apply	Process data through this node

Proportions

Source

Plot stacked bar chart of one categorical variable vs. another categorical variable.

For information on the plot widget see Proportions Plot

ScatterPlot

Source

Create scatter plot of numerical data containing [X, Y] values

For information on the plot widget see Scatter Plot

TimeSeries

Plot the means along with confidence intervals or standard eviation of numerical arrays representing time series data.

For more information see plot_TimeSeries

---

Signal

Routine signal processing functions

I recommend this book by Tom O’Haver if you are unfamiliar with basic signal processing: https://terpconnect.umd.edu/~toh/spectrum/TOC.html

Butterworth

Source

Creates a Butterworth filter using scipy.signal.butter and applies it using scipy.signal.filtfilt.

The Wn parameter of scipy.signal.butter is calculated by dividing the sampling rate of the data by the freq_divisor parameter (see below).

Output Data Column (numerical): _BUTTERWORTH

Terminal	Description
In	Input Transmission
Out	Transmission with filtered signals in the output data column

Parameters	Description
data_column	Data column containing numerical arrays to be filtered
order	Order of the filter
freq_divisor	Divisor for dividing the sampling frequency of the data to get Wn
Apply	Process data through this node

SavitzkyGolay

Source

Savitzky Golay filter. Uses scipy.signal.savgol_filter.

Output Data Column (numerical): _SAVITZKY_GOLAY

Terminal	Description
In	Input Transmission
Out	Transmission with filtered signals in the output data column

Parameters	Description
data_column	Data column containing numerical arrays to be filtered
window_length	Size of windows for fitting the polynomials. Must be an odd number.
polyorder	Order of polynomials to fit into the windows. Must be less than window_length
Apply	Process data through this node

PowSpecDens

Resample

Source

Resample the data in numerical arrays. Uses scipy.signal.resample.

Output Data Column (numerical): _RESAMPLE

Terminal	Description
In	Input Transmission
Out	Transmission with resampled signals in the output data column

Parameters	Description
data_column	Data column containing numerical arrays to be resampled
Rs	New sampling rate in Tu units of time.
Tu	Time unit
Apply	Process data through this node

Note

If Tu = 1, then Rs is the new sampling rate in Hertz.

ScalerMeanVariance

Source

Uses tslearn.preprocessing.TimeSeriesScalerMeanVariance

Output Data Column (numerical): _SCALER_MEAN_VARIANCE

Terminal	Description
In	Input Transmission
Out	Transmission with scaled signals in the output column

Parameters	Description
data_column	Data column containing numerical arrays to be scaled
mu	Mean of the output time series
std	Standard Deviation of the output time series
Apply	Process data through this node

Note

if mu = 0 and std = 1, the output is the z-score of the signal.

Normalize

Source

Normalize the signal so that all values are between 0 and 1 based on the min and max of the signal.

Output Data Column (numerical): _NORMALIZE

Terminal	Description
In	Input Transmission
Out	Transmission with scaled signals in the output column

Parameters	Description
data_column	Data column containing numerical arrays to be scaled
Apply	Process data through this node

RFFT

Source

Uses scipy.fftpack.rfft. “Discrete Fourier transform of a real sequence”

Output Data Column (numerical): _RFFT

Terminal	Description
In	Input Transmission
Out	Transmission with the RFT of signals in the output column

Parameters	Description
data_column	Data column containing numerical arrays
Apply	Process data through this node

iRFFT

Source

Uses scipy.fftpack.irfft. “inverse discrete Fourier transform of real sequence x”

Output Data Column (numerical): _IRFFT

PeakDetect

Source

Simple Peak Detection using derivatives. The “Differentiation” chapter of Tom O’Haver’s book has a section on Peak Detection which I recommend reading. https://terpconnect.umd.edu/~toh/spectrum/TOC.html

Output Data Column (DataFrame): peaks_bases

See also

Peak Editor GUI

Terminal	Description
Derivative	Transmission with derivatives of signals. Must have _DERIVATIVE column. It’s recommended to use a derivative from a normalized filtered signal.
Normalized	Transmission containing Normalized signals, used for thresholding See Normalize node
Curve	Transmission containing original signals. Usually not filtered to avoid distortions caused by filtering
PB_Input (optional)	Transmission containing peaks & bases data (peaks_bases column). Useful for visualizing a saved Transmission that has peaks & bases data
Out	Transmission with the detected peaks & bases as DataFrames in the output column

Warning

The PB_Input terminal overrides all other terminals. Do not connect inputs to PB_Input and other terminals simultaneously.

Parameter	Description
data_column	Data column of the input Curve Transmission for placing peaks & bases onto
Fictional_Bases	Add bases to beginning and end of signal if first or last peak is lonely
Edit	Open Peak Editor GUI, see Peak Editor
SlopeThr	Slope threshold
AmplThrAbs	Absolute amplitude threshold
AmplThrRel	Relative amplitude threshold
Apply	Process data through this node

PeakFeatures

Source

Compute peak features. The DataFrame of the ouput Transmission contains one row for each peak.

Output Data Column	Description
_pf_peak_curve	array representing the peak
_pf_ampl_rel_b_ix_l	peak amplitude relative to its left base
_pf_ampl_rel_b_ix_r	peak amplitude relative to its right base
_pf_ampl_rel_b_mean	peak amplitude relative to the mean of its bases
_pf_ampl_rel_zero	peak amplitude relative to zero
_pf_area_rel_zero	Simpson’s Rule Integral of the curve
_pf_area_rel_min	Simpson’s Rule Integral relative to the minimum value of the curve Substracts the minimum values of the peak curve before computing the integral
_pf_rising_slope_avg	slope of the line drawn from the left base to the peak
_pf_falling_slope_avg	slope of the line drawn from the right base to the peak
_pf_duration_base	distance between the left and right base
_pf_p_ix	index of the peak maxima in the parent curve
_pf_uuid	peak UUID
_pf_b_ix_l	index of the left base in the parent curve
_pf_b_ix_r	index of the right base in the parent curve

See also

mesmerize/analysis/compute_peak_features for the code that computes the peak features.

Terminal	Description
In	Input Transmission. Must contain peak_bases column that contains peak_bases DataFrames.
Out	Transmission with peak features in various output columns

Parameter	Description
data_column	Data column containing numerical arrays from which to compute peak features.
Apply	Process data through this node

---

Math

Nodes for performing basic Math functions

Derivative

Source

Computes the first derivative.

Output Data Column (numerical): _DERIVATIVE

Terminal	Description
In	Input Transmission
Out	Transmission with the derivative placed in the output column

Parameter	Description
data_column	Data column containing numerical arrays
Apply	Process data through this node

TVDiff

Source

Based on Numerical Differentiation of Noisy, Nonsmooth Data. Rick Chartrand. (2011).. Translated to Python by Simone Sturniolo.

XpowerY

Source

Raises each element of the numerical arrays in the data_column to the exponent Y

Output Data Column (numerical): _X_POWER_Y

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
data_column	Data column containing numerical arrays
Y	Exponent
Apply	Process data through this node

AbsoluteValue

Source

Element-wise absolute values of the input arrays. Computes root mean squares if input arrays are complex.

Output Data Column (numerical): _ABSOLUTE_VALUE

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
data_column	Data column containing numerical arrays
Apply	Process data through this node

LogTransform

Source

Perform Logarithmic transformation of the data.

Output Data Column (numerical): _LOG_TRANSFORM

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
data_column	Data column containing numerical arrays
transform	log10: Base 10 logarithm ln: Natural logarithm modlog10: \(sign(x) * \log_{10} (|x| + 1)\) modln: \(sign(x) * \ln (|x| + 1)\)
Apply	Process data through this node

ArrayStats

Source

Perform a few basic statistical functions.

Output Data Column (numerical): Customizable by user entry

Output data are single numbers, not arrays

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

The desired function is applied to each 1D array in the data_column and the output is placed in the Output Data Column.

Parameter	Description
data_column	Data column containing numerical arrays
function	amin: Return the minimum of the input array amax: Return the maximum of the input array nanmin: Return the minimum of the input array, ignore NaNs nanmax: Return the maximum of the input array, ignore NaNs ptp: Return the range (max - min) of the values of the input array median: Return the median of the input array mean: Return the mean of the input array std: Return the standard deviation of the input array var: Return the variance of the input array nanmedian: Return the median of the input array, ignore NaNs nanmean: Return the mean of the input array, ignore NaNs nanstd: Return the standard deviation of the input array, ignore NaNs nanvar: Return the variance of the input array, ignore NaNs
group_by (Optional)	Group by a categorial variable, for example get the mean array of a group
group_by_sec (Optional)	Group by a secondary categorical variable
output_col	Enter a name for the output column
Apply	Process data through this node

ArgGroupStat

Source

Group by a categorial variable and return the value of any other column based on a statistic. Basically creates sub-dataframes for each group and then returns based on the sub-dataframe.

Group by column “group_by” and return value from column “return_col” where data in data_column fits “stat”

Output Data Column (Any): ARG_STAT

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
data_column	Data column containing single numbers (not arrays for now)
group_by	Group by column (categorical variables)
return_col	Return value from this column (any data)
stat	“max” or “min”
Apply	Process data through this node

ZScore

Source

Compute Z-Scores of the data. Uses scipy.stats.zscore. The input data are are divided into groups according to the group_by parameter. Z-Scores are computed for the data in each group with respect to the data only in that group.

Output Data Column (numerical): _ZSCORE

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
data_column	Input data column containing numerical arrays
group_by	Categorial data column to group by.
Apply	Process data through this node

LinRegress

Source

Basically uses scipy.stats.linregress

Performs Linear Regression on numerical arrays and returns slope, intercept, r-value, p-value and standard error

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
data_column	Data column containing 1D numerical arrays. The values are used as the y values and indices as the x values for the regression

Output Columnns: Single numbers, _SLOPE, _INTERCEPT, _R-VALUE, _P-VALUE, _STDERR as decribed in scipy.stats.linregress

---

Biology

Nodes for some biologically useful things which I couldn’t categorize elsewhere

ExtractStim

Source

Extract the portions of a trace corresponding to stimuli that have been temporally mapped onto it. It outputs one row per stimulus period.

Note: Stimulus extraction is currently quite slow, will be optimized after some planned changes in the Transmission object.

Output Data Column	Description
ST_TYPE	Stimulus type, corresponds to your Project Config
ST_NAME	Name of the stimulus
_ST_CURVE	The extracted array based on the parameters
_ST_START_IX	Start index of the stimulus period in the parent curve
_ST_END_IX	End index of the stimulus period in the parent curve
ST_uuid	UUID assigned for the extracted stimulus period

Parameter	Description
data_column	Data column containing the signals to be extracted based on the stimulus maps
Stim_Type	Type of stimulus to extract
start_offset	Offset the start index of the stimulus mapping by a value (in frames)
end_offset	Offset the end index of the stimulus mapping by a value (in frames)
zero_pos	Zero index of the extracted signal start_offset: extraction begins at the start_offset value, stops at the end_offset stim_end: extraction begins at the end of the stimulus, stops at the end_offset. stim_center: extraction begins at the midpoint of the stimulus period plus the start_offset, stops at end_offset

DetrendDFoF

Source

Uses the detrend_df_f function from the CaImAn library. This node does not use any of the numerical data in a Transmission DataFrame to compute the detrended \(\Delta F / F_0\). It directly uses the CNMF output data for the Samples that are present in the Transmission DataFrame.

Output Data Column (numerical): _DETREND_DF_O_F

StaticDFoFo

Source

Perform \(\frac{F - F_0}{F_0}\) without a rolling window. \(F\) is an input array and \(F_0\) is the minimum value of the input array.

Output Data Column (numerical): _STATIC_DF_O_F

Terminal	Description
In	Input Transmission
Out	Transmission with the result placed in the output column

Parameter	Description
data_column	Data column containing numerical arrays
Apply	Process data through this node

---

Clustering

KShape

Source

Perform KShape clustering. For more information see KShape plot.

KMeans

Source

Basically sklearn.cluster.KMeans.

---

Hierarchical

These nodes allow you to perform Hierarchical Clustering using scipy.cluster.hierarchy.

If you are unfamiliar with Hierarchical Clustering I recommend going through this chapter from Michael Greenacre: http://www.econ.upf.edu/~michael/stanford/maeb7.pdf

Note

Some of these nodes do not use Transmission objects for some inputs/outputs.

Linkage

Source

Compute a linkage matrix which can be used to form flat clusters using the FCluster node.

Based on scipy.cluster.hierarchy.linkage

Terminal	Description
In	Input Transmission
Out	dict containing the Linkage matrix and parameters, not a Transmission object

Parameters	Description
data_column	Numerical data column used for computing linkage matrix
method	linkage method
metric	metric for computing distance matrix
optimal_order	minimize distance between successive leaves, more intuitive visualization Click here for more info
Apply	Process data through this node

FCluster

Source

“Form flat clusters from the hierarchical clustering defined by the given linkage matrix.”

Based on scipy.cluster.hierarchy.fcluster

Output Data Column (categorial): FCLUSTER_LABELS

Terminal	Description
Linkage	Linkage matrix, output from Linkage node.
Data	Input Transmission, usually the same input Transmission used for the Linkage node.
IncM (optional)	Inconsistency matrix, output from Inconsistent
Monocrit (optional)	Output from MaxIncStat or MaxInconsistent
Out	Transmission with clustering data that can be visualized using the Heatmap

Parameters: Exactly as desribed in scipy.cluster.hierarchy.fcluster

HistoryTrace output structure: Dict of all the parameters for this node, as well as the parameters used for creating the linkage matrix and the linkage matrix itself from the Linkage node.

Inconsistent

MaxIncStat

MaxInconsistent

---

Transform

Nodes for transforming data

LDA

Source

Perform Linear Discriminant Analysis. Uses sklearn.discriminant_analysis.LinearDiscriminantAnalysis

Terminal	Description
train_data	Input Transmission containing the training data
predict	Input Transmission containing data on which to predict
T	Transmission with Transformed data and decision function. Output columns outlined below: _LDA_TRANSFORM: The transformed data, can be visualized with a Scatter Plot for instance _LDA_DFUNC: Decision function (confidence scores). Can be visualized with a Heatmap
coef	Transmission with LDA Coefficients. Output columns outlined below: classes: The categorical labels that were trained against _COEF: LDA Coefficients (weight vectors) for the classes. Can be visualized with a Heatmap
means	Transmission with LDA Means. Output columns outlined below: classes: The categorical labels that were trained against _MEANS: LDA means for the classes. Can be visualized with a Heatmap
predicted	Transmission containing predicted class labels for the data. The class labels are placed in a column named LDA_PREDICTED_LABELS The names of the class labels correspond to the labels from the training labels optional

Parameter	Description
train_data	Single or multiple data columns that contain the input features.
labels	Data column containing categorical labels to train to
solver	svd: Singular Value Decomposition lsqr: Least Squares solution eigen: Eigen decomposition
shrinkage	Can be used with lsqr or eigen solvers.
shrinkage_val	shrinkage value if shrinkage is set to “value”
n_components	Number of components to output
tol	Tolereance threshold exponent. The used value is 10^<tol>
score	Displays mean score of the classification (read only)
predict_on	Single or multiple data columns that contain the data that are used for predicting on Usually the same name as the data column(s) used for the training data. optional

HistoryTrace output structure: Dict of all the parameters for this node

Examples

You can view examples of flowcharts in the demo dataset or one of the other datasets associated with the paper:

Demo dataset: https://doi.org/10.6084/m9.figshare.11370183

3. intestinalis dataset: https://doi.org/10.6084/m9.figshare.10289162

3. elegans dataset: https://doi.org/10.6084/m9.figshare.10287113

PVC-7 as a Mesmerize dataset: https://doi.org/10.6084/m9.figshare.10293041

Screenshots

Flowchart screenshots from the C. intestinalis dataset.

Z-score

Peak detection

Hierarchical clustering

Beeswarm

Used for visualization of data points using a pseudo-scatter and violin plots.

Layout

You can click on individual datapoints and view the associated data using the Datapoint Tracer. To show the Datapoint Tracer, in the menubar go to View -> Live datapoint tracer

Parameters

Parameter	Description
Data columns	Multi-select data columns to plot They must have single numerical values, not arrays
Group based on	Categorical data column used for grouping the data
Datapoint tracer curve	Data column, containing numerical arrays, that is shown in the Datapoint Tracer
UUID column	Column containing the UUIDs that correspond to the data in the selected data column(s)
Apply all	Apply the plot parameters and draw the plot

Consoles

Currently the Heatmap, Scatter, Proportions plot and KShape follow a uniform structure allowing internal access to the data and plot axes. Refer to their Base API. For example, through their consoles you can access the Transmission containing data for the current plot, manually save the plot, etc.

Cross Correlation

Explore Cross-correlation functions of all curves from a sample. Normalized cross-correlations are computed using tslearn.cycc.normalized_cc

This is an interactive widget. You can click on the individual cells in the heatmap to view the individual curves, the cross-correlation function of the two curves, and the spatial localization of the ROI that they originate from.

Layout

Left: Lag or Maxima Matrix (see below) with thresholds applied and visualized as a heatmap. When you click on the individual cells it will open/update the Datapoint Tracer according to the two curves the cell corresponds to.

Top Center: Parameters.

Center: When you click on a cell in the heatmap you will see Curve 1 (x-axis of heatmap), and Curve 2 (y-axis of heatmap) and their cross-correlation function. The units are in seconds for all of these

Right: List of Samples. Click on a Sample to select it as the current sample.

Lag Matrix

Computed as follows:

1. A 2D array is created where each element is a cross-correlation function (represented by a 1D numerical array).

2. The x-distance (time) between zero and the global maxima of the cross-correlation function (called \(lag\)) is computed for each of these elements.

3. The 2D array of cross-correlation functions is reduced to a 2D array of these \(lag\) values.

The result is a matrix where each element is the x-distance between zero and the global maxima of the cross-correlation of the two curves the element represents.

Maxima Matrix

Similar to computation of the Lag Matrix above, but instead of using the \(lag\) between zero and the global maxima it uses the y-value of the global maxima.

Parameters

Data column: The data column, containing numerical arrays, that are used as the input curves for computing cross-correlations.

Labels column: The labels column, containing categorical labels, that are used for the row and column labels in the heatmaps.

Pre-normalization: Option to perform 0 - 1 Normalization (Same method as the Normalize) or Z-Score of the input curves prior to computing their cross-correlation functions.

Compute all data: Appy the parameters and compute cross-correlation data for all Samples in the DataFrame of the input transmission.

Thresholds

Apply thresholds for \(lag\) and the maxima value. The colormap limits of the heatmap are set according to this threshold and all data under are set to white on the heatmap (you can still click and explore them).

Thresholds are applied live onto the heatmap.

Datapoint Tracer

API Reference

The Datapoint Tracer is attached to many plots, allowing you to interactively explore the data associated to the datapoints. You can explore the analysis history, the spatial localization of the ROI it originates from, associated numerical or categorical data, and view an additional numerical column (such as the raw trace).

The Datapoint Tracer is embedded in some plots, and in others you can open it by going to View -> Live Datapoint Tracer.

Datapoint Tracer Layout

Top right: Max Projection or Standard Deviation Project of the image sequence.

Bottom right: Numerical data, based on the “DPT Curve column” that the user has specified in the plot controls. If exploring peak feature based data the temporal span of the peak will be highlighted.

Top left: Analysis log, a ordered list of operations and their parameters.

Bottom left: All other data associated with this datapoint (the data present in the other columns of the row this datapoint is present in, see Transmission)

Open in viewer button: Open the parent Sample of the current datapoint in the viewer.

Heatmap

API Reference

Note

This plot can be saved in an interactive form, see Saving plots

Visualize numerical arrays in the form of a heatmap. Also used for visualization of Hierarchical clusterting dendrograms. Datapoint Tracer is embedded.

Layout

Left: The heatmap. Clicking the heatmap highlights the selected row and upates the Datapoint Tracer. Right click on the heatmap to clear the selection highlight on the heatmap. You can zoom and pan the heatmap using the tools above the plot area. You can zoom/pan in the legend and heatmap. The up and down keys on your keyboard can be used to move the current row selection.

Bottom left: Set the row order of the heatmap according to a categorical column.

Middle: Plot controls.

Very bottom: Status label - displays any issues that were raised while setting the plot data. Click on the status label to see more information.

Parameters

Data column: Data column, numerical arrays, that contain the data for the heatmap. Each row of this data column (a 1D array) is represented as a row on the heatmap.

Labels column: Column containing categorical labels that are used to create the row legend for the heatmap.

DPT curve column: Data column, containing numerical arrays, that is shown in the Datapoint Tracer.

Data colormap: Colormap used for representing the data in the heatmap. Default is ‘jet’.

Legend colormap: Colormap used for the row legend.

Live update from input transmission: If checked this plots receives live updates from the flowchart.

Plot: Updates data input from the flowchart.

Save: Save the plot data and state in an interactive form <save_ptrn>

Load: Load a plot that has been saved as a “.ptrn” file.

Layout to visualize Hierarchical Clustering

This plot widget can also be used to visualize a dendrogram on top of a heatmap of data.

The differences are:

1. There are two legend bars

   • Left: Cluster label

   • Right: Corresponds to Labels column parameter.

2. You can also zoom/pan the dendrogram in addition to the legends and heatmap.

3. Sorting the heatmap rows is disabled because this wouldn’t make sense

Console

You can directly access the heatmap widget through the console. This is useful for plot customization and exporting with specific parameters.

Toggle the console’s visibility by clicking on the “Show/Hide Console” button at the bottom of the controls.

See also

API Reference

Namespace

reference	Description
this	The higher-level HeatmapTracerWidget instance, i.e. the entire widget
this.transmission	Current input Transmission
get_plot_area()	Returns the lower-level Heatmap variant instance, basically the actual plot area
get_plot_area().plot	Returns the seaborn ClusterGrid instance containing the axes
get_plot_area().fig	Returns the matplotlib Figure instance

Attributes of get_plot_area().plot

For example, the heatmap axes object can be retrieved through get_plot_area().plot.ax_heatmap. See the usage examples.

ax_heatmap	Heatmap axes
ax_row_dendrogram	Row dendrogram axes
ax_col_dendrogram	Used for the legend
cax	Colorbar axes

Examples

Export

See also

matplotlib API for: Figure.savefig, Figure.set_size_inches, Figure.get_size_inches

# Desired size (width, height)
size = (2.0, 2.5)

# Get the figure
fig = get_plot_area().fig

# original size to reset the figure after we save it
orig_size = fig.get_size_inches()

#Set the desired size
fig.set_size_inches(size)

# Save the figure as a png file with 1200 dpi
fig.savefig('/share/data/temp/kushal/amazing_heatmap.png', dpi=1200, bbox_inches='tight', pad_inches=0)

# Reset the figure size and draw()
fig.set_size_inches(orig_size)
get_plot_area().draw()

Note

The entire plot area might go gray after the figure is reset to the original size. I think this is a Qt-matplotlib issue. Just resize the window a bit and the plot will be visible again!

Warning

From my experience I have not been able to open clustermap SVG files saved with very high DPI (600+). Even with 32 cores & 128GB of RAM both inkscape and illustrator just hang ¯\_(ツ)_/¯. Try png or other formats.

x tick labels

See also

matplotlib.axes.Axes.set_xticklabels, matplotlib.axes.Axes.set_xticks.

If the data are in the time domain:

from mesmerize.analysis import get_sampling_rate
import numpy as np

# Get the sampling rate of the data
sampling_rate = get_sampling_rate(this.transmission)

# Number of frames currently displayed in the heatmap
num_frames = get_plot_area().data.shape[1]

# Set an appropriate interval
interval = 30 # This is in seconds, not frames

# Get the recording time in seconds
recording_time = int(num_frames / sampling_rate)

# Set the new ticks
get_plot_area().plot.ax_heatmap.set_xticks(np.arange(0, num_frames, interval * sampling_rate))

# Set the tick labels
# You can change the fontsize here
get_plot_area().plot.ax_heatmap.set_xticklabels(np.arange(0, recording_time, interval), fontdict={'fontsize': 4})

# Set a title for the x axis. You can change the fontsize here
get_plot_area().plot.ax_heatmap.set_xlabel('Time (seconds)', fontdict={'fontsize': 6})

# Draw the plot with these changes
get_plot_area().draw()

Note

You may need to resize the dock widget that the plot is present in to display the newly drawn plot, this is a Qt-matplotlib issue.

If the data are in the frequency domain:

from mesmerize.analysis import get_frequency_linspace
import numpy as np

# Get frequency linspace and Nyquist frequency
freqs, nf = get_frequency_linspace(this.transmission)

# Get the number of frequencies currently shown in the heatmap
num_freqs = get_plot_area().data.shape[1]

# The max frequency currently display in the heatmap
max_freq = freqs[num_freqs - 1]

# Set an appropriate interval
interval = 0.25 # This is in Hertz

# Set the tick labels
# Set the new ticks
get_plot_area().plot.ax_heatmap.set_xticks(np.arange(0, num_freqs, (num_freqs * interval) / max_freq))

# You can change the fontsize here
get_plot_area().plot.ax_heatmap.set_xticklabels(np.arange(0, max_freq, interval), fontdict={'fontsize': 4})

# Set a title for the x axis. You can change the fontsize here
get_plot_area().plot.ax_heatmap.set_xlabel('Frequency (Hertz)', fontdict={'fontsize': 6})

# Draw the plot with these changes
get_plot_area().draw()

Note

You may need to resize the dock widget that the plot is present in to display the newly drawn plot, this is a Qt-matplotlib issue.

Colorbar label

get_plot_area().plot.cax.set_title('norm. z-score', x=-0.25, y=0.65, fontdict={'fontsize': 6}, rotation=90)
get_plot_area().draw()

Axes visibility

Hide/show legend

get_plot_area().plot.ax_col_dendrogram.set_visible(False)
get_plot_area().draw()

Hide/show y axis (similar for x axis)

get_plot_area().plot.ax_heatmap.get_yaxis().set_visible(False)
get_plot_area().draw()

Hide/show colorbar

get_plot_area().plot.cax.set_visible(False)
get_plot_area().draw()

KShape

Perform KShape clustering.

I recommend reading the paper on it: Paparrizos, John, and Luis Gravano. “k-Shape: Efficient and Accurate Clustering of Time Series.” In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 1855-1870. ACM, 2015.

This GUI uses the tslearn.clustering.KShape implementation.

See also

API reference

Note

This plot can be saved in an interactive form, see Saving plots

Layout

Left: KShape parameters and Plot parameters

Bottom left: Plot of a random sample of input data from a cluster.

Center: Plot of cluster mean and either confidence interval, standard deviation, or neither. Uses seaborn.lineplot

Right: Proportions plot. Exactly the same as Proportions.

Bottom Right: Console

KShape Parameters

The parameters and input data are simply fed to tslearn.clustering.KShape

Parameters outlined here are simply as they appear in the tslearn docs.

data_column: Input data for clustering.

n_clusters: Number of clusters to form.

max_iter: Maximum number of iterations of the k-Shape algorithm.

tol: Inertia variation threshold. If at some point, inertia varies less than this threshold between two consecutive iterations, the model is considered to have converged and the algorithm stops.

n_init: Number of times the k-Shape algorithm will be run with different centroid seeds. The final results will be the best output of n_init consecutive runs in terms of inertia.

random_state: Generator used to initialize the centers. If an integer is given, it fixes the seed. Defaults to the global numpy random number generator.

training subset: The subset of the input data that are used for used for training. After training, the predictions are fit on all the input data.

Plot Options

Plot cluster: The cluster from which to plot random samples of input data in the bottom left plot

Show centers: Show the centroids returned by the KShape model

Warning

There’s currently an issue where cluster centroids don’t appear to be index correctly. See https://github.com/rtavenar/tslearn/issues/114

max num curves: Maximum number of input data samples to plot

Error band: The type of data to show for the the error band in the means plots.

set x = 0 at: The zero position of a means plots with respect to the cluster members in the plot.

Save: Save the plot data and state in an interactive form

Console

The console can be useful for formatting plots, inspecting the underlying data etc.

See also

API reference

Namespace

reference	Description
this	The higher-level KShape widget instance, i.e. the entire widget
this.transmission	Current input Transmission
get_plot_means()	Returns the means plot
get_plot_raw()	Returns the raw plot
get_plot_proportions()	Returns the proportions plot, which is an instance of Proportions Widget

Examples

See also

matplotlib Axes

Set axis ranges

Set equal x & y axis ranges for the means plots. Also removes the top & right spines.

from itertools import product

# Get the means plot
plot = get_plot_means()

# Get the indices of the subplots
ixs = product(range(plot.nrows), range(plot.ncols))

# Set the same x & y axis limits for all subplots
for ix in ixs:

    # The subplot axes
    ax = plot.axs[ix]

    # Set the y limits
    ax.set_ylim([-2, 15.5])

    # Set the x limits
    ax.set_xlim([-30, 1000])

    # Remove the top & right plot spins
    ax.spines['right'].set_visible(False)
    ax.spines['top'].set_visible(False)

# Set a tight layout
plot.fig.tight_layout()

# Draw the plot
plot.draw()

Note

You may need to resize the dock widget that the plot is present in to display the newly drawn plot, this is a Qt-matplotlib issue.

x tick labels

Set the x tick labels in time units instead of frames

See also

matplotlib.axes.Axes.set_xticklabels | matplotlib.axes.Axes.set_xticks.

import numpy as np
from itertools import product
from mesmerize.analysis import get_sampling_rate

# Get the sampling rate of the data
sampling_rate = get_sampling_rate(this.transmission)

# Get the padded number of frames that are shown in the plots
num_frames = this.cluster_centers.shape[1]

# Set an appropriate interval
interval = 5 # This is in seconds, not frames

# Convert the padded frame number to time units
total_time = int(num_frames / sampling_rate)

ixs = product(range(4), range(3))

# Set these time units for all the means plots
# For the raw plots just remove the loop
for ix in ixs:
    # Get the axes
    ax = get_plot_means().axs[ix]

    # Set the new ticks
    ax.set_xticks(np.arange(0, num_frames, interval * sampling_rate))

    # Set the tick labels
    # You can change the fontsize here
    ax.set_xticklabels(np.arange(0, total_time, interval), fontdict={'fontsize': 4}, rotation=90)

    # Set a title for the x axis. You can change the fontsize here
    ax.set_xlabel('Time (seconds)', fontdict={'fontsize': 6})

    # Set ylabel as well
    ax.set_ylabel('z-score', fontdict={'fontsize': 6})

# Set a tight layout
get_plot_means().fig.tight_layout()

# Draw the plot with these changes
get_plot_means().draw()

Note

You may need to resize the dock widget that the plot is present in to display the newly drawn plot, this is a Qt-matplotlib issue.

Hide legend

Hide/show legend in the proportions plot

get_plot_proportions().ax.legend().set_visible(True)
get_plot_proportions().draw()

Export

You can export any of the plots with a specific size & DPI.

Replace the get_<plot>().fig on line 5 with the desired plot.

See also

matplotlib API for: Figure.savefig, Figure.set_size_inches, Figure.get_size_inches

# Desired size (width, height)
size = (7.0, 10.0)

# Get the figure
fig = get_<plot>().fig

# original size to reset the figure after we save it
orig_size = fig.get_size_inches()

#Set the desired size
fig.set_size_inches(size)

# Save the figure as an png file with 600 dpi
fig.savefig('/share/data/temp/kushal/amazing_shapes.png', dpi=600, bbox_inches='tight', pad_inches=0)

# Reset the figure size and draw
fig.set_size_inches(orig_size)
get_<plot>().draw()

Note

The entire plot area might go gray after the figure is reset to the original size. I think this is a Qt-matplotlib issue. Just resize the window a bit and the plot will be visible again!

Peak Editor

Visualize and edit detected peaks & bases. This GUI is accessible through the PeakDetect node.

Usage

• Optimize your peaks/bases detection through the datastreams that feed into the Derivative and Normalize terminals of the parent PeakDetect node. For example, play with filtering parameters for the ButterWorth node or SavitzkyGolay node.

• Optimize amplitude thresholds of the parent PeakDetect node.

• Disconnect from the flowchart (see below).

• Edit your peaks/bases

• Click “Send all curves to flowchart” (see below) to set the edited data as the output of the parent PeakDetect node.

Layout

Bottom

List of curves from the Transmission inputted to the Curve or PB_Input terminal. See PeakDetect node

Top

Mode buttons: Set the current interactive mode for mouse events.

View: Just view, pan, and zoom the plot.

Drag: Click and drag peaks/bases along the curve.

Add Peak/Base: Click to add a peak/base onto the curve.

Delete: Delete a peak or base.

Dot Size: Move the slider to change the size of the dots representing peaks/bases.

Set Peaks at relative maxima: Not implemented yet.

Save Curve: Save the current curve. A curve auto-saved when you switch to another one.

Clear Curve: Not implemented.

Disconnect from flowchart: Disconnect the GUI from changes in the flowchart. Edits to the peaks/bases will be lost if this GUI is not disconnected while changes occur in the flowchart.

Send all curves to flowchart: Set the edited data as the output of the parent PeakDetect node

Open in viewer: Open the parent Sample of this curve in a Viewer.

Right

History Tree Widget

Proportions

API Reference

Note

This plot can be saved in an interactive form

Compare proportions of categorical variables between different groups using bar charts.

Parameter	Description
X column	DataFrame column containing the categorical labels used for grouping the data Data in each X column sums to 100% if Show percentages is checked
Y column	DataFrame column containing the categorical labels that are counted for each group
Show percentages	When unchecked shows raw counts
bar plot	Visualize as bar plots
heatmap	Visualize as a heatmap
Update Plot	Update plot
Swap X-Y	Swap X & Y columns
Save	Save this plot as a ptrn file
Load	Load from a ptrn file
Export CSV	Export the data for the current plot as to a csv file.
Show Console	Show/hide the console

Scatter

API Reference

Interactive scatter plot

Note

This plot can be saved in an interactive form

Layout

Left: Controls

Control
Data Column	Data column containing numerical arrays of size 2, X & Y values [x, y]
X	Data column containing only X values
Y	Data column containing only Y values
log x	Use \(log_{10}\) of the X data
log y	Use \(log_{10}\) of the Y data
Colors based on	Set spot colors based on categorical labels in this column
Choose colormap	Colromap for the the spot colors
Shapes based on	Set spot shapes based on categorical labels in this column
UUID Column	Column containing UUIDs that correspond to the plot data
DPT Curve column	Data column containing numerical arrays to show in the Datapoint Tracer
Spot size	Size of the spots
Alpha	Not implemented yet
Live update…	Update the plot with live inputs from the flowchart
Update Plot	Update the plot according to the input data from the flowchart and the parameters
Save	Save the plot as a ptrn file
Load	Load a saved ptrn file
Export to ma…	Not implemented yet
Export data	Not implemented yet

Below the plot: Status label that displays plotting issues. Click the label to see more information.

Right: Datapoint Tracer. Click datapoints in the plot to set the Datapoint Tracer.

Bottom: Console

Simple Plot

Just a very basic time-series plot. It will plot all the data in the selected data column.

SpaceMap

API Reference

Note

This plot can be saved in an interactive form, see Saving plots

Spatially map a categorical variable onto a projection of a Sample’s image sequence

Note

Image produced from the following dataset: Garner, Aleena (2014): In vivo calcium imaging of layer 4 cells in the mouse using sinusoidal grating stimuli. CRCNS.org. http://dx.doi.org/10.6080/K0C8276G

Controls

Parameter	Description
Patch labels	Categorical column to use for the patch labels
Image Colormap	Colormap for the image
Patches Colormap	Colormap for the patches
Projection	Show the image as a “Max” or “Standard Deviation” projection
Fill Patches	Fill the patches
Line width	Line width of the patches
Alpha	Alpha level of the patches
Samples	Click on the sample to plot
Save	Save the plot data and state in an interactive form
Load	Load a plot that has been saved as a “.ptrn” file.

Console

See also

API Reference

Namespace

reference	Description
this	The SpaceMapWidget instance, i.e. the entire widget
this.transmission	Current input Transmission
get_plot()	Returns the plot area
get_plot().fig	Returns the matplotlib Figure instance
get_plot().ax	Returns the Axes for the current plot matplotlib Axes

Examples

Export

See also

matplotlib API for: Figure.savefig, Figure.set_size_inches, Figure.get_size_inches

# Desired size (width, height)
size = (6,5)

# Get the figure
fig = get_plot().fig

# original size to reset the figure after we save it
orig_size = fig.get_size_inches()

#Set the desired size
fig.set_size_inches(size)

# Save the figure as a png file with 600 dpi
fig.savefig('/share/data/temp/kushal/spacemap.png', dpi=600, bbox_inches='tight', pad_inches=0)

# Reset to original size and draw
fig.set_size_inches(orig_size)
get_plot().draw()

Note

The entire plot area might go gray after the figure is reset to the original size. I think this is a Qt-matplotlib issue. Just resize the window a bit and the plot will be visible again!

Legend Title

See also

matplotlib API for matplotlib.axes.Axes.get_legend

get_plot().ax.get_legend().set_title('New Title')
get_plot().draw()

Hide Axis Borders

See also

matplotlib API for matplotlib.axes.Axes.axis

get_plot().ax.axis('off')
get_plot().draw()

Welcome Window

The Welcome Window is the first window that you are presented with when you launch Mesmerize.

• Use the large buttons for opening new Viewer or Flowchart windows.

• Open a project using the button, or double-click a recent project from the list.

• Create a new project using the button.

• You basically have access to all objects in the Mesmerize instance through this console.

See also

User guide on creating new projects and Consoles

Project Structure

A Mesmerize project is encapsulated within a single directory. It contains the following:

• config file - contains configuration data, such as roi type columns, stimulus type columns, and custom columns with their datatypes.

Warning

Do not manually modify the config file

Directories

Dir	Purpose
dataframes	Contains an file storing the project dataframe, root.dfr, and backups. A new backup is created every time a Sample is added to the project. Restore a backup by renaming it to “root.dfr”.
images	Contains the image sequences and work environment data for all samples in the project
curves	Contains the curves for every sample in the project
batches	Used for storing batches used by the Batch Manager if you wish.
flowcharts	Used for storing .fc flowchart files that save the layout of nodes in a flowchart.
plots	Used for storing .ptrn interactive plot files.

See also

Flowchart Overview and Saving Plots

Warning

Do not manually modify the data under the images or curves directories

Project Sample

Consoles

A Python console is embedded in many parts of Mesmerize. You can use it to perform very specific operations, further automate tasks, save an analysis object, format plots, etc.

The console is accessible in many windows through View -> Console. Within the console namespace this refers to the window. For example this refers to the Project Browser Window instance in the Project Browser’s console. A list of useful object references and helper functions are listed when you open most consoles.

You can run entire scripts within the console. You can also use import statements to import libraries that you have in your Python environment.

Keyboard controls:

Execute: Shift + Enter

New line: Enter

Scroll up through history: Page Up

Scroll down through history: Page Down

The history is stored in ~/.mesmerize

Saving plots

Some plots allow you to save them in an interactive form, along with the plot data and the plot state as a “.ptrn” file. If you save the file in the “plots” directory of your project it will be listed in the Welcome Window when you open your project.

This is currently possible with the following plots: Heatmap, KShape, Proportions, Scatter, and SpaceMap

Plot Navbar

Many plots have a navigation toolbar which you can use to zoom, pan, configure plots, and export plots as images.

Official matplotlib docs about the navigation toolbar: https://matplotlib.org/2.1.2/users/navigation_toolbar.html

Home: Reset the plot (not applicable for all plots)

Pan: Pan the plot

Zoom: Zoom in/out a selection using the left/right mouse button respectively

Subplot-configuration: Options to adjust spacing, borders, set tight layout.

Edit axis, curve…: For some plots. Options for formating x & y axis limits, labels, select line style, color, etc.

Save: Export the figure as an image. This is not the same as saving an interactive plot, see “Saving Plots” above.

System Configuration

Set system configuration options

This window is accessible through the Welcome Window menubar at Configuration -> System Configuration.

Maximum number of threads to use: The maximum number of threads that Mesmerize is allowed to use, this includes proccesses started by the Batch Manager, various analysis proccesses in the flowchart, and the viewer as well.

Python call: Many parts of Mesmerize, such as the Batch Manager use external processes to run a python script. This setting sets which python call should be used. The default setting of “python3” should work for both snap and pip installations unless you have written a custom expansion that uses python2.

Use CUDA: Use CUDA accerlation if you have a GPU with CUDA cores. Only works with the pip installation, and you must have pycuda and scikit-cuda (as well as the nvidia CUDA toolkit) installed. In Mesmerize CUDA is currently used only by Caiman Motion Correction. We plan to expand CUDA support to computationally intensive tasks that are performed by flowchart nodes.

Work dir: Many parts of Mesmerize use a working directory for temporary files. If you have a fast filesystem you can use that for this purpose.

Pre-run commands (large text entry): Mesmerize runs some computationally intensive tasks in subprocesses. These commands are run prior to the python script that performs the task.

• If you are using Mesmerize in a virtual environment (such as a pip installed Mesmerize) you will need activate the environment so you must include the line source /path_to_venv/bin/activate to the pre-run commands

• Simiarly if you are using Mesmerize in an Anaconda environment you will need include commands to activate the environment.

• If you are using an Intel CPU you should get optimal performance by installing Math Kernel Library (MKL) and including export MKL_NUM_THREADS=1 to the pre-run commands.

• If you are using an AMD CPU make sure you have OpenBLAS installed for optimal performance and include export OPENBLAS_NUM_THREADS=1 to the pre-run commands. You may better performance by installing the AMD specific libraries.

Plugins directory: If you have a plugins dir include enter its path here.

Indices and tables

• Index

• Module Index

• Search Page