derotation.analysis.full_derotation_pipeline#

FullPipeline is the main pipeline meant to be used for the full rotation protocol, which involves rotating the sample by 360 degrees at various speeds and directions.

Classes

FullPipeline(_config)

FullPipeline is a class that derotates an image stack acquired with a rotating sample under a microscope.

class derotation.analysis.full_derotation_pipeline.FullPipeline(_config)[source]#

FullPipeline is a class that derotates an image stack acquired with a rotating sample under a microscope.

It is meant to be used for the full rotation protocol, in which the sample is rotated by 360 degrees at various speeds and directions.

It involves:

processing the analog signals
finding the offset of the image stack
setting the optimal center of rotation
derotating the image stack
masking the images
calculating the mean images
evaluating the quality of the derotation
saving the derotated image stack and the csv file

In the constructor, it loads the config file, starts the logging process, and loads the data.

Parameters:: _config (Union[dict, str]) – Name of the config file without extension that will be retrieved in the derotation/config folder, or the config dictionary.

Methods

`__call__`()	Execute the steps necessary to derotate the image stack from start to finish.
`add_circle_mask`(image_stack[, diameter])	Adds a circular mask to the rotated image stack.
`adjust_rotation_increment`()	It calculates the new rotation increment for each rotation, given the number of ticks in each rotation.
`calculate_angles_by_line_and_frame`()	From the interpolated angles, it calculates the rotation angles by line and frame.
`calculate_velocity`()	Calculates the velocity of the rotation by line.
`check_number_of_rotations`()	Checks that the number of rotations is as expected.
`check_rotation_number_after_interpolation`(...)	Checks that the number of rotations is as expected.
`clock_to_latest_frame_start`(clock_time)	Get the index of the frame that is being acquired at the given clock time.
`clock_to_latest_line_start`(clock_time)	Get the index of the line that is being scanned at the given clock time.
`clock_to_latest_rotation_start`(clock_time)	Get the index of the latest rotation that happened.
`correct_start_and_end_rotation_signal`(start, end)	Removes artifacts from the start and end times of the on periods of the rotation signal.
`create_signed_rotation_array`()	Reconstructs an array that has the same length as the full rotation signal.
`derotate_frames_line_by_line`()	Wrapper for the function `derotate_an_image_array_line_by_line`.
`drop_ticks_outside_of_rotation`()	Drops the rotation ticks that are outside of the rotation periods.
`find_image_offset`(img)	Find the "F0", also called "image offset" for a given image.
`find_missing_rotation_on_periods`()	Find the missing rotation on periods by looking at the rotation ticks and the rotation on signal.
`find_optimal_parameters`()	Finds the optimal parameters for the derotation.
`find_rotation_peaks`()	Finds the peaks of the rotation ticks signal using scipy.signal.find_peaks.
`get_config`(config_name)	Loads config file from derotation/config folder.
`get_interpolated_angles`()	Starting from the rotation ticks and knowing the rotation increment, it calculates the rotation angles for each clock time.
`get_peaks_in_rotation`(start, end)	Counts the number of ticks in a rotation given the start and end times of the rotation.
`get_start_end_times_with_threshold`(signal, ...)	Finds the start and end times of the on periods of the signal.
`is_number_of_ticks_correct`()	Compares the total number of ticks with the expected number of ticks, which is calculated from the number of rotations and the rotation increment.
`load_data`()	Loads data from the paths specified in the config file.
`mean_image_for_each_rotation`(...)	Calculates the mean image for each rotation and saves it in the `debug_plots` folder.
`plot_max_projection_with_center`(stack[, name])	Plots the maximum projection of the image stack with the center of rotation.
`plot_rotation_angles_and_velocity`()	Plots example rotation angles by line and frame for each speed.
`plot_rotation_on_and_ticks`()	Plots the rotation ticks and the rotation on signal.
`plot_rotation_speeds`()	Plots the velocity of the rotation for each speed.
`process_analog_signals`()	From the analog signals (frame clock, line clock, full rotation, rotation ticks) calculates the rotation angles by line and frame.
`remove_artifacts_from_interpolated_angles`()	Removes artifacts from the interpolated angles, coming from an inconsistency between the number of ticks and cumulative sum.
`save`(masked)	Saves the masked image stack in the saving folder specified in the config file.
`save_csv_with_derotation_data`()	Saves a csv file with the rotation angles by line and frame, and the rotation on signal.
`set_optimal_center`()	Checks if the optimal center of rotation is calculated.
`start_logging`()	Starts logging process using fancylog package.

get_config(config_name)[source]#

Loads config file from derotation/config folder. Please edit it to change the parameters of the analysis.

Parameters:: config_name (str) – Name of the config file without extension. Either “full_rotation” or “incremental_rotation”.
Returns:: Config dictionary.
Return type:: dict

start_logging()[source]#: Starts logging process using fancylog package. Logs saved where specified in the config file.

load_data()[source]#

Loads data from the paths specified in the config file. Data is stored in the class attributes.

What is loaded:

various parameters from config file
image stack (tif file)
direction and speed of rotation (from randperm file, uses custom_data_loaders.read_randomized_stim_table)
analog signals (from aux file, uses custom_data_loaders.get_analog_signals)

Analog signals are four files, measured in “clock_time”:

frame_clock: on during acquisition of a new frame, off otherwise
line_clock: on during acquisition of a new line, off otherwise
full_rotation: when the motor is rotating
rotation_ticks: peaks at every given increment of rotation

The data is loaded using the custom_data_loaders module, which are specific to the setup used in the lab. Please edit them to load data from your setup.

process_analog_signals()[source]#

From the analog signals (frame clock, line clock, full rotation, rotation ticks) calculates the rotation angles by line and frame.

It involves:

finding rotation ticks peaks
identifying the rotation ticks that correspond to clockwise and counter clockwise rotations
removing various kinds of artifacts that derive from wrong ticks
interpolating the angles between the ticks
calculating the angles by line and frame

If debugging_plots is True, it also plots:

rotation ticks and the rotation on signal
rotation angles by line and frame

find_rotation_peaks()[source]#

Finds the peaks of the rotation ticks signal using scipy.signal.find_peaks. It filters the peaks using the height and distance parameters specified in the config file.

Returns:: the clock times of the rotation ticks peaks
Return type:: np.ndarray

static get_start_end_times_with_threshold(signal, std_coef)[source]#

Finds the start and end times of the on periods of the signal. Works for analog signals that have a squared pulse shape.

Parameters:

signal (np.ndarray) – An analog signal.
std_coef (float) – The factor used to quantify the threshold.

Returns:

The start and end times of the on periods of the signal.

Return type:

Tuple(np.ndarray, np.ndarray)

correct_start_and_end_rotation_signal(start, end)[source]#

Removes artifacts from the start and end times of the on periods of the rotation signal. These artifacts appear as very brief off periods that are not plausible given the experimental setup. The two surrounding on periods are merged.

Used the inter_rotation_interval_min_len parameter from the config file: the minimum length of the time in between two rotations. It is important to remove artifacts.

Parameters:

start (np.ndarray) – The start times of the on periods of rotation signal.
end (np.ndarray) – The end times of the on periods of rotation signal.

Returns:

Corrected start and end times of the on periods of rotation signal.

Return type:

dict

create_signed_rotation_array()[source]#

Reconstructs an array that has the same length as the full rotation signal. It is 0 when the motor is off, and it is 1 or -1 when the motor is on, depending on the direction of rotation. 1 is clockwise, -1 is counter clockwise. Uses the start and end times of the on periods of rotation signal, and the direction of rotation to reconstruct the array.

Returns:: The rotation on signal.
Return type:: np.ndarray

drop_ticks_outside_of_rotation()[source]#

Drops the rotation ticks that are outside of the rotation periods.

Returns:: The clock times of the rotation ticks peaks, without the ticks outside of the rotation periods.
Return type:: np.ndarray

check_number_of_rotations()[source]#

Checks that the number of rotations is as expected.

Raises:

ValueError – if the number of start and end of rotations is different
ValueError – if the number of rotations is not as expected

find_missing_rotation_on_periods()[source]#: Find the missing rotation on periods by looking at the rotation ticks and the rotation on signal. This is useful when the number of rotations is not as expected. Uses k-means to cluster the ticks and pick the first and last for each cluster. These are the starting and ending times.

is_number_of_ticks_correct()[source]#

Compares the total number of ticks with the expected number of ticks, which is calculated from the number of rotations and the rotation increment.

Returns:: whether the number of ticks is as expected
Return type:: bool

get_peaks_in_rotation(start, end)[source]#

Counts the number of ticks in a rotation given the start and end times of the rotation.

Parameters:

start (int) – Start clock time of the rotation.
end (int) – End clock time of the rotation.

Returns:

The number of ticks in the rotation.

Return type:

int

adjust_rotation_increment()[source]#

It calculates the new rotation increment for each rotation, given the number of ticks in each rotation. It also outputs the number of ticks in each rotation.

Returns:: The new rotation increment for each rotation, and the number of ticks in each rotation.
Return type:: Tuple[np.ndarray, np.ndarray]

get_interpolated_angles()[source]#

Starting from the rotation ticks and knowing the rotation increment, it calculates the rotation angles for each clock time.

Returns:: The rotation angles for each clock time.
Return type:: np.ndarray

remove_artifacts_from_interpolated_angles()[source]#: Removes artifacts from the interpolated angles, coming from an inconsistency between the number of ticks and cumulative sum. These artifacts appear as very brief rotation periods that are not plausible given the experimental setup.

check_rotation_number_after_interpolation(start, end)[source]#

Checks that the number of rotations is as expected.

Raises:

ValueError – if the number of start and end of rotations is different
ValueError – if the number of rotations is not as expected

calculate_angles_by_line_and_frame()[source]#

From the interpolated angles, it calculates the rotation angles by line and frame. It can use the start or the end of the line/frame to infer the angle.

Returns:: The rotation angles by line and frame.
Return type:: Tuple[np.ndarray, np.ndarray]

clock_to_latest_line_start(clock_time)[source]#

Get the index of the line that is being scanned at the given clock time.

Parameters:: clock_time (int) – The clock time.
Returns:: The index of the line
Return type:: int

clock_to_latest_frame_start(clock_time)[source]#

Get the index of the frame that is being acquired at the given clock time.

Parameters:: clock_time (int) – The clock time.
Returns:: The index of the frame
Return type:: int

clock_to_latest_rotation_start(clock_time)[source]#

Get the index of the latest rotation that happened.

Parameters:: clock_time (int) – The clock time.
Returns:: The index of the latest rotation
Return type:: int

calculate_velocity()[source]#: Calculates the velocity of the rotation by line.

plot_rotation_on_and_ticks()[source]#: Plots the rotation ticks and the rotation on signal. This plot will be saved in the debug_plots folder. Please inspect it to check that the rotation ticks are correctly placed during the times in which the motor is rotating.

plot_rotation_angles_and_velocity()[source]#

Plots example rotation angles by line and frame for each speed. The velocity is also plotted on top of the rotation angles.

This plot will be saved in the debug_plots folder. Please inspect it to check that the rotation angles are correctly calculated.

plot_rotation_speeds()[source]#: Plots the velocity of the rotation for each speed. This plot will be saved in the debug_plots folder. Please inspect it to check that the velocity is correctly calculated.

find_optimal_parameters()[source]#: Finds the optimal parameters for the derotation. It calls the Bayesian Optimization algorithm implemented in BO_for_derotation.

set_optimal_center()[source]#: Checks if the optimal center of rotation is calculated. If it is not calculated, it will calculate it.

plot_max_projection_with_center(stack, name='max_projection_with_center')[source]#: Plots the maximum projection of the image stack with the center of rotation. This plot will be saved in the debug_plots folder. Please inspect it to check that the center of rotation is correctly placed.

derotate_frames_line_by_line()[source]#

Wrapper for the function derotate_an_image_array_line_by_line. Before calling the function, it finds the F0 image offset with find_image_offset.

Returns:: The rotated image stack.
Return type:: np.ndarray

static find_image_offset(img)[source]#

Find the “F0”, also called “image offset” for a given image.

Explanations: What is the image offset? The PMT (photo-multiplier tube) adds an arbitrary offset to the image that corresponds to 0 photons received. We can use a Gaussian Mixture Model to find this offset by assuming that it will be the smallest mean of the Gaussian components.

Why do we need to find it? When we rotate the image, the pixels of the image that are not sampled will be filled with the offset is order to correctly express “0 photons received”.

Parameters:: img (np.ndarray) – The image for which you want to find the offset.
Returns:: The offset.
Return type:: float

mean_image_for_each_rotation(derotated_image_stack)[source]#: Calculates the mean image for each rotation and saves it in the debug_plots folder. This plot will be saved in the debug_plots folder. Please inspect it to check that the mean images are correctly calculated.

static add_circle_mask(image_stack, diameter=256)[source]#

Adds a circular mask to the rotated image stack. It is useful to hide the portions of the image that are not sampled equally during the rotation. If a diameter is specified, the image stack is cropped to match the diameter. The mask is then added to the cropped image stack, and the cropped image stack is padded to match the original size. This is important when the images are registered to correct from motion artifacts.

Parameters:

image_stack (np.ndarray) – The image stack that you want to mask.
diameter (int, optional) – The diameter of the circular mask, by default 256

Returns:

The masked image stack.

Return type:

np.ndarray

save(masked)[source]#

Saves the masked image stack in the saving folder specified in the config file.

Parameters:: masked (np.ndarray) – The masked derotated image stack.

save_csv_with_derotation_data()[source]#: Saves a csv file with the rotation angles by line and frame, and the rotation on signal. It is saved in the saving folder specified in the config file.