Recording Optical Flow with the Zeiss LSM software Multi Time Series module

Acquisition in Zeiss LSM Multi Time Series module

Recording noise characteristics

The aim here is to record a set of evenly illuminated fields at different intensities from zero to close to saturation. This have to be performed at completely identical detector settings to the Optical Flow recording.

1. To record noise characteristics use completely identical detector settings as used in experiments, and use the same PMT.
2. If Optical Flow experiments have been already recorded use Reuse to load their settings.
3. Pinhole, lens, filters and laser lines do not have to be identical.
4. Mount a fluorescent plastic slide on the microscope. Select a proper configuration for the fluorescence of the slide.
5. Focus the slide to have as even illumination in the image as possible (in the middle of the slide).
   
6. Record a time series using the Time Series Control window setting sufficient delay in between frames, to be able to change laser intensity settings:
   1. Start with zero illumination; no laser turned on.
   2. Turn on, and gradually increase laser intensity during the wait phases between frames.
   3. It is sufficient to use illumination levels up to it results in similar pixel intensities to the observed intensities during experiments. Do not saturate.
7. Save acquired time lapse lsm file.

Recording Optical Flow using the Multi Time Series module

The best way for Optical Flow recording in the Zeiss LSM software to use the Multi Time Series module Block mode feature.
Alternatively Optical Flow can be recorded in continuous mode in single positions using the Time Series Control window.

Settings for regular Time Series:

1. Acquisition parameters should result (optimally) 512x512 pixels images at ~0.2-0.3 mm/pixel resolution and the scan time is around 1s or less to achieve similar time lapse acquisition interval.
2. Setting up Scan Configurations
   1. Use Single Track to prevent motion due to motion of optical elements.
   2. Because the Optical Flow calculation depends on the noise parameters of the detector, the gains, offsets, scan speed, averaging and image size should not be varied between experiments, unless the noise characteristics is measured for each setting. The noise parameters do not depend on the pinhole settings and on the filter/ dichroic mirror settings, or laser intensities.
   3. Pinhole: Optical Flow benefits from (relatively) open pinhole. Do not use small pinhole unless the experiment benefits from the optical sectioning.
   4. If using averaging, use line averaging. Do not use frame averaging. Be careful if using bi-directional scanning, the two scan directions have to perfectly register.
   5. Set the scan speed close to, but shorter the intended acquisition period.
3. Start Time lapse.
4. Experiments can be stopped for additions, and continued as new experiments, these can be merged in Image Analyst MKII.

Settings for Multi Time Series:

1. Acquisition parameters should result (optimally) 512x512 pixels images at ~0.2-0.3 mm/pixel resolution and the scan time is around 1s or less to achieve similar short time lapse acquisition interval.
2. Setting up Scan Configurations
   1. Use Multi Track configuration, and save separate configurations for:
      1. Channels to record before Optical Flow
      2. Optical Flow. It is important to use the same color associations as for #1, because the blocks will be concatenated based on color. Use only one track in the Optical Flow configuration.
      3. Autofocus
   2. Because the Optical Flow calculation depends on the noise parameters of the detector, the gains, offsets, scan speed, averaging and image size should not be varied between experiments, unless the noise characteristics is measured for each setting. The noise parameters do not depend on the pinhole settings and on the filter/ dichroic mirror settings, or laser intensities.
   3. Pinhole: Optical Flow benefits from (relatively) open pinhole. Do not use small pinhole unless the experiment benefits from the optical sectioning.
   4. If using averaging, use line averaging. Do not use frame averaging. Be careful if using bi-directional scanning, the two scan directions have to perfectly register.
   5. Set the scan speed close to, but shorter the intended acquisition period.
3. Start the Multi Time Series macro module
4. In the List of Blocks panel:
   1. Set it to L-GR-G mode
   2. Set the number of time points in the experiment at the Experiment Repetitions.
   3. If other channels are to be recorded before Optical Flow recording, press Add Block.
      • All blocks have to be in the same Group, so select the first block, and press Start Group, then select the last (second) block and press End Group.
   4. Select the Block 1
      1. In the Block 1 Parameters/Wait Interval set the acquisition interval of the experiment (not the short time lapses) here. This value will appear in the first row of the List of Blocks/BkIntv
      2. Define Configuration and Autofocus for Block 1.
      3. If acquiring only Optical Flow frames, in the Configuration panel set the No of Scans to 2 and set the delay time of the short time lapse:
         The frame interval is given by the delay time plus the time to acquire the frame. E.g. to keep 1 s interval, if the acquisition time (see at the Scan Control) is ~1s then the delay is 0.
   5. Select the Block 2 (if present)
      1. Block 1 Parameters/Wait Interval=0
      2. Define Configuration but add no Autofocus here
      3. In the Configuration set No of Scans to 2 and set the delay time of the short time lapse:
         The frame interval is given by the delay time plus the time to acquire the frame. E.g. to keep 1 s interval, if the acquisition time (see at the Scan Control) is ~1s then the delay is 0.
5. If Multiple Locations were set up before block configuration, copy the above settings to each location by switching to Fixed Location, then back to Multiple Locations
6. In the Options dialog enable the following items: Use channel color as criteria to Concatenate...., Wait Interval
7. Set up autofocus offsets 
8. Start Time lapse.
9. Experiments can be stopped for additions, and continued as new experiments, these can be merged in Image Analyst MKII.

Use Multi Track to define configurations. This is example for the configuration associated to Block 1, that is acquired before optical flow. In Block 2 only those channels will be recorded which have a color that is present in Block 1.	This configures GFP emission for the Optical Flow frames. The green color of Ch2 was used in the TMRM configuration as Ch1 on on the left.	Settings for Optical Flow: 512x512 resolution Line averaging or no averaging Scan time will set the short time lapse interval, so it's around 1 s. Proper zoom to achieve ~0.2-0.3 mm/pixel resolution.

Block 1 Press Start Group Enter Parameters/Wait Interval Define Autofocus	Block 1 (if channels to be recorded before Optical Flow, in this case TMRM) Define Configuration for channels preceding Optical Flow acquisition Number of Scans (XY/View): 1 Delay (XY/View): 0	Block 2 Press End Group (make sure that the values in the Groups and G.Rep columns in the List of Blocks table are all 1) Enter Parameters/Wait Interval Define Configuration for Optical Flow Number of Scans (XY/View): 2 Delay (XY/View): delay of Optical Flow (see above)

Enable: Use channel color as criteria to Concatenate Wait Interval

The protocol is based on Zeiss LSM 510 V4.2 SP1  and Multi Time Series 4.0.23Beta

Analysis in Image Analyst MKII

Analyzing noise characteristics

1. Open the noise characteristics file recorded above
2. Set LUT scaling to frame-by-frame in the Set scaling menu point of context menu of the Image Window (check Scale each frame independently)
3. Look for a small part of the image where the illumination is the most even. Draw a small ROI here (~20x20 pixels, or larger if the field is quite even)
4. Select the Sensor Noise Characteristics in the Special main menu.
5. In the Parameter Bar, set the 'Set values in Optical Flow functions' parameter to Yes.
6. In the context menu of the Image Window click Process This with Noise Characteristics; A Plot and a Text window appear.
7. The content of the Plot window is the intensity-variance relationship of the pixels within the ROI. This has to be a straight line. If it is not linear:
   1. Frames have to be in the order of increasing intensity
   2. Delete any saturated frames.
   3. Nonlinearity may be caused by uneven illumination. Move the ROI around to find a linear spot.
   4. Try to draw a smaller ROI.
8. The function automatically sets the following parameters of the Optical Flow function:
   • Detector offset (mean of the zero illumination image intensity)
   • Detector variance vs. intensity Slope (slope of the Plot Window)
   • Detector Read out Variance (variance at the zero  illumination)
9. The above values will be stored when exiting Image Analyst, or click Edit/Save Preferences in the main menu.

Noise curve of a Zeiss LSM 510 laser scanning confocal microscope at scan speed 9, bidirectional scanning, 2x line averaging, detector gain: 500 offset:0 amplifier gain 1. The image on the left was scaled between its 1 and 99 percentiles, therefore shows inhomogeneities amplified.		Offset: 214.33 Variance vs. intensity Slope: 9.0728 Readout noise (variance σ2): 4.9604 ------------------------------------------------------- Electrons per gray unit: 0.1102 Readout noise (e-;RMS): 0.7394

Analyzing Optical Flow from Multi Time Series recordings

1. Open the first  (*_Sum.lsm) file. Image Analyst will scan the folder for all stage positions. If the experiment consists multiple *_Sum.lsm file sets recorded sequentially in time then they can be merged in time by multiple selection. The Multi-Dimensional Open dialog appears.
2. Switch to the Settings tab:
   1. Check Separate Blocks... and Treat as Z or OF.
   2. Enter the number of frames in each block. These are the numbers in the Scan column of the List of Blocks in the Zeiss Multi Time Series window. Enter these numbers separated by commas (in the above example 1,2).
   3. The Load specified frames of each stack... has to be unchecked, unless you have recorded more frames in the short time lapses than you will use for analysis. If more frames are processed than the width of the dt (temporal differentiation) kernel, thus when the recording is longer than the width of dt kernel and Load specified frames of each stack is not set to match the width, multiple velocity images are calculated, and the result will be obtained by using projection as given in the Project Z field.
   4. The Load specified frames of the time lapse feature can be used.
3. In Open tab: the channels are now split to show separate blocks separately. Select only the channel/block  used for the Optical Flow recording. Select Optical Flow in the Processing panel.
4. In the OpticalFlow load tab the parameters of the Optical Flow function are listed. The following parameters may have to be set here:
   1. Select dt kernel: [1,-1]  (to match the length of short time lapses of two frames)
      • If the block size is greater than 2, set [Savitzky-Golay first derivative] here and enter the size of the block at the SG kernel for dt width, and enter No at #2 below.
   2. Average OF for dt width: Yes (dt kernel of width of two always used with averaging to avoid biasing between leading and trailing edges)
   3. Block mode: No (each short time lapse is separately processed, so there is no need for block mode when using the Multi-Dimensional Open dialog)
   4. Pixel size: (the mm/pixel calibration can be given here to obtain velocities in mm/s rather than in pixel/s. 1 results output in pixels/s. Use the context menu  Show Image Info of an Image Window, or the Tools/Setup DFT filter to determine scaling)
   5. Output as... (enable the desired kind of outputs; as default only absolute velocities are calculated)
   6. Output as Absolute value of Projected Vectors: If Yes, velocity vectors are projected to a point ROI. This can be used to assess anterograde transport (away from the point ROI) by positive velocities and retrograde transport (towards the ROI) by negative values. When using this feature first (before #3) load the image series by setting the Processing panel to None in the Open tab. Draw ROI on the opened image. Then follow the above protocol form #3. Set the ROI No. in the Projection ROI parameter. The ROIs are automatically copied from the last open image during Optical Flow open.
   7. Other parameters: noise parameters were filled in above. Fine tuning of other parameters see here.
5. Above settings are valid as long as the dialog is open, or can be stored by the Set as Default button.
6. Click Open to perform loading and processing.
7. The default LUT of the Optical Flow image is pseudocolor, and can be set in the Preferences dialog.

The resultant Optical Flow image consists of pseudocolored pixels where Optical Flow determination was feasible based on the noise characteristics (there was enough image detail to distinguish movement from noise), and black mask where not. The unit of the Optical Flow image is pixel/s, or mm/s  if the Pixel size is set above.

Analyzing Optical Flow from simple time lapse recordings (see figure about block mode)

1. Open lsm file in the File/Open image series/measurement.  Importantly, this section is only valid for time lapses recorded without stage movement.
2. Background must not be subtracted. The original background level is required masking of Optical Flow images.
3. Select the Optical Flow function in the main menu Special are listed. The following parameters may have to be set in the parameter bar:
   1. Select dt kernel: [1,-1] or set the width of blocks if the recording was in block mode.
      • If the block size is greater than 2, set [Savitzky-Golay first derivative] here and enter the size of the block at the SG kernel for dt width, and enter No at #2 below.
   2. Average OF for dt width: Yes (dt kernel of width of two always used with averaging to avoid biasing between leading and trailing edges. Set No if using wider kernel)
   3. Block mode: if the experiment was recorded with an even frame rate around 1s/frame or less set No. If the experiment was recorded as frames (equal number of the width of the dt kernel at short interval, then pause for an arbitrary time, and then this is cyclically repeating, set Yes.
   4. Pixel size: (the mm/pixel calibration can be given here to obtain velocities in mm/s rather than in pixel/s. 1 results output in pixels/s. Use the context menu  Show Image Info of an Image Window to determine scaling)
   5. Output as... (enable the desired kind of outputs; as default only absolute velocities are calculated)
   6. Output as Absolute value of Projected Vectors: If Yes, velocity vectors are projected to a point ROI. This can be used to assess anterograde transport (away from the point ROI) by positive velocities and retrograde transport (towards the ROI) by negative values. When using this feature first (before #3) load the image series by setting the Processing panel to None in the Open tab. Draw ROI on the opened image. Then follow the above protocol form #3. Set the ROI No. in the Projection ROI parameter. The ROIs are automatically copied from the last open image during Optical Flow open.
   7. Other parameters: noise parameters were filled in above. Fine tuning of other parameters see here.
4. In the context menu of the Image Window click Process This with Optical Flow.
5. The default LUT of the Optical Flow image is pseudocolor, and can be set in the Preferences dialog.

Select the Optical Flow function in the Special menu. if the experiment was recorded with an even frame rate around 1s/frame or less set No for the Block Mode. If the experiment was recorded as frames (equal number of the width of the dt kernel at short interval, then pause for an arbitrary time, and then this is cyclically repeating, set Yes for the Block Mode.

Fine tuning optical flow (see here)

Example

Example lsm files (27MB, zip compressed)
Download and uncompress data on your hard drive.
See more details of working with Optical Flow images here.
Calculation of Optical Flow from the example image set:

1. In the main menu select File/Open image series/measurement ,set file type to "*.lsm"  and open noise.lsm in the “Noise Characteristics” folder.
2. Cut the last two frames because of nonlinearity close to saturation using the toolbar icon.
3. Follow the points in the Analyzing noise characteristics section above.
4. Close images by File/Close all.
5. In the main menu select File/Open image series/measurement, set file type to "*_Sum.lsm" and open the file in the “Mitochondrial Motion” folder.
6. Set Settings and and Optical Flow tabs as shown above (the noise parameters should be automatically entered by now)
7. Switch back to the Open tab, select stage position 1, only channel 1 block 2 and Click Open. Inspect the image sequence.
   Channel 1 block 2 contains the short time lapse recorded as an image stack for each time point. (technically the example recording had short time lapses of 3 frames and no other recording before the short time lapse)
8. The Pixels size can be obtained by the Show image info in the context menu of the Image Window.
9. Select Optical Flow in Processing and press Open again.
10. Draw a ROI around the neuron and press .

To process the same image file as a regular, non-Multi-Dimensionalrecording:

1. Follow the noise analysis above.
2. In the main menu select File/Open image series/measurement, set file type to "*.lsm" and open the file in the “Mitochondrial Motion” folder. This recording was performed in block mode, with 3 frames per block, so the short time lapses consists of 3 frames. Of note, recording only 2 frames is sufficient for Optical Flow calculation.
3. Select the Optical Flow and set parameters similarly as above plus set:
   1. Select dt kernel: [Savitzky-Golay first derivative]
   2. Average OF for dt width: No
   3. Block mode: Yes
   4. SG kernel for dt width: 3
4. Process e.g. by using the context menu of the Image Window.
5. Draw a ROI around the neuron and press .

Frame 10 of projection image Hippocampal neuron expressing mito-roGFP1, acquired by a Zeiss LSM 510	Frame 10 of Absolute Velocity Image	Mean absolute velocity over the encircled area in the images. At the end of the time lapse 4% paraformaldheide was added to the cells. Not do to remaining restricted Brownian motion the motion is not zero at the end. The y-axis is scaled in mm/sec.

Protocol by Akos A. Gerencser 08/10/2010 V1.1        

References

1. Gerencser A. A. and Nicholls D. G. (2008) Measurement of Instantaneous Velocity Vectors of Organelle Transport: Mitochondrial Transport and Bioenergetics in Hippocampal Neurons. Biophys J. 2008 Sep 15;95(6):3079-99.