Recording Optical Flow with Metamorph

Acquisition in Metamorph

Recording noise characteristics

The aim here is to record a set of evenly illuminated fields at different intensities from zero to close to saturation. This have to be performed at completely identical camera settings to the Optical Flow recording.

1. To record noise characteristics use completely identical settings of the camera to the settings used in experiments.
2. Lens and filters do not have to be identical.
3. Mount a fluorescent plastic slide on the microscope. Select a proper optical configuration for the fluorescence spectrum.
4. Focus it to have as even illumination in the image as possible.
   Alternatively the objective lens can be removed and the slide is placed on the nosepiece, in the back focal plane of the lens.
5. Use the Acquire dialog of the Metamorph.
   1. Set proper illumination
   2. Set frames to be added to the existing stack
6. Start with zero illumination; simply acquire images while the light path is diverted from the camera, e.g. 100% to eyepiece.
7. Set light path to the camera and gradually increase illumination, frame-by-frame. For this:
   1. Use the intensity control of the illumination unit, e.g. Sutter Instruments Smart shutter
   2. or use the aperture diaphragm in the fluorescence illumination pathway
   3. or pull out and then gradually push back the fiber optics light guide
8. It is sufficient to use illumination levels up to it results in similar pixel intensities to the observed intensities during experiments. Do not saturate.
9. Save acquired set (time lapse) of images as an stk file.

Recording Optical Flow in Multi-DimensionalAcquisition (MDA)

There are two ways of recording Optical Flow with Metamorph

1. The best way for Optical Flow recording in Metamorph is to use a journal (macro) to record a short time lapse after the last channel was recorded. Pros: accurate timing and fast short time lapses are possible. Cons: needs more complicated journal programming.
2. Alternative way is to record z-stacks with zero step size. Pros: simple. Cons: significant hardware delay time, not suitable for fast motion (like axonal mitochondria when using with an Olympus IX-81 microscope)

The exact journal varies by the version of the Metamorph and with the actual hardware, but the principles are the following:

1. Short time lapse from journal

Create a journal: ‘OFacquire.jnl’ (this will record an extra time lapse at the end of each stage position):

1. Select Illumination – give the illumination name to be used for OF recording.
2. Acquire Timelapse
   • Time Interval 1 sec
   • Duration 1 sec
   • Number of planes 2
   • Image storage: stack
   • Destination: OF
   • Update Image Window
   • Illum: [Current Shutter]
3. Rename Image New Name= MDA.BaseName+"_w3OptFlow_s"+ STR(MDA.Status.StagePosNum)+"_t"+ STR(MDA.Status.TimePointNum)
   • Use instead of ‘OptFlow’ whatever describes the recording
   • The ‘w3’ in the name has to be set to the number of wavelengths set in the ‘MDA’ dialog + 1
4. Save [Last Result] // unfortunately in Metamorph 6.3 the only way of getting through the save dialog during measurement is putting a weight on the enter key.
   //In later versions there should be a setting to override the save dialog.
5. Close

Note, that this journal does not control exposure time. In Metamorph 6.3 the  Acquire form Digital Camera/Load Acquisition Settings and Activate Digital Camera Setting journals are useful to achieve full control over acquisition settings before the Acquire Timelapse journal command.
The advantage of this approach is that the recorded stk files provide proper timing.

Settings in the Multi-DimensionalAcquisition dialog of the Metamorph for short time lapsing from journal

1. Acquisition parameters should result (optimally) 512x512 pixels images at ~0.2-0.3 mm/pixel resolution. For example:
   • 60x lens, 1.0 Zoom, no binning of a 16mm pixel camera: 16/60=0.267 mm/pixel
   • 40x lens, 1.0 Zoom, 2x2 binning of a 6.4mm pixel camera: 2*6.4/40=0.32 mm/pixel
2. In Main tab check  Do Timelapse, Multiple Stage Positions, Multiple Wavelengths, Run Journals
3. Set up wavelengths to be acquired, but skip the wavelength of the Optical Flow.
   
4. Because the Optical Flow calculation depends on the noise parameters of the camera, the gain, multiplier, AD conversion clock (MHz) settings (if applicable for the camera in use) should not be varied between experiments, unless the noise characteristics is measured for each setting. The exposure time may be varied. The safest is to set intensities by varying illumination intensities.
5. In the above created OFacquire.jnl journal make sure that the renaming follows the number of wavelengths set in the MDA dialog. e.g. 3 wavelength recorded in the MDA dialog, and the OF is the fourth, then  New Name= MDA.BaseName+"_w4OptFlow_s"+ STR(MDA.Status.StagePosNum)+"_t"+ STR(MDA.Status.TimePointNum)
6. The time interval of the short time lapse is set in the OFacquire.jnl, and it has to be tuned according the application (see here).
7. In Journal tab set: End of Stage position: ‘OFacquire.jnl
8. The Time and Stage tabs can be arbitrarily set.
9. It is advised to use some kind of auto focusing mechanism. If the microscope lacks of active auto focus, set up a separate journal to perform software autofocus on fluorescence beads placed somewhere in the specimen. An example journal is shown below for auto focusing on beads.
10. If the focus of the stage has active feedback these may have to be turned off for the duration of the acquisition of the Optical Flow frames, to ensure that no focal change or stage movement happens due to fluttering of the active feedback mechanisms.
11. If the actual version of Metamorph is not capable of overriding save dialogs, don't forget to put a weight over the Enter key once the acquisition is started. It is also useful to turn off other softwares, including Microsoft Windows auto update to prevent pop up windows interfering with the image acquisition.

2. Short time lapse from z-stack

Create a journal: ‘delay.jnl’ (this will record an extra time lapse at the end of each stage position):

1. Add an IF-ELSE clause, with (MDA.Status.WaveName="rogfp438") AND (MDA.Status.ZStepNum=1) as condition,
   where the "rogfp438" is the name of the illumination configuration used to record Optical Flow.
2. In between the IF-THEN and the ELSE add Delay, specifying the delay in between the two frames of the short time lapse. Anticipate a significant hardware delay, depending on the make of the microscope.

This journal introduces a wait period only after the first frame of the short time lapse.

Settings in the Multi-DimensionalAcquisition dialog of the Metamorph for short time lapsing as z-stacking

1. Acquisition parameters should result (optimally) 512x512 pixels images at ~0.2-0.3 mm/pixel resolution. For example:
   • 60x lens, 1.0 Zoom, no binning of a 16mm pixel camera: 16/60=0.267 mm/pixel
   • 40x lens, 1.0 Zoom, 2x2 binning of a 6.4mm pixel camera: 2*6.4/40=0.32 mm/pixel
2. In Main tab check  Do Timelapse, Multiple Stage Positions, Multiple Wavelengths, Do Z Series, Run Journals
3. Set up wavelengths to be acquired, including the wavelength of the Optical Flow (this should be the last)
4. In the Wavelengths tab check Do Z Series only for the wavelength of the Optical Flow.
5. Because the Optical Flow calculation depends on the noise parameters of the camera, the gain, multiplier, AD conversion clock (MHz) settings (if applicable for the camera in use) should not be varied between experiments, unless the noise characteristics is measured for each setting. The exposure time may be varied. The safest is to set intensities by varying illumination intensities.
6. In Journal tab set: After each image: ‘delay.jnl
7. The time interval of the short time lapse is set in the delay.jnl, and it has to be tuned according the application (see here).
8. The Time and Stage tabs can be arbitrarily set.
9. It is advised to use some kind of auto focusing mechanism. If the microscope lacks of active auto focus, set up a separate journal to perform software autofocus on fluorescence beads placed somewhere in the specimen. An example journal is shown above for auto focusing on beads.
10. If the focus of the stage has active feedback these may have to be turned off for the duration of the acquisition of the Optical Flow frames, to ensure that no focal change or stage movement happens due to fluttering of the active feedback mechanisms.

The protocol is based on Metamorph 6.3

Analysis in Image Analyst MKII

Analyzing noise characteristics

1. Open the noise characteristics file recorded above
2. Set LUT scaling to frame-by-frame in the Set scaling menu point of context menu of the Image Window (check Scale each frame independently)
3. Look for a small part of the image where the illumination is the most even. Draw a small ROI here (~20x20 pixels)
4. Select the Sensor Noise Characteristics in the Special main menu.
5. In the Parameter Bar, set the 'Set values in Optical Flow functions' parameter to Yes.
6. In the context menu of the Image Window click Process This with Noise Characteristics; A Plot and a Text window appear.
7. The content of the Plot window is the intensity-variance relationship of the pixels within the ROI. This has to be a straight line. If it is not linear:
   1. Frames have to be in the order of increasing intensity
   2. Delete any saturated frames.
   3. Nonlinearity may be caused by uneven illumination. Move the ROI around to find a linear spot.
   4. Try to draw a smaller ROI.
8. The function automatically sets the following parameters of the Optical Flow function:
   • Detector offset (mean of the zero illumination image intensity)
   • Detector variance vs. intensity Slope (slope of the Plot Window)
   • Detector Read out Variance (variance at the zero  illumination)
9. The above values will be stored when exiting Image Analyst, or click Edit/Save Preferences in the main menu.

Analyzing Optical Flow

1. Open the Multi-DimensionalAcquisition file (*.nd). If the experiment consists multiple *.nd files they can be merged in time by multiple selection. The Multi-Dimensional Open dialog appears. The analysis is identical for both acquisition methods described above.
2. In Open tab: select only the channel that will be used for Optical Flow calculation. Select Optical Flow in the Processing panel.
3. In the Settings tab make sure that the Separate Blocks... and Ignore STK file time stamps are not checked. The Load specified frames of the time lapse feature can be used. If more frames than the number intended to be processed as Optical Flow were recorded, use the Load specified frames of each stack... to enter frame numbers  (within the short time lapse) to be used, separated by commas. If more frames are processed than the width of the dt (temporal differentiation) kernel, thus when the recording is longer than the width of dt kernel and Load specified frames of each stack is not set to match the width, multiple velocity images are calculated, and the result will be obtained by using projection as given in the Project Z field.
4. In the OpticalFlow load tab the parameters of the Optical Flow function are listed. The following parameters may have to be set here:
   1. Select dt kernel: [1,-1]  (to match the length of short time lapses of two frames)
   2. Average OF for dt width: Yes (dt kernel of width of two always used with averaging to avoid biasing between leading and trailing edges)
   3. Block mode: No (each short time lapse is separately processed, so there is no need for block mode)
   4. Pixel size: (the mm/pixel calibration can be given here to obtain velocities in mm/s rather than in pixel/s. 1 results output in pixels/s)
   5. Output as... (enable the desired kind of outputs; as default only absolute velocities are calculated)
   6. Output as Absolute value of Projected Vectors: If Yes, velocity vectors are projected to a point ROI. This can be used to assess anterograde transport (away from the point ROI) by positive velocities and retrograde transport (towards the ROI) by negative values. When using this feature first (before #2) load the image series by setting the Processing panel to None in the Open tab. Draw ROI on the opened image. Then follow the above protocol form #2. Set the ROI No. in the Projection ROI parameter. The ROIs are automatically copied from the last open image during Optical Flow open.
   7. Other parameters: noise parameters were filled in above. Fine tuning of other parameters see here.
5. Above settings are valid as long as the dialog is open, or can be stored by the Set as Default button.
6. Click Open to perform loading and processing.
7. The default LUT of the Optical Flow image is pseudocolor, and can be set in the Preferences dialog.

The resultant Optical Flow image consists of pseudocolored pixels where Optical Flow determination was feasible based on the noise characteristics (there was enough image detail to distinguish movement from noise), and black mask where not. The unit of the Optical Flow image is pixel/s, or mm/s  if the Pixel size is set above.

Analyzing Optical Flow from simple time lapse recordings (see figure about block mode)

1. Open stk file in the File/Open image series/measurement. Importantly, this section is only valid for time lapses recorded without stage movement.
2. Background must not be subtracted. The original background level is required masking of Optical Flow images.
3. Select the Optical Flow function in the main menu Special are listed. The following parameters may have to be set in the parameter bar:
   1. Select dt kernel: [1,-1] or set the width of blocks if the recording was in block mode.
      • If the block size is greater than 2, set [Savitzky-Golay first derivative] here and enter the size of the block at the SG kernel for dt width, and enter No at #2 below.
   2. Average OF for dt width: Yes (dt kernel of width of two always used with averaging to avoid biasing between leading and trailing edges. Set No if using wider kernel)
   3. Block mode: if the experiment was recorded with an even frame rate around 1s/frame or less set No. If the experiment was recorded as frames (equal number of the width of the dt kernel at short interval, then pause for an arbitrary time, and then this is cyclically repeating, set Yes.
   4. Pixel size: (the mm/pixel calibration can be given here to obtain velocities in mm/s rather than in pixel/s. 1 results output in pixels/s. Use the context menu  Show Image Info of an Image Window to determine scaling)
   5. Output as... (enable the desired kind of outputs; as default only absolute velocities are calculated)
   6. Output as Absolute value of Projected Vectors: If Yes, velocity vectors are projected to a point ROI. This can be used to assess anterograde transport (away from the point ROI) by positive velocities and retrograde transport (towards the ROI) by negative values. When using this feature first (before #3) load the image series by setting the Processing panel to None in the Open tab. Draw ROI on the opened image. Then follow the above protocol form #3. Set the ROI No. in the Projection ROI parameter. The ROIs are automatically copied from the last open image during Optical Flow open.
   7. Other parameters: noise parameters were filled in above. Fine tuning of other parameters see here.
4. In the context menu of the Image Window click Process This with Optical Flow.
5. The default LUT of the Optical Flow image is pseudocolor, and can be set in the Preferences dialog.

Fine tuning optical flow (see here)

Example

Example nd data set file (107MB, zip compressed)
Download and uncompress data on your hard drive.
See more details of working with Optical Flow images here.
Calculation of Optical Flow from the example image set:

1. In the main menu select File/Open image series/measurement, set the file type to "*.stk" and open noise.stk in the “Noise Characteristics” folder.
2. Follow the points in the Analyzing noise characteristics section above.
3. Close images by File/Close all.
4. In the main menu select File/Open image series/measurement, set the file type to "*.nd" and select all files in the “Mitochondrial Motion” folder.
5. In  the “File Order” tab of the Multi-Dimensional Open dialog make sure, that the order is correctly incrementing
6. In the Settings tab uncheck everything except for the Short Image Window Name.
7. Switch back to the Open tab, select stage position 7 and Click Open. Inspect images.
8. Wavelength 3 contains the short time lapse recorded as an image stack for each time point. Unselect wavelengths 1-2 and select only wavelength 3.
9. Select Optical Flow in Processing and switch to the “Optical Flow Load” tab.
10. Inspect parameters. Set parameters as discussed in the Analyzing Optical Flow section above . The default values can be seen here.
11. The resolution was not set in Metamorph. The Pixel size is 0.28 mm/pixel.
12. Click Open. 
13. Draw a ROI around the neuron and press .

Protocol by Akos A. Gerencser 08/10/2010 V1.1        
updated on 11/06/2015

References

1. Gerencser A. A. and Nicholls D. G. (2008) Measurement of Instantaneous Velocity Vectors of Organelle Transport: Mitochondrial Transport and Bioenergetics in Hippocampal Neurons. Biophys J. 2008 Sep 15;95(6):3079-99.