Films¶

Transient HDR Film (transient_hdr_film)¶

mitransient’s equivalent to Mitsuba 3’s HDRFilm. The HDRFilm plugin creates a data structure that stores one image. Our transient version extends this idea to store a list of images (the transient video).

Specifying the start and end times of the video: You need to specify the exposure time for each frame of the video, and the start time of the video. These values are specified in optical path length and not in time. All the lights of the scene emit at t=0. Thus, for example, if you want to capture an event in your scene that starts when light has travelled 1 meter, and ends when light has travelled 2 meters, you should set start_opl=1.0, bin_width_opl=0.01 and temporal_bins=100. This will store a video of 100 frames that starts when light has travelled 1 meter, and ends when light has travelled 2 meters.

<film type="transient_hdr_film">
    <integer name="width"  value="256"/>
    <integer name="height" value="256"/>
    <integer name="temporal_bins" value="400"/>
    <float name="start_opl" value="1000"/>
    <float name="bin_width_opl" value="6.5"/>
    <rfilter type="box"/>
</film>

{
    'type': 'transient_hdr_film',
    'width': 256,
    'height': 256,
    'temporal_bins': 400,
    'start_opl': 1000,
    'bin_width_opl': 6.5,
    'rfilter': {'type': 'box'}
}

We stores two image blocks simultaneously:

Steady block: Accumulates all samples (sum over all the time dimension)
Transient block: Accumulates samples separating them in time bins (histogram)

The results can be retrieved using the develop(raw=True) method, which returns a (steady, transient) tuple. The transient image will have shape (width, height, temporal_bins, channels).

Parameter	Type	Description
temporal_bins	integer	Number of bins in the time dimension (histogram representation)
bin_width_opl	float	Width of each bin in the time dimension (histogram representation), measured in optical path length
start_opl	float	Start of the time dimension (histogram representation), measured in optical path length
exhaustive_scan	boolean	If set to true, the transient film is modified to take into account two different scanning grids, one for the points scanned by the sensor, and another for the points illuminated by the laser. This is required (and supported) only for Exhaustive NLOS captures, as these captures take into account the contribution of each illuminated point for each scanned point. See also `transient_nlos_path` and its `capture_type` parameter. NOTE: this type of scanning does not support cropping. (default: false)
laser_scan_width	float	Horizontal resolution of the laser scanning pattern. Only used if `exhaustive_scan` is true. (default: 0)
laser_scan_height	float	Vertical resolution of the laser scanning pattern. Only used if `exhaustive_scan` is true. (default: 0)

Phasor HDR Film (phasor_hdr_film)¶

Equivalent to transient_hdr_film, but stores the transient information in the frequency domain for a sparse set of frequencies.

Specifying the start and end times of the video: See the documentation for transient_hdr_film.

Choosing which frequencies to store: This plugin was made with the work “Non-line-of-sight imaging using phasor-field virtual wave optics” in mind. Thus, the parameters wl_mean and wl_sigma, which appear in that work, are used to choose which frequencies to store based on a Morlet wavelet filter. Roughly speaking, wl_mean is the central wavelength of the filter, and 6*wl_sigma will be be width of the filter. That way you can choose a range of frequencies to store.

<film type="phasor_hdr_film">
    <float name="wl_mean" value="100"/>
    <float name="wl_sigma" value="100"/>
    <integer name="width"  value="256"/>
    <integer name="height" value="256"/>
    <integer name="temporal_bins" value="400"/>
    <float name="start_opl" value="1000"/>
    <float name="bin_width_opl" value="6.5"/>
    <rfilter type="box"/>
</film>

{
    'type': 'phasor_hdr_film',
    'wl_mean': 100,
    'wl_sigma': 100,
    'width': 256,
    'height': 256,
    'temporal_bins': 400,
    'start_opl': 1000,
    'bin_width_opl': 6.5,
    'rfilter': {'type': 'box'}
}

We stores two image blocks simultaneously:

Steady block: Accumulates all samples (sum over all the time dimension)
Phasor block: Accumulates samples separating them in frequency components

The results can be retrieved using the develop(raw=True) method, which returns a (steady, phasors) tuple. The transient image will have shape (width, height, num_frequencies, 2). The last dimension stores the real and imaginary components of the phasors for their respective frequencies.

Parameter	Type	Description
wl_mean	float	Central wavelength for the Morlet wavelet filter, measured in optical path length
wl_sigma	float	Width of the Morlet wavelet filter, measured in optical path length
temporal_bins	integer	Number of bins in the time dimension (histogram representation)
bin_width_opl	float	Width of each bin in the time dimension (histogram representation), measured in optical path length
start_opl	float	Start of the time dimension (histogram representation), measured in optical path length