System simulation

Forest-related stuff.

Forest

This module contains system simulation functionality in case of a forest simulation.

Note

If you plan to build a custom system simulation say, a conveyor belt with plastic bits, you should create a similar file to this as much of the stuff here is specific for a forest simulation. That is why this file is not simpy called something like system_simulation.interface. It is simply not so general.

_gaussian(param_dict: dict, rng, is_seed=False) → dict

Applies gaussian random value to a value that is represented as a dictionary in system_simulation control file.

Parameters:

param_dict – Dict representation of the value.
rng – A Numpy random generator object to be used for randomization.
is_seed – If True, new value is drawn from discrete uniform distribution.

Returns:

A new dict with randomized value based on standard deviation that was present in given dict. The std field is removed from returned dict representation.

_m2s(control_dict: dict, rng, is_seed=False) → dict

Rewrites given master control dictionary into a slave control dictionary.

New values are drawn from Gaussian distribution based on standard deviations present in the control. New random seeds (for the Blender file) are drawn from uniform distribution.

Parameters:

control_dict – Master control dictionary.
rng – Numpy random generator object used to randomize values in the slave control file.
is_seed – If True, new value is drawn from discrete uniform distribution.

Returns:

New slave control dictionary.

:raises: RuntimeError if given dict is not master control.

_s2m(control_dict: dict) → dict

Change slave control into master control.

Basically, just adds default STD to each item that needs it. Almost opposite operation of m2s().

Parameters:: control_dict – Slave dict to be masterfied. If given control is already master, it will be returned without modifications.
Returns:: New master control dict.

construct_spectral_cube(system_sim_name: str, system_sim_name_for_white_signal: str = None)

Constructs an ENVI-style hyperspectral image cube out of rendered images.

Can be used after the scene has been rendered (at least spectral and visibility maps).

White reference for reflectance calculation is searched automatically from available visibility maps if system_sim_name_for_white_signal is not given . Note that the maps must be named like Reference 0.00 material_0001.tif.

Saves white signal used in reflectance calculation as a toml file.

Default RGB bands for ENVI metadata are inferred if in visible range. Otherwise first, middle, and last bands are used.

Parameters:

system_sim_name – See system_sim_name.
system_sim_name_for_white_signal – Optionally, give system simulation name from where to fetch the white signal. This is useful when you have a lot of scenes with the same illumination, so the white signal does not have to be inferred for every scene separately. If None, it is inferred from the data. If there are no visibility maps in the current system simulation, this will fail and raise an error.

Raises:

FileNotFoundError – if the rendered frames directory does not exist or is empty. Also, if the sun data file does not exist, which is needed for wavelength info.

create_scene_bundle(bundle_name: str, system_sim_name_ancestor: str, rng, count=10, leaves=None, soil_name: str = None, sun_file_name: str = None, sky_file_name: str = None)

Creates a bundle of scene variants and initializes them.

Loop count times and create slave scenes with initialization. Gather and write generated scene names into a toml. Return path to the toml so it can be run in a bundle by system_simulation.forest.run_scene_bundle().

Parameters without explanation are passed directrly to system_simulation.forest.init().

Parameters:

bundle_name – Name of the bundle (file name).
system_sim_name_ancestor – Which scene is to be used as an ancestor for the new scenes. It must be a master scene (indicated in the scene control file).
rng – Numpy random number generator.
count – How many scene variations are to be created.

Returns:

The path to the created bundle file.

init(leaves=None, soil_name: str = None, sun_file_name: str = None, sky_file_name: str = None, system_sim_name_to_copy_from: str = None, new_system_sim_name: str = None, conf_type: str = None, rng=None) → str

Create a new forest system simulation by copying a template.

If you want to produce a non-working (i.e., you cannot actually run the system simulation) copy, do not provide any of the arguments. For a working copy, you have to produce at least the leaves. Other arguments can still be left to None and the default values will be used.

Loads slab (leaf) material parameters for each leaf. They must have the same spectral sampling, but they do not have to originate from a single Slab simulation.

Loads sun and sky scatter spectra and resample them to match the range and resolution of the leaves. The spectra are normalized so that the brightest band of the brightest spectra will be unity, and the other spectrum is normalized with that too. In other words, the sky scatter may be brighter than the direct sunlight (overcast sky) and the sky spectrum would reach the value of 1 but the sun spectrum would not (unless they are equal). Same works if the sunlight is brighter than sky scattered light. These resampled spectra are saved locally for this system simulation for later use.

Parameters:

leaves – Leaves should be given as list of tuples [(slab_simulation_name: str, sample_id: int, slab_material_name: str), (),…]. If None, the scene will be copied, but no sun, sky, or leaf spectra will be copied. In this case, an empty string is returned.
soil_name – Uses the soil spectrum from a file that includes this string. First found occurrence is used.
sun_file_name – Name of the file with the sun spectrum to be used.
sky_file_name – Name of the file with the sky spectrum to be used.
system_sim_name_to_copy_from – If given, a system_simulation scene with this name will be copied instead of the default system_simulation template.
new_system_sim_name – If given, this will be the name for the new system_simulation instead of the default generated name, which is meant for creating an arbitrary number of randomized clones.
conf_type –
How to produce configuration file: string on of “m2m”, “m2s”, or “s2m”. The configuration file is most useful in randomization.
- ”m2m” (from master to master) makes a pure copy of the scene configuration file from the source scene. This is the default behavior.
- ”m2s” (from master to slave) will generate (gaussian) random values based on standard deviations defined in the source master configuration file.
- ”s2m” (from slave to master) will create a new master configuration from the source scene configuration with default standard deviation.
Note

Configuration type s2s does not exist as there is no standard deviations present in slave configs.
rng – Numpy random number generator for randomizing forest and tree parameters (the geometry of the scene). This is only needed if conf_type="m2m" and failing to provide it will raise an error.

Returns:

New system_sim_name that is generated if new_system_sim_name is not given.

process_forest_control(runtime: RuntimeEnvironment, system_sim_name: str, generate=True)

Either (re)generates the control file (if generate=True) or applies it to the scene (if generate=False).

This should be called after making any changes to the Blender scene file manually, i.e., using Blender directly rather than calling the HyperBlend’s internal scripts to reflect those changes in the control file. And in reverse, if you manually change the control file, this method will apply the changes to the scene file (.blend).

Parameters:

runtime – See runtime.
system_sim_name – See system_sim_name.
generate – If True, generate the control file based on the scene file. If False, does the opposite, i.e., applies the control file to the scene file.

render_forest(runtime: RuntimeEnvironment, system_sim_name: str, render_mode: str, silent=True)

Renders the forest scene into a spectral image.

Parameters:

runtime – See runtime.
system_sim_name – See system_sim_name.
render_mode – One of the following ‘preview’, ‘spectral’ or ‘visibility’. ‘preview’ renders only some preview images that can give an idea of the scene geometry without having to open the Blender scene itself. ‘spectral’ renders all spectral bands (one rendered image per band). ‘visibility’ renders visibility maps that show which object is visible in each pixel. This must be done before you can construct the reflectance image cube as the white correction relies on this information to find the reference plate locations.
silent – If True, Blender output is redirected to null stream to avoid cluttering of console.

run_scene_bundle(runtime: RuntimeEnvironment, bundle_name: str, slab_material_names, render_spectral=True, render_visibility_maps=True, render_preview=True, construct_cube=True)

Runs the scene bundle either by rendering or by constructing the spectral image cube.

All of these can be done with a single call.

See other parameters from system_simulation.forest.render_forest() .

Parameters:

runtime – See runtime.
bundle_name – Name of the bundle (file name).
slab_material_names – Slab material name list as accepted by system_simulation.forest.setup_forest_for_rendering().
render_spectral – If True, renders spectral bands.
render_visibility_maps – If True, renders visibility maps (needed for spectral cube construction).
render_preview – If True, renders preview images.
construct_cube – If True, constructs the spectral cube out of rendered frames. Cannot be called before spectral bands and visibility maps are rendered.

setup_forest_for_rendering(runtime: RuntimeEnvironment, system_sim_name: str, slab_material_names=None)

Set up the system simulation for rendering.

Most importantly, this applies the material and light values frame by frame to reflect the values needed to each spectral band.

Warning

If this is not called before rendering, the results are arbitrary.

Parameters:

runtime – Runtime environment object that contains the Blender executable path.
system_sim_name – Name of the system simulation to be set up.
slab_material_names – Names of the slab materials (must mach the ones used in the Blender scene) as a list of strings like: [‘Slab material 1’, ‘Slab material 2’,…].

Lighting

This module is an access point to load light spectra to be used in system level simulation.

If you are happy with the default light spectra, you can simply call load_light() and be done with it. If you want to use one of the integrated spectra generation tools, follow the instructions below.

How to generate sun and sky spectra with SSolar GOA

Download Windows version of GOA from goa.uva.es/ssolar_goa-model/ or use their GitHub version from github.com/GOA-UVa/SSolar-GOA.

(Assuming the Windows version) Click on “Load default settings”

Change wavelength range to 400-2500 nm

Click on “Run”

Move the generated file to where HyperBLend’s light files are stored and call load_light() once. It will the split the sun and sky spectra to separate files that can be read by calling the load_light() again.

How to generate sun spectra with NASA PSG

Select template “Sun from Earth” and click “Load Spectra”.

If you want to select certain location and date on Earth, click “Change Object” button. Remember to click save settings!

Click “Change Instrument”

Set “Spectral range” to 400 - 2500 nm. Change units from \(\mu m\) to nm.

Set “Resolution” to 1 nm and units from “Resolving power” to nm.

Set “Spectrum intensity unit” to W/m2/um (spectral irradiance)

Change other settings as you see fit, but know what you are doing!

Click “Generate Spectra”

From the first image, click “Download Spectra” and put the file as in GOA instructions above.

How to use fully custom light spectra

If you have a spectra that your grandpa recorded and wrote down on a piece of paper, you can use it as a light source in HyperBLend. Just write the values to a text file. The only requirement is that the file contains wavelength-irradiance pairs in two columns, separated by a space. You can add comments to the file by starting the line with a hash sign (#). All comments must occur before the actual data.

load_light(file_name: str = None, system_sim_name: str = None, sampling=None, lighting_type='sun')

Loads a lighting file and returns wavelengths and corresponding irradiances.

Files formatted so that comment lines are prefixed with ‘#’ and rest of the lines contain wavelength-irradiance pairs that can be cast to floats can be read directly. For spectra generated with either NASA PSG or SSolar GOA, the files are fixed so that they can be read directly later.

Parameters:

file_name – Passed directly to data.path_handling.find_light_file().
system_sim_name – Passed directly to data.path_handling.find_light_file().
sampling – List of wavelengths as floats. If given, sun data is resampled to wavelengths specified in the list. If sampling is None (default), the data is returned as raw.
lighting_type – Lighting type either “sun” or “sky”, default is “sun”. This is used to get the default sun or sky file if file_name was not given. If both are given, the file_name has precedence.

Returns:

Tuple (np.array[wls], np.array[irradiances]) where wls is a list of wavelengths and irradiances are corresponding list of irradiances. The length of the lists vary depending on given sampling.

Raises:

ValueError if file_name was not provided and lighting_type is not either “sun” or “sky”.

Soil

find_gsv_soil_path(soil_name)

Finds path to a soil file that includes given soil name.

Returns the first occurrence if multiple files match the name.

load_soil(soil_name: str, forest_id: str = None, sampling=None)

Find a soil csv from root/soil/ and place a resampled copy of it to given system_simulation scene.

Parameters:

soil_name – Soil name to be searched. Must be one that exists in root/soil/.
forest_id – Forest scene id where the result should be writeen.
sampling – Resampling data as a list of wavelengths.

Misc

Temporarily store system_simulation scene related stuff here, so they don’t mess with 0.2.0 release.

These are mostly hacks and should be rewritten into proper code.

load_into_cube(scene_id)

maxdiff(set_name='prospect_randoms')

ndvi(scene_id)

show_forest_rend(band, scene_id)

white_ref(frame, percent=2)

Developer Actions

This file contains actions that are meant only for developers! Do not call these unless you know what you are doing.

apply_global_master_forest_control_to_template(runtime: RuntimeEnvironment)

Apply a global master control file to the system simulation template.

Counterpart of system_simulation.dev_actions.generate_global_master_forest_control_from_template()

Parameters:: runtime – Runtime environment object that contains the Blender executable path.

generate_global_master_forest_control_from_template(runtime: RuntimeEnvironment)

Writes a new global master control file.

Warning

This changes the control file that is in the repository as a base for all new simulation scenes. You should only call this if you make changes to the forest simulation template and need to have those changes reflected in the control file. Or if there is something that needs to fixed in the stored control file. Push the new control file to your repo after ensuring the new one works as expected.

Parameters:: runtime – Runtime environment object that contains the Blender executable path.