Source root

Note

constants and plotter should be removed from here

The root of the source code.

Main

Entry point of the program.

There is no user interface to the program (yet) so code your calls here and run in your favourite IDE.

Constants

Constants used throughout the program. Mostly file names and dictionary keys.

Do not change these, it may break reading of existing optimization results and rendering.

ad_coeffs = 'ad_coeffs'

List of coefficients of absorption particle density.

ai_coeffs = 'ai_coeffs'

List of coefficients of scattering anisotropy.

blender_executable_path_linux = '/snap/bin/blender'

Default location of Blender on Ubuntu.

blender_executable_path_win = 'C:\\Program Files\\Blender Foundation\\Blender 3.6/blender.exe'

change this to where your Blender is installed.

blender_foundation_win = 'C:\\Program Files\\Blender Foundation'

By default, all Blender versions on Windows are installed here.

blender_scene_slab_simulation_template = 'slab_sim_template.blend'

Name of the slab simulation template Blender scene.

default_B_wl = 465.0

Default Blue wavelength for false color images.

default_G_wl = 532.0

Default Green wavelength for false color images.

default_R_wl = 630.0

Default Red wavelength for false color images.

dirname_opt_target_signal = 'Target signal'

Target signal files are stored to this directory.

dirname_optimization_results = 'Optimization results'

Wavelength-wise results for each signal solved by optimization method are stored here (numerical and png images).

dirname_result_signal = 'Result signal'

Stores simulated signals matching to target signals.

dirname_slab_sim_rend = 'rend'

Folder name for leaf target renders.

dirname_slab_sim_working_temp = 'working_temp'

Top level folder where rendered images of slab simulation are saved in their own subdirectories.

dirname_slab_simulation = 'Slab simulation'

Top level slab simulation directory name.

dirname_slab_solvers = 'Slab solvers'

Top level Slab solver directory name.

dirname_slab_solvers_default = 'Default slab'

Default slab solver directory name.

dirname_system_simulation = 'System simulation'

Top level system simulation directory name.

file_app_info = 'app_info'

Toml formatted file that application info such as version number.

file_default_sky = 'default_sky.txt'

Default sky spectrum file name that is included in the repository.

file_default_sun = 'default_sun.txt'

Default sun spectrum file name that is included in the repository.

file_directory_structure = 'directory_structure'

Toml formatted file that defines directory structure and core files.

file_dynamic_definitions = 'dynamic'

Toml formatted file that dynamically inferred info such as operating system.

file_opt_res = 'final_result'

Set result file name.

file_opt_target = 'target'

Target file name prefix as in ‘target_0.toml’.

file_sampling_data = 'sampling'

Toml formatted file storing wavelengths for resampling.

filename_result_signal = 'result_signal'

Simulated result signal of a slab simulation. This is postfixed with the signal id by file_names.

filename_slab_sim_error_plot = 'slab_sim_error'

File name of the error plot of a slab simulation.

filename_slab_sim_result = 'slab_sim_result'

File name of the final result toml and plot of a slab simulation..

filename_system_sim_control = 'system_sim_control'

Toml formatted file for system simulation scene control parameters.

folder_rend_ref_refl = 'rend_refl_ref'

Folder name for reflectance white reference renders.

folder_rend_ref_tran = 'rend_tran_ref'

Folder name for transmittance white reference renders.

imaging_type_refl = 'refl'

Imaging type reflectance (for rendering).

imaging_type_tran = 'tran'

Imaging type transmittance (for rendering).

key_sample_result_ad = 'absorption_density'

List of optimized absorption densities.

key_sample_result_ai = 'scattering_anisotropy'

List of optimized scattering anisotropies.

key_sample_result_mf = 'mix_factor'

List of optimized mix factors.

key_sample_result_process_elapsed_min = 'process_elapsed_min'

Processor time used to optimize one sample.

key_sample_result_r = 'refls_modeled'

List of modeled reflectances.

key_sample_result_r_RMSE = 'r_RMSE'

Root mean squared error of reflectance of a sample.

key_sample_result_re = 'refls_error'

List of errors in reflectance.

key_sample_result_rm = 'refls_measured'

List of measured (target) reflectances.

key_sample_result_sd = 'scattering_density'

List of optimized scattering densities.

key_sample_result_t = 'trans_modeled'

List of modeled transmittances.

key_sample_result_t_RMSE = 't_RMSE'

Root mean squared error of transmittance of a sample.

key_sample_result_te = 'trans_error'

List of errors in transmittance.

key_sample_result_tm = 'trans_measured'

List of measured (target) transmittances.

key_sample_result_wall_clock_elapsed_min = 'wall_clock_time_elapsed_min'

Wall clock time used to optimize one sample.

key_sample_result_wls = 'wls'

List of wavelengths used in a sample.

key_set_result_processor_time_per_sample_hours = 'processor_time_per_sample_hours'

Average processor time per sample in hours.

key_set_result_r_mean = 'set_result_r_mean'

Total mean of modeled reflectances over all samples and wavelengths.

key_set_result_r_std = 'set_result_tm_std'

Total standard deviation of modeled reflectances over all samples and wavelengths.

key_set_result_re_mean = 'refl_error_mean'

Total mean of RMSEs of reflectance over all samples and wavelengths.

key_set_result_re_std = 'refl_error_std'

Total standard deviation of RMSEs of reflectance over all samples and wavelengths.

key_set_result_rm_mean = 'set_result_rm_mean'

Total mean of measured reflectances over all samples and wavelengths.

key_set_result_rm_std = 'set_result_tm_std'

Total standard deviation of measured reflectances over all samples and wavelengths.

key_set_result_sample_count = 'sample_count'

Number of samples in a set.

key_set_result_t_mean = 'set_result_t_mean'

Total mean of modeled transmittances over all samples and wavelengths.

key_set_result_t_std = 'set_result_tm_std'

Total standard deviation of modeled transmittances over all samples and wavelengths.

key_set_result_te_mean = 'tran_error_mean'

Total mean of RMSEs of transmittance over all samples and wavelengths.

key_set_result_te_std = 'tran_error_std'

Total standard deviation of RMSEs of transmittance over all samples and wavelengths.

key_set_result_time_per_sample_hours = 'time_per_sample_hours'

Average wall clock optimization time per sample in hours.

key_set_result_tm_mean = 'set_result_tm_mean'

Total mean of measured transmittances over all samples and wavelengths.

key_set_result_tm_std = 'set_result_tm_std'

Total standard deviation of measured transmittances over all samples and wavelengths.

key_set_result_total_processor_time_hours = 'total_processor_time_hours'

Total processor time of all samples in hours.

key_set_result_total_time_hours = 'total_time_hours'

Total wall clock optimization time of all samples in hours.

key_set_result_wl_ad_mean = 'set_result_wl_ad_mean'

Wavelength-wise mean of absorption particle density.

key_set_result_wl_ad_std = 'set_result_wl_ad_std'

Wavelength-wise standard deviation of absorption particle density. Set to zero is only one sample was used.

key_set_result_wl_ai_mean = 'set_result_wl_ai_mean'

Wavelength-wise mean of scattering anisotropy.

key_set_result_wl_ai_std = 'set_result_wl_ai_std'

Wavelength-wise standard deviation of scattering anisotropy. Set to zero is only one sample was used.

key_set_result_wl_mf_mean = 'set_result_wl_mf_mean'

Wavelength-wise mean of mix factor.

key_set_result_wl_mf_std = 'set_result_wl_mf_std'

Wavelength-wise standard deviation of mix factor. Set to zero is only one sample was used.

key_set_result_wl_r_mean = 'set_result_wl_r_mean'

Wavelength-wise mean of modeled reflectance.

key_set_result_wl_r_std = 'set_result_wl_r_std'

Wavelength-wise standard deviation of modeled reflectance. Set to zero is only one sample was used.

key_set_result_wl_re_mean = 'set_result_wl_re_mean'

Wavelength-wise mean of reflectance error.

key_set_result_wl_re_std = 'set_result_wl_re_std'

Wavelength-wise standard deviation of reflectance error. Set to zero is only one sample was used.

key_set_result_wl_rm_mean = 'set_result_wl_rm_mean'

Wavelength-wise mean of measured reflectance.

key_set_result_wl_rm_std = 'set_result_wl_rm_std'

Wavelength-wise standard deviation of measured reflectance. Set to zero is only one sample was used.

key_set_result_wl_sd_mean = 'set_result_wl_sd_mean'

Wavelength-wise mean of scattering particle density.

key_set_result_wl_sd_std = 'set_result_wl_sd_std'

Wavelength-wise standard deviation of scattering particle density. Set to zero is only one sample was used.

key_set_result_wl_t_mean = 'set_result_wl_t_mean'

Wavelength-wise mean of modeled transmittance.

key_set_result_wl_t_std = 'set_result_wl_t_std'

Wavelength-wise standard deviation of modeled transmittance. Set to zero is only one sample was used.

key_set_result_wl_te_mean = 'set_result_wl_te_mean'

Wavelength-wise mean of transmittance error.

key_set_result_wl_te_std = 'set_result_wl_te_std'

Wavelength-wise standard deviation of transmittance error. Set to zero is only one sample was used.

key_set_result_wl_tm_mean = 'set_result_wl_tm_mean'

Wavelength-wise mean of measured transmittance.

key_set_result_wl_tm_std = 'set_result_wl_tm_std'

Wavelength-wise standard deviation of measured transmittance. Set to zero is only one sample was used.

key_set_result_wls = 'wls'

List of wavelengths used in the set.

key_wl_result_elapsed_time_s = 'elapsed_time_s'

Time in seconds to optimize a single wavelength.

key_wl_result_history_ad = 'history_absorption_density'

History of absorption density listed for each render run.

key_wl_result_history_ai = 'history_scattering_anisotropy'

History of scattering anisotropy listed for each render run.

key_wl_result_history_loss_total = 'loss'

History of loss listed for each render run.

key_wl_result_history_mf = 'history_mix_factor'

History of mix factor listed for each render run.

key_wl_result_history_r = 'history_reflectance'

History of reflectance listed for each render run.

key_wl_result_history_sd = 'history_scattering_density'

History of scattering density listed for each render run.

key_wl_result_history_t = 'history_transmittance'

History of transmittance listed for each render run.

key_wl_result_optimizer = 'optimizer'

Optimizer name used in optimization.

key_wl_result_optimizer_diffstep = 'diffstep'

Used relative step size when approximating the Jacobian in optimizer.

key_wl_result_optimizer_ftol = 'ftol'

Used function value (error to measured) tolerance to stop optimization.

key_wl_result_optimizer_result = 'optimizer_result'

The content of the optimizer result depends on the optimizer used.

key_wl_result_optimizer_xtol = 'xtol'

Used variable change tolerance (between iterations) to stop optimization.

key_wl_result_refl_error = 'reflectance_error'

Error of modeled reflectance.

key_wl_result_refl_measured = 'reflectance_measured'

Measured (target) reflectance.

key_wl_result_refl_modeled = 'reflectance_modeled'

Modeled reflectance.

key_wl_result_render_calls = 'render_calls'

Render calls. Used in optimization history.

key_wl_result_tran_error = 'transmittance_error'

Error of modeled transmittance.

key_wl_result_tran_measured = 'transmittance_measured'

Measured (target) transmittance.

key_wl_result_tran_modeled = 'transmittance_modeled'

Modeled transmittance.

key_wl_result_wl = 'wl'

Wavelength.

key_wl_result_x0 = 'x0'

Starting guess used for the wavelength optimization.

key_wl_result_x_best = 'x_best'

Optimized leaf material variables for the wavelength.

mf_coeffs = 'mf_coeffs'

List of coefficients of mix factor.

path_project_root = '../'

Project root relative to working folder that is assumed to be project_root/src/.

postfix_plot_image_format = '.png'

Image format of plots.

postfix_render_image_format = '.tif'

Image format of images rendered with Blender.

postfix_text_data_format = '.toml'

Text file format with witch the results are saved.

sd_coeffs = 'sd_coeffs'

List of coefficients of scattering particle density.

signal_directory_prefix = 'Signal'

Prefix for signal directories within a single slab simulation such as ‘signal_0’.

slab_nn_name = 'slab_nn.pt'

Name of the neural network solver.

slab_surf_name = 'slab_surf.toml'

Name of the surface solver.

starting_guess_set_name = 'linear_starting_guess'

Name of the starting guess file.

target_type_ref = 'reference'

Rendering target (white) reference.

target_type_slab = 'slab'

Rendering target is the slab object.

Plotter

This file contains plotting-related code.

Tips for plotting: https://towardsdatascience.com/5-powerful-tricks-to-visualize-your-data-with-matplotlib-16bc33747e05

_plot_refl_tran_to_axis(axis_object, refl, tran, x_values, x_label, invert_tran=False, refl_color='royalblue', tran_color='deeppink')

Plots reflectance and transmittance to given axis object.

Parameters:

• axis_object – matplotlib.axis object to plot to.

• refl – List of reflectance values to be plotted.

• tran – List of transmittance values to be plotted.

• x_values – Essentially a list of wavelengths.

• x_label – Label of x-axis.

• invert_tran – If True, transmittance is plotted on separate y-axis ‘upside down’ as is common.

• refl_color – Color of reflectance points.

• tran_color – Color of transmittance points.

Returns:

None

_plot_starting_guess_coeffs_fitting(dont_show=True, save_thumbnail=True, slab_sim_name: str = None, solver_name: str = None) → None

Plot starting guess poynomial fit with data.

Used only when generating the starting guess.

Parameters:

slab_sim_name – See slab_sim_name.

_plot_with_shadow(ax_obj, x_data, y_data, y_data_std, color, label, ls='-') → None

Plot data with standard deviation as shadow.

Data must be sorted to show correctly.

Parameters:

• ax_obj – Pyplot axes object to plot to.

• x_data – Data x values (wavelengths).

• y_data – Data y values as numpy.array.

• y_data_std – Standard deviation as numpy array. The shadow is drawn as +- std/2.

• color – Color of the plot and shadow.

• label – Label of the value.

• ls – Line style. See pyplot linestyle documentation.

alpha_error = 0.2

Alpha for std shadow.

axis_label_font_size = 16

Axis label font size

fig_title_font_size = 18

Title font size.

figsize = (12, 6)

Figure size for two plot figures.

figsize_single = (7, 6)

Figure size for single plot figures.

figsize_triple_width = (22, 6)

Figure size for two plot figures.

max_ticks = 8

Max tick count for wavelength.

plot_blender_soil(wls, reflectances, soil_name, wls_resampled=None, reflectances_resampled=None, forest_id=None, dont_show=True, save=True)

Plots possibly resampled soil reflectance.

Parameters:

• wls – Original wavelengths as a list.

• reflectances – Original soil reflectances.

• soil_name – Name of the soil used. This will be shown in the plot and in resulting file’s name.

• wls_resampled – Optional, resampled wavelengths.

• reflectances_resampled – Optional, resampled reflectances corresponding to resampled wavelengths.

• forest_id – Forest id is needed if save=True.

• dont_show – If True (default), the plot is not shown, otherwise, show the plot using pyplot show(), which will halt the execution of the program until the window is manually shut.

• save – If True (default), save image to system_simulation scene directory (forest_id must then be given too).

:raises

AttributeError if save=True but forest_id=None.

plot_default_soil_visualization(wls, reflectances, labels, save=True, dont_show=True)

Plot visualization (reflectances) of the default soil types.

Parameters:

• wls – List of wavelengths [400,401,…2500].

• reflectances – 2D-array of reflectances. One row for one soil type.

• labels – List of soil names to be used as labels. These are also used to infer plot colors.

• save – If True (default), save image to disc.

• dont_show – If True (default), the plot is not shown, otherwise, show the plot using pyplot show(), which will halt the execution of the program until the window is manually shut.

plot_light_data(wls, irradiances, wls_binned=None, irradiances_binned=None, forest_id=None, light_plot_name=None, show=False, lighting_type='sun')

Plot used sun or sky data for a scene.

Plot can either be shown or saved. Plot is saved if scene_id is given.

Three main uses:

1. just original spectrum

2. just binned spectrum

3. both

Parameters:

• wls – Either original (1 nm resolution) or binned wavelengths.

• irradiances – Irradiances corresponding to wls in [W/m2/nm].

• wls_binned – Optional. Binned (integrated) wavelengths over certain bandwidth. Used bandwidth is calculated by wls_binned[1] - wls_binned[0].

• irradiances_binned – Optional. Binned (integrated) irradiances correcponding to wls_binned. Both must be given so that binned irradiances can be plotted.

• forest_id – Optional. Forest id for saving the plot to the scene directory.

• light_plot_name – Optional. Sun filename for naming the image file.

• show – If True, the plot is shown to the user. Default is False.

• lighting_type – String either ‘sun’ or ‘sky’.

plot_nn_train_history(train_loss, test_loss, best_epoch_idx, dont_show=True, save_thumbnail=True, solver_name=None) → None

Plot training history of neural network.

Parameters:

• train_loss – List of training losses (per epoch).

• test_loss – List of test losses (per epoch).

• best_epoch_idx – Index of best epoch for highlighting.

• dont_show – If true, does not show the interactive plot (that halts excecution). Always use True when running multiple times in a loop (hyperparameter tuning). Default True.

• save_thumbnail – If True, save plot to disk. Default True.

Returns:

plot_reflectance_lab(HSV_value, reflectance, powers, plot_name=None, show=False, save=True)

Plot simulation result of virtual reflectance lab.

Parameters:

• HSV_value – Used hue-saturation-value values as a list of floats.

• reflectance – List of list of reflectances. Length of the first list must match the sun powers used.

• powers – List of sun powers.

• plot_name – Name for the plot file.

• show – If True, show interactive plot. Default is False.

• save – If True, save plot to project root directory. Default is True.

Raises:

ValueError – If len(powers) != len(reflectance).

plot_resampling(slab_sim_name: str, dont_show=True, save_thumbnail=True) → None

Plots leaf resampled spectra along with the original for all leaf samples in given set.

Parameters:

• slab_sim_name – See slab_sim_name.

• save_thumbnail – If True, a PNG image is saved. Default is True.

• dont_show – If True, the plot is not plotted on the monitor. Use together with save_thumbnail. Default is True.

plot_signal_result(slab_sim_name: str, sample_id: int, dont_show=True, save_thumbnail=True) → None

Plots sample result.

Parameters:

• slab_sim_name – See slab_sim_name.

• sample_id – Sample id.

• save_thumbnail – If True, a PNG image is saved. Default is True.

• dont_show – If True, the plot is not plotted on the monitor. Use together with save_thumbnail. Default is True.

plot_slab_sim_errors(slab_sim_name: str, dont_show=True, save_thumbnail=True)

Plots averaged optimization errors of a sample.

plot_slab_sim_result(slab_sim_name: str, dont_show=True, save_thumbnail=True) → None

Plot average of sample results as the set result.

Parameters:

• slab_sim_name – See slab_sim_name.

• dont_show – If False, pyplot.show() is called, otherwise nothing is shown. Default is True.

• save_thumbnail – If True, save plot to disk. Default is True.

plot_trained_leaf_models(slab_sim_name='training_data', save_thumbnail=True, show_plot=False, plot_surf=True, plot_nn=True, plot_points=True, solver_name: str = None)

plot_training_data_set(r_good, t_good, r_bad=None, t_bad=None, k1=None, b1=None, k2=None, b2=None, show=False, save=True, save_name='training_data', solver_name=None)

Plot training data either interactively or save to disk.

Constants k1,2 and b1,2 are used to visualize cutting lines along equation k*r + b.

Parameters:

• solver_name

• r_good – List of reflectances of good points.

• t_good – List of transmittances of good points.

• r_bad – List of reflectances of bad points.

• t_bad – List of transmittances of bad points.

• k1 – First cutting line k.

• b1 – First cutting line b.

• k2 – Second cutting line k.

• b2 – First cutting line b.

• show – Show interactive plot to user. Default is `False`.

• save – Save plot to disk. Default is `True`.

• save_name – Filename to be used when the image is saved. Only used if ‘save’ is True.

plot_wl_optimization_history(slab_sim_name: str, wl: float, sample_id, dont_show=True, save_thumbnail=True) → None

Plots optimization history of a single wavelength using existing wavelength result toml file.

Parameters:

• slab_sim_name – See slab_sim_name.

• wl – Wavelength of the optimization.

• sample_id – Sample id.

• save_thumbnail – If True, a PNG image is saved to result/plot folder. Default is True.

• dont_show – If True, the plot is not plotted on the monitor. Use together with save_thumbnail. Default is True.

replot_wl_results(slab_sim_name: str)

Replot wavelength results.

Overwrites existing plots.

save_resolution = 600

Save resolution for plots in dots per inch.

variable_space_ylim = [0.0, 1.0]

Y-axis limit for leaf material parameter plot.