Name Last Update
..
color_tables Loading commit data...
parameters_files Loading commit data...
L1C_band_composition.py Loading commit data...
OTB_workflow.py Loading commit data...
all_run_alcd.py Loading commit data...
confidence_map_exploitation.py Loading commit data...
contour_from_labeled.py Loading commit data...
expand_point_region.py Loading commit data...
find_directory_names.py Loading commit data...
layers_creation.py Loading commit data...
masks_preprocessing.py Loading commit data...
merge_shapefiles.py Loading commit data...
metrics_exploitation.py Loading commit data...
quicklook_generator.py Loading commit data...
readme.md Loading commit data...
split_samples.py Loading commit data...
synthese_alcd_runs.py Loading commit data...

readme.md

Active Learning for Cloud Detection (ALCD)

This tools allows the user to create cloud masks with an active learning process. These masks can be used as reference masks.

This readme describes quickly the use of each python file, the general use of the tool is described in the user manual in the parent directory.

Description of each file

  • all_run_alcd.py main file, calls the other files. Run the full ALCD code.
    Called by: /

  • L1C_band_composition.py creates the features of the full image, taking into account the original image, the cloud-free one and the DEM.
    Called by: all_run_alcd.py, layers_creation.py

  • OTB_workflow.py core of the program, relies heavily on OTB. Computes statistics on the image, trains a Random Forest, classifies the image.
    Called by: all_run_alcd.py

  • confidence_map_exploitation.py creates the enhanced version of the confidence map. Also computes the evolution of the confidence and of the samples number across the iterations.
    Called by: all_run_alcd.py, OTB_workflow.py

  • contour_from_labeled.py creates the contours of the classes from the classification map.
    Called by: OTB_workflow.py

  • expand_point_region.py increases the number of samples by taking their neighborhood.
    Called by: masks_preprocessing.py

  • find_directory_names.py gets the path of the L1C directory, and check if the dates given by the user are valid.
    Called by: all_run_alcd.py, L1C_band_composition.py

  • layers_creation.py generates empty shapefiles layer, one for each class, so the user can populate them afterwards.
    Called by: all_run_alcd.py

  • masks_preprocessing.py calls other codes to augment the samples quantity and merge them into one shapefile, and split them between a training and validation set.
    Called by: all_run_alcd.py

  • merge_shapefiles.py merge multiple shapefiles into one, and gives a class number to each point based on the shapefile it comes from.
    Called by: confidence_map_exploitation.py, masks_preprocessing.py

  • metrics_exploitation.py computes various metrics, including accuracy and f1-score for the k-fold.
    Called by: all_run_alcd.py

  • quicklook_generator.py generates quicklooks of all the dates in the L1C directory for given locations.
    Called by: /

  • split_samples.py separates point from a shapefile in two different shapefiles (training and validation sets). Also used to split into k different sets, for the k-fold cross-validation.
    Called by: masks_preprocessing.py

  • synthese_alcd_runs.py makes a synthesis of the scenes proceeded with ALCD.
    Called by: /

Getting Started

You need to set the parameters as described in the user manual. Then you can simply run

python all_run_alcd.py -f true -s 0 -l Arles -d 20171002 -c 20171005

Modify the shapefiles to populate them. Run

python all_run_alcd.py -f true -s 1 -l Arles -d 20171002 -c 20171005

While the classification is not satisfactory, run

python all_run_alcd.py -f false -s 0

Add more points to the shapefiles. Run

python all_run_alcd.py -f false -s 1

Outputs

For each scene, a root_scene/ output dir will be created. In this directory you will find after having run ALCD, among other things:

  • root_scene/In_data/Image/ --- the original image and the composition of all the bands serving as features
  • root_scene/In_data/Masks/ --- the layers with the samples you placed manually
  • root_scene/Model/ --- the model you trained on your samples
  • root_scene/Out/confidence_enhanced.tif --- the modified confidence map that helps you understand where the confidence is low
  • root_scene/Out/labeled_img_regular.tif --- the current classification map, with the following classes: 0: null value, 1: background, 2: low_clouds, 3: high_clouds, 4: clouds_shadows, 5: land, 6: water, 7: snow
  • root_scene/Out/contours_lables.tif --- the contours from the classes of the labeled_img_regular.tif
  • root_scene/Statistics/ --- contains the various statistics, such as the one from the K-fold if you ran it, or the confusion matrix