Commit 1950a90b7ce08a139bf320b6357283128b91491e

Authored by Erwan Motte
1 parent dc66561e
Exists in master

Many updates, started to include autodoc from code

source/PRO/PRO_L0btoL1a.rst 0 → 100644
... ... @@ -0,0 +1,165 @@
  1 +.. raw:: latex
  2 +
  3 + \clearpage
  4 +
  5 +.. _PRO_L0b_L1a:
  6 +
  7 +*********************
  8 +Level 0b to Level 1a
  9 +*********************
  10 +
  11 +l0b_to_l1a.py
  12 +====================
  13 +
  14 +Module description
  15 +------------------
  16 +The goal of this module is to compute the basic observables for
  17 +land applications. Nadir (nad) and Zenith (zen) subscript are used only when
  18 +the processing is not generic to all channels.
  19 +
  20 +#. Performs additional waveform coherent averaging by arithmetic averaging.
  21 + (Currently not done, Ti = t_coh)
  22 +
  23 + code::
  24 +
  25 + n_shots = int(round(t_coh/Ti))
  26 + wf_co = reshape(wf, n_shots).mean(axis=1)
  27 +
  28 +#. Incoherent averaging is performed in power as the square of the modulus of
  29 + the coherently averaged amplitude: :math:`P = <|Y|>`
  30 +
  31 + code::
  32 +
  33 + n_shots = int(round(t_incoh/t_coh))
  34 + wf_pow = reshape(np.square(np.abs(wf_co)), n_shots).mean(axis=1)
  35 +
  36 +#. And the variance of the absolute value of the ICF
  37 + :math:`\sigma(|Y_{nad}/Y_{zen|})^2` is also computed:
  38 +
  39 + code::
  40 +
  41 + n_shots = int(round(t_incoh/t_coh))
  42 + icf_var = np.square(reshape(np.abs(wf_co_nad/wf_co_zen), n_shots).std
  43 + (axis=1))
  44 +
  45 +#. The noise level is computed as the mean of channel 9 over the whole file
  46 + (36s). This is expected to be representative of the noise, and then
  47 + subtracted to the power: (:math:`P_{corr} = P-B`).
  48 +
  49 + code::
  50 +
  51 + # Noise level estimation
  52 + pow_noise_sq = wf_pow[:, 9].mean(axis=0)
  53 +
  54 + # power correction with noise
  55 + pow_corr = np.clip(pow - pow_noise_sq)
  56 +
  57 +#. Correct signal according to antenna gain pattern G as
  58 + :math:`P_{ant} = P_{corr} / G`
  59 +
  60 + code::
  61 +
  62 + # Find pointing direction in antenna reference frame according
  63 + # to aircraft attitude and satellite elevation and azimuth.
  64 + theta_zen_res, theta_nad_res, phi_res = theta_phi_from_df(df_anc)
  65 +
  66 + # Interpolate antenna co-pol and cross-pol gain according to pointing
  67 + G, X = find_gain('ZR', cfg, df_anc)
  68 +
  69 + # Correct measurement for gain
  70 + pow_ant = pow_corr / G
  71 +
  72 +#. Compute the ICF as
  73 + :math:`\frac{P_{ant,nad}}{P_{ant,zen}}`
  74 +
  75 + code::
  76 +
  77 + ICF = pow_ant_nad / pow_ant_zen
  78 +
  79 +#. Compute the apparent reflectivity as
  80 + :math:`\Gamma' = ICF - \sigma_{ICF}^2`
  81 +
  82 + code::
  83 +
  84 + gamma = ICF - icf_var
  85 +
  86 +
  87 +#. Finally, the total processing can be summarized by this equation
  88 +
  89 + .. math::
  90 +
  91 + \Gamma' = \frac{(<|Y_{nad}|> - B_{nad}) / G_{nad}}
  92 + {(<|Y_{zen}|> - B_{zen}) / G_{zen}}
  93 + - \sigma(|Y_{nad}/Y_{zen|})^2
  94 +
  95 +
  96 +#. Merge in one file:
  97 +
  98 + - The Direct RHCP signal peak correlation complex amplitude
  99 + - The direct estimated Doppler
  100 + - The reflected LHCP signal peak correlation complex amplitude
  101 + - The reflected RHCP signal peak correlation complex amplitude
  102 + - The estimated delay of the reflected LHCP signal in lags
  103 + - Ancillary info from L0 files
  104 + - PRN
  105 + - StartSOW, ifSOWread, WN,
  106 + - fs,
  107 + - El, Az
  108 +
  109 +
  110 +.. csv-table:: Processing steps
  111 + :stub-columns: 1
  112 + :widths: 1 10 8 6 4
  113 + :header: Step, Processing, Output, Variable, format
  114 +
  115 + 1, Read waveforms , Raw WFs , WF_NL , 2D cpx WF
  116 + 2, Additional coh. avging , Coherently avgd WFs, WF_NL_co , 2D cpx WF
  117 + 3, Find coh peak , peak position , NL_loc_co , 1D int
  118 + 4, Store coh peak value , peak value TS , NL_pow_co , 1D cpx
  119 + 5, Compute coh noise floor, noise floor level , grass_NL_co , 1D float
  120 + 6, abs and correct for NF , noise free TS , NL_pow_corr_co, 1D float
  121 +
  122 +Inputs
  123 +------
  124 +
  125 +Outputs
  126 +-------
  127 +
  128 +.. csv-table:: L0 to L1a output file
  129 + :stub-columns: 1
  130 + :widths: 4 20
  131 + :header: Field, Description
  132 +
  133 + DateTime , Python Datetime
  134 + lat_spec , Latitude of specular point
  135 + lon_spec , Longitude of specular point
  136 + DEM_spec , DEM at specular point location
  137 + SV_elev , Elevation at specular point
  138 + SV_azimuth , Azimuth at specular point
  139 + ZR_amp_res , Incoherently Averaged amplitude
  140 + ZR_pow_res , Incoherently averaged power
  141 + ZR_var_res , Incoherently averaged variance
  142 + ZR_ph_res , Incoherently averaged phase
  143 + grass_ZR_sq , Mean of noise floor squared
  144 + ZR_pow_corr , Power corrected for noise floor
  145 + grass_ZR , Mean of noise floor
  146 + ZR_amp_corr , Amplitude corrected for noise floor
  147 + G_ZR , Copol Gain of antenna
  148 + X_ZR , Xpol Gain of antenna
  149 + ZR_amp_cal , Amplitude corrected for antenna pattern
  150 + ZR_pow_cal , Power corrected for antenna pattern
  151 + NL_pow_cal_xp, Power corrected for antenna gain + xpol contribution
  152 + ZR_pow_coh , Coherent contribution of the power
  153 + NL_pow_coh_xp, Coherent contribution of the power corrected for xpol
  154 + Gamma_L , Reflection coeff : Ratio of reflected over direct
  155 + Gamma_LR , Polarimetric ratio: Ratio of LHCP over RHCP
  156 + Gamma_L_xp , Reflection coeff : Ratio of reflected over direct (xpol)
  157 + Gamma_LR_xp , Polarimetric ratio: Ratio of LHCP over RHCP (xpol)
  158 +
  159 +
  160 +l0b_to_l1a.py
  161 +=============
  162 +
  163 +
  164 +.. automodule:: pyGLORI.process.l0b_to_l1a
  165 + :members:
... ...
source/PRO/PRO_L0toL0b.rst 0 → 100644
... ... @@ -0,0 +1,82 @@
  1 +.. raw:: latex
  2 +
  3 + \clearpage
  4 +
  5 +.. _PRO_L0_L0b:
  6 +
  7 +*********************
  8 +Level 0 to Level 0b
  9 +*********************
  10 +
  11 +
  12 +Module description
  13 +------------------
  14 +The goal of this module is to compute the basic observables for
  15 +land applications. Nadir (nad) and Zenith (zen) subscript are used only when
  16 +the processing is not generic to all channels.
  17 +
  18 +#. Resynchronize the data acquisitions by reading the SOW info from the
  19 + navigation message and time tagging of the L0b files
  20 +
  21 +#. Performs waveform peak search using various techniques
  22 +
  23 +#. Store position of peaks and max values
  24 +
  25 +#. Compute and store waveforms noise floor.
  26 +
  27 +
  28 +
  29 +Inputs
  30 +------
  31 +
  32 +The required inputs for this routine are:
  33 +
  34 +- A valid configuration files
  35 +- L0 files in the cStarlight netcdf format (see :ref:`PRO_raw_L0`: for details)
  36 +
  37 +Outputs
  38 +-------
  39 +
  40 +.. csv-table:: L0 to L1a output file
  41 + :stub-columns: 1
  42 + :widths: 4 20
  43 + :header: Field, Description
  44 +
  45 + DateTime , Python Datetime
  46 + lat_spec , Latitude of specular point
  47 + lon_spec , Longitude of specular point
  48 + DEM_spec , DEM at specular point location
  49 + SV_elev , Elevation at specular point
  50 + SV_azimuth , Azimuth at specular point
  51 + ZR_amp_res , Incoherently Averaged amplitude
  52 + ZR_pow_res , Incoherently averaged power
  53 + ZR_var_res , Incoherently averaged variance
  54 + ZR_ph_res , Incoherently averaged phase
  55 + grass_ZR_sq , Mean of noise floor squared
  56 + ZR_pow_corr , Power corrected for noise floor
  57 + grass_ZR , Mean of noise floor
  58 + ZR_amp_corr , Amplitude corrected for noise floor
  59 + G_ZR , Copol Gain of antenna
  60 + X_ZR , Xpol Gain of antenna
  61 + ZR_amp_cal , Amplitude corrected for antenna pattern
  62 + ZR_pow_cal , Power corrected for antenna pattern
  63 + NL_pow_cal_xp, Power corrected for antenna gain + xpol contribution
  64 + ZR_pow_coh , Coherent contribution of the power
  65 + NL_pow_coh_xp, Coherent contribution of the power corrected for xpol
  66 + Gamma_L , Reflection coeff : Ratio of reflected over direct
  67 + Gamma_LR , Polarimetric ratio: Ratio of LHCP over RHCP
  68 + Gamma_L_xp , Reflection coeff : Ratio of reflected over direct (xpol)
  69 + Gamma_LR_xp , Polarimetric ratio: Ratio of LHCP over RHCP (xpol)
  70 +
  71 +
  72 +Functions and routines
  73 +----------------------
  74 +
  75 +.. automodule:: pyGLORI.process.l0_to_l0b
  76 + :members:
  77 +
  78 +
  79 +
  80 +Issues / recommendations
  81 +------------------------
  82 +
... ...
source/PRO/PRO_L0toL1a.rst
... ... @@ -1,91 +0,0 @@
1   -.. _PRO_L0_L1a:
2   -
3   -*********************
4   -Level 0 to Level 1a
5   -*********************
6   -
7   -Process_L0_to_L1A.py
8   -====================
9   -
10   -Module description
11   -------------------
12   -The goal of this module is to compute the basic observables for land applications:
13   -
14   -#. Resynchronize the data acquisitions by reading the SOW info from the navigation message
15   -#. Performs additional coherent averaging by performing phase aware integration
16   -#. Correct coherent signal according to Antenna gain pattern and measurement noise.
17   -#. Compute ICF as
18   -
19   - .. math::
20   -
21   - ICF_{corr} = \frac{G_d}{G_r} \frac{Y_{r,max}-B_r}{Y_{d,max}-B_d}
22   - e^{j(\phi_{r,max}-\phi_{d,max})}
23   -#. Performs incoherent averaging
24   -#. Compute phase variance
25   -#. Merge in one file:
26   -
27   - - The Direct RHCP signal peak correlation complex amplitude
28   - - The direct estimated Doppler
29   - - The reflected LHCP signal peak correlation complex amplitude
30   - - The reflected RHCP signal peak correlation complex amplitude
31   - - The estimated delay of the reflected LHCP signal in lags
32   - - Ancillary info from L0 files
33   - - PRN
34   - - StartSOW, ifSOWread, WN,
35   - - fs,
36   - - El, Az
37   -
38   -
39   -.. csv-table:: Processing steps
40   - :stub-columns: 1
41   - :widths: 1 10 8 6 4
42   - :header: Step, Processing, Output, Variable, format
43   -
44   - 1, Read waveforms , Raw WFs , WF_NL , 2D cpx WF
45   - 2, Additional coh. avging , Coherently avgd WFs, WF_NL_co , 2D cpx WF
46   - 3, Find coh peak , peak position , NL_loc_co , 1D int
47   - 4, Store coh peak value , peak value TS , NL_pow_co , 1D cpx
48   - 5, Compute coh noise floor, noise floor level , grass_NL_co , 1D float
49   - 6, abs and correct for NF , noise free TS , NL_pow_corr_co, 1D float
50   -
51   -Inputs
52   -------
53   -
54   -Outputs
55   --------
56   -
57   -.. csv-table:: L0 to L1a output file
58   - :stub-columns: 1
59   - :widths: 4 20
60   - :header: Field, Description
61   -
62   - DateTime , Python Datetime
63   - lat_spec , Latitude of specular point
64   - lon_spec , Longitude of specular point
65   - DEM_spec , DEM at specular point location
66   - SV_elev , Elevation at specular point
67   - SV_azimuth , Azimuth at specular point
68   - ZR_amp_res , Incoherently Averaged amplitude
69   - ZR_pow_res , Incoherently averaged power
70   - ZR_var_res , Incoherently averaged variance
71   - ZR_ph_res , Incoherently averaged phase
72   - grass_ZR_sq , Mean of noise floor squared
73   - ZR_pow_corr , Power corrected for noise floor
74   - grass_ZR , Mean of noise floor
75   - ZR_amp_corr , Amplitude corrected for noise floor
76   - G_ZR , Copol Gain of antenna
77   - X_ZR , Xpol Gain of antenna
78   - ZR_amp_cal , Amplitude corrected for antenna pattern
79   - ZR_pow_cal , Power corrected for antenna pattern
80   - NL_pow_cal_xp, Power corrected for antenna gain + xpol contribution
81   - ZR_pow_coh , Coherent contribution of the power
82   - NL_pow_coh_xp, Coherent contribution of the power corrected for xpol
83   - Gamma_L , Reflection coeff : Ratio of reflected over direct
84   - Gamma_LR , Polarimetric ratio: Ratio of LHCP over RHCP
85   - Gamma_L_xp , Reflection coeff : Ratio of reflected over direct (xpol)
86   - Gamma_LR_xp , Polarimetric ratio: Ratio of LHCP over RHCP (xpol)
87   -
88   -
89   -Issues / recommendations
90   -------------------------
91   -
source/PRO/PRO_L1atoL1b.rst
  1 +.. raw:: latex
  2 +
  3 + \clearpage
  4 +
1 5 .. _PRO_L1a_L1b:
2 6  
3 7 *********************
... ...
source/PRO/PRO_common.rst
... ... @@ -14,11 +14,17 @@ geom.py
14 14  
15 15 Geometry related functions
16 16  
  17 +.. automodule:: pyGLORI.common.glori_geom
  18 + :members:
  19 +
17 20  
18 21 glori_antennas.py
19 22 =================
20 23  
21   -antenna related functions
  24 +Antennas related functions
  25 +
  26 +.. automodule:: pyGLORI.common.glori_antennas
  27 + :members:
22 28  
23 29  
24 30 glori_common.py
... ... @@ -26,6 +32,9 @@ glori_common.py
26 32  
27 33 general purpose functions (related with time manipulation among others)
28 34  
  35 +.. automodule:: pyGLORI.common.glori_common
  36 + :members:
  37 +
29 38  
30 39 glori_io.py
31 40 ===========
... ... @@ -33,8 +42,6 @@ glori_io.py
33 42 I/O related routines, linked to netCDF, HDF and Pickle specific formats for
34 43 L0, L1a and L1b
35 44  
  45 +.. automodule:: pyGLORI.common.glori_io
  46 + :members:
36 47  
37   -local_angle.py
38   -==============
39   -
40   -Functions related to calculation of angle differences and rotations.
... ...
source/PRO/PRO_index.rst
... ... @@ -13,7 +13,8 @@ GLORI Processing Chain
13 13 PRO_common
14 14 PRO_ancillary
15 15 PRO_rawtoL0
16   - PRO_L0toL1a
  16 + PRO_L0toL0b
  17 + PRO_L0btoL1a
17 18 PRO_L1atoL1b
18 19 PRO_plot
19 20 PRO_ToDo
20 21 \ No newline at end of file
... ...
source/PRO/PRO_install.rst
... ... @@ -83,6 +83,11 @@ Linux packages install
83 83 - hdf5-devel
84 84 - netcdf-devel
85 85  
  86 +for documentation
  87 +
  88 +- texlive
  89 +- texlive-collections-latexextra
  90 +
86 91 All these packages can be installed at once with the following commands::
87 92  
88 93 sudo dnf install gcc-gfortran blas-devel lapack-devel atlas-devel
... ... @@ -95,9 +100,11 @@ Python module installation
95 100 ..........................
96 101  
97 102 - h5py, netCDF4
98   -- pandas
99   -- geopandas
  103 +- pandas, geopandas
100 104 - fiona, shapely, pyproj, transforms3d
  105 +- SRTM.py, requests
  106 +- PyUblox (in process folder), pynmea2
  107 +- simplekml
101 108  
102 109 Python module installation with anaconda
103 110 ----------------------------------------
... ...
source/PRO/PRO_intro.rst
... ... @@ -39,7 +39,10 @@ The Processing chain is based in the following subsystems
39 39 - :ref:`PRO_raw_L0`: Reads, converts and processes Raw ADC data
40 40 to provide uncalibrated waveforms.
41 41  
42   -- :ref:`PRO_L0_L1a`: Integrates L0 data to reduce noise, add ancillary
  42 +- :ref:`PRO_L0_L0b`: Time tag the files with GPS time read from the data.
  43 + Extract waveform maximums.
  44 +
  45 +- :ref:`PRO_L0b_L1a`: Integrates L0 data to reduce noise, add ancillary
43 46 information to the file. Check and interpolate time information.
44 47  
45 48 - :ref:`PRO_L1a_L1b`: calibrates L1A signal for attitude,
... ...
source/PRO/PRO_rawtoL0.rst
... ... @@ -4,9 +4,6 @@
4 4 Raw to Level 0
5 5 *********************
6 6  
7   -raw_to_l0.py
8   -============
9   -
10 7 Module description
11 8 ------------------
12 9 This module performs the basic observable computation (correlation waveforms)
... ... @@ -48,17 +45,16 @@ Outputs
48 45 Issues / recommendations
49 46 ------------------------
50 47  
51   -cStarlight.py
52   -=============
53 48  
54   -Module description
55   -------------------
  49 +Functions and routines
  50 +----------------------
56 51  
57   -Inputs
58   -------
  52 +.. automodule:: pyGLORI.process.raw_to_l0
  53 + :members:
59 54  
60   -Outputs
61   --------
62 55  
63   -Issues / recommendations
64   -------------------------
  56 +Functions and routines
  57 +----------------------
  58 +
  59 +.. automodule:: pyGLORI.process.cstarlight
  60 + :members:
65 61 \ No newline at end of file
... ...
source/conf.py
... ... @@ -18,7 +18,7 @@ import os
18 18 # If extensions (or modules to document with autodoc) are in another directory,
19 19 # add these directories to sys.path here. If the directory is relative to the
20 20 # documentation root, use os.path.abspath to make it absolute, like shown here.
21   -#sys.path.insert(0, os.path.abspath('.'))
  21 +sys.path.insert(0, os.path.abspath('../../processing/python_code/'))
22 22  
23 23 # -- General configuration ------------------------------------------------
24 24  
... ... @@ -36,6 +36,7 @@ extensions = [
36 36 'sphinx.ext.coverage',
37 37 'sphinx.ext.mathjax',
38 38 'sphinx.ext.ifconfig',
  39 + 'sphinx.ext.napoleon',
39 40 # 'rst2pdf.pdfbuilder',
40 41 ]
41 42  
... ... @@ -215,7 +216,7 @@ htmlhelp_basename = &#39;GLORI_docdoc&#39;
215 216  
216 217 latex_elements = {
217 218 # The paper size ('letterpaper' or 'a4paper').
218   -#'papersize': 'letterpaper',
  219 +'papersize': 'a4paper',
219 220  
220 221 # The font size ('10pt', '11pt' or '12pt').
221 222 #'pointsize': '10pt',
... ...
source/index.rst
... ... @@ -10,6 +10,7 @@ Contents:
10 10  
11 11 .. toctree::
12 12 :maxdepth: 2
  13 + :numbered:
13 14  
14 15 intro
15 16 blocks
... ...