Commit 14a564f72be9815bd7f7da3f89cc1ff14ab99f89

Authored by Erwan Motte
1 parent 38cab1d3
Exists in master

Finished converting the available processing chain Google Doc documentation to S…

…phinx documentation. Many blank parts to be populated
source/PRO/PRO_L0toL1a.rst
... ... @@ -2,4 +2,90 @@
2 2  
3 3 *********************
4 4 Level 0 to Level 1a
5   -*********************
6 5 \ No newline at end of file
  6 +*********************
  7 +
  8 +Process_L0_to_L1A.py
  9 +====================
  10 +
  11 +Module description
  12 +------------------
  13 +The goal of this module is to compute the basic observables for land applications:
  14 +
  15 +#. Resynchronize the data acquisitions by reading the SOW info from the navigation message
  16 +#. Performs additional coherent averaging by performing phase aware integration
  17 +#. Correct coherent signal according to Antenna gain pattern and measurement noise.
  18 +#. Compute ICF as
  19 +
  20 + .. math::
  21 +
  22 + ICF_{corr} = \frac{G_d}{G_r} \frac{Y_{r,max}-B_r}{Y_{d,max}-B_d}
  23 + e^{j(\phi_{r,max}-\phi_{d,max})}
  24 +#. Performs incoherent averaging
  25 +#. Compute phase variance
  26 +#. Merge in one file:
  27 +
  28 + - The Direct RHCP signal peak correlation complex amplitude
  29 + - The direct estimated Doppler
  30 + - The reflected LHCP signal peak correlation complex amplitude
  31 + - The reflected RHCP signal peak correlation complex amplitude
  32 + - The estimated delay of the reflected LHCP signal in lags
  33 + - Ancillary info from L0 files
  34 + - PRN
  35 + - StartSOW, ifSOWread, WN,
  36 + - fs,
  37 + - El, Az
  38 +
  39 +
  40 +.. csv-table:: Processing steps
  41 + :stub-columns: 1
  42 + :widths: 1 10 8 6 4
  43 + :header: Step, Processing, Output, Variable, format
  44 +
  45 + 1, Read waveforms , Raw WFs , WF_NL , 2D cpx WF
  46 + 2, Additional coh. avging , Coherently avgd WFs, WF_NL_co , 2D cpx WF
  47 + 3, Find coh peak , peak position , NL_loc_co , 1D int
  48 + 4, Store coh peak value , peak value TS , NL_pow_co , 1D cpx
  49 + 5, Compute coh noise floor, noise floor level , grass_NL_co , 1D float
  50 + 6, abs and correct for NF , noise free TS , NL_pow_corr_co, 1D float
  51 +
  52 +Inputs
  53 +------
  54 +
  55 +Outputs
  56 +-------
  57 +
  58 +.. csv-table:: L0 to L1a output file
  59 + :stub-columns: 1
  60 + :widths: 4 20
  61 + :header: Field, Description
  62 +
  63 + DateTime , Python Datetime
  64 + lat_spec , Latitude of specular point
  65 + lon_spec , Longitude of specular point
  66 + DEM_spec , DEM at specular point location
  67 + SV_elev , Elevation at specular point
  68 + SV_azimuth , Azimuth at specular point
  69 + ZR_amp_res , Incoherently Averaged amplitude
  70 + ZR_pow_res , Incoherently averaged power
  71 + ZR_var_res , Incoherently averaged variance
  72 + ZR_ph_res , Incoherently averaged phase
  73 + grass_ZR_sq , Mean of noise floor squared
  74 + ZR_pow_corr , Power corrected for noise floor
  75 + grass_ZR , Mean of noise floor
  76 + ZR_amp_corr , Amplitude corrected for noise floor
  77 + G_ZR , Copol Gain of antenna
  78 + X_ZR , Xpol Gain of antenna
  79 + ZR_amp_cal , Amplitude corrected for antenna pattern
  80 + ZR_pow_cal , Power corrected for antenna pattern
  81 + NL_pow_cal_xp, Power corrected for antenna gain + xpol contribution
  82 + ZR_pow_coh , Coherent contribution of the power
  83 + NL_pow_coh_xp, Coherent contribution of the power corrected for xpol
  84 + Gamma_L , Reflection coeff : Ratio of reflected over direct
  85 + Gamma_LR , Polarimetric ratio: Ratio of LHCP over RHCP
  86 + Gamma_L_xp , Reflection coeff : Ratio of reflected over direct (xpol)
  87 + Gamma_LR_xp , Polarimetric ratio: Ratio of LHCP over RHCP (xpol)
  88 +
  89 +
  90 +Issues / recommendations
  91 +------------------------
  92 +
... ...
source/PRO/PRO_L1atoL1b.rst
... ... @@ -2,4 +2,23 @@
2 2  
3 3 *********************
4 4 Level 1a to Level 1b
5   -*********************
6 5 \ No newline at end of file
  6 +*********************
  7 +
  8 +Process_L1A_to_L1B.py
  9 +=====================
  10 +
  11 +Module description
  12 +------------------
  13 +
  14 +- Ancillary info from external files (time dependant)
  15 +- SV Elevation and Azimuth
  16 +- Aircraft, latitude, longitude and altitude
  17 +- heading, pitch, roll
  18 +- Zenith and Nadir depointing angle
  19 +
  20 +
  21 +Inputs
  22 +------
  23 +
  24 +Outputs
  25 +-------
... ...
source/PRO/PRO_ToDo.rst 0 → 100644
... ... @@ -0,0 +1,60 @@
  1 +************************
  2 +TODO List
  3 +************************
  4 +
  5 +
  6 +General Requirements / Issues / Optimization
  7 +=============================================
  8 +
  9 +Platform
  10 +--------
  11 +Currently running in a virtualBox
  12 +- Fedora 22 ok
  13 +- Fedora 23: Issues with pyproj 1.9.2 (azimuth error in the fwd function)
  14 +
  15 +
  16 +Optimizing data containers
  17 +--------------------------
  18 +
  19 +HDF vs. Pickle
  20 +
  21 +- For one full dataset (Flight 6):
  22 +
  23 + - Write
  24 +
  25 + - 14:50:15 : INFO : pyGLORI.process.l1a_to_l1b : h5 write : 7.608558 s
  26 + - 14:50:23 : INFO : pyGLORI.process.l1a_to_l1b : pickle write : 7.875547 s
  27 +
  28 + - Read
  29 +
  30 + - 14:45:37 : INFO : pyGLORI.plot.plot_l1b_kml : pickle read : 4.510629 s
  31 + - 14:45:41 : INFO : pyGLORI.plot.plot_l1b_kml : h5 read : 4.451701 s
  32 +
  33 +Conclusions:
  34 +
  35 +- No winner in terms of size and performance
  36 +- H5 maximizes allows for cross-application exchange
  37 +- pickle maximizes python and pandas version compatibility
  38 +
  39 +
  40 +
  41 +Plotting
  42 +---------
  43 +
  44 +Using something else than KML?
  45 +
  46 +- Python + openstreet map, such as in http://www.mdpi.com/1424-8220/15/11/28287
  47 +- Interactive maps
  48 +
  49 + - Use folium python package!!
  50 + http://folium.readthedocs.org/en/latest/
  51 + examples here, including legends and all:
  52 + http://nbviewer.ipython.org/github/ocefpaf/folium_notebooks/tree/master/
  53 + - data format in geoJSON: http://geojson.org/geojson-spec.html
  54 +
  55 +
  56 +- Non interactive
  57 +
  58 + - Cartopy
  59 + - Basemaps
  60 +
... ...
source/PRO/PRO_ancillary.rst
... ... @@ -37,14 +37,14 @@ for specular point calculation and altimetric corrections.
37 37 The module just reads a plain-text file using pandas read_table() function
38 38 and store it into a pickled pandas file.
39 39  
40   -inputs
  40 +Inputs
41 41 ------
42   -
43 42 The processing chain configuration file specifying paths and parameters
44 43 is the only needed input.
45 44 The related fields in the configuration files are the following:
46 45  
47   -ubx_dir, ubx_prefix
  46 +- ubx_dir
  47 +- ubx_prefix
48 48  
49 49 The routine will be looking for the file in the folder “ubx_dir”
50 50 with filename “ubx_prefix” and extension “.azel”
... ... @@ -90,283 +90,230 @@ the raw ubx file with the following command:
90 90 Outputs
91 91 -------
92 92 One file in the pickled pandas format is generated:
93   -**{Flight_id}_rtk.p**: Space Vehicles Azimuth and elevation from
  93 +- **{Flight_id}_rtk.p**: Space Vehicles Azimuth and elevation from
94 94 GLORI Ublox receiver processed by RTKLib
95   -Field
96   -Unit
97   -Description
98   -DateTime
99   -Python DateTime
100   -Date and time of the sample
101   -sv
102   -N/A
103   -Satellite Space Vehicle ID (PRN ID for GPS)
104   -azim_te
105 95  
  96 +.. csv-table:: Anc_read_rtk.py output file
  97 + :stub-columns: 1
  98 + :widths: 2 4 8
  99 + :header: Field, Unit, Description
  100 +
  101 + DateTime, Python DateTime, Date and time of the sample
  102 + sv, N/A, Satellite Space Vehicle ID (PRN ID for GPS)
  103 + azim_te, integer degrees, GLORI UBlox RTKLib processed SV azimuth
  104 + elev_te, integer degrees, GLORI UBlox RTKLib processed SV elevation
106 105  
107   -integer degrees
108   -GLORI UBlox RTKLib processed SV azimuth relative to receiver location
109   -elev_te
110   -integer degrees
111   -GLORI UBlox RTKLib processed SV elevation relative to receiver location
112 106  
113 107 Anc_read_UBX_v2.py
114 108 ==================
115 109  
116 110 Module description
117   -The purpose of this module is to read the data recorded by the Ublox GPS receiver included in the GLORI instrument. Depending on the version of the instrument, data was saved in different formats (UBX + NMEA for the first campaign, UBX only for the second campaign). The module is based on the pyUblox (https://github.com/tridge/pyUblox) and pynmea2 (https://github.com/Knio/pynmea2) packages.
118   -inputs
119   -The processing chain configuration file specifying paths and parameters is the only needed input.
  111 +------------------
  112 +The purpose of this module is to read the data recorded by the
  113 +Ublox GPS receiver included in the GLORI instrument.
  114 +Depending on the version of the instrument, data was saved
  115 +in different formats (UBX + NMEA for the first campaign,
  116 +UBX only for the second campaign). The module is based on
  117 +the pyUblox_ and pynmea2_ packages.
  118 +
  119 +.. _pyUblox: https://github.com/tridge/pyUblox
  120 +.. _pynmea2: https://github.com/Knio/pynmea2
  121 +
  122 +Inputs
  123 +------
  124 +The processing chain configuration file specifying paths and parameters
  125 +is the only needed input.
120 126 The related fields in the configuration files are the following:
121   -Field
122   -Description
123   -Example
124   -ubx_dir
125   -directory of the ubx files relative to the base directory
126   -‘UBlox/’
127   -ubx_prefix
128   -Name of the ublox file, without the ‘.ubx’ extension
129   -ubx_type
130   -Type of data format 'UBX+NMEA' or ‘UBX_ONLY’
131   -‘UBX_ONLY’
  127 +
  128 +- ubx_dir
  129 +- ubx_prefix
  130 +- ubx_type ('UBX+NMEA' or ‘UBX_ONLY’)
132 131  
133 132 Outputs
134   -2 files are generated by the anc_read_UBX_v2.py module, in the pickled pandas format:
135   -{Flight_id}_ubx_pos.p: 3D position GLORI from Ublox receiver
136   -Field
137   -Unit
138   -Description
139   -DateTime
140   -Python DateTime
141   -Date and time of the sample
142   -sow
143   -seconds
144   -GPS Time of the Week of the sample
145   -lat
146   -decimal degrees
147   -GLORI UBlox-based latitude
148   -lon
149   -decimal degrees
150   -GLORI UBlox-based longitude
151   -hMSL
152   -m
153   -GLORI UBlox-based height over mean sea level
154   -hOE
155   -m
156   -GLORI UBlox-based height over Ellipsoid
157   -course
158   -decimal degrees
159   -GLORI UBlox-based course
160   -gSpeed
161   -m/s
162   -GLORI UBlox-based ground speed
163   -hAcc
164   -m
165   -GLORI UBlox-based horizontal Accuracy
166   -vAcc
167   -m
168   -GLORI UBlox-based vertical Accuracy
169   -
170   -{Flight_id}_ubx_SVs.p: Space Vehicles information from GLORI Ublox receiver
171   -Field
172   -Unit
173   -Description
174   -DateTime
175   -Python DateTime
176   -Date and time of the sample
177   -sow
178   -seconds
179   -GPS Time of the Week of the sample
180   -sv
181   -N/A
182   -Satellite Space Vehicle ID (PRN ID for GPS)
183   -azim
184   -
185   -
186   -integer degrees
187   -GLORI UBlox-based SV azimuth relative to receiver location
188   -elev
189   -integer degrees
190   -GLORI UBlox-based SV elevation relative to receiver location
191   -cn0
192   -dB
193   -GLORI UBlox-based SV Carrier to Noise Ratio
194   -cpMes
195   -radians
196   -GLORI UBlox-based SV Code Phase
197   -doMes
198   -Hz
199   -GLORI UBlox-based SV Doppler
200   -prMes
201   -m
202   -GLORI UBlox-based SV PseudoRange
  133 +-------
  134 +2 files are generated by the anc_read_UBX_v2.py module,
  135 +in the pickled pandas format:
  136 +
  137 +- **{Flight_id}_ubx_pos.p**: 3D position GLORI from Ublox receiver
  138 +
  139 +.. csv-table:: {Flight_id}_ubx_pos.p output file
  140 + :stub-columns: 1
  141 + :widths: 2 4 8
  142 + :header: Field, Unit, Description
  143 +
  144 +
  145 + DateTime, Python DateTime, Date and time of the sample
  146 + sow , seconds , GPS Time of the Week of the sample
  147 + lat , decimal degrees, GLORI UBlox-based latitude
  148 + lon , decimal degrees, GLORI UBlox-based longitude
  149 + hMSL , m , GLORI UBlox-based height over mean sea level
  150 + hOE , m , GLORI UBlox-based height over Ellipsoid
  151 + course , decimal degrees, GLORI UBlox-based course
  152 + gSpeed , m/s , GLORI UBlox-based ground speed
  153 + hAcc , m , GLORI UBlox-based horizontal Accuracy
  154 + vAcc , m , GLORI UBlox-based vertical Accuracy
  155 +
  156 +- **{Flight_id}_ubx_SVs.p**: Space Vehicles information from GLORI Ublox receiver
  157 +
  158 +.. csv-table:: {Flight_id}_ubx_SVs.p output file
  159 + :stub-columns: 1
  160 + :widths: 2 4 8
  161 + :header: Field, Unit, Description
  162 +
  163 + DateTime, Python DateTime, Date and time of the sample
  164 + sow , seconds , GPS Time of the Week of the sample
  165 + sv , N/A , Satellite Space Vehicle ID (PRN ID for GPS)
  166 + azim , integer degrees, GLORI UBlox-based SV azimuth
  167 + elev , integer degrees, GLORI UBlox-based SV elevation
  168 + cn0 , dB , GLORI UBlox-based SV Carrier to Noise Ratio
  169 + cpMes , radians , GLORI UBlox-based SV Code Phase
  170 + doMes , Hz , GLORI UBlox-based SV Doppler
  171 + prMes , m , GLORI UBlox-based SV PseudoRange
  172 +
203 173 Issues / Recommendations
204   -Various format and time resolution depending on campaigns (UBX+NMEA, UBX only)
205   -Solution: Create specific cases for each configuration
206   -Issues in the computation of the UTC time (jumps of 16s in one case.)
207   -Recompute time from received iTOW
  174 +------------------------
  175 +- Various format and time resolution depending on campaigns (UBX+NMEA, UBX only)
  176 +
  177 + - Solution: Create specific cases for each configuration
  178 +
  179 +- Issues in the computation of the UTC time (jumps of 16s in one case.)
  180 +
  181 + - Recompute time from received iTOW
208 182  
209 183 Anc_read_SAFIRE_v2.py
210 184 =====================
211 185  
212 186 Module description
213   -The purpose of this module is to read the data recorded by SAFIRE ATR-42 on-board instrumentation, i.e. from an AIRINS system. This mainly include information from an independant GPS receiver attached to an Inertial Navigation System (INS) as well as a Radio Altimeter providing independent estimate of range under the plane track. Is has to be noted that depending on the flights, data fields, data frequency and data format was changed. See Post-Campaign instrument analysis document for more information.
214   -inputs
215   -The processing chain configuration file specifying paths and parameters is the only needed input.
  187 +------------------
  188 +The purpose of this module is to read the data recorded by SAFIRE ATR-42
  189 +on-board instrumentation, i.e. from an AIRINS system.
  190 +This mainly include information from an independant GPS receiver attached
  191 +to an Inertial Navigation System (INS) as well as a Radio Altimeter
  192 +providing independent estimate of range under the plane track.
  193 +It has to be noted that depending on the flights, data fields,
  194 +data frequency and data format was changed.
  195 +See Post-Campaign instrument analysis document for more information.
  196 +
  197 +Inputs
  198 +------
  199 +The processing chain configuration file specifying paths
  200 +and parameters is the only needed input.
216 201 The related fields in the configuration files are the following:
217   -Field
218   -Description
219   -Example
220   -SAFIRE_dir
221   -directory of the SAFIRE files relative to the base directory
222   -'SAFIRE/'
223   -SAFIRE_file
224   -Name of the SAFIRE file, with the extension
225   -‘ATR_10Hz-as_kuros_20141120_as140040.txt'
226   -SAFIRE_type
227   -Type of data format 'xls', ‘txt_v1’ or ‘txt_v2’
228   -'xls'
  202 +
  203 +- SAFIRE_dir
  204 +- SAFIRE_file
  205 +- SAFIRE_type ('xls', ‘txt_v1’ or ‘txt_v2’)
  206 +
229 207  
230 208 Outputs
231   -1 file is generated by the anc_read_SAFIRE_v2.py module, in the pickled pandas format:
232   -{Flight_id}_saf.p: SAFIRE ancillary information
233   -Field
234   -Unit
235   -Description
236   -DateTime
237   -Python DateTime
238   -Date and time of the sample
239   -lat_ins
240   -decimal degrees
241   -SAFIRE AIRINS-based latitude
242   -lon_ins
243   -decimal degrees
244   -SAFIRE AIRINS-based longitude
245   -hMSL_ins
246   -m
247   -SAFIRE AIRINS-based height over mean sea level
248   -heading_ins
249   -m
250   -SAFIRE AIRINS-based aircraft heading
251   -pitch_ins
252   -decimal degrees
253   -SAFIRE AIRINS-based aircraft pitch
254   -roll_ins
255   -m/s
256   -SAFIRE AIRINS-based aircraft roll
257   -alt_ra
258   -m
259   -SAFIRE Radio altimeter estimated range under the plane track
  209 +-------
  210 +1 file is generated by the anc_read_SAFIRE_v2.py module,
  211 +in the pickled pandas format:
  212 +- **{Flight_id}_saf.p**: SAFIRE ancillary information
  213 +
  214 +.. csv-table:: {Flight_id}_saf.p output file
  215 + :stub-columns: 1
  216 + :widths: 2 4 8
  217 + :header: Field, Unit, Description
  218 +
  219 + DateTime , Python DateTime, Date and time of the sample
  220 + lat_ins , decimal degrees, SAFIRE AIRINS-based latitude
  221 + lon_ins , decimal degrees, SAFIRE AIRINS-based longitude
  222 + hMSL_ins , m , SAFIRE AIRINS-based height over mean sea level
  223 + heading_ins, m , SAFIRE AIRINS-based aircraft heading
  224 + pitch_ins , decimal degrees, SAFIRE AIRINS-based aircraft pitch
  225 + roll_ins , m/s , SAFIRE AIRINS-based aircraft roll
  226 + alt_ra , m , SAFIRE Radio altimeter range
260 227  
261 228 Anc_post_process.py
262 229 ===================
263 230  
264 231 Module description
265   -This modules takes the output of the Anc_read_UBX.py and Anc_read_SAFIRE.py modules in order to generate consolidated ancillary data files ready to be used by other procedures. It performs the following tasks:
266   -Process Plane position info
267   -Read Ublox information
268   -Filter data for a minimum flight altitude
269   -SRTM DEM lookup for the flight path (using https://github.com/tkrajina/srtm.py)
270   -Read Safire information
271   -Compute Zenith Antenna Elevation and Azimuth from Attitude information
272   -Interpolate SAFIRE according to Ublox position info
273   -Process SVs info
274   -Filter for SV elevation angle and SV type
275   -Compute the approximate footprint location for the filtered SVs, using pyProj
276   -SRTM DEM lookup for the Specular point location
  232 +------------------
  233 +
  234 +This modules takes the output of the Anc_read_UBX.py and Anc_read_SAFIRE.py
  235 +modules in order to generate consolidated ancillary data files
  236 +ready to be used by other procedures. It performs the following tasks:
  237 +
  238 +- Process Plane position info
  239 +- Read Ublox information
  240 +- Filter data for a minimum flight altitude
  241 +- SRTM DEM lookup for the flight path (using `strm.py`_ )
  242 +- Read Safire information
  243 +- Compute Zenith Antenna Elevation and Azimuth from Attitude information
  244 +- Interpolate SAFIRE according to Ublox position info
  245 +- Process SVs info
  246 +- Filter for SV elevation angle and SV type
  247 +- Compute the approximate footprint location for the filtered SVs, using pyProj
  248 +- SRTM DEM lookup for the Specular point location
  249 +
  250 +.. _strm.py: https://github.com/tkrajina/srtm.py
  251 +
277 252 Inputs
278   -The processing chain configuration file specifying paths and parameters is the only needed input
  253 +------
  254 +The processing chain configuration file specifying
  255 +paths and parameters is the only needed input
  256 +
279 257 Outputs
280   -3 files are generated by the anc_post_process.py module, all of them inthe pickled pandas format:
281   -{Flight_id}_post_pos.p: GLORI Ublox position + SAFIRE INS attitude information + DEM information interpolated (nearest neighbour) at the Ublox GPS rate
282   -Field
283   -Unit
284   -Description
285   -DateTime
286   -Python DateTime
287   -Date and time of the sample
288   -sow
289   -seconds
290   -GPS Time of the Week of the sample
291   -lat
292   -decimal degrees
293   -GLORI UBlox-based latitude
294   -lon
295   -decimal degrees
296   -GLORI UBlox-based longitude
297   -gSpeed
298   -m/s
299   -GLORI UBlox-based ground speed
300   -hMSL
301   -m
302   -GLORI UBlox-based height over mean sea level
303   -course
304   -decimal degrees
305   -UBlox-based true course
306   -DEM
307   -m
308   -Digital Elevation Model (SRTM) height under the flight track
309   -heading_ins
310   -decimal degrees
311   -SAFIRE AIRINS-based aircraft heading
312   -pitch_ins
313   -decimal degrees
314   -SAFIRE AIRINS-based aircraft pitch
315   -roll_ins
316   -decimal degrees
317   -SAFIRE AIRINS-based aircraft roll
318   -lat_ins
319   -decimal degrees
320   -SAFIRE AIRINS-based aircraft latitude
321   -lon_ins
322   -decimal degrees
323   -SAFIRE AIRINS-based aircraft longitude
324   -hMSL_ins
325   -m
326   -SAFIRE AIRINS-based aircraft altitude over mean sea level
327   -alt_ra
328   -m
329   -SAFIRE Radio altimeter estimated range under the plane track
330   -el_zen
331   -decimal degrees
332   -Zenith antenna boresight elevation angle (computed from SAFIRE attitude information)
333   -az_zen
334   -decimal degrees
335   -Zenith antenna boresight azimuth direction (computed from SAFIRE attitude information)
336   -
337   -{Flight_id}_post_SVs.p: GLORI Ublox-based Space Vehicles information: Approximate azimuth and elevation relative to plane, and related computed position of specular point on ground.
338   -Field
339   -Unit
340   -Description
341   -sow
342   -seconds
343   -GPS Time of the Week of the sample
344   -sv
345   -N/A
346   -Satellite Space Vehicle ID (PRN ID for GPS)
347   -azim
348   -
349   -
350   -integer degrees
351   -GLORI UBlox-based satellite azimuth relative to receiver location
352   -elev
353   -integer degrees
354   -GLORI UBlox-based satellite elevation relative to receiver location
355   -speclat
356   -decimal degrees
357   -Computed specular point approximate latitude
358   -speclon
359   -decimal degrees
360   -Computed specular point approximate longitude
361   -specDEM
362   -m
363   -Digital Elevation Model (SRTM) value at the specular point location
364   -heightOG
365   -m
366   -Height difference between DEM at specular point location and flight altitude (can be used for altimetric validation purposes)
367   -
368   -{Flight_id}_post_SVs.p: Same fields as above, but with a reduced temporal resolution defined by cfg[‘lowres_step’], typically 2 seconds. File used for generating kml files.
  258 +-------
  259 +3 files are generated by the anc_post_process.py module,
  260 +all of them inthe pickled pandas format:
  261 +
  262 +- **{Flight_id}_post_pos.p**: GLORI Ublox position +
  263 + SAFIRE INS attitude information +
  264 + DEM information interpolated (nearest neighbour) at the Ublox GPS rate
  265 +
  266 +.. csv-table:: {Flight_id}_post_pos.p output file
  267 + :stub-columns: 1
  268 + :widths: 2 4 8
  269 + :header: Field, Unit, Description
  270 +
  271 + DateTime , Python DateTime, Date and time of the sample
  272 + sow , seconds , GPS Time of the Week of the sample
  273 + lat , decimal degrees, GLORI UBlox-based latitude
  274 + lon , decimal degrees, GLORI UBlox-based longitude
  275 + gSpeed , m/s , GLORI UBlox-based ground speed
  276 + hMSL , m , GLORI UBlox-based height over mean sea level
  277 + course , decimal degrees, UBlox-based true course
  278 + DEM , m , Digital Elevation Model (SRTM) under the track
  279 + heading_ins, decimal degrees, SAFIRE AIRINS-based aircraft heading
  280 + pitch_ins , decimal degrees, SAFIRE AIRINS-based aircraft pitch
  281 + roll_ins , decimal degrees, SAFIRE AIRINS-based aircraft roll
  282 + lat_ins , decimal degrees, SAFIRE AIRINS-based aircraft latitude
  283 + lon_ins , decimal degrees, SAFIRE AIRINS-based aircraft longitude
  284 + hMSL_ins , m , SAFIRE AIRINS-based aircraft altitude over MSL
  285 + alt_ra , m , SAFIRE Radio altimeter estimated range
  286 + el_zen , decimal degrees, Zenith antenna boresight elevation angle
  287 + az_zen , decimal degrees, Zenith antenna boresight azimuth direction
  288 +
  289 +
  290 +- **{Flight_id}_post_SVs.p**: GLORI Ublox-based Space Vehicles information:
  291 + Approximate azimuth and elevation relative to plane,
  292 + and related computed position of specular point on ground.
  293 +
  294 +.. csv-table:: {Flight_id}_post_SVs.p output file
  295 + :stub-columns: 1
  296 + :widths: 2 4 8
  297 + :header: Field, Unit, Description
  298 +
  299 + sow , seconds , GPS Time of the Week of the sample
  300 + sv , N/A , Satellite Space Vehicle ID (PRN ID for GPS)
  301 + azim , integer degrees, GLORI UBlox-based satellite azimuth
  302 + elev , integer degrees, GLORI UBlox-based satellite elevation
  303 + speclat , decimal degrees, Computed specular point approximate latitude
  304 + speclon , decimal degrees, Computed specular point approximate longitude
  305 + specDEM , m , DEM (SRTM) value at the specular point location
  306 + heightOG, m , Height difference between DEM and flight altitude
  307 +
  308 +- **{Flight_id}_post_SVs.p**: Same fields as above, but with a reduced
  309 + temporal resolution defined by cfg[‘lowres_step’], typically 2 seconds.
  310 + File used for generating kml files.
  311 +
369 312 Issues / recommendations
370   -Data rate and interpolation need to be checked for output files. Maybe set a common frequency (5Hz?) for each output measurement could be a good way to get uniform ancillary datasets.
371   -Compare SV az/el against CStarlight-derived az/el
  313 +------------------------
  314 +- Data rate and interpolation need to be checked for output files.
  315 + Maybe set a common frequency (5Hz?) for each output measurement
  316 + could be a good way to get uniform ancillary datasets.
  317 +
  318 + - Compare SV az/el against CStarlight-derived az/el
372 319  
373 320 \ No newline at end of file
... ...
source/PRO/PRO_index.rst
1 1 .. _Processing:
2 2  
3   -**********
4   -Processing
5   -**********
  3 +**********************
  4 +GLORI Processing Chain
  5 +**********************
6 6  
7 7 .. toctree::
8 8 :maxdepth: 2
... ... @@ -14,4 +14,5 @@ Processing
14 14 PRO_rawtoL0
15 15 PRO_L0toL1a
16 16 PRO_L1atoL1b
17   - PRO_plot
18 17 \ No newline at end of file
  18 + PRO_plot
  19 + PRO_ToDo
19 20 \ No newline at end of file
... ...
source/PRO/PRO_rawtoL0.rst
... ... @@ -2,4 +2,63 @@
2 2  
3 3 *********************
4 4 Raw to Level 0
5   -*********************
6 5 \ No newline at end of file
  6 +*********************
  7 +
  8 +raw_to_l0.py
  9 +============
  10 +
  11 +Module description
  12 +------------------
  13 +This module performs the basic observable computation (correlation waveforms)
  14 +from raw data. As for september 2015, it is based on the cStarlight package
  15 +provided with Starlab’s Oceanpal instrument.
  16 +Inputs and outputs are documented in Oceanpal user manual,
  17 +but will be described again in the present document for the sake of clarity.
  18 +
  19 +- Open the plane track file and keep unique points
  20 +- Check and list the raw files covered by ancillary data
  21 + (already filtered on altitude)
  22 +- Download the yuma file corresponding to the beginning of the flight
  23 +- for each raw file in the list
  24 +
  25 + - Generate a generic string based on the raw file name
  26 + - Check for the existence of a timepos file,
  27 + that would tell that the file has already processed
  28 + - if it is not the case, or if the “reprocess” keyword is set to True
  29 + in the config file, start the actual processing:
  30 +
  31 + - Generate a new timepos file with segment median
  32 + latitude, longitude and altitude.
  33 + - Decompress and convert the raw datafiles into
  34 + a cStarlight compatible format (1Bit real non zero IF),
  35 + using the binary pointed by ‘decompress_exe’ in the config file
  36 + - Update the cStarlight L0_gps_settings.txt config file with the altitude
  37 + - Erase previously generated L0 files
  38 + - Build cStarlight command for both LHCP and RHCP
  39 + - Run both channels at the same time and wait for their completion
  40 + - Delete uncompressed files
  41 +
  42 +
  43 +Inputs
  44 +------
  45 +
  46 +Outputs
  47 +-------
  48 +
  49 +Issues / recommendations
  50 +------------------------
  51 +
  52 +cStarlight.py
  53 +=============
  54 +
  55 +Module description
  56 +------------------
  57 +
  58 +Inputs
  59 +------
  60 +
  61 +Outputs
  62 +-------
  63 +
  64 +Issues / recommendations
  65 +------------------------
... ...