export NetCDF: handle missing data
While working on the installing the this in PAR008, the Parsivel logger main.py stored a few entries without telegram data, and some entries that contained the VERSION info
2024-04-08T13:58:00.279988+00:00|01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-9.999; 08:20000; 09:00061; 10:1
2024-04-08T13:59:00.550884+00:00|
2024-04-08T14:00:00.650024+00:00|
2024-04-08T14:01:00.746481+00:00|
2024-04-08T14:02:00.839956+00:00|
2024-04-08T14:03:00.934467+00:00|VERSION:2.11.6; BUILD:2112151; 01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-
2024-04-08T14:04:00.210604+00:00|01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-9.999; 08:20000; 09:00053; 10:1
2024-04-08T14:05:00.478993+00:00|01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-9.999; 08:20000; 09:00060; 10:1
2024-04-08T14:06:00.738552+00:00|01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-9.999; 08:20000; 09:00060; 10:1
2024-04-08T14:08:00.033223+00:00|VERSION:2.11.6; BUILD:2112151; 01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-
2024-04-08T14:11:00.811345+00:00|VERSION:2.11.6; BUILD:2112151; 01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-
2024-04-08T14:13:00.909964+00:00|VERSION:2.11.6; BUILD:2112151; 01:0000.000; 02:0000.00; 03:00; 04:00; 05:NP; 06:C; 07:-
cat log_capture_disdro_data.py.json.20240408 | grep ERROR
{"date": "2024-04-08 13:59:01,554", "name": "capture_disdro_data.py: PAR008", "level": "ERROR", "msg": "parsivel_lines is EMPTY"}
{"date": "2024-04-08 14:00:01,653", "name": "capture_disdro_data.py: PAR008", "level": "ERROR", "msg": "parsivel_lines is EMPTY"}
{"date": "2024-04-08 14:01:01,749", "name": "capture_disdro_data.py: PAR008", "level": "ERROR", "msg": "parsivel_lines is EMPTY"}
{"date": "2024-04-08 14:02:01,842", "name": "capture_disdro_data.py: PAR008", "level": "ERROR", "msg": "parsivel_lines is EMPTY"}
When trying to export that day: /usr/local/src/disdrodlv2/venv/bin/python /usr/local/src/disdrodlv2/export_disdrodlDB2NC.py -d 2024-04-08 -c /usr/local/src/disdrodlv2/configs_netcdf/config_008_GV.yml
A error is returned and the NetCDF is not created
Traceback (most recent call last):
File "/usr/local/src/disdrodlv2/export_disdrodlDB2NC.py", line 67, in <module>
row_telegram.parse_telegram_row()
File "/usr/local/src/disdrodlv2/modules/classes.py", line 101, in parse_telegram_row
self.__str2list(field='90', separator=',')
File "/usr/local/src/disdrodlv2/modules/classes.py", line 150, in __str2list
str_val = self.telegram_data[field]
KeyError: '90'
However, the error is not logged! It must be!
And the export script have to be able to handle the missing telegram, not by trowing back an error, but by not entering that DB entry to the NetCDF.
@mschleiss do you agree with my logic?