bed43d6785
* remove usage of imc::object
...
* introduce imc_channel with collection of affiliate blocks
* imc::keys as list instead of map with custom check/get methods
* imc_raw: start generation of affiliate blocks
2021-02-11 12:48:49 +01:00
f4ecf48218
start group - channel - component - buffer association
2021-02-10 20:01:11 +01:00
9bac0f1063
* list specific blocks
...
* key comparison operator
* imc::object: fix timestamp, fix analog vs. digital index
* imc_raw: convert datatype
2021-02-10 18:17:54 +01:00
0eb034196d
imc::object: all objects parsed
2021-02-10 15:37:24 +01:00
fe3befc5f2
start parsing imc::objects
2021-02-10 13:08:32 +01:00
eb2c49549a
integrate imc::object into parsing workflow
2021-02-10 11:08:08 +01:00
4f3e816dbf
wrapper for imc::object, include imc::object in imc::block
2021-02-09 20:57:44 +01:00
a808a001a9
imc::objects: preliminary complete
2021-02-09 19:31:27 +01:00
e952764e4f
reduce computational complexity
2021-02-09 18:11:06 +01:00
79c1b19e33
check block consistency, reset block.end for length that exceeds buffer size (e.g. final CS)
2021-02-09 16:40:11 +01:00
ee7f70d36f
add imc_meta.hpp
2021-02-08 18:09:02 +01:00
09737a93fb
imcdatatype: simplify class name
2021-02-08 17:27:17 +01:00
cb8365a897
implement imcdatatype
2021-02-08 17:21:37 +01:00
cf7d51e2fb
imc::block: fix parameters parsing
2021-02-08 16:48:22 +01:00
cf6f91a324
extract all block/parameters data
2021-02-08 16:22:28 +01:00
584243f466
full imc raw format revision
2021-02-05 19:45:33 +01:00
09011b50ba
tidy up parquet from lib
2021-02-05 12:30:01 +01:00
19ff83f1d5
lib/raweat.hpp: reset validity flag when setting next file
2020-11-17 15:28:50 +01:00
Mario Fink
f99c37e6dd
raweat.hpp: introduce error queue, find_markers: check for/consider corrupted markers
2020-10-14 15:30:46 +02:00
Mario Fink
043784d278
* rawmerge.hpp: generalize/fix condition for consistent timeseries
...
* introduce propagation of C++ exceptions to Cython/Python
* convert all cout/cerr output to exceptions
* Cython: introduce separate 'do_conversion' => avoid constructor to
trigger conversion
2020-09-18 18:43:25 +02:00
Mario Fink
3611e43276
raw_merge.hpp: fix condition for consistent time series
2020-09-15 19:34:06 +02:00
Mario Fink
956bf8ee58
lib/raweat.hpp: constructor: only start conversion for non-empty file names
2020-08-14 11:46:49 +02:00
Mario Fink
4ce51b55c7
rawmerger: write_table_all() for direct csv output of merged table
2020-08-13 16:01:54 +02:00
Mario Fink
0bd96a1426
fix bugs in lib/rawmerge.hpp: merge_channels(...) and improve example.py
...
- raweat.hpp: find_markers(): properly (re)adjust valid_ flag when doing multiple conversion
- rawmerge.hpp: improve logging, merge_channels(): add condition to avoid range error in vector
- example.py: obtain return flag from add_channel(), write csv output from parquet table
- main.cpp: return to pure raw_eater test version
2020-08-12 11:55:38 +02:00
Mario Fink
4843dcf774
- cython: raw_meat: add/adjust getter method
...
- implement full app example in pyt/example.py
- merges time series of all consistent channels
- extract them into pyarrow table
- save pyarrow table as compressed parquet file
2020-08-11 17:40:11 +02:00
Mario Fink
9bbbada6c9
- adjust example main.cpp for test of rawmerge.hpp
...
- add getter methods including get_num_channels in rawmerge.hpp
2020-08-11 16:07:48 +02:00
Mario Fink
99e10a4f41
rawmerge.hpp: merge core method working/debugged
2020-08-11 15:36:23 +02:00
Mario Fink
1ee05c03a2
rawmerge.hpp: start implementation of merge
2020-08-10 17:29:49 +02:00
Mario Fink
7316d59c2c
- add cython .cpp files to .gitignore
...
- raweat.hpp: add empty constructor and introduce "set_file"
- rawmerge.hpp: add initial channel and write_table method
- add rawmeat example to pyt/example.py
- add raw_merger.hpp to main.cpp
2020-08-10 14:54:15 +02:00
Mario Fink
23f933f68e
check markers resolving #1
2020-08-10 12:33:11 +02:00
Mario Fink
f8dcc6493f
initialize new cython module raw_meat
2020-08-10 12:03:35 +02:00
Mario Fink
ecd364d482
fix typo
2020-08-10 10:28:58 +02:00
2e8fea74a6
pyarrow_test.py: changed compression to BROTLI, fixed encoding in text
2020-08-05 11:58:17 +02:00
3618ed9ee8
-> correction write_table vs. write_data in main.cpp
...
-> container build of pyarrow (i.a. for armv7l build)
2020-08-05 11:32:02 +02:00
Mario Fink
26930e960c
switch to cmake build
2020-08-03 17:51:27 +02:00
Mario Fink
232d376287
get dependencies
2020-08-03 01:19:24 +02:00
9e78f9dde0
next one
2020-07-31 20:33:11 +02:00
64090344c7
parquet: setup_sources.sh
2020-07-31 16:40:02 +02:00
aee2c180a7
extract essential dependecies
2020-07-31 14:01:19 +02:00
441745d894
start adding parquet support
2020-07-30 18:34:53 +02:00
Mario Fink
53377ec75b
fix get_unit with correct marker name, generalize example.py
2020-07-29 17:17:01 +02:00
Mario Fink
6b3aac87fe
show found markers, hex::show: include last incomplete line
2020-07-29 15:33:53 +02:00
Mario Fink
ebda13e738
raweat.hpp: get_unit(): check existence of marker
2020-07-28 17:25:41 +02:00
31e2d7a82e
offer multiple csv-like output formats
2020-07-24 12:33:56 +02:00
4ef68695b3
remove assertions, check validity of file format
2020-07-23 18:09:42 +02:00
3711a94f8b
proper log managment, disabled
2020-07-23 16:41:10 +02:00
4ebd965114
remove redundant headers, correct 2-byte-word datatype
2020-07-03 18:36:30 +02:00
a546a0a6fb
add support for signed long int => int datatype
2020-07-03 18:19:30 +02:00
9c1e9d1468
fix conversion of with CP marker : Zahlenformat = 11 (2-Byte Word digital)
2020-07-03 17:55:59 +02:00
Mario Fink
494c0bc6b1
add extension to convert and temporal unify multiple raw files with different measruemtn frequencies. to BE DEBUGGED
2020-02-17 17:24:06 +00:00
