Mario Fink
4ce51b55c7
rawmerger: write_table_all() for direct csv output of merged table
2020-08-13 16:01:54 +02:00
Mario Fink
0bd96a1426
fix bugs in lib/rawmerge.hpp: merge_channels(...) and improve example.py
...
- raweat.hpp: find_markers(): properly (re)adjust valid_ flag when doing multiple conversion
- rawmerge.hpp: improve logging, merge_channels(): add condition to avoid range error in vector
- example.py: obtain return flag from add_channel(), write csv output from parquet table
- main.cpp: return to pure raw_eater test version
2020-08-12 11:55:38 +02:00
Mario Fink
8d92bce9f0
simplify directory structure, adjust makefile
2020-08-12 09:30:34 +02:00
Mario Fink
4843dcf774
- cython: raw_meat: add/adjust getter method
...
- implement full app example in pyt/example.py
- merges time series of all consistent channels
- extract them into pyarrow table
- save pyarrow table as compressed parquet file
2020-08-11 17:40:11 +02:00
Mario Fink
9bbbada6c9
- adjust example main.cpp for test of rawmerge.hpp
...
- add getter methods including get_num_channels in rawmerge.hpp
2020-08-11 16:07:48 +02:00
Mario Fink
99e10a4f41
rawmerge.hpp: merge core method working/debugged
2020-08-11 15:36:23 +02:00
Mario Fink
9f739d68fd
python example: numpy to pyarrow and parquet
2020-08-11 11:39:17 +02:00
Mario Fink
1ee05c03a2
rawmerge.hpp: start implementation of merge
2020-08-10 17:29:49 +02:00
Mario Fink
7316d59c2c
- add cython .cpp files to .gitignore
...
- raweat.hpp: add empty constructor and introduce "set_file"
- rawmerge.hpp: add initial channel and write_table method
- add rawmeat example to pyt/example.py
- add raw_merger.hpp to main.cpp
2020-08-10 14:54:15 +02:00
Mario Fink
23f933f68e
check markers resolving #1
2020-08-10 12:33:11 +02:00
Mario Fink
f8dcc6493f
initialize new cython module raw_meat
2020-08-10 12:03:35 +02:00
Mario Fink
ecd364d482
fix typo
2020-08-10 10:28:58 +02:00
2e8fea74a6
pyarrow_test.py: changed compression to BROTLI, fixed encoding in text
2020-08-05 11:58:17 +02:00
3618ed9ee8
-> correction write_table vs. write_data in main.cpp
...
-> container build of pyarrow (i.a. for armv7l build)
2020-08-05 11:32:02 +02:00
Mario Fink
26930e960c
switch to cmake build
2020-08-03 17:51:27 +02:00
Mario Fink
232d376287
get dependencies
2020-08-03 01:19:24 +02:00
9e78f9dde0
next one
2020-07-31 20:33:11 +02:00
64090344c7
parquet: setup_sources.sh
2020-07-31 16:40:02 +02:00
aee2c180a7
extract essential dependecies
2020-07-31 14:01:19 +02:00
984895a4f2
document/improve makefile
2020-07-31 09:05:12 +02:00
441745d894
start adding parquet support
2020-07-30 18:34:53 +02:00
194990c62f
improve/debug python example
2020-07-30 12:08:46 +02:00
Mario Fink
53377ec75b
fix get_unit with correct marker name, generalize example.py
2020-07-29 17:17:01 +02:00
Mario Fink
6b3aac87fe
show found markers, hex::show: include last incomplete line
2020-07-29 15:33:53 +02:00
Mario Fink
ebda13e738
raweat.hpp: get_unit(): check existence of marker
2020-07-28 17:25:41 +02:00
31e2d7a82e
offer multiple csv-like output formats
2020-07-24 12:33:56 +02:00
4ef68695b3
remove assertions, check validity of file format
2020-07-23 18:09:42 +02:00
3711a94f8b
proper log managment, disabled
2020-07-23 16:41:10 +02:00
4ebd965114
remove redundant headers, correct 2-byte-word datatype
2020-07-03 18:36:30 +02:00
a546a0a6fb
add support for signed long int => int datatype
2020-07-03 18:19:30 +02:00
9c1e9d1468
fix conversion of with CP marker : Zahlenformat = 11 (2-Byte Word digital)
2020-07-03 17:55:59 +02:00
Mario Fink
25b203aabd
adjust gitignore
2020-07-02 17:59:41 +02:00
Mario Fink
0805d5a152
ignore executable, add more samples
2020-07-02 16:26:41 +02:00
2dcd67c194
makefile: fix some paths/dependencies and setup + gitignore
2020-06-22 10:00:44 +02:00
2772515dea
add manual imc studio
2020-06-03 11:33:31 +02:00
8b0e4a995e
eatall: convert set of raw file with unified time scale, bug fixed
2020-02-18 10:43:41 +01:00
Mario Fink
494c0bc6b1
add extension to convert and temporal unify multiple raw files with different measruemtn frequencies. to BE DEBUGGED
2020-02-17 17:24:06 +00:00
Mario Fink
fe5aae4e83
fixed bug of converting twice, added asserts to avoid double conversion
2020-02-17 14:08:35 +00:00
Mario Fink
a08fab03ba
added cython exension
2020-02-17 13:10:52 +00:00
Mario Fink
46044ec969
avoid sudo inside of makefile
2020-02-17 11:23:19 +00:00
Mario Fink
b9064e2c82
installation target in makefile
2020-02-17 10:49:47 +00:00
Mario Fink
9a588570cc
added help message and usage to executable
2020-02-14 08:30:12 +00:00
Mario Fink
438f96d17b
added check for correct file type, remove all logs
2020-02-13 10:15:38 +00:00
Mario Fink
4a64f3a0e5
check markers/segments before conversion, clean up
2020-02-13 08:42:24 +00:00
Mario Fink
60ff91ee65
preliminary final with template datatype converter
2020-02-12 16:52:17 +00:00
Mario Fink
b588dfb79f
sufficient docu with official docu of raw format
2020-02-12 15:16:44 +01:00
Mario Fink
68378ef8fa
read and parse all segment marker elements and start distinction between datatype based on header info
2020-02-12 11:54:25 +00:00
Mario Fink
56438bb967
trying to adapt to weird 16bbit encoding of raws
2020-02-11 16:04:04 +00:00
Mario Fink
ccf04c42f7
clean main.cpp
2020-02-11 11:14:11 +01:00
Mario Fink
24c7aa4846
integrated support for binary16 = 16bit floating point numbers via half.hpp, IEEE 754-based half-precision floating-point library
2020-02-11 10:57:50 +01:00
