Compare commits

..

48 Commits

Author SHA1 Message Date
06b4c4f870 fix README section about python install 2025-03-09 22:01:43 +01:00
a2f61f7405 comply with PEP 625: adjust README.md 2025-02-12 23:36:40 +01:00
bb10d80489 comply with PEP 625: lower case distribution filenames; bump version 2.1.18 2025-02-12 23:18:49 +01:00
1eea219361 github actions: matrix os ubuntu, win; bump version 2.1.17 2025-02-12 22:57:58 +01:00
400b6eaab9 github actions: separate strategy matrix os; bump version 2.1.16 2025-02-12 22:34:01 +01:00
6f4511e1c2 github actions: remove macos; bump version 2.1.15 2025-02-12 13:53:55 +01:00
fe042b531f github actions: use os latest + macos; bump version 2.1.14 2025-02-12 13:46:58 +01:00
9d5f0cff01 bump version 2.1.13; reuse and extract various text fields, take care of backslash in text w.r.t. to python dict 2025-02-12 01:04:51 +01:00
720acd3b62 github actions: fix Upload binary wheels PyPI 2025-02-03 21:11:21 +01:00
8ca6ee3ddc github actions: fix Upload binary wheels PyPI 2025-02-03 20:56:17 +01:00
379feaa85a github actions: fix Upload binary wheels PyPI 2025-02-02 00:53:16 +01:00
cff2e913fc github actions: fix Upload binary wheels PyPI 2025-02-02 00:47:13 +01:00
75d70a9521 * github actions: fix Unable to download artifact(s): Artifact not found for name: binary-wheels
* README.md: structure and sort references
* bump v2.1.8
2025-02-02 00:30:56 +01:00
mario-fink
36cf0c9c18
Merge pull request #32 from jgoedeke/master
Bugfix trigger-time with XY Datasets
2025-01-25 01:15:19 +01:00
jgoedeke
2326725756 bugfix: set xprec to 9 for XY datasets 2025-01-23 15:07:12 +00:00
jgoedeke
59de48424e Fix absolute_trigger_time for XY datasets and remove addtime from output
Add-time is already implicitly provided by the difference between `trigger-time-nt` and `trigger-time` for normal datasets. For XY datasets the `addtime` parameter is not applicable.
2025-01-23 14:18:08 +00:00
jgoedeke
a44461cba6 Fix wrong keys in runtime errors 2025-01-23 14:15:52 +00:00
cee146593b github actions: fix Unable to download artifact(s): Artifact not found for name: binary-wheels 2025-01-22 19:58:58 +01:00
223f25b6e0 github actions: fix Failed to CreateArtifact: Received non-retryable error: Failed request: (409) 2025-01-22 00:40:24 +01:00
8ec02d21c5 github actions: fix Failed to CreateArtifact: Received non-retryable error: Failed request: (409) 2025-01-22 00:20:57 +01:00
df445bfd7f github actions: update runner-images 2025-01-22 00:04:49 +01:00
af6622492a deprecated version of actions/upload-artifact, upgrade to v4 2025-01-21 00:40:05 +01:00
ef67e14ee7 update samples 2025-01-21 00:34:10 +01:00
Mario Fink
1500943cc4
Merge pull request #29 from RecordEvolution/dependabot/github_actions/dot-github/workflows/actions/download-artifact-4.1.7
Bump actions/download-artifact from 2 to 4.1.7 in /.github/workflows
2025-01-21 00:23:00 +01:00
Mario Fink
be282a8aae
Merge pull request #31 from jgoedeke/master
Add multi component parsing to support XY-datasets
2025-01-20 20:22:12 +01:00
jgoedeke
71121834d5
Add factor and offset transformation for both components and parse common NT and CD keys properly 2024-11-05 12:17:08 +01:00
jgoedeke
0a17b87452
Add multi component parsing to support XY-datasets 2024-11-05 10:47:36 +01:00
dependabot[bot]
9ea7186090
Bump actions/download-artifact from 2 to 4.1.7 in /.github/workflows
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 4.1.7.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...v4.1.7)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-03 22:06:04 +00:00
befda072e5 include origin comment of NO block 2024-06-12 20:53:59 +02:00
0b1cb2f230 reference correct name of data file format in README.md (issue #14) and add link to imc documentation 2024-06-12 20:32:01 +02:00
ce444814c2 reference correct name of data file format in README.md (issue #14) and add link to imc documentation 2024-06-12 20:31:08 +02:00
eec3853af9 by default use utf-8 for metadata including non-ascii characters while missing codepage info 2024-06-12 12:02:26 +02:00
25e114dbea Fix Github Workflow Badge:
* https://github.com/badges/shields/issues/8733
* https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/adding-a-workflow-status-badge
2024-06-12 11:50:35 +02:00
740116c7ba bump version 2.0.20 2024-06-12 11:23:12 +02:00
5899bd646b bump version 2.0.19 2024-06-12 10:39:22 +02:00
1d33a81ad2 bump version 2.0.18 2024-06-12 10:37:54 +02:00
b58efae107 bump version 2.0.17 2024-06-12 10:33:51 +02:00
cc0dd9978f update and fix badges 2024-06-12 10:22:53 +02:00
4834f63a67 update and fix badges 2024-06-12 10:16:56 +02:00
1be78a2c5e Revert "update and fix badges"
This reverts commit f5581557bca14d8e6c200e8b3b57661a18a957cc.
2024-06-12 10:15:23 +02:00
f5581557bc update and fix badges 2024-06-12 10:14:51 +02:00
8cde88848b update and fix badges 2024-06-12 10:08:33 +02:00
0799513ea2 by default use codepage1252 for metadata including non-ascii characters while missing codepage info 2024-06-12 09:45:30 +02:00
effeee105c win32: replace timegm by _mkgmtime, bump version 2.0.15 2023-08-08 23:46:10 +02:00
ed5b366341 imc_channel.hpp: usage of iconv for unix only, bump version 2.0.15 2023-08-08 23:29:48 +02:00
9a520ddd9c bump version 2.0.14 2023-08-08 00:54:07 +02:00
2c43087d15 bump version 2.0.13 2023-08-08 00:52:27 +02:00
60ac1365a5 * imc_channel.hpp: usage of iconv for unix only
* IMCtermite.pyx: add codepage conversion for windows
* bump VERSION
2023-08-08 00:50:52 +02:00
23 changed files with 645 additions and 345 deletions

View File

@ -1,5 +1,5 @@
name: Build Python Wheels name: CI Build Wheel
on: on:
push: push:
@ -10,16 +10,18 @@ jobs:
build_setup: build_setup:
name: Prepare environment for wheel builds name: Prepare environment for wheel builds
runs-on: ubuntu-20.04 runs-on: ubuntu-24.04
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
- name: Prepare wheel build - name: Prepare wheel build
run: make -C python/ setup run: make -C python/ setup
- name: Store wheel configuration files - name: Store wheel configuration files
uses: actions/upload-artifact@v2 uses: actions/upload-artifact@v4.6.0
with: with:
name: wheel-config name: wheel-config
path: python/ path: python/
- name: Display files
run: ls -lR
build_wheels: build_wheels:
name: Build binary wheels on ${{ matrix.os }} name: Build binary wheels on ${{ matrix.os }}
@ -27,8 +29,7 @@ jobs:
needs: [build_setup] needs: [build_setup]
strategy: strategy:
matrix: matrix:
os: [ubuntu-20.04, windows-2019] os: [ubuntu-latest, windows-latest]
arch: [auto32, auto64, aarch64]
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@ -36,37 +37,29 @@ jobs:
- name: Install cibuildwheel - name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.1.2 run: python -m pip install cibuildwheel==2.1.2
- name: Get wheel configuration files - name: Get wheel configuration files
uses: actions/download-artifact@v2 uses: actions/download-artifact@v4.1.7
with: with:
name: wheel-config name: wheel-config
path: python/ path: python/
- name: Install libiconv for windows
if: matrix.os == 'windows-2019'
run: |
git clone https://github.com/Microsoft/vcpkg.git
.\vcpkg\bootstrap-vcpkg.bat
vcpkg install libiconv
- name: Build wheels - name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse run: python -m cibuildwheel --output-dir wheelhouse
working-directory: python/ working-directory: python/
# env:
# CIBW_BUILD: cp36-* cp37-* cp38-* cp39-* cp310-*
- name: Store binary wheels - name: Store binary wheels
uses: actions/upload-artifact@v2 uses: actions/upload-artifact@v4.6.0
with: with:
name: binary-wheels name: binary-wheels-${{matrix.os}}-${{ strategy.job-index }}
path: python/wheelhouse/*.whl path: python/wheelhouse/*.whl
build_sdist: build_sdist:
name: Build source distribution name: Build source distribution
runs-on: ubuntu-latest runs-on: ubuntu-24.04
needs: [build_setup] needs: [build_setup]
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
- name: Install cython - name: Install cython
run: python -m pip install cython==0.29.24 run: python -m pip install cython==0.29.24
- name: Get wheel configuration files - name: Get wheel configuration files
uses: actions/download-artifact@v2 uses: actions/download-artifact@v4.1.7
with: with:
name: wheel-config name: wheel-config
path: python/ path: python/
@ -74,28 +67,33 @@ jobs:
run: python setup.py sdist run: python setup.py sdist
working-directory: python/ working-directory: python/
- name: Store source wheels - name: Store source wheels
uses: actions/upload-artifact@v2 uses: actions/upload-artifact@v4.6.0
with: with:
name: source-wheels name: source-wheels
path: python/dist/*.tar.gz path: python/dist/*.tar.gz
- name: Display files
run: ls -lR
upload_pypi: upload_pypi:
name: Upload binary wheels to PyPI name: Upload wheels to PyPI
runs-on: ubuntu-20.04 runs-on: ubuntu-24.04
needs: [build_wheels, build_sdist] needs: [build_wheels, build_sdist]
steps: steps:
- name: Get source wheels - name: Get source wheels
uses: actions/download-artifact@v2 uses: actions/download-artifact@v4.1.7
with: with:
name: source-wheels name: source-wheels
path: dist/ path: dist/
- name: Get binary wheels - name: Get binary wheels
uses: actions/download-artifact@v2 uses: actions/download-artifact@v4.1.7
with: with:
name: binary-wheels
path: dist/ path: dist/
pattern: binary-wheels-*
merge-multiple: true
- name: Display files
run: ls -lR
- uses: pypa/gh-action-pypi-publish@release/v1 - uses: pypa/gh-action-pypi-publish@release/v1
with: with:

View File

@ -1,15 +1,13 @@
[![Total alerts](https://img.shields.io/lgtm/alerts/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/alerts/)
[![Language grade: C/C++](https://img.shields.io/lgtm/grade/cpp/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/context:cpp)
[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/context:python)
[![LICENSE](https://img.shields.io/github/license/RecordEvolution/IMCtermite)](https://img.shields.io/github/license/RecordEvolution/IMCtermite) [![LICENSE](https://img.shields.io/github/license/RecordEvolution/IMCtermite)](https://img.shields.io/github/license/RecordEvolution/IMCtermite)
[![STARS](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)](https://img.shields.io/github/stars/RecordEvolution/IMCtermite) [![STARS](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)
![Build Python Wheels](https://github.com/RecordEvolution/IMCtermite/actions/workflows/pypi-deploy.yml/badge.svg) ![CI Build Wheel](https://github.com/RecordEvolution/IMCtermite/actions/workflows/pypi-deploy.yml/badge.svg?branch=&event=push)
[![PYPI](https://img.shields.io/pypi/v/IMCtermite.svg)](https://pypi.org/project/imctermite/)
# IMCtermite # IMCtermite
_IMCtermite_ provides access to the proprietary data format _IMCtermite_ provides access to the proprietary data format
_IMC Bus Format_ with the file extension _.raw_ introduced and developed by _IMC2 Data Format_ with the file extension _.raw_ (or .dat) introduced and developed by
[imc Test & Measurement GmbH](https://www.imc-tm.de/). This data format is [imc Test & Measurement GmbH](https://www.imc-tm.de/). This data format is
employed i.a. by the measurement hardware employed i.a. by the measurement hardware
[imc CRONOSflex](https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/) [imc CRONOSflex](https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/)
@ -20,7 +18,9 @@ for measurement data control and analysis. Thanks to the integrated Python modul
the extracted measurement data can be stored in any open-source file format the extracted measurement data can be stored in any open-source file format
accessible by Python like i.a. _csv_, _json_ or _parquet_. accessible by Python like i.a. _csv_, _json_ or _parquet_.
On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/), the library can be used both as a command line tool for interactive usage and as a Python module to integrate the _.raw_ format into any ETL workflow. On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/),
the library can be used both as a command line tool for interactive usage and as a
Python module to integrate the _.raw_ format into any ETL workflow.
## Overview ## Overview
@ -31,12 +31,11 @@ On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/),
## File format ## File format
[Warning: Take a look at [this issue](https://github.com/RecordEvolution/IMCtermite/issues/14) when reading this section regarding the file format.] A file of the _IMC2 Data Format_ type with extension _.raw_ (or .dat) is a _mixed text/binary
A data file of the _IMC Bus Format_ type with the extension _.raw_ is a _mixed text/binary
file_ featuring a set of markers (keys) that indicate the start of various blocks file_ featuring a set of markers (keys) that indicate the start of various blocks
of data that provide meta information and the actual measurement data. Every single of data that provide meta information and the actual measurement data. Every single
marker is introduced by the character `"|" = 0x 7c` followed by two uppercase letters that characterize the type of marker. Each block is further divided into several marker is introduced by the character `"|" = 0x 7c` followed by two uppercase letters that
characterize the type of marker. Each block is further divided into several
parameters separated by commata `"," = 0x 2c` and terminated by a semicolon parameters separated by commata `"," = 0x 2c` and terminated by a semicolon
`";" = 0x 3b`. For instance, the header - first 600 bytes - of a raw file may `";" = 0x 3b`. For instance, the header - first 600 bytes - of a raw file may
look like this (in UTF-8 encoding): look like this (in UTF-8 encoding):
@ -132,24 +131,21 @@ which may require root permissions.
### Python ### Python
To integrate the library into a customized ETL toolchain, several cython targets To integrate the library into a customized ETL toolchain, several python targets
are available. For a local build that enables you to run the examples, use: are available. For a local build that enables you to run the examples, use:
``` ```
make cython-build make python-build
``` ```
However, in a production environment, a proper installation of the module with
`make cython-install` is recommended for system-wide availability of the module.
#### Installation with pip #### Installation with pip
The package is also available in the [Python Package Index](https://pypi.org) The package is also available in the [Python Package Index](https://pypi.org)
at [IMCtermite](https://pypi.org/project/IMCtermite/). at [imctermite](https://pypi.org/project/imctermite/).
To install the latest version simply do To install the latest version simply do
```Shell ```Shell
python3 -m pip install IMCtermite python3 -m pip install imctermite
``` ```
which provides binary wheels for multiple architectures on _Windows_ and _Linux_ which provides binary wheels for multiple architectures on _Windows_ and _Linux_
@ -196,9 +192,9 @@ Given the `IMCtermite` module is available, we can import it and declare an inst
of it by passing a _raw_ file to the constructor: of it by passing a _raw_ file to the constructor:
```Python ```Python
import IMCtermite import imctermite
imcraw = IMCtermite.imctermite(b"sample/sampleA.raw") imcraw = imctermite.imctermite(b"sample/sampleA.raw")
``` ```
An example of how to create an instance and obtain the list of channels is: An example of how to create an instance and obtain the list of channels is:
@ -223,16 +219,31 @@ can be found in the `python/examples` folder.
## References ## References
### IMC
- https://www.imc-tm.de/produkte/messtechnik-software/imc-famos/funktionen/im-und-export/ - https://www.imc-tm.de/produkte/messtechnik-software/imc-famos/funktionen/im-und-export/
- https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/ - https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/
- https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html - https://www.imc-tm.de/download-center/produkt-downloads/imc-famos/handbuecher
- https://www.imc-tm.de/fileadmin/Public/Downloads/Manuals/imc_FAMOS/imcGemeinsameKomponenten.pdf
- https://github.com/Apollo3zehn/ImcFamosFile - https://github.com/Apollo3zehn/ImcFamosFile
- https://apollo3zehn.github.io/ImcFamosFile/api/ImcFamosFile.FamosFileKeyType.html - https://apollo3zehn.github.io/ImcFamosFile/api/ImcFamosFile.FamosFileKeyType.html
### Cython
- https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html
### PyPI
- https://pypi.org/help/#apitoken - https://pypi.org/help/#apitoken
- https://sgoel.dev/posts/uploading-binary-wheels-to-pypi-from-github-actions/ - https://sgoel.dev/posts/uploading-binary-wheels-to-pypi-from-github-actions/
- https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsrun - https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsrun
- https://github.com/pypa/cibuildwheel/blob/main/examples/github-deploy.yml - https://github.com/pypa/cibuildwheel/blob/main/examples/github-deploy.yml
- https://cibuildwheel.readthedocs.io/en/stable/deliver-to-pypi/ - https://cibuildwheel.readthedocs.io/en/stable/deliver-to-pypi/
- https://github.com/actions/download-artifact#download-all-artifacts
- https://github.com/actions/download-artifact?tab=readme-ov-file#download-multiple-filtered-artifacts-to-the-same-directory
### iconv
- https://www.gnu.org/software/libiconv/ - https://www.gnu.org/software/libiconv/
- https://vcpkg.io/en/packages.html - https://vcpkg.io/en/packages.html
- https://vcpkg.io/en/getting-started - https://vcpkg.io/en/getting-started

View File

@ -5,45 +5,78 @@
#include "imc_datatype.hpp" #include "imc_datatype.hpp"
#include "imc_conversion.hpp" #include "imc_conversion.hpp"
#include "imc_block.hpp"
#include <sstream> #include <sstream>
#include <math.h> #include <math.h>
#include <chrono> #include <chrono>
#include <ctime> #include <ctime>
#include <time.h> #include <time.h>
#if defined(__linux__) || defined(__APPLE__)
#include <iconv.h> #include <iconv.h>
#elif defined(__WIN32__) || defined(_WIN32)
#define timegm _mkgmtime
#endif
//---------------------------------------------------------------------------// //---------------------------------------------------------------------------//
namespace imc namespace imc
{ {
struct component_env
{
std::string uuid_;
// required channel components for CG channels only
std::string CCuuid_, CPuuid_;
// optional channel components for CG channels only
std::string CDuuid_, NTuuid_;
std::string Cbuuid_, CRuuid_;
// reset all members
void reset()
{
uuid_.clear();
CCuuid_.clear();
CPuuid_.clear();
CDuuid_.clear();
Cbuuid_.clear();
CRuuid_.clear();
NTuuid_.clear();
}
};
// collect uuid's of blocks required for full channel reconstruction // collect uuid's of blocks required for full channel reconstruction
struct channel_env struct channel_env
{ {
// define unique identifer for channel_env // define unique identifer for channel_env
std::string uuid_; std::string uuid_;
// collect common affiliate blocks for every channel
std::string NOuuid_, NLuuid_;
// collect affiliate blocks for a single channel // collect affiliate blocks for a single channel
std::string CBuuid_, CGuuid_, CCuuid_, CNuuid_; // channel types
std::string CDuuid_, CTuuid_, Cbuuid_, CPuuid_, CRuuid_, CSuuid_; std::string CBuuid_, CGuuid_, CIuuid_, CTuuid_;
std::string NTuuid_, NOuuid_, NLuuid_; std::string CNuuid_, CDuuid_, NTuuid_;
std::string CSuuid_;
component_env compenv1_;
component_env compenv2_;
// reset all members // reset all members
void reset() void reset()
{ {
uuid_.clear(); uuid_.clear();
CBuuid_.clear();
CGuuid_.clear();
CCuuid_.clear();
CNuuid_.clear();
CDuuid_.clear();
CTuuid_.clear();
Cbuuid_.clear();
CPuuid_.clear();
CRuuid_.clear();
CSuuid_.clear();
NTuuid_.clear();
NOuuid_.clear(); NOuuid_.clear();
NLuuid_.clear(); NLuuid_.clear();
CBuuid_.clear();
CGuuid_.clear();
CIuuid_.clear();
CTuuid_.clear();
CNuuid_.clear();
CDuuid_.clear();
NTuuid_.clear();
CSuuid_.clear();
compenv1_.reset();
compenv2_.reset();
} }
// get info // get info
@ -51,21 +84,23 @@ namespace imc
{ {
std::stringstream ss; std::stringstream ss;
ss<<std::setw(width)<<std::left<<"uuid:"<<uuid_<<"\n" ss<<std::setw(width)<<std::left<<"uuid:"<<uuid_<<"\n"
<<std::setw(width)<<std::left<<"NOuuid:"<<NOuuid_<<"\n"
<<std::setw(width)<<std::left<<"NLuuid:"<<NLuuid_<<"\n"
//
<<std::setw(width)<<std::left<<"CBuuid:"<<CBuuid_<<"\n" <<std::setw(width)<<std::left<<"CBuuid:"<<CBuuid_<<"\n"
<<std::setw(width)<<std::left<<"CGuuid:"<<CGuuid_<<"\n" <<std::setw(width)<<std::left<<"CGuuid:"<<CGuuid_<<"\n"
<<std::setw(width)<<std::left<<"CCuuid:"<<CCuuid_<<"\n" <<std::setw(width)<<std::left<<"CIuuid:"<<CIuuid_<<"\n"
<<std::setw(width)<<std::left<<"CTuuid:"<<CTuuid_<<"\n"
<<std::setw(width)<<std::left<<"CNuuid:"<<CNuuid_<<"\n" <<std::setw(width)<<std::left<<"CNuuid:"<<CNuuid_<<"\n"
// //
<<std::setw(width)<<std::left<<"CDuuid:"<<CDuuid_<<"\n" <<std::setw(width)<<std::left<<"CCuuid:"<<compenv1_.CCuuid_<<"\n"
<<std::setw(width)<<std::left<<"CTuuid:"<<CTuuid_<<"\n" <<std::setw(width)<<std::left<<"CPuuid:"<<compenv1_.CPuuid_<<"\n"
<<std::setw(width)<<std::left<<"Cbuuid:"<<Cbuuid_<<"\n"
<<std::setw(width)<<std::left<<"CPuuid:"<<CPuuid_<<"\n"
<<std::setw(width)<<std::left<<"CRuuid:"<<CRuuid_<<"\n"
<<std::setw(width)<<std::left<<"CSuuid:"<<CSuuid_<<"\n"
// //
<<std::setw(width)<<std::left<<"NTuuid:"<<NTuuid_<<"\n" <<std::setw(width)<<std::left<<"CDuuid:"<<compenv1_.CDuuid_<<"\n"
<<std::setw(width)<<std::left<<"NOuuid:"<<NOuuid_<<"\n" <<std::setw(width)<<std::left<<"Cbuuid:"<<compenv1_.Cbuuid_<<"\n"
<<std::setw(width)<<std::left<<"NLuuid:"<<NLuuid_<<"\n"; <<std::setw(width)<<std::left<<"CRuuid:"<<compenv1_.CRuuid_<<"\n"
<<std::setw(width)<<std::left<<"NTuuid:"<<compenv1_.NTuuid_<<"\n"
<<std::setw(width)<<std::left<<"CSuuid:"<<CSuuid_<<"\n";
return ss.str(); return ss.str();
} }
@ -74,19 +109,20 @@ namespace imc
{ {
std::stringstream ss; std::stringstream ss;
ss<<"{"<<"\"uuid\":\""<<uuid_ ss<<"{"<<"\"uuid\":\""<<uuid_
<<"\",\"CBuuid\":\""<<CBuuid_
<<"\",\"CGuuid\":\""<<CGuuid_
<<"\",\"CCuuid\":\""<<CCuuid_
<<"\",\"CNuuid\":\""<<CNuuid_
<<"\",\"CDuuid\":\""<<CDuuid_
<<"\",\"CTuuid\":\""<<CTuuid_
<<"\",\"Cbuuid\":\""<<Cbuuid_
<<"\",\"CPuuid\":\""<<CPuuid_
<<"\",\"CRuuid\":\""<<CRuuid_
<<"\",\"CSuuid\":\""<<CSuuid_
<<"\",\"NTuuid\":\""<<NTuuid_
<<"\",\"NOuuid\":\""<<NOuuid_ <<"\",\"NOuuid\":\""<<NOuuid_
<<"\",\"NLuuid\":\""<<NLuuid_ <<"\",\"NLuuid\":\""<<NLuuid_
<<"\",\"CBuuid\":\""<<CBuuid_
<<"\",\"CGuuid\":\""<<CGuuid_
<<"\",\"CIuuid\":\""<<CIuuid_
<<"\",\"CTuuid\":\""<<CTuuid_
<<"\",\"CNuuid\":\""<<CNuuid_
<<"\",\"CCuuid\":\""<<compenv1_.CCuuid_
<<"\",\"CPuuid\":\""<<compenv1_.CPuuid_
<<"\",\"CDuuid\":\""<<compenv1_.CDuuid_
<<"\",\"Cbuuid\":\""<<compenv1_.Cbuuid_
<<"\",\"CRuuid\":\""<<compenv1_.CRuuid_
<<"\",\"NTuuid\":\""<<compenv1_.NTuuid_
<<"\",\"CSuuid\":\""<<CSuuid_
<<"\"}"; <<"\"}";
return ss.str(); return ss.str();
} }
@ -144,6 +180,7 @@ namespace imc
return sumstr; return sumstr;
} }
#if defined(__linux__) || defined(__APPLE__)
// convert encoding of any descriptions, channel-names, units etc. // convert encoding of any descriptions, channel-names, units etc.
class iconverter class iconverter
{ {
@ -215,6 +252,58 @@ namespace imc
astring = outstring; astring = outstring;
} }
}; };
#elif defined(__WIN32__) || defined(_WIN32)
class iconverter
{
public:
iconverter(std::string in_enc, std::string out_enc, size_t out_buffer_size = 1024) {}
void convert(std::string &astring) {}
};
#endif
struct component_group
{
imc::component CC_;
imc::packaging CP_;
imc::abscissa CD_;
imc::buffer Cb_;
imc::range CR_;
imc::channelobj CN_;
imc::triggertime NT_;
component_env compenv_;
// Constructor to parse the associated blocks
component_group(component_env &compenv, std::map<std::string, imc::block>* blocks, std::vector<unsigned char>* buffer)
: compenv_(compenv)
{
if (blocks->count(compenv.CCuuid_) == 1)
{
CC_.parse(buffer, blocks->at(compenv.CCuuid_).get_parameters());
}
if (blocks->count(compenv.CPuuid_) == 1)
{
CP_.parse(buffer, blocks->at(compenv.CPuuid_).get_parameters());
}
if (blocks->count(compenv.CDuuid_) == 1)
{
CD_.parse(buffer, blocks->at(compenv.CDuuid_).get_parameters());
}
if (blocks->count(compenv.Cbuuid_) == 1)
{
Cb_.parse(buffer, blocks->at(compenv.Cbuuid_).get_parameters());
}
if (blocks->count(compenv.CRuuid_) == 1)
{
CR_.parse(buffer, blocks->at(compenv.CRuuid_).get_parameters());
}
if (blocks->count(compenv.NTuuid_) == 1)
{
NT_.parse(buffer, blocks->at(compenv.NTuuid_).get_parameters());
}
}
};
// channel // channel
struct channel struct channel
@ -224,8 +313,16 @@ namespace imc
std::map<std::string,imc::block>* blocks_; std::map<std::string,imc::block>* blocks_;
std::vector<unsigned char>* buffer_; std::vector<unsigned char>* buffer_;
imc::origin_data NO_;
imc::language NL_;
imc::text CT_;
imc::groupobj CB_;
imc::datafield CG_;
imc::channelobj CN_;
// collect meta-data of channels according to env, // collect meta-data of channels according to env,
// just everything valueable in here // just everything valueable in here
// TODO: is this necessary?
std::string uuid_; std::string uuid_;
std::string name_, comment_; std::string name_, comment_;
std::string origin_, origin_comment_, text_; std::string origin_, origin_comment_, text_;
@ -234,165 +331,155 @@ namespace imc
std::string language_code_, codepage_; std::string language_code_, codepage_;
std::string yname_, yunit_; std::string yname_, yunit_;
std::string xname_, xunit_; std::string xname_, xunit_;
double xstepwidth_, xoffset_; double xstepwidth_, xstart_;
int xprec_; int xprec_;
int dimension_;
// buffer and data // buffer and data
int signbits_, num_bytes_; int xsignbits_, xnum_bytes_;
int ysignbits_, ynum_bytes_;
// unsigned long int byte_offset_; // unsigned long int byte_offset_;
unsigned long int buffer_offset_, buffer_size_; unsigned long int xbuffer_offset_, ybuffer_offset_;
unsigned long int xbuffer_size_, ybuffer_size_;
long int addtime_; long int addtime_;
int datatp_; imc::numtype xdatatp_, ydatatp_;
imc::datatype dattyp_; std::vector<imc::datatype> xdata_, ydata_;
std::vector<imc::datatype> ydata_;
std::vector<double> xdata_;
// range, factor and offset // range, factor and offset
double factor_, offset_; double xfactor_, yfactor_;
double xoffset_, yoffset_;
// group reference the channel belongs to // group reference the channel belongs to
int group_index_; unsigned long int group_index_;
std::string group_uuid_, group_name_, group_comment_; std::string group_uuid_, group_name_, group_comment_;
// constructor takes channel's block environment // constructor takes channel's block environment
channel(channel_env &chnenv, std::map<std::string,imc::block>* blocks, channel(channel_env &chnenv, std::map<std::string,imc::block>* blocks,
std::vector<unsigned char>* buffer): std::vector<unsigned char>* buffer):
chnenv_(chnenv), blocks_(blocks), buffer_(buffer), chnenv_(chnenv), blocks_(blocks), buffer_(buffer),
factor_(1.), offset_(0.), xfactor_(1.), yfactor_(1.), xoffset_(0.), yoffset_(0.),
group_index_(-1) group_index_(-1)
{ {
// declare list of block parameters
std::vector<imc::parameter> prms;
// use uuid from CN block // use uuid from CN block
uuid_ = chnenv_.CNuuid_; uuid_ = chnenv_.CNuuid_;
// extract associated NO data
if ( blocks_->count(chnenv_.NOuuid_) == 1 )
{
NO_.parse(buffer_, blocks_->at(chnenv_.NOuuid_).get_parameters());
origin_ = NO_.generator_;
comment_ = NO_.comment_;
}
// extract associated NL data
if ( blocks_->count(chnenv_.NLuuid_) == 1 )
{
NL_.parse(buffer_, blocks_->at(chnenv_.NLuuid_).get_parameters());
codepage_ = NL_.codepage_;
language_code_ = NL_.language_code_;
}
// extract associated CB data // extract associated CB data
if ( blocks_->count(chnenv_.CBuuid_) == 1 ) if ( blocks_->count(chnenv_.CBuuid_) == 1 )
{ {
prms = blocks_->at(chnenv_.CBuuid_).get_parameters(); CB_.parse(buffer_, blocks_->at(chnenv_.CBuuid_).get_parameters());
group_index_ = std::stoi(blocks_->at(chnenv_.CBuuid_).get_parameter(prms[2]));
group_name_ = blocks_->at(chnenv_.CBuuid_).get_parameter(prms[4]);
group_comment_ = blocks_->at(chnenv_.CBuuid_).get_parameter(prms[6]);
} }
// extract associated CT data // extract associated CT data
if ( blocks_->count(chnenv_.CTuuid_) == 1 ) if ( blocks_->count(chnenv_.CTuuid_) == 1 )
{ {
prms = blocks_->at(chnenv_.CTuuid_).get_parameters(); CT_.parse(buffer_, blocks_->at(chnenv_.CTuuid_).get_parameters());
text_ = blocks_->at(chnenv_.CTuuid_).get_parameter(prms[4]) + std::string(" - ") text_ = CT_.name_ + std::string(" - ")
+ blocks_->at(chnenv_.CTuuid_).get_parameter(prms[6]) + std::string(" - ") + CT_.text_ + std::string(" - ")
+ blocks_->at(chnenv_.CTuuid_).get_parameter(prms[8]); + CT_.comment_;
}
// extract associated CD data
if ( blocks_->count(chnenv_.CDuuid_) == 1 )
{
prms = blocks_->at(chnenv_.CDuuid_).get_parameters();
xstepwidth_ = std::stod(blocks_->at(chnenv_.CDuuid_).get_parameter(prms[2]));
xunit_ = blocks_->at(chnenv_.CDuuid_).get_parameter(prms[5]);
// TODO
// xname_ = std::string("time");
// find appropriate precision for "xdata_" by means of "xstepwidth_"
xprec_ = (xstepwidth_ > 0 ) ? (int)ceil(fabs(log10(xstepwidth_))) : 10;
}
// extract associated CP data
if ( blocks_->count(chnenv_.CPuuid_) == 1 )
{
prms = blocks_->at(chnenv_.CPuuid_).get_parameters();
num_bytes_ = std::stoi(blocks_->at(chnenv_.CPuuid_).get_parameter(prms[3]));
datatp_ = std::stoi(blocks_->at(chnenv_.CPuuid_).get_parameter(prms[4]));
signbits_ = std::stoi(blocks_->at(chnenv_.CPuuid_).get_parameter(prms[5]));
// byte_offset_ = std::stoul(blocks_->at(chnenv_.CPuuid_).get_parameter(prms[7]));
}
// extract associated Cb data
if ( blocks_->count(chnenv_.Cbuuid_) == 1 )
{
prms = blocks_->at(chnenv_.Cbuuid_).get_parameters();
buffer_offset_ = std::stoul(blocks_->at(chnenv_.Cbuuid_).get_parameter(prms[6]));
buffer_size_ = std::stoul(blocks_->at(chnenv_.Cbuuid_).get_parameter(prms[7]));
xoffset_ = std::stod(blocks_->at(chnenv_.Cbuuid_).get_parameter(prms[11]));
addtime_ = (long int)std::stod(blocks_->at(chnenv_.Cbuuid_).get_parameter(prms[12]));
}
// extract associated CR data
if ( blocks_->count(chnenv_.CRuuid_) == 1 )
{
prms = blocks_->at(chnenv_.CRuuid_).get_parameters();
factor_ = std::stod(blocks_->at(chnenv_.CRuuid_).get_parameter(prms[3]));
offset_ = std::stod(blocks_->at(chnenv_.CRuuid_).get_parameter(prms[4]));
yunit_ = blocks_->at(chnenv_.CRuuid_).get_parameter(prms[7]);
} }
// extract associated CN data // extract associated CN data
if ( blocks_->count(chnenv_.CNuuid_) == 1 ) if ( blocks_->count(chnenv_.CNuuid_) == 1 )
{ {
prms = blocks_->at(chnenv_.CNuuid_).get_parameters(); CN_.parse(buffer_, blocks_->at(chnenv_.CNuuid_).get_parameters());
name_ = blocks_->at(chnenv_.CNuuid_).get_parameter(prms[6]); group_index_ = CN_.group_index_;
yname_ = name_; group_name_ = CN_.name_;
comment_ = blocks_->at(chnenv_.CNuuid_).get_parameter(prms[8]); group_comment_ = CN_.comment_;
// group_index_ = std::stoi(blocks_->at(chnenv_.CNuuid_).get_parameter(prms[2]));
} }
// extract associated NO data if ( !chnenv_.compenv1_.uuid_.empty() && chnenv_.compenv2_.uuid_.empty() )
if ( blocks_->count(chnenv_.NOuuid_) == 1 )
{ {
prms = blocks_->at(chnenv_.NOuuid_).get_parameters(); // normal dataset (single component)
origin_ = blocks_->at(chnenv_.NOuuid_).get_parameter(prms[4]); // set common NT and CD keys if no others are specified
origin_comment_ = blocks_->at(chnenv_.NOuuid_).get_parameter(prms[6]); if (chnenv_.compenv1_.NTuuid_.empty()) chnenv_.compenv1_.NTuuid_ = chnenv_.NTuuid_;
} if (chnenv_.compenv1_.CDuuid_.empty()) chnenv_.compenv1_.CDuuid_ = chnenv_.CDuuid_;
// extract associated NL data // comp_group1 contains y-data, x-data is based on xstepwidth_, xstart_ and the length of y-data
// codepage: component_group comp_group1(chnenv_.compenv1_, blocks_, buffer_);
// - http://www.iana.org/assignments/character-sets/character-sets.xhtml dimension_ = 1;
// - https://de.wikipedia.org/wiki/Zeichensatztabelle
// language-code:
// - https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c?redirectedfrom=MSDN
if ( blocks_->count(chnenv_.NLuuid_) == 1 )
{
prms = blocks_->at(chnenv_.NLuuid_).get_parameters();
codepage_ = blocks_->at(chnenv_.NLuuid_).get_parameter(prms[2]);
language_code_ = blocks_->at(chnenv_.NLuuid_).get_parameter(prms[3]);
}
// obtain NT data
// - https://en.cppreference.com/w/cpp/chrono/c/tm
// - https://en.cppreference.com/w/cpp/io/manip/put_time
if ( blocks_->count(chnenv_.NTuuid_) == 1 )
{
prms = blocks_->at(chnenv_.NTuuid_).get_parameters();
//std::tm tm{};
std::tm tms = std::tm();
tms.tm_mday = std::stoi(blocks_->at(chnenv_.NTuuid_).get_parameter(prms[2]));
tms.tm_mon = std::stoi(blocks_->at(chnenv_.NTuuid_).get_parameter(prms[3])) - 1;
tms.tm_year = std::stoi(blocks_->at(chnenv_.NTuuid_).get_parameter(prms[4])) - 1900;
tms.tm_hour = std::stoi(blocks_->at(chnenv_.NTuuid_).get_parameter(prms[5]));
tms.tm_min = std::stoi(blocks_->at(chnenv_.NTuuid_).get_parameter(prms[6]));
long double secs = std::stold(blocks_->at(chnenv_.NTuuid_).get_parameter(prms[7]));
double secs_int;
trigger_time_frac_secs_ = modf((double)secs,&secs_int);
tms.tm_sec = (int)secs_int;
//tms.tm_isdst = -1;
xstepwidth_ = comp_group1.CD_.dx_;
xunit_ = comp_group1.CD_.unit_;
ybuffer_offset_ = comp_group1.Cb_.offset_buffer_;
ybuffer_size_ = comp_group1.Cb_.number_bytes_;
xstart_ = comp_group1.Cb_.x0_;
yfactor_ = comp_group1.CR_.factor_;
yoffset_ = comp_group1.CR_.offset_;
yunit_ = comp_group1.CR_.unit_;
name_ = comp_group1.CN_.name_;
yname_ = comp_group1.CN_.name_;
comment_ = comp_group1.CN_.comment_;
ynum_bytes_ = comp_group1.CP_.bytes_;
ydatatp_ = comp_group1.CP_.numeric_type_;
ysignbits_ = comp_group1.CP_.signbits_;
// generate std::chrono::system_clock::time_point type // generate std::chrono::system_clock::time_point type
// ( - https://www.gnu.org/software/libc/manual/html_node/Broken_002ddown-Time.html std::time_t ts = timegm(&comp_group1.NT_.tms_); // std::mktime(&tms);
// - https://man7.org/linux/man-pages/man3/tzset.3.html )
std::time_t ts = timegm(&tms); //std::mktime(&tms);
trigger_time_ = std::chrono::system_clock::from_time_t(ts); trigger_time_ = std::chrono::system_clock::from_time_t(ts);
trigger_time_frac_secs_ = comp_group1.NT_.trigger_time_frac_secs_;
// calculate absolute trigger-time
addtime_ = static_cast<long int>(comp_group1.Cb_.add_time_);
absolute_trigger_time_ = trigger_time_ + std::chrono::seconds(addtime_);
// + std::chrono::nanoseconds((long int)(trigger_time_frac_secs_*1.e9));
}
else if ( !chnenv_.compenv1_.uuid_.empty() && !chnenv_.compenv2_.uuid_.empty() )
{
// XY dataset (two components)
// set common NT and CD keys if no others are specified
if (chnenv_.compenv1_.NTuuid_.empty()) chnenv_.compenv1_.NTuuid_ = chnenv_.NTuuid_;
if (chnenv_.compenv1_.CDuuid_.empty()) chnenv_.compenv1_.CDuuid_ = chnenv_.CDuuid_;
if (chnenv_.compenv2_.NTuuid_.empty()) chnenv_.compenv2_.NTuuid_ = chnenv_.NTuuid_;
if (chnenv_.compenv2_.CDuuid_.empty()) chnenv_.compenv2_.CDuuid_ = chnenv_.CDuuid_;
// comp_group1 contains x-data, comp_group2 contains y-data
component_group comp_group1(chnenv_.compenv1_, blocks_, buffer_);
component_group comp_group2(chnenv_.compenv2_, blocks_, buffer_);
dimension_ = 2;
xbuffer_offset_ = comp_group2.Cb_.offset_buffer_;
xbuffer_size_ = comp_group2.Cb_.number_bytes_;
ybuffer_offset_ = comp_group1.Cb_.offset_buffer_;
ybuffer_size_ = comp_group1.Cb_.number_bytes_;
xfactor_ = comp_group2.CR_.factor_;
xoffset_ = comp_group2.CR_.offset_;
yfactor_ = comp_group1.CR_.factor_;
yoffset_ = comp_group1.CR_.offset_;
xdatatp_ = comp_group2.CP_.numeric_type_;
xsignbits_ = comp_group2.CP_.signbits_;
ydatatp_ = comp_group1.CP_.numeric_type_;
ysignbits_ = comp_group1.CP_.signbits_;
// generate std::chrono::system_clock::time_point type
std::time_t ts = timegm(&comp_group2.NT_.tms_); // std::mktime(&tms);
trigger_time_ = std::chrono::system_clock::from_time_t(ts);
trigger_time_frac_secs_ = comp_group2.NT_.trigger_time_frac_secs_;
absolute_trigger_time_ = trigger_time_;
}
else
{
// no datafield
} }
// start converting binary buffer to imc::datatype // start converting binary buffer to imc::datatype
if ( !chnenv_.CSuuid_.empty() ) convert_buffer(); if ( !chnenv_.CSuuid_.empty() ) convert_buffer();
// calculate absolute trigger-time // convert any non-UTF-8 codepage to UTF-8 and cleanse any text
absolute_trigger_time_ = trigger_time_ + std::chrono::seconds(addtime_);
// + std::chrono::nanoseconds((long int)(trigger_time_frac_secs_*1.e9));
// convert any non-UTF-8 codepage to UTF-8
convert_encoding(); convert_encoding();
cleanse_text();
} }
// convert buffer to actual datatype // convert buffer to actual datatype
@ -406,76 +493,111 @@ namespace imc
// extract (channel dependent) part of buffer // extract (channel dependent) part of buffer
unsigned long int buffstrt = prms[3].begin(); unsigned long int buffstrt = prms[3].begin();
std::vector<unsigned char> CSbuffer( buffer_->begin()+buffstrt+buffer_offset_+1, std::vector<unsigned char> yCSbuffer( buffer_->begin()+buffstrt+ybuffer_offset_+1,
buffer_->begin()+buffstrt+buffer_offset_+buffer_size_+1 ); buffer_->begin()+buffstrt+ybuffer_offset_+ybuffer_size_+1 );
// determine number of values in buffer // determine number of values in buffer
unsigned long int num_values = (unsigned long int)(CSbuffer.size()/(signbits_/8)); unsigned long int ynum_values = (unsigned long int)(yCSbuffer.size()/(ysignbits_/8));
if ( num_values*(signbits_/8) != CSbuffer.size() ) if ( ynum_values*(ysignbits_/8) != yCSbuffer.size() )
{ {
throw std::runtime_error("CSbuffer and significant bits of datatype don't match"); throw std::runtime_error("CSbuffer and significant bits of y datatype don't match");
} }
// adjust size of ydata
ydata_.resize(num_values);
// distinguish numeric datatypes included in "imc_datatype" if (dimension_ == 1)
if ( datatp_ == 1 )
{ {
imc::convert_data_to_type<imc_Ubyte>(CSbuffer,ydata_); // process y-data
process_data(ydata_, ynum_values, ydatatp_, yCSbuffer);
// find appropriate precision for "xdata_" by means of "xstepwidth_"
xprec_ = (xstepwidth_ > 0 ) ? (int)ceil(fabs(log10(xstepwidth_))) : 10;
// fill xdata_
for ( unsigned long int i = 0; i < ynum_values; i++ )
{
xdata_.push_back(xstart_+(double)i*xstepwidth_);
} }
else if ( datatp_ == 2 )
{
imc::convert_data_to_type<imc_Sbyte>(CSbuffer,ydata_);
} }
else if ( datatp_ == 3 ) else if (dimension_ == 2)
{ {
imc::convert_data_to_type<imc_Ushort>(CSbuffer,ydata_); // process x- and y-data
std::vector<unsigned char> xCSbuffer( buffer_->begin()+buffstrt+xbuffer_offset_+1,
buffer_->begin()+buffstrt+xbuffer_offset_+xbuffer_size_+1 );
// determine number of values in buffer
unsigned long int xnum_values = (unsigned long int)(xCSbuffer.size()/(xsignbits_/8));
if ( xnum_values*(xsignbits_/8) != xCSbuffer.size() )
{
throw std::runtime_error("CSbuffer and significant bits of x datatype don't match");
} }
else if ( datatp_ == 4 ) if ( xnum_values != ynum_values )
{ {
imc::convert_data_to_type<imc_Sshort>(CSbuffer,ydata_); throw std::runtime_error("x and y data have different number of values");
} }
else if ( datatp_ == 5 )
{ xprec_ = 9;
imc::convert_data_to_type<imc_Ulongint>(CSbuffer,ydata_);
} process_data(xdata_, xnum_values, xdatatp_, xCSbuffer);
else if ( datatp_ == 6 ) process_data(ydata_, ynum_values, ydatatp_, yCSbuffer);
{
imc::convert_data_to_type<imc_Slongint>(CSbuffer,ydata_);
}
else if ( datatp_ == 7 )
{
imc::convert_data_to_type<imc_float>(CSbuffer,ydata_);
}
else if ( datatp_ == 8 )
{
imc::convert_data_to_type<imc_double>(CSbuffer,ydata_);
}
// ...
else if ( datatp_ == 11 )
{
imc::convert_data_to_type<imc_digital>(CSbuffer,ydata_);
} }
else else
{ {
throw std::runtime_error("unsupported dimension");
}
transformData(xdata_, xfactor_, xoffset_);
transformData(ydata_, yfactor_, yoffset_);
}
// handle data type conversion
void process_data(std::vector<imc::datatype>& data_, size_t num_values, numtype datatp_, std::vector<unsigned char>& CSbuffer)
{
// adjust size of data
data_.resize(num_values);
// handle data type conversion
switch (datatp_)
{
case numtype::unsigned_byte:
imc::convert_data_to_type<imc_Ubyte>(CSbuffer, data_);
break;
case numtype::signed_byte:
imc::convert_data_to_type<imc_Sbyte>(CSbuffer, data_);
break;
case numtype::unsigned_short:
imc::convert_data_to_type<imc_Ushort>(CSbuffer, data_);
break;
case numtype::signed_short:
imc::convert_data_to_type<imc_Sshort>(CSbuffer, data_);
break;
case numtype::unsigned_long:
imc::convert_data_to_type<imc_Ulongint>(CSbuffer, data_);
break;
case numtype::signed_long:
imc::convert_data_to_type<imc_Slongint>(CSbuffer, data_);
break;
case numtype::ffloat:
imc::convert_data_to_type<imc_float>(CSbuffer, data_);
break;
case numtype::ddouble:
imc::convert_data_to_type<imc_double>(CSbuffer, data_);
break;
case numtype::two_byte_word_digital:
imc::convert_data_to_type<imc_digital>(CSbuffer, data_);
break;
case numtype::six_byte_unsigned_long:
imc::convert_data_to_type<imc_sixbyte>(CSbuffer, data_);
break;
default:
throw std::runtime_error(std::string("unsupported/unknown datatype ") + std::to_string(datatp_)); throw std::runtime_error(std::string("unsupported/unknown datatype ") + std::to_string(datatp_));
} }
// fill xdata_
for ( unsigned long int i = 0; i < num_values; i++ )
{
xdata_.push_back(xoffset_+(double)i*xstepwidth_);
} }
// employ data transformation void transformData(std::vector<imc::datatype>& data, double factor, double offset) {
if ( factor_ != 1.0 || offset_ != 0.0 ) if (factor != 1.0 || offset != 0.0) {
{ for (imc::datatype& el : data) {
for ( imc::datatype& el: ydata_ ) double fact = (factor == 0.0) ? 1.0 : factor;
{ el = imc::datatype(el.as_double() * fact + offset);
//std::cout<<"value:"<<el.as_double()<<"\n";
double fact = ( factor_ == 0.0 ) ? 1.0 : factor_;
el = imc::datatype(el.as_double()*fact + offset_);
} }
} }
} }
@ -507,6 +629,35 @@ namespace imc
} }
} }
void cleanse_text()
{
escape_backslash(name_);
escape_backslash(comment_);
escape_backslash(origin_);
escape_backslash(origin_comment_);
escape_backslash(text_);
escape_backslash(language_code_);
escape_backslash(yname_);
escape_backslash(yunit_);
escape_backslash(xname_);
escape_backslash(xunit_);
escape_backslash(group_name_);
escape_backslash(group_comment_);
}
void escape_backslash(std::string &text)
{
char backslash = 0x5c;
std::string doublebackslash("\\\\");
for ( std::string::iterator it = text.begin(); it != text.end(); ++it )
{
if ( int(*it) == backslash ) {
text.replace(it,it+1,doublebackslash);
++it;
}
}
}
// get info string // get info string
std::string get_info(int width = 20) std::string get_info(int width = 20)
{ {
@ -519,6 +670,7 @@ namespace imc
<<std::setw(width)<<std::left<<"name:"<<name_<<"\n" <<std::setw(width)<<std::left<<"name:"<<name_<<"\n"
<<std::setw(width)<<std::left<<"comment:"<<comment_<<"\n" <<std::setw(width)<<std::left<<"comment:"<<comment_<<"\n"
<<std::setw(width)<<std::left<<"origin:"<<origin_<<"\n" <<std::setw(width)<<std::left<<"origin:"<<origin_<<"\n"
<<std::setw(width)<<std::left<<"origin-comment:"<<origin_comment_<<"\n"
<<std::setw(width)<<std::left<<"description:"<<text_<<"\n" <<std::setw(width)<<std::left<<"description:"<<text_<<"\n"
<<std::setw(width)<<std::left<<"trigger-time-nt:"<<std::put_time(std::gmtime(&tt),"%FT%T")<<"\n" <<std::setw(width)<<std::left<<"trigger-time-nt:"<<std::put_time(std::gmtime(&tt),"%FT%T")<<"\n"
<<std::setw(width)<<std::left<<"trigger-time:"<<std::put_time(std::gmtime(&att),"%FT%T")<<"\n" <<std::setw(width)<<std::left<<"trigger-time:"<<std::put_time(std::gmtime(&att),"%FT%T")<<"\n"
@ -526,21 +678,20 @@ namespace imc
<<std::setw(width)<<std::left<<"codepage:"<<codepage_<<"\n" <<std::setw(width)<<std::left<<"codepage:"<<codepage_<<"\n"
<<std::setw(width)<<std::left<<"yname:"<<yname_<<"\n" <<std::setw(width)<<std::left<<"yname:"<<yname_<<"\n"
<<std::setw(width)<<std::left<<"yunit:"<<yunit_<<"\n" <<std::setw(width)<<std::left<<"yunit:"<<yunit_<<"\n"
<<std::setw(width)<<std::left<<"datatype:"<<datatp_<<"\n" <<std::setw(width)<<std::left<<"datatype:"<<ydatatp_<<"\n"
<<std::setw(width)<<std::left<<"significant bits:"<<signbits_<<"\n" <<std::setw(width)<<std::left<<"significant bits:"<<ysignbits_<<"\n"
<<std::setw(width)<<std::left<<"buffer-offset:"<<buffer_offset_<<"\n" <<std::setw(width)<<std::left<<"buffer-offset:"<<ybuffer_offset_<<"\n"
<<std::setw(width)<<std::left<<"buffer-size:"<<buffer_size_<<"\n" <<std::setw(width)<<std::left<<"buffer-size:"<<ybuffer_size_<<"\n"
<<std::setw(width)<<std::left<<"add-time:"<<addtime_<<"\n"
<<std::setw(width)<<std::left<<"xname:"<<xname_<<"\n" <<std::setw(width)<<std::left<<"xname:"<<xname_<<"\n"
<<std::setw(width)<<std::left<<"xunit:"<<xunit_<<"\n" <<std::setw(width)<<std::left<<"xunit:"<<xunit_<<"\n"
<<std::setw(width)<<std::left<<"xstepwidth:"<<xstepwidth_<<"\n" <<std::setw(width)<<std::left<<"xstepwidth:"<<xstepwidth_<<"\n"
<<std::setw(width)<<std::left<<"xoffset:"<<xoffset_<<"\n" <<std::setw(width)<<std::left<<"xoffset:"<<xstart_<<"\n"
<<std::setw(width)<<std::left<<"factor:"<<factor_<<"\n" <<std::setw(width)<<std::left<<"factor:"<<yfactor_<<"\n"
<<std::setw(width)<<std::left<<"offset:"<<offset_<<"\n" <<std::setw(width)<<std::left<<"offset:"<<yoffset_<<"\n"
<<std::setw(width)<<std::left<<"group:"<<"("<<group_index_<<","<<group_name_ <<std::setw(width)<<std::left<<"group:"<<"("<<group_index_<<","<<group_name_
<<","<<group_comment_<<")"<<"\n" <<","<<group_comment_<<")"<<"\n"
<<std::setw(width)<<std::left<<"ydata:"<<imc::joinvec<imc::datatype>(ydata_,6,9,true)<<"\n" <<std::setw(width)<<std::left<<"ydata:"<<imc::joinvec<imc::datatype>(ydata_,6,9,true)<<"\n"
<<std::setw(width)<<std::left<<"xdata:"<<imc::joinvec<double>(xdata_,6,xprec_,true)<<"\n"; <<std::setw(width)<<std::left<<"xdata:"<<imc::joinvec<imc::datatype>(xdata_,6,xprec_,true)<<"\n";
// <<std::setw(width)<<std::left<<"aff. blocks:"<<chnenv_.get_json()<<"\n"; // <<std::setw(width)<<std::left<<"aff. blocks:"<<chnenv_.get_json()<<"\n";
return ss.str(); return ss.str();
} }
@ -557,6 +708,7 @@ namespace imc
<<"\",\"name\":\""<<name_ <<"\",\"name\":\""<<name_
<<"\",\"comment\":\""<<comment_ <<"\",\"comment\":\""<<comment_
<<"\",\"origin\":\""<<origin_ <<"\",\"origin\":\""<<origin_
<<"\",\"origin-comment\":\""<<origin_comment_
<<"\",\"description\":\""<<text_ <<"\",\"description\":\""<<text_
<<"\",\"trigger-time-nt\":\""<<std::put_time(std::gmtime(&tt),"%FT%T") <<"\",\"trigger-time-nt\":\""<<std::put_time(std::gmtime(&tt),"%FT%T")
<<"\",\"trigger-time\":\""<<std::put_time(std::gmtime(&att),"%FT%T") <<"\",\"trigger-time\":\""<<std::put_time(std::gmtime(&att),"%FT%T")
@ -564,19 +716,19 @@ namespace imc
<<"\",\"codepage\":\""<<codepage_ <<"\",\"codepage\":\""<<codepage_
<<"\",\"yname\":\""<<prepjsonstr(yname_) <<"\",\"yname\":\""<<prepjsonstr(yname_)
<<"\",\"yunit\":\""<<prepjsonstr(yunit_) <<"\",\"yunit\":\""<<prepjsonstr(yunit_)
<<"\",\"significantbits\":\""<<signbits_ <<"\",\"significantbits\":\""<<ysignbits_
<<"\",\"addtime\":\""<<addtime_ <<"\",\"buffer-size\":\""<<ybuffer_size_
<<"\",\"xname\":\""<<prepjsonstr(xname_) <<"\",\"xname\":\""<<prepjsonstr(xname_)
<<"\",\"xunit\":\""<<prepjsonstr(xunit_) <<"\",\"xunit\":\""<<prepjsonstr(xunit_)
<<"\",\"xstepwidth\":\""<<xstepwidth_ <<"\",\"xstepwidth\":\""<<xstepwidth_
<<"\",\"xoffset\":\""<<xoffset_ <<"\",\"xoffset\":\""<<xstart_
<<"\",\"group\":{"<<"\"index\":\""<<group_index_ <<"\",\"group\":{"<<"\"index\":\""<<group_index_
<<"\",\"name\":\""<<group_name_ <<"\",\"name\":\""<<group_name_
<<"\",\"comment\":\""<<group_comment_<<"\""<<"}"; <<"\",\"comment\":\""<<group_comment_<<"\""<<"}";
if ( include_data ) if ( include_data )
{ {
ss<<",\"ydata\":"<<imc::joinvec<imc::datatype>(ydata_,0,9,true) ss<<",\"ydata\":"<<imc::joinvec<imc::datatype>(ydata_,0,9,true)
<<",\"xdata\":"<<imc::joinvec<double>(xdata_,0,xprec_,true); <<",\"xdata\":"<<imc::joinvec<imc::datatype>(xdata_,0,xprec_,true);
} }
// ss<<"\",\"aff. blocks\":\""<<chnenv_.get_json() // ss<<"\",\"aff. blocks\":\""<<chnenv_.get_json()
ss<<"}"; ss<<"}";

View File

@ -16,12 +16,12 @@ namespace imc
// //
// e.g. ARM Cortex-A72 armv7l gcc version 10.2.0 (Ubuntu 10.2.0-13ubuntu1) // e.g. ARM Cortex-A72 armv7l gcc version 10.2.0 (Ubuntu 10.2.0-13ubuntu1)
// #ifdef __arm__ // #ifdef __arm__
typedef unsigned long int imc_Ulongint; // typedef unsigned long int imc_Ulongint;
typedef signed long int imc_Slongint; // typedef signed long int imc_Slongint;
// e.g. Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz x86_64 gcc version 10.2.0 (Ubuntu 10.2.0-13ubuntu1) // e.g. Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz x86_64 gcc version 10.2.0 (Ubuntu 10.2.0-13ubuntu1)
// #ifdef i386 __i386 __i386__ // #ifdef i386 __i386 __i386__
// typedef unsigned int imc_Ulongint; typedef unsigned int imc_Ulongint;
// typedef signed int imc_Slongint; typedef signed int imc_Slongint;
// //
typedef float imc_float; typedef float imc_float;
typedef double imc_double; typedef double imc_double;
@ -30,7 +30,11 @@ namespace imc
// typedef <whatever that is ->... > "imc Devices Transitional Recording" // typedef <whatever that is ->... > "imc Devices Transitional Recording"
// typedf <sometimestamptype> "Timestamp Ascii" // typedf <sometimestamptype> "Timestamp Ascii"
typedef char16_t imc_digital; typedef char16_t imc_digital;
// typedef < > imc_sixbyte "6byte unsigned long" //
typedef struct {
unsigned char bytes[6];
} imc_sixbyte;
class datatype class datatype
{ {
@ -44,13 +48,14 @@ namespace imc
imc_float sfloat_; // 6 imc_float sfloat_; // 6
imc_double sdouble_; // 7 imc_double sdouble_; // 7
imc_digital sdigital_; // 10 imc_digital sdigital_; // 10
short int dtidx_; // \in \{0,...,7,10\} imc_sixbyte sixbyte_; // 13
short int dtidx_; // \in \{0,...,7,10,13\}
public: public:
datatype(): ubyte_(0), sbyte_(0), datatype(): ubyte_(0), sbyte_(0),
ushort_(0), sshort_(0), ushort_(0), sshort_(0),
ulint_(0), slint_(0), ulint_(0), slint_(0),
sfloat_(0.0), sdouble_(0.0), sfloat_(0.0), sdouble_(0.0),
sdigital_(0), sdigital_(0), sixbyte_({0}),
dtidx_(0) { }; dtidx_(0) { };
// every supported datatype gets its own constructor // every supported datatype gets its own constructor
datatype(imc_Ubyte num): ubyte_(num), dtidx_(0) {}; datatype(imc_Ubyte num): ubyte_(num), dtidx_(0) {};
@ -62,10 +67,13 @@ namespace imc
datatype(imc_float num): sfloat_(num), dtidx_(6) {}; datatype(imc_float num): sfloat_(num), dtidx_(6) {};
datatype(imc_double num): ubyte_(0), sbyte_(0), ushort_(0), sshort_(0), datatype(imc_double num): ubyte_(0), sbyte_(0), ushort_(0), sshort_(0),
ulint_(0), slint_(0), sfloat_(0.0), sdouble_(num), ulint_(0), slint_(0), sfloat_(0.0), sdouble_(num),
sdigital_(0), dtidx_(7) {}; sdigital_(0), sixbyte_({0}), dtidx_(7) {};
datatype(imc_digital num): ubyte_(0), sbyte_(0), ushort_(0), sshort_(0), datatype(imc_digital num): ubyte_(0), sbyte_(0), ushort_(0), sshort_(0),
ulint_(0), slint_(0), sfloat_(0.0), sdouble_(num), ulint_(0), slint_(0), sfloat_(0.0), sdouble_(0.0),
sdigital_(num), dtidx_(10) {}; sdigital_(num), sixbyte_({0}), dtidx_(10) {};
datatype(imc_sixbyte num): ubyte_(0), sbyte_(0), ushort_(0), sshort_(0),
ulint_(0), slint_(0), sfloat_(0.0), sdouble_(0.0),
sdigital_(0), sixbyte_(num), dtidx_(13) {};
// identify type // identify type
short int& dtype() { return dtidx_; } short int& dtype() { return dtidx_; }
@ -82,6 +90,7 @@ namespace imc
this->sfloat_ = num.sfloat_; this->sfloat_ = num.sfloat_;
this->sdouble_ = num.sdouble_; this->sdouble_ = num.sdouble_;
this->sdigital_ = num.sdigital_; this->sdigital_ = num.sdigital_;
this->sixbyte_ = num.sixbyte_;
this->dtidx_ = num.dtidx_; this->dtidx_ = num.dtidx_;
} }
@ -99,6 +108,7 @@ namespace imc
this->sfloat_ = num.sfloat_; this->sfloat_ = num.sfloat_;
this->sdouble_ = num.sdouble_; this->sdouble_ = num.sdouble_;
this->sdigital_ = num.sdigital_; this->sdigital_ = num.sdigital_;
this->sixbyte_ = num.sixbyte_;
this->dtidx_ = num.dtidx_; this->dtidx_ = num.dtidx_;
} }
@ -160,6 +170,12 @@ namespace imc
this->dtidx_ = 10; this->dtidx_ = 10;
return *this; return *this;
} }
datatype& operator=(const imc_sixbyte &num)
{
this->sixbyte_ = num;
this->dtidx_ = 13;
return *this;
}
// obtain number as double // obtain number as double
double as_double() double as_double()
@ -174,6 +190,13 @@ namespace imc
else if ( dtidx_ == 6 ) num = (double)sfloat_; else if ( dtidx_ == 6 ) num = (double)sfloat_;
else if ( dtidx_ == 7 ) num = (double)sdouble_; else if ( dtidx_ == 7 ) num = (double)sdouble_;
else if ( dtidx_ == 10 ) num = static_cast<double>(sdigital_); else if ( dtidx_ == 10 ) num = static_cast<double>(sdigital_);
else if ( dtidx_ == 13 ) {
unsigned long long value = 0;
for (int i = 0; i < 6; ++i) {
value |= static_cast<unsigned long long>(sixbyte_.bytes[i]) << (8 * i);
}
num = static_cast<double>(value);
}
return num; return num;
} }
@ -189,6 +212,13 @@ namespace imc
else if ( num.dtidx_ == 6 ) out<<num.sfloat_; else if ( num.dtidx_ == 6 ) out<<num.sfloat_;
else if ( num.dtidx_ == 7 ) out<<num.sdouble_; else if ( num.dtidx_ == 7 ) out<<num.sdouble_;
else if ( num.dtidx_ == 10 ) out<<static_cast<double>(num.sdigital_); else if ( num.dtidx_ == 10 ) out<<static_cast<double>(num.sdigital_);
else if ( num.dtidx_ == 13 ) {
unsigned long long value = 0;
for (int i = 0; i < 6; ++i) {
value |= static_cast<unsigned long long>(num.sixbyte_.bytes[i]) << (8 * i);
}
out<<static_cast<double>(value);
}
return out; return out;
} }

View File

@ -4,6 +4,7 @@
#define IMCOBJECT #define IMCOBJECT
#include <time.h> #include <time.h>
#include <math.h>
#include "imc_key.hpp" #include "imc_key.hpp"
//---------------------------------------------------------------------------// //---------------------------------------------------------------------------//
@ -245,7 +246,7 @@ namespace imc
// construct members by parsing particular parameters from buffer // construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters) void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{ {
if ( parameters.size() < 4 ) throw std::runtime_error("invalid number of parameters in CD2"); if ( parameters.size() < 4 ) throw std::runtime_error("invalid number of parameters in CC");
component_index_ = std::stoi(get_parameter(buffer,&parameters[2])); component_index_ = std::stoi(get_parameter(buffer,&parameters[2]));
analog_digital_ = ( std::stoi(get_parameter(buffer,&parameters[3])) == 2 ); analog_digital_ = ( std::stoi(get_parameter(buffer,&parameters[3])) == 2 );
} }
@ -272,7 +273,9 @@ namespace imc
imc_devices_transitional_recording, imc_devices_transitional_recording,
timestamp_ascii, timestamp_ascii,
two_byte_word_digital, two_byte_word_digital,
six_byte_unsigned_long eight_byte_unsigned_long,
six_byte_unsigned_long,
eight_byte_signed_long
}; };
// packaging information of component (corresponds to key CP) // packaging information of component (corresponds to key CP)
@ -292,8 +295,8 @@ namespace imc
{ {
if ( parameters.size() < 10 ) throw std::runtime_error("invalid number of parameters in CP"); if ( parameters.size() < 10 ) throw std::runtime_error("invalid number of parameters in CP");
buffer_reference_ = std::stoi(get_parameter(buffer,&parameters[2])); buffer_reference_ = std::stoi(get_parameter(buffer,&parameters[2]));
numeric_type_ = (numtype)std::stoi(get_parameter(buffer,&parameters[3])); bytes_ = std::stoi(get_parameter(buffer,&parameters[3]));
bytes_ = std::stoi(get_parameter(buffer,&parameters[4])); numeric_type_ = (numtype)std::stoi(get_parameter(buffer,&parameters[4]));
signbits_ = std::stoi(get_parameter(buffer,&parameters[5])); signbits_ = std::stoi(get_parameter(buffer,&parameters[5]));
mask_ = std::stoi(get_parameter(buffer,&parameters[6])); mask_ = std::stoi(get_parameter(buffer,&parameters[6]));
offset_ = std::stoul(get_parameter(buffer,&parameters[7])); offset_ = std::stoul(get_parameter(buffer,&parameters[7]));
@ -336,7 +339,7 @@ namespace imc
// construct members by parsing particular parameters from buffer // construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters) void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{ {
if ( parameters.size() < 13 ) throw std::runtime_error("invalid number of parameters in CD2"); if ( parameters.size() < 13 ) throw std::runtime_error("invalid number of parameters in Cb");
number_buffers_ = std::stoul(get_parameter(buffer,&parameters[2])); number_buffers_ = std::stoul(get_parameter(buffer,&parameters[2]));
bytes_userinfo_ = std::stoul(get_parameter(buffer,&parameters[3])); bytes_userinfo_ = std::stoul(get_parameter(buffer,&parameters[3]));
buffer_reference_ = std::stoul(get_parameter(buffer,&parameters[4])); buffer_reference_ = std::stoul(get_parameter(buffer,&parameters[4]));
@ -378,7 +381,7 @@ namespace imc
// construct members by parsing particular parameters from buffer // construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters) void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{ {
if ( parameters.size() < 8 ) throw std::runtime_error("invalid number of parameters in CD2"); if ( parameters.size() < 8 ) throw std::runtime_error("invalid number of parameters in CR");
transform_ = (get_parameter(buffer,&parameters[2]) == std::string("1")); transform_ = (get_parameter(buffer,&parameters[2]) == std::string("1"));
factor_ = std::stod(get_parameter(buffer,&parameters[3])); factor_ = std::stod(get_parameter(buffer,&parameters[3]));
offset_ = std::stod(get_parameter(buffer,&parameters[4])); offset_ = std::stod(get_parameter(buffer,&parameters[4]));
@ -410,7 +413,7 @@ namespace imc
// construct members by parsing particular parameters from buffer // construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters) void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{ {
if ( parameters.size() < 9 ) throw std::runtime_error("invalid number of parameters in CD2"); if ( parameters.size() < 9 ) throw std::runtime_error("invalid number of parameters in CN");
group_index_ = std::stoul(get_parameter(buffer,&parameters[2])); group_index_ = std::stoul(get_parameter(buffer,&parameters[2]));
index_bit_ = (get_parameter(buffer,&parameters[4]) == std::string("1")); index_bit_ = (get_parameter(buffer,&parameters[4]) == std::string("1"));
name_ = get_parameter(buffer,&parameters[6]); name_ = get_parameter(buffer,&parameters[6]);
@ -439,7 +442,7 @@ namespace imc
// construct members by parsing particular parameters from buffer // construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters) void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{ {
if ( parameters.size() < 4 ) throw std::runtime_error("invalid number of parameters in CD2"); if ( parameters.size() < 4 ) throw std::runtime_error("invalid number of parameters in CS");
index_ = std::stoul(get_parameter(buffer,&parameters[2])); index_ = std::stoul(get_parameter(buffer,&parameters[2]));
} }
@ -454,6 +457,21 @@ namespace imc
} }
}; };
// language (corresponds to key NL)
struct language
{
std::string codepage_;
std::string language_code_;
// construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{
if (parameters.size() < 4) throw std::runtime_error("invalid number of parameters in NL");
codepage_ = get_parameter(buffer, &parameters[2]);
language_code_ = get_parameter(buffer, &parameters[3]);
}
};
// origin of data (corresponds to key NO) // origin of data (corresponds to key NO)
struct origin_data struct origin_data
{ {
@ -464,7 +482,7 @@ namespace imc
// construct members by parsing particular parameters from buffer // construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters) void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{ {
if ( parameters.size() < 7 ) throw std::runtime_error("invalid number of parameters in CD2"); if ( parameters.size() < 7 ) throw std::runtime_error("invalid number of parameters in NO");
origin_ = ( get_parameter(buffer,&parameters[2]) == std::string("1") ); origin_ = ( get_parameter(buffer,&parameters[2]) == std::string("1") );
generator_ = get_parameter(buffer,&parameters[4]); generator_ = get_parameter(buffer,&parameters[4]);
comment_ = get_parameter(buffer,&parameters[6]); comment_ = get_parameter(buffer,&parameters[6]);
@ -484,45 +502,30 @@ namespace imc
// trigger timestamp (corresponds to key NT1) // trigger timestamp (corresponds to key NT1)
struct triggertime struct triggertime
{ {
int day_, month_, year_; std::tm tms_;
int hour_, minute_; double trigger_time_frac_secs_;
double second_;
std::string timestamp_;
// construct members by parsing particular parameters from buffer // construct members by parsing particular parameters from buffer
void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters) void parse(const std::vector<unsigned char>* buffer, const std::vector<parameter>& parameters)
{ {
if ( parameters.size() < 8 ) throw std::runtime_error("invalid number of parameters in CD2"); if ( parameters.size() < 8 ) throw std::runtime_error("invalid number of parameters in NT1");
day_ = std::stoi( get_parameter(buffer,&parameters[2]) ); tms_ = std::tm();
month_ = std::stoi( get_parameter(buffer,&parameters[3]) ); tms_.tm_mday = std::stoi( get_parameter(buffer,&parameters[2]) );
year_ = std::stoi( get_parameter(buffer,&parameters[4]) ); tms_.tm_mon = std::stoi( get_parameter(buffer,&parameters[3]) ) - 1;
hour_ = std::stoi( get_parameter(buffer,&parameters[5]) ); tms_.tm_year = std::stoi( get_parameter(buffer,&parameters[4]) ) - 1900;
minute_ = std::stoi( get_parameter(buffer,&parameters[6]) ); tms_.tm_hour = std::stoi( get_parameter(buffer,&parameters[5]) );
second_ = std::stod( get_parameter(buffer,&parameters[7]) ); tms_.tm_min = std::stoi( get_parameter(buffer,&parameters[6]) );
long double secs = std::stold( get_parameter(buffer,&parameters[7]) );
//time_t rawtime; double secs_int;
//struct tm ts; trigger_time_frac_secs_ = modf((double)secs,&secs_int);
//time(&rawtime); tms_.tm_sec = (int)secs_int;
//localtime_r(&rawtime,&ts);
//ts.tm_mday = day_;
//ts.tm_mon = month_-1;
//ts.tm_year = year_-1900;
//ts.tm_hour = hour_;
//ts.tm_min = minute_;
//ts.tm_sec = (int)second_;
//asctime_r(&ts,&timestamp_[0]);
timestamp_ = std::to_string(year_) + std::string("-") + std::to_string(month_)
+ std::string("-") + std::to_string(day_)
+ std::string("T") + std::to_string(hour_)
+ std::string(":") + std::to_string(minute_)
+ std::string(":") + std::to_string(second_);
} }
// get info string // get info string
std::string get_info(int width = 20) std::string get_info(int width = 20)
{ {
std::stringstream ss; std::stringstream ss;
ss<<std::setw(width)<<std::left<<"timestamp:"<<timestamp_<<"\n"; ss<<std::setw(width)<<std::left<<"timestamp:"<<std::put_time(&tms_, "%Y-%m-%dT%H:%M:%S")<<"\n";
return ss.str(); return ss.str();
} }
}; };

View File

@ -211,29 +211,61 @@ namespace imc
imc::channel_env chnenv; imc::channel_env chnenv;
chnenv.reset(); chnenv.reset();
imc::component_env *compenv_ptr = nullptr;
// collect affiliate blocks for every channel WITH CHANNEL and AFFILIATE // collect affiliate blocks for every channel WITH CHANNEL and AFFILIATE
// BLOCK CORRESPONDENCE GOVERNED BY BLOCK ORDER IN BUFFER!! // BLOCK CORRESPONDENCE GOVERNED BY BLOCK ORDER IN BUFFER!!
for ( imc::block blk: rawblocks_ ) for ( imc::block blk: rawblocks_ )
{ {
if ( blk.get_key().name_ == "CN" ) chnenv.CNuuid_ = blk.get_uuid(); if ( blk.get_key().name_ == "NO" ) chnenv.NOuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CD" ) chnenv.CDuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CT" ) chnenv.CTuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "Cb" ) chnenv.Cbuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CP" ) chnenv.CPuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CR" ) chnenv.CRuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CS" ) chnenv.CSuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "NT" ) chnenv.NTuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "NO" ) chnenv.NOuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "NL" ) chnenv.NLuuid_ = blk.get_uuid(); else if ( blk.get_key().name_ == "NL" ) chnenv.NLuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CB" ) chnenv.CBuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CG" ) chnenv.CGuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CI" ) chnenv.CIuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CT" ) chnenv.CTuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CN" ) chnenv.CNuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CS" ) chnenv.CSuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CC" )
{
// a new component group is started
// TODO: can we avoid to parse the whole component here?
imc::component component;
component.parse(&buffer_, blk.get_parameters());
if ( component.component_index_ == 1 ) compenv_ptr = &chnenv.compenv1_;
else if ( component.component_index_ == 2 ) compenv_ptr = &chnenv.compenv2_;
else throw std::runtime_error("invalid component index in CC block");
compenv_ptr->CCuuid_ = blk.get_uuid();
compenv_ptr->uuid_ = compenv_ptr->CCuuid_;
}
else if ( blk.get_key().name_ == "CD" )
{
if (compenv_ptr == nullptr) chnenv.CDuuid_ = blk.get_uuid();
else compenv_ptr->CDuuid_ = blk.get_uuid();
}
else if ( blk.get_key().name_ == "NT" )
{
if (compenv_ptr == nullptr) chnenv.NTuuid_ = blk.get_uuid();
else compenv_ptr->NTuuid_ = blk.get_uuid();
}
else if ( blk.get_key().name_ == "Cb" ) compenv_ptr->Cbuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CP" ) compenv_ptr->CPuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CR" ) compenv_ptr->CRuuid_ = blk.get_uuid();
// check for currently associated channel // check for currently associated channel
// TODO: CNuuid is not unique for multichannel data
if ( !chnenv.CNuuid_.empty() ) if ( !chnenv.CNuuid_.empty() )
{ {
// any component/channel is closed by any of {CS, CC, CG, CB} // at the moment only a single channel is supported
if ( blk.get_key().name_ == "CS" || blk.get_key().name_ == "CC" // any channel is closed by any of {CB, CG, CI, CT, CS}
|| blk.get_key().name_ == "CG" || blk.get_key().name_ == "CB" ) if ( blk.get_key().name_ == "CB" || blk.get_key().name_ == "CG"
|| blk.get_key().name_ == "CI" || blk.get_key().name_ == "CT"
|| blk.get_key().name_ == "CS" )
{ {
// provide UUID for channel // provide UUID for channel
// for multi component channels exactly one CN is available
chnenv.uuid_ = chnenv.CNuuid_; chnenv.uuid_ = chnenv.CNuuid_;
// for multichannel data there may be multiple channels referring to // for multichannel data there may be multiple channels referring to
@ -255,8 +287,15 @@ namespace imc
); );
// reset channel uuid // reset channel uuid
chnenv.reset(); chnenv.CNuuid_.clear();
//chnenv.CNuuid_.clear();
chnenv.CBuuid_.clear();
chnenv.CGuuid_.clear();
chnenv.CIuuid_.clear();
chnenv.CTuuid_.clear();
chnenv.CSuuid_.clear();
compenv_ptr = nullptr;
} }
} }
@ -264,7 +303,8 @@ namespace imc
// already belong to NEXT component // already belong to NEXT component
if ( blk.get_key().name_ == "CB" ) chnenv.CBuuid_ = blk.get_uuid(); if ( blk.get_key().name_ == "CB" ) chnenv.CBuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CG" ) chnenv.CGuuid_ = blk.get_uuid(); else if ( blk.get_key().name_ == "CG" ) chnenv.CGuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CC" ) chnenv.CCuuid_ = blk.get_uuid(); else if ( blk.get_key().name_ == "CI" ) chnenv.CIuuid_ = blk.get_uuid();
else if ( blk.get_key().name_ == "CT" ) chnenv.CTuuid_ = blk.get_uuid();
} }
} }

View File

@ -88,11 +88,11 @@ docker-run:
python-build: check-tags $(GVSN) python-build: check-tags $(GVSN)
make -C python/ build-inplace make -C python/ build-inplace
cp python/IMCtermite*.so ./ -v cp python/imctermite*.so ./ -v
python-clean: python-clean:
make -C python/ clean make -C python/ clean
rm -vf IMCtermite*.so rm -vf imctermite*.so
python-test: python-test:
PYTHONPATH=./ python python/examples/usage.py PYTHONPATH=./ python python/examples/usage.py

View File

@ -1 +1 @@
2.0.9 2.1.18

View File

@ -1,5 +1,5 @@
import IMCtermite import imctermite
import pandas import pandas
import datetime import datetime
@ -11,7 +11,7 @@ def add_trigger_time(trigger_time, add_time) :
if __name__ == "__main__" : if __name__ == "__main__" :
# read file and extract data # read file and extract data
imctm = IMCtermite.imctermite(b"Measurement.raw") imctm = imctermite.imctermite(b"Measurement.raw")
chns = imctm.get_channels(True) chns = imctm.get_channels(True)
# prepare abscissa # prepare abscissa

View File

@ -1,11 +1,11 @@
import IMCtermite import imctermite
import json import json
import os import os
# declare and initialize instance of "imctermite" by passing a raw-file # declare and initialize instance of "imctermite" by passing a raw-file
try : try :
imcraw = IMCtermite.imctermite(b"samples/exampleB.raw") imcraw = imctermite.imctermite(b"samples/exampleB.raw")
except RuntimeError as e : except RuntimeError as e :
raise Exception("failed to load/parse raw-file: " + str(e)) raise Exception("failed to load/parse raw-file: " + str(e))

View File

@ -1,5 +1,5 @@
import IMCtermite import imctermite
import json import json
import os import os
@ -15,7 +15,7 @@ for fl in rawlist1:
# declare and initialize instance of "imctermite" by passing a raw-file # declare and initialize instance of "imctermite" by passing a raw-file
try : try :
imcraw = IMCtermite.imctermite(fl.encode()) imcraw = imctermite.imctermite(fl.encode())
except RuntimeError as e : except RuntimeError as e :
raise Exception("failed to load/parse raw-file: " + str(e)) raise Exception("failed to load/parse raw-file: " + str(e))

View File

@ -1,12 +1,12 @@
import IMCtermite import imctermite
import json import json
import os import os
import datetime import datetime
# declare and initialize instance of "imctermite" by passing a raw-file # declare and initialize instance of "imctermite" by passing a raw-file
try : try :
imcraw = IMCtermite.imctermite(b"samples/sampleB.raw") imcraw = imctermite.imctermite(b"samples/sampleB.raw")
except RuntimeError as e : except RuntimeError as e :
raise Exception("failed to load/parse raw-file: " + str(e)) raise Exception("failed to load/parse raw-file: " + str(e))

View File

@ -1,5 +1,5 @@
from IMCtermite import imctermite import imctermite import imctermite
def show_results(imcraw) : def show_results(imcraw) :

View File

@ -1,10 +1,20 @@
# distutils: language = c++ # distutils: language = c++
# cython: language_level = 3 # cython: language_level = 3
from IMCtermite cimport cppimctermite from imctermite cimport cppimctermite
import json as jn import json as jn
import decimal import decimal
import platform
# auxiliary function for codepage conversion
def get_codepage(chn) :
if platform == 'Windows' :
chndec = jn.loads(chn.decode(errors="ignore"))
chncdp = chndec["codepage"]
return 'utf-8' if chncdp is None else chncdp
else :
return 'utf-8'
cdef class imctermite: cdef class imctermite:
@ -22,7 +32,7 @@ cdef class imctermite:
# get JSON list of channels # get JSON list of channels
def get_channels(self, bool include_data): def get_channels(self, bool include_data):
chnlst = self.cppimc.get_channels(True,include_data) chnlst = self.cppimc.get_channels(True,include_data)
chnlstjn = [jn.loads(chn.decode(errors="ignore")) for chn in chnlst] chnlstjn = [jn.loads(chn.decode(get_codepage(chn),errors="ignore")) for chn in chnlst]
return chnlstjn return chnlstjn
# print single channel/all channels # print single channel/all channels

View File

@ -29,6 +29,7 @@ build-clean:
rm -vf imctermite*.so imctermite*.cpp rm -vf imctermite*.so imctermite*.cpp
rm -vf IMCtermite*.so IMCtermite*.cpp rm -vf IMCtermite*.so IMCtermite*.cpp
rm -rvf dist/ IMCtermite.egg-info/ rm -rvf dist/ IMCtermite.egg-info/
rm -rvf dist/ imctermite.egg-info/
cibuildwheel-build: setup cibuildwheel-build: setup
cibuildwheel --platform linux cibuildwheel --platform linux

View File

@ -1,6 +1,6 @@
[metadata] [metadata]
name = IMCtermite name = imctermite
description = Enables extraction of measurement data from binary files with extension 'raw' used by proprietary software imcFAMOS and imcSTUDIO and facilitates its storage in open source file formats description = Enables extraction of measurement data from binary files with extension 'raw' used by proprietary software imcFAMOS and imcSTUDIO and facilitates its storage in open source file formats
long_description = file: README.md long_description = file: README.md
# long_description_content_type = text/x-rst # long_description_content_type = text/x-rst

View File

@ -11,8 +11,8 @@ cmpArgs = {
} }
extension = Extension( extension = Extension(
"IMCtermite", "imctermite",
sources=["IMCtermite.pyx"], sources=["imctermite.pyx"],
extra_compile_args=cmpArgs[sys.platform] extra_compile_args=cmpArgs[sys.platform]
) )

Binary file not shown.

View File

@ -0,0 +1,14 @@
|CF,2,1,1;|CK,1,3,1,1;
|Nv,1,12,7,3,4,64,1,0;
|NO,1,12,1,5,FAMOS,0,;
|NL,1,10,1252,0x407;
|CG,1,5,1,1,1;
|CD,1,13,1,1,1,s,0,0,0;
|NT,1,19, 6, 4,2018,11,33,54;
|CC,1,3,1,1;
|CP,1,16,1,8,8,64,0,0,1,0;
|Cb,1,22,1,0,1,1,0,8,0,8,1,0,0,;
|CR,1,11,0,0,0,1,1,V;
|CN,1,35,0,0,0,7,Average,16,Measurement 0815;
|CT,1,43,0,8,TxTester,8,E. Smith,16,Measurement 0815;
|CS,1,10,1,ÍÌÌÌÌÌ(@;

View File

@ -0,0 +1,24 @@
|CF,2,1,1;|CK,1,3,1,1;
|Nv,1,12,7,3,4,64,1,0;
|NO,1,12,1,5,FAMOS,0,;
|NL,1,10,1252,0x407;
|CB,1,12,1,5,Meas1,0,;
|CG,1,5,1,1,1;
|CD,1,16,5E-1,1,1,2,0,0,0;
|NT,1,19, 6, 4,2018,11,24,18;
|CC,1,3,1,1;
|CP,1,15,1,1,2,8,0,0,1,0;
|Cb,1,22,1,0,1,1,0,3,0,3,1,3,0,;
|CR,1,30,1,3.937007874015748E-2,5,1,1,V;
|ND,1,15,-1,-1,-1,0,1E+1;
|CN,1,16,1,0,0,5,Chan1,0,;
|CG,1,5,1,1,1;
|CD,1,16,5E-1,1,1,2,0,0,0;
|NT,1,19, 6, 4,2018,11,24,18;
|CC,1,3,1,1;
|CP,1,15,2,1,2,8,0,0,1,0;
|Cb,1,22,1,0,2,1,3,3,0,3,1,3,0,;
|CR,1,30,1,3.937007874015748E-2,5,1,1,V;
|ND,1,15,-1,-1,-1,0,1E+1;
|CN,1,16,1,0,0,5,Chan2,0,;
|CS,1,8,1, šš šš;

View File

@ -0,0 +1,17 @@
|CF,2,1,1;|CK,1,3,1,1;
|Nv,1,12,7,3,4,64,1,0;
|NO,1,12,1,5,FAMOS,0,;
|NL,1,10,1252,0x407;
|CG,1,5,2,2,2;
|CD,1,12,1,1,0,,2,0,0;
|NT,1,18, 6, 4,2018,11,37,1;
|CC,1,3,1,1;
|CP,1,16,1,4,7,32,0,0,1,0;
|Cb,1,24,1,0,1,1,0,16,0,16,1,0,0,;
|CR,1,11,0,0,0,1,1,V;
|CN,1,20,0,0,0,9,MyXY_plot,0,;
|CC,1,3,2,1;
|CP,1,16,2,2,3,16,0,0,1,0;
|Cb,1,23,1,0,2,1,16,8,0,8,1,0,0,;
|CR,1,30,1,4.577706569008927E-5,0,1,1,s;
|CS,1,26,1, @ €? @ffF@ UUªªÿÿ;