Compare commits

...

20 Commits

Author SHA1 Message Date
befda072e5 include origin comment of NO block 2024-06-12 20:53:59 +02:00
0b1cb2f230 reference correct name of data file format in README.md (issue #14) and add link to imc documentation 2024-06-12 20:32:01 +02:00
ce444814c2 reference correct name of data file format in README.md (issue #14) and add link to imc documentation 2024-06-12 20:31:08 +02:00
eec3853af9 by default use utf-8 for metadata including non-ascii characters while missing codepage info 2024-06-12 12:02:26 +02:00
25e114dbea Fix Github Workflow Badge:
* https://github.com/badges/shields/issues/8733
* https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/adding-a-workflow-status-badge
2024-06-12 11:50:35 +02:00
740116c7ba bump version 2.0.20 2024-06-12 11:23:12 +02:00
5899bd646b bump version 2.0.19 2024-06-12 10:39:22 +02:00
1d33a81ad2 bump version 2.0.18 2024-06-12 10:37:54 +02:00
b58efae107 bump version 2.0.17 2024-06-12 10:33:51 +02:00
cc0dd9978f update and fix badges 2024-06-12 10:22:53 +02:00
4834f63a67 update and fix badges 2024-06-12 10:16:56 +02:00
1be78a2c5e Revert "update and fix badges"
This reverts commit f5581557bc.
2024-06-12 10:15:23 +02:00
f5581557bc update and fix badges 2024-06-12 10:14:51 +02:00
8cde88848b update and fix badges 2024-06-12 10:08:33 +02:00
0799513ea2 by default use codepage1252 for metadata including non-ascii characters while missing codepage info 2024-06-12 09:45:30 +02:00
effeee105c win32: replace timegm by _mkgmtime, bump version 2.0.15 2023-08-08 23:46:10 +02:00
ed5b366341 imc_channel.hpp: usage of iconv for unix only, bump version 2.0.15 2023-08-08 23:29:48 +02:00
9a520ddd9c bump version 2.0.14 2023-08-08 00:54:07 +02:00
2c43087d15 bump version 2.0.13 2023-08-08 00:52:27 +02:00
60ac1365a5 * imc_channel.hpp: usage of iconv for unix only
* IMCtermite.pyx: add codepage conversion for windows
* bump VERSION
2023-08-08 00:50:52 +02:00
6 changed files with 44 additions and 24 deletions

View File

@@ -1,9 +1,9 @@
name: Build Python Wheels
name: CI Build Wheel
on:
push:
#branches: [master]
# branches: [master]
tags: ["v[0-9]+.[0-9]+.[0-9]+"]
jobs:
@@ -40,12 +40,6 @@ jobs:
with:
name: wheel-config
path: python/
- name: Install libiconv for windows
if: matrix.os == 'windows-2019'
run: |
git clone https://github.com/Microsoft/vcpkg.git
.\vcpkg\bootstrap-vcpkg.bat
vcpkg install libiconv
- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
working-directory: python/

View File

@@ -1,15 +1,13 @@
[![Total alerts](https://img.shields.io/lgtm/alerts/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/alerts/)
[![Language grade: C/C++](https://img.shields.io/lgtm/grade/cpp/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/context:cpp)
[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/context:python)
[![LICENSE](https://img.shields.io/github/license/RecordEvolution/IMCtermite)](https://img.shields.io/github/license/RecordEvolution/IMCtermite)
[![STARS](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)
![Build Python Wheels](https://github.com/RecordEvolution/IMCtermite/actions/workflows/pypi-deploy.yml/badge.svg)
![CI Build Wheel](https://github.com/RecordEvolution/IMCtermite/actions/workflows/pypi-deploy.yml/badge.svg?branch=&event=push)
[![PYPI](https://img.shields.io/pypi/v/IMCtermite.svg)](https://pypi.org/project/IMCtermite/)
# IMCtermite
_IMCtermite_ provides access to the proprietary data format
_IMC Bus Format_ with the file extension _.raw_ introduced and developed by
_IMC2 Data Format_ with the file extension _.raw_ (or .dat) introduced and developed by
[imc Test & Measurement GmbH](https://www.imc-tm.de/). This data format is
employed i.a. by the measurement hardware
[imc CRONOSflex](https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/)
@@ -20,7 +18,9 @@ for measurement data control and analysis. Thanks to the integrated Python modul
the extracted measurement data can be stored in any open-source file format
accessible by Python like i.a. _csv_, _json_ or _parquet_.
On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/), the library can be used both as a command line tool for interactive usage and as a Python module to integrate the _.raw_ format into any ETL workflow.
On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/),
the library can be used both as a command line tool for interactive usage and as a
Python module to integrate the _.raw_ format into any ETL workflow.
## Overview
@@ -31,12 +31,11 @@ On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/),
## File format
[Warning: Take a look at [this issue](https://github.com/RecordEvolution/IMCtermite/issues/14) when reading this section regarding the file format.]
A data file of the _IMC Bus Format_ type with the extension _.raw_ is a _mixed text/binary
A file of the _IMC2 Data Format_ type with extension _.raw_ (or .dat) is a _mixed text/binary
file_ featuring a set of markers (keys) that indicate the start of various blocks
of data that provide meta information and the actual measurement data. Every single
marker is introduced by the character `"|" = 0x 7c` followed by two uppercase letters that characterize the type of marker. Each block is further divided into several
marker is introduced by the character `"|" = 0x 7c` followed by two uppercase letters that
characterize the type of marker. Each block is further divided into several
parameters separated by commata `"," = 0x 2c` and terminated by a semicolon
`";" = 0x 3b`. For instance, the header - first 600 bytes - of a raw file may
look like this (in UTF-8 encoding):
@@ -225,6 +224,8 @@ can be found in the `python/examples` folder.
- https://www.imc-tm.de/produkte/messtechnik-software/imc-famos/funktionen/im-und-export/
- https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/
- https://www.imc-tm.de/download-center/produkt-downloads/imc-famos/handbuecher
- https://www.imc-tm.de/fileadmin/Public/Downloads/Manuals/imc_FAMOS/imcGemeinsameKomponenten.pdf
- https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html
- https://github.com/Apollo3zehn/ImcFamosFile
- https://apollo3zehn.github.io/ImcFamosFile/api/ImcFamosFile.FamosFileKeyType.html

View File

@@ -10,7 +10,11 @@
#include <chrono>
#include <ctime>
#include <time.h>
#if defined(__linux__) || defined(__APPLE__)
#include <iconv.h>
#elif defined(__WIN32__) || defined(_WIN32)
#define timegm _mkgmtime
#endif
//---------------------------------------------------------------------------//
@@ -144,12 +148,13 @@ namespace imc
return sumstr;
}
#if defined(__linux__) || defined(__APPLE__)
// convert encoding of any descriptions, channel-names, units etc.
class iconverter
{
std::string in_enc_, out_enc_;
iconv_t cd_;
size_t out_buffer_size_;
std::string in_enc_, out_enc_;
iconv_t cd_;
size_t out_buffer_size_;
public:
@@ -215,6 +220,14 @@ namespace imc
astring = outstring;
}
};
#elif defined(__WIN32__) || defined(_WIN32)
class iconverter
{
public:
iconverter(std::string in_enc, std::string out_enc, size_t out_buffer_size = 1024) {}
void convert(std::string &astring) {}
};
#endif
// channel
struct channel
@@ -519,6 +532,7 @@ namespace imc
<<std::setw(width)<<std::left<<"name:"<<name_<<"\n"
<<std::setw(width)<<std::left<<"comment:"<<comment_<<"\n"
<<std::setw(width)<<std::left<<"origin:"<<origin_<<"\n"
<<std::setw(width)<<std::left<<"origin-comment:"<<origin_comment_<<"\n"
<<std::setw(width)<<std::left<<"description:"<<text_<<"\n"
<<std::setw(width)<<std::left<<"trigger-time-nt:"<<std::put_time(std::gmtime(&tt),"%FT%T")<<"\n"
<<std::setw(width)<<std::left<<"trigger-time:"<<std::put_time(std::gmtime(&att),"%FT%T")<<"\n"
@@ -557,6 +571,7 @@ namespace imc
<<"\",\"name\":\""<<name_
<<"\",\"comment\":\""<<comment_
<<"\",\"origin\":\""<<origin_
<<"\",\"origin-comment\":\""<<origin_comment_
<<"\",\"description\":\""<<text_
<<"\",\"trigger-time-nt\":\""<<std::put_time(std::gmtime(&tt),"%FT%T")
<<"\",\"trigger-time\":\""<<std::put_time(std::gmtime(&att),"%FT%T")

View File

@@ -5,6 +5,16 @@ from IMCtermite cimport cppimctermite
import json as jn
import decimal
import platform
# auxiliary function for codepage conversion
def get_codepage(chn) :
if platform == 'Windows' :
chndec = jn.loads(chn.decode(errors="ignore"))
chncdp = chndec["codepage"]
return 'utf-8' if chncdp is None else chncdp
else :
return 'utf-8'
cdef class imctermite:
@@ -22,7 +32,7 @@ cdef class imctermite:
# get JSON list of channels
def get_channels(self, bool include_data):
chnlst = self.cppimc.get_channels(True,include_data)
chnlstjn = [jn.loads(chn.decode(errors="ignore")) for chn in chnlst]
chnlstjn = [jn.loads(chn.decode(get_codepage(chn),errors="ignore")) for chn in chnlst]
return chnlstjn
# print single channel/all channels

View File

@@ -1 +1 @@
2.0.9
2.0.20

View File

@@ -8,7 +8,7 @@
|CR,1,49,1,3.921568627450980E-2,0.000000000000000E+0,1,1,V;
|ND,1,50,-1,-1,-1,0.000000000000000E+0,1.000000000000000E+1;|CN,1,17,1,0,0,6,kanal2,0,;
|CS,1,8,1,leclass="Code Example" translate="true">
|NO,1,37,1,9, imc-FAMOS,20,Erzeugt:E.Mustermann; |CG,1,5,2,2,2;
|NO,1,37,1,9,imc-FAMOS,20,Erzeugt:E.Mustermann; |CG,1,5,2,2,2;
|CD,1,31,1.000000000000000E-1,1,0,,0,0,0;|NT,1,27,10, 6,1993,19,18,20.0000000;
|CC,1,3,1,1;|CP,1,16,1,4,7,32,0,0,1,0;|Cb,1,40,1,0,1,1,0,16,0,16,1,0.000000000000E+0,0,;
|CR,1,11,0,0,0,1,1,V;|CN,1,20,0,0,0,9,E06_6_121,0,;|CC,1,3,2,1;|CP,1,16,2,4,5,32,0,0,1,0;