Compare commits

..

22 Commits

Author SHA1 Message Date
befda072e5 include origin comment of NO block 2024-06-12 20:53:59 +02:00
0b1cb2f230 reference correct name of data file format in README.md (issue #14) and add link to imc documentation 2024-06-12 20:32:01 +02:00
ce444814c2 reference correct name of data file format in README.md (issue #14) and add link to imc documentation 2024-06-12 20:31:08 +02:00
eec3853af9 by default use utf-8 for metadata including non-ascii characters while missing codepage info 2024-06-12 12:02:26 +02:00
25e114dbea Fix Github Workflow Badge:
* https://github.com/badges/shields/issues/8733
* https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/adding-a-workflow-status-badge
2024-06-12 11:50:35 +02:00
740116c7ba bump version 2.0.20 2024-06-12 11:23:12 +02:00
5899bd646b bump version 2.0.19 2024-06-12 10:39:22 +02:00
1d33a81ad2 bump version 2.0.18 2024-06-12 10:37:54 +02:00
b58efae107 bump version 2.0.17 2024-06-12 10:33:51 +02:00
cc0dd9978f update and fix badges 2024-06-12 10:22:53 +02:00
4834f63a67 update and fix badges 2024-06-12 10:16:56 +02:00
1be78a2c5e Revert "update and fix badges"
This reverts commit f5581557bc.
2024-06-12 10:15:23 +02:00
f5581557bc update and fix badges 2024-06-12 10:14:51 +02:00
8cde88848b update and fix badges 2024-06-12 10:08:33 +02:00
0799513ea2 by default use codepage1252 for metadata including non-ascii characters while missing codepage info 2024-06-12 09:45:30 +02:00
effeee105c win32: replace timegm by _mkgmtime, bump version 2.0.15 2023-08-08 23:46:10 +02:00
ed5b366341 imc_channel.hpp: usage of iconv for unix only, bump version 2.0.15 2023-08-08 23:29:48 +02:00
9a520ddd9c bump version 2.0.14 2023-08-08 00:54:07 +02:00
2c43087d15 bump version 2.0.13 2023-08-08 00:52:27 +02:00
60ac1365a5 * imc_channel.hpp: usage of iconv for unix only
* IMCtermite.pyx: add codepage conversion for windows
* bump VERSION
2023-08-08 00:50:52 +02:00
57027e234e fix workflow pypi-deploy.yml for installing libiconv 2023-08-07 23:03:51 +02:00
887d5db635 add docu and fix github workflow pypi-deploy.yml for installing libiconv 2023-08-07 22:50:00 +02:00
6 changed files with 47 additions and 28 deletions

View File

@@ -1,9 +1,9 @@
name: Build Python Wheels name: CI Build Wheel
on: on:
push: push:
#branches: [master] # branches: [master]
tags: ["v[0-9]+.[0-9]+.[0-9]+"] tags: ["v[0-9]+.[0-9]+.[0-9]+"]
jobs: jobs:
@@ -40,16 +40,6 @@ jobs:
with: with:
name: wheel-config name: wheel-config
path: python/ path: python/
- name: Install iconv for Windows
if: matrix.os == 'windows-2019'
steps:
- name: Get vcpkg repo
run: git clone https://github.com/Microsoft/vcpkg.git
- name: Install vcpkg
run: .\vcpkg\bootstrap-vcpkg.bat
working-directory: vcpkg/
- name: Install required package libiconv
run: vcpkg install libiconv
- name: Build wheels - name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse run: python -m cibuildwheel --output-dir wheelhouse
working-directory: python/ working-directory: python/

View File

@@ -1,15 +1,13 @@
[![Total alerts](https://img.shields.io/lgtm/alerts/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/alerts/)
[![Language grade: C/C++](https://img.shields.io/lgtm/grade/cpp/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/context:cpp)
[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/RecordEvolution/IMCtermite.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/RecordEvolution/IMCtermite/context:python)
[![LICENSE](https://img.shields.io/github/license/RecordEvolution/IMCtermite)](https://img.shields.io/github/license/RecordEvolution/IMCtermite) [![LICENSE](https://img.shields.io/github/license/RecordEvolution/IMCtermite)](https://img.shields.io/github/license/RecordEvolution/IMCtermite)
[![STARS](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)](https://img.shields.io/github/stars/RecordEvolution/IMCtermite) [![STARS](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)](https://img.shields.io/github/stars/RecordEvolution/IMCtermite)
![Build Python Wheels](https://github.com/RecordEvolution/IMCtermite/actions/workflows/pypi-deploy.yml/badge.svg) ![CI Build Wheel](https://github.com/RecordEvolution/IMCtermite/actions/workflows/pypi-deploy.yml/badge.svg?branch=&event=push)
[![PYPI](https://img.shields.io/pypi/v/IMCtermite.svg)](https://pypi.org/project/IMCtermite/)
# IMCtermite # IMCtermite
_IMCtermite_ provides access to the proprietary data format _IMCtermite_ provides access to the proprietary data format
_IMC Bus Format_ with the file extension _.raw_ introduced and developed by _IMC2 Data Format_ with the file extension _.raw_ (or .dat) introduced and developed by
[imc Test & Measurement GmbH](https://www.imc-tm.de/). This data format is [imc Test & Measurement GmbH](https://www.imc-tm.de/). This data format is
employed i.a. by the measurement hardware employed i.a. by the measurement hardware
[imc CRONOSflex](https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/) [imc CRONOSflex](https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/)
@@ -20,7 +18,9 @@ for measurement data control and analysis. Thanks to the integrated Python modul
the extracted measurement data can be stored in any open-source file format the extracted measurement data can be stored in any open-source file format
accessible by Python like i.a. _csv_, _json_ or _parquet_. accessible by Python like i.a. _csv_, _json_ or _parquet_.
On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/), the library can be used both as a command line tool for interactive usage and as a Python module to integrate the _.raw_ format into any ETL workflow. On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/),
the library can be used both as a command line tool for interactive usage and as a
Python module to integrate the _.raw_ format into any ETL workflow.
## Overview ## Overview
@@ -31,12 +31,11 @@ On the [Record Evolution Platform](https://www.record-evolution.de/en/home-en/),
## File format ## File format
[Warning: Take a look at [this issue](https://github.com/RecordEvolution/IMCtermite/issues/14) when reading this section regarding the file format.] A file of the _IMC2 Data Format_ type with extension _.raw_ (or .dat) is a _mixed text/binary
A data file of the _IMC Bus Format_ type with the extension _.raw_ is a _mixed text/binary
file_ featuring a set of markers (keys) that indicate the start of various blocks file_ featuring a set of markers (keys) that indicate the start of various blocks
of data that provide meta information and the actual measurement data. Every single of data that provide meta information and the actual measurement data. Every single
marker is introduced by the character `"|" = 0x 7c` followed by two uppercase letters that characterize the type of marker. Each block is further divided into several marker is introduced by the character `"|" = 0x 7c` followed by two uppercase letters that
characterize the type of marker. Each block is further divided into several
parameters separated by commata `"," = 0x 2c` and terminated by a semicolon parameters separated by commata `"," = 0x 2c` and terminated by a semicolon
`";" = 0x 3b`. For instance, the header - first 600 bytes - of a raw file may `";" = 0x 3b`. For instance, the header - first 600 bytes - of a raw file may
look like this (in UTF-8 encoding): look like this (in UTF-8 encoding):
@@ -225,6 +224,8 @@ can be found in the `python/examples` folder.
- https://www.imc-tm.de/produkte/messtechnik-software/imc-famos/funktionen/im-und-export/ - https://www.imc-tm.de/produkte/messtechnik-software/imc-famos/funktionen/im-und-export/
- https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/ - https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/
- https://www.imc-tm.de/download-center/produkt-downloads/imc-famos/handbuecher
- https://www.imc-tm.de/fileadmin/Public/Downloads/Manuals/imc_FAMOS/imcGemeinsameKomponenten.pdf
- https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html - https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html
- https://github.com/Apollo3zehn/ImcFamosFile - https://github.com/Apollo3zehn/ImcFamosFile
- https://apollo3zehn.github.io/ImcFamosFile/api/ImcFamosFile.FamosFileKeyType.html - https://apollo3zehn.github.io/ImcFamosFile/api/ImcFamosFile.FamosFileKeyType.html
@@ -233,3 +234,6 @@ can be found in the `python/examples` folder.
- https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsrun - https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsrun
- https://github.com/pypa/cibuildwheel/blob/main/examples/github-deploy.yml - https://github.com/pypa/cibuildwheel/blob/main/examples/github-deploy.yml
- https://cibuildwheel.readthedocs.io/en/stable/deliver-to-pypi/ - https://cibuildwheel.readthedocs.io/en/stable/deliver-to-pypi/
- https://www.gnu.org/software/libiconv/
- https://vcpkg.io/en/packages.html
- https://vcpkg.io/en/getting-started

View File

@@ -10,7 +10,11 @@
#include <chrono> #include <chrono>
#include <ctime> #include <ctime>
#include <time.h> #include <time.h>
#if defined(__linux__) || defined(__APPLE__)
#include <iconv.h> #include <iconv.h>
#elif defined(__WIN32__) || defined(_WIN32)
#define timegm _mkgmtime
#endif
//---------------------------------------------------------------------------// //---------------------------------------------------------------------------//
@@ -144,6 +148,7 @@ namespace imc
return sumstr; return sumstr;
} }
#if defined(__linux__) || defined(__APPLE__)
// convert encoding of any descriptions, channel-names, units etc. // convert encoding of any descriptions, channel-names, units etc.
class iconverter class iconverter
{ {
@@ -215,6 +220,14 @@ namespace imc
astring = outstring; astring = outstring;
} }
}; };
#elif defined(__WIN32__) || defined(_WIN32)
class iconverter
{
public:
iconverter(std::string in_enc, std::string out_enc, size_t out_buffer_size = 1024) {}
void convert(std::string &astring) {}
};
#endif
// channel // channel
struct channel struct channel
@@ -519,6 +532,7 @@ namespace imc
<<std::setw(width)<<std::left<<"name:"<<name_<<"\n" <<std::setw(width)<<std::left<<"name:"<<name_<<"\n"
<<std::setw(width)<<std::left<<"comment:"<<comment_<<"\n" <<std::setw(width)<<std::left<<"comment:"<<comment_<<"\n"
<<std::setw(width)<<std::left<<"origin:"<<origin_<<"\n" <<std::setw(width)<<std::left<<"origin:"<<origin_<<"\n"
<<std::setw(width)<<std::left<<"origin-comment:"<<origin_comment_<<"\n"
<<std::setw(width)<<std::left<<"description:"<<text_<<"\n" <<std::setw(width)<<std::left<<"description:"<<text_<<"\n"
<<std::setw(width)<<std::left<<"trigger-time-nt:"<<std::put_time(std::gmtime(&tt),"%FT%T")<<"\n" <<std::setw(width)<<std::left<<"trigger-time-nt:"<<std::put_time(std::gmtime(&tt),"%FT%T")<<"\n"
<<std::setw(width)<<std::left<<"trigger-time:"<<std::put_time(std::gmtime(&att),"%FT%T")<<"\n" <<std::setw(width)<<std::left<<"trigger-time:"<<std::put_time(std::gmtime(&att),"%FT%T")<<"\n"
@@ -557,6 +571,7 @@ namespace imc
<<"\",\"name\":\""<<name_ <<"\",\"name\":\""<<name_
<<"\",\"comment\":\""<<comment_ <<"\",\"comment\":\""<<comment_
<<"\",\"origin\":\""<<origin_ <<"\",\"origin\":\""<<origin_
<<"\",\"origin-comment\":\""<<origin_comment_
<<"\",\"description\":\""<<text_ <<"\",\"description\":\""<<text_
<<"\",\"trigger-time-nt\":\""<<std::put_time(std::gmtime(&tt),"%FT%T") <<"\",\"trigger-time-nt\":\""<<std::put_time(std::gmtime(&tt),"%FT%T")
<<"\",\"trigger-time\":\""<<std::put_time(std::gmtime(&att),"%FT%T") <<"\",\"trigger-time\":\""<<std::put_time(std::gmtime(&att),"%FT%T")

View File

@@ -5,6 +5,16 @@ from IMCtermite cimport cppimctermite
import json as jn import json as jn
import decimal import decimal
import platform
# auxiliary function for codepage conversion
def get_codepage(chn) :
if platform == 'Windows' :
chndec = jn.loads(chn.decode(errors="ignore"))
chncdp = chndec["codepage"]
return 'utf-8' if chncdp is None else chncdp
else :
return 'utf-8'
cdef class imctermite: cdef class imctermite:
@@ -22,7 +32,7 @@ cdef class imctermite:
# get JSON list of channels # get JSON list of channels
def get_channels(self, bool include_data): def get_channels(self, bool include_data):
chnlst = self.cppimc.get_channels(True,include_data) chnlst = self.cppimc.get_channels(True,include_data)
chnlstjn = [jn.loads(chn.decode(errors="ignore")) for chn in chnlst] chnlstjn = [jn.loads(chn.decode(get_codepage(chn),errors="ignore")) for chn in chnlst]
return chnlstjn return chnlstjn
# print single channel/all channels # print single channel/all channels

View File

@@ -1 +1 @@
2.0.9 2.0.20

View File

@@ -8,7 +8,7 @@
|CR,1,49,1,3.921568627450980E-2,0.000000000000000E+0,1,1,V; |CR,1,49,1,3.921568627450980E-2,0.000000000000000E+0,1,1,V;
|ND,1,50,-1,-1,-1,0.000000000000000E+0,1.000000000000000E+1;|CN,1,17,1,0,0,6,kanal2,0,; |ND,1,50,-1,-1,-1,0.000000000000000E+0,1.000000000000000E+1;|CN,1,17,1,0,0,6,kanal2,0,;
|CS,1,8,1,leclass="Code Example" translate="true"> |CS,1,8,1,leclass="Code Example" translate="true">
|NO,1,37,1,9, imc-FAMOS,20,Erzeugt:E.Mustermann; |CG,1,5,2,2,2; |NO,1,37,1,9,imc-FAMOS,20,Erzeugt:E.Mustermann; |CG,1,5,2,2,2;
|CD,1,31,1.000000000000000E-1,1,0,,0,0,0;|NT,1,27,10, 6,1993,19,18,20.0000000; |CD,1,31,1.000000000000000E-1,1,0,,0,0,0;|NT,1,27,10, 6,1993,19,18,20.0000000;
|CC,1,3,1,1;|CP,1,16,1,4,7,32,0,0,1,0;|Cb,1,40,1,0,1,1,0,16,0,16,1,0.000000000000E+0,0,; |CC,1,3,1,1;|CP,1,16,1,4,7,32,0,0,1,0;|Cb,1,40,1,0,1,1,0,16,0,16,1,0.000000000000E+0,0,;
|CR,1,11,0,0,0,1,1,V;|CN,1,20,0,0,0,9,E06_6_121,0,;|CC,1,3,2,1;|CP,1,16,2,4,5,32,0,0,1,0; |CR,1,11,0,0,0,1,1,V;|CN,1,20,0,0,0,9,E06_6_121,0,;|CC,1,3,2,1;|CP,1,16,2,4,5,32,0,0,1,0;