remove redundant docu
This commit is contained in:
parent
2697e0b578
commit
26ff196676
Binary file not shown.
Binary file not shown.
@ -1,96 +0,0 @@
|
|||||||
|
|
||||||
## Fileformat
|
|
||||||
|
|
||||||
A data file of _IMC Bus Format_ type with extension _.raw_ is a _mixed text/binary
|
|
||||||
file_ featuring a set of markers (keys) that indicate the start of various blocks
|
|
||||||
of data providing meta information and the actual measurement data. Every single
|
|
||||||
marker is introduced by character `"|" = 0x 7c` followed by two uppercase letters,
|
|
||||||
which characterize the type of marker. Each block is further divided into several
|
|
||||||
parameters separated by commata `"," = 0x 2c` and terminated by a semicolon
|
|
||||||
`";" = 0x 3b`. For instance, the header - first 600 bytes - of a raw file may
|
|
||||||
look like this (in UTF-8 encoding)
|
|
||||||
|
|
||||||
```
|
|
||||||
|CF,2,1,1;|CK,1,3,1,1;
|
|
||||||
|NO,1,86,0,78,imc STUDIO 5.0 R10 (04.08.2017)@imc DEVICES 2.9R7 (25.7.2017)@imcDev__15190567,0,;
|
|
||||||
|CG,1,5,1,1,1; |CD,2, 63, 5.0000000000000001E-03,1,1,s,0,0,0, 0.0000000000000000E+00,1;
|
|
||||||
|NT,1,16,1,1,1980,0,0,0.0; |CC,1,3,1,1;|CP,1,16,1,4,7,32,0,0,1,0;
|
|
||||||
|CR,1,60,0, 1.0000000000000000E+00, 0.0000000000000000E+00,1,4,mbar;|CN,1,27,0,0,0,15,pressure_Vacuum,0,;
|
|
||||||
|Cb,1, 117,1,0, 1, 1, 0, 9608, 0, 9608,1, 2.0440300000000000E+03, 1.2416717060000000E+09,;
|
|
||||||
|CS,1, 9619, 1,<2C>oD <09>nD6<44>nD)<29>nD<6E>
|
|
||||||
```
|
|
||||||
|
|
||||||
where line breaks where introduced for readability. Most of the markers introduce
|
|
||||||
blocks of text, while only the last one identified by `|CS` contains binary data.
|
|
||||||
The format supports the storage of _multiple data sets (channels)_ in a single
|
|
||||||
file. The channels may be ordered in _multiplex_ mode (ordering w.r.t. time) or
|
|
||||||
_block_ mode (ordering w.r.t. to channels).
|
|
||||||
|
|
||||||
### Markers
|
|
||||||
|
|
||||||
The markers (keys) are introduced by `"|" = 0x 7c` followed by two uppercase
|
|
||||||
letters. There are _two types_ of markers distinguished by the first letter:
|
|
||||||
|
|
||||||
1. _critical_ markers: introduced by `|C` featuring uppercase `C`
|
|
||||||
1. _noncritical_ markers: introduced by `|N` featuring uppercase `N`
|
|
||||||
|
|
||||||
The second letter represents further details of the specific key. Note, that
|
|
||||||
while the _noncritical_ keys are optional, any _.raw_ file _cannot be_ correctly
|
|
||||||
decoded if any of the _critical_ markers are misinterpreted, invalid or damaged.
|
|
||||||
The second uppercase letter is followed by the first comma and the _version_
|
|
||||||
of the key starting from 1. After the next comma, an _(long) integer_ (in text
|
|
||||||
representation) specifies the length of the entire block, i.e. the number of
|
|
||||||
bytes between the following comma and the block-terminating semicolon. The further
|
|
||||||
structure of a block is not defined and may feature different numbers of additional
|
|
||||||
parameters. The format allows for any number of carriage returns (`CR = 0x0d`)
|
|
||||||
and line feeds (`LF = 0x 0a`) between keys, i.e. the block terminating semicolon
|
|
||||||
and the vertical bar (pipe) of the next key. The following _critical markers_
|
|
||||||
are defined
|
|
||||||
|
|
||||||
| marker | structure (example) | description |
|
|
||||||
|--------|-----------------------------------------------------------------------------------------------|---------------------------------------------------------|
|
|
||||||
| CF | CF,2,1,1; | format version and processor |
|
|
||||||
| CK | CK,1,3,1,<closed>; | start of group of keys |
|
|
||||||
| | | length is always 3 |
|
|
||||||
| | | <closed> must be 0 or 1 depending correct closure of the measurment series |
|
|
||||||
| CB | CB,1,<length>,<groupindex>,<namelength>,<name>,<commentlength>,<comment>; | defines a group of cannels |
|
|
||||||
| CT | CT,1,<length>,<groupindex>,<namelength>,<name>,<textlength>,<text>,<commentlength>,<comment>; | text definition |
|
|
||||||
| CG | CG,1,<length>,<numberofcompontent>,<fieldtype>,<dimension>; | introduces group of components corresponding to CC keys |
|
|
||||||
| | | field type is one of: 1 = real number |
|
|
||||||
| | | 2 = XY |
|
|
||||||
| | | 3 = 3XY |
|
|
||||||
| | | 4 = complex with Re + Im |
|
|
||||||
| | | 5 = complex with Abs + Phase |
|
|
||||||
| | | 6 = complex: Abs in dB + Phase |
|
|
||||||
| | | dimension: 1 for fieldtype 1, 2 for fieldtype 2-6 |
|
|
||||||
| CD | CD,1,<length>,<dx>,<calibration>,<unitlength>,<unit>,0,0,0; | dx = distance between two samples |
|
|
||||||
| | | calibration = 1 : calibrated time-basis |
|
|
||||||
| | | = 0 : noncalibrated time-basis |
|
|
||||||
| | | unit: unit of this axis |
|
|
||||||
| CZ | CZ,1,<length>,<dz>,<dzcalibration>,<z0>,<z0calibration>,<unitlength>,<unit>,<segmentlength>; | scaling of z-axis for segments |
|
|
||||||
| CC | CC,1,<length>,<componentindex>,<analogdigital>; | start of single component: |
|
|
||||||
| | | - componentindex = 1 real, real part, abs |
|
|
||||||
| | | = 2 complex, imaginar part, phase |
|
|
||||||
| | | - analogdigital = 1 analog = 2 digital |
|
|
||||||
| CP | CP,1,<lenght>,<buffref>,<bytes>,<datatype>,<signbits>,<mask>,<offset>,<numsamples>,<bytedist>;| buffref = int > 0 corresp.to buffer descript. in Cb key |
|
|
||||||
| | | bytes: number of bytes for singel value |
|
|
||||||
| | | datatype: 1 = unsigned byte, 2 = signed byte, |
|
|
||||||
| | | 3 = unsigned short, 4 = signed short |
|
|
||||||
| | | 5 = unsigned long, 6 = signed long |
|
|
||||||
| | | 7 = float, 8 = double |
|
|
||||||
| | | 9 = imc Devices transitional recording |
|
|
||||||
| | | 10 = timestamp ascii, 11 = 2byte word digital |
|
|
||||||
| | | 13 = 6byte unsigned long |
|
|
||||||
| | | signbits: number of significant bits |
|
|
||||||
| | | mask: default = 0 |
|
|
||||||
| | | offset: offset of first sample in data block |
|
|
||||||
| | | numsamples: number of subsequent samples in channel |
|
|
||||||
| | | bytedist: number of bytes between last sample of channel|
|
|
||||||
| | | to start of next channel |
|
|
||||||
| Cb | Cb,1,<lenght>,<numbuff>,<bytesinuserinfo>,<buffref>,<indexsamplekey>,<offsetbuffer>, | |
|
|
||||||
| | <bufferlength>,<offset>,<fillbytes>,0,X0,<addtime>,<userinfo>,<newevent>,[buffref,...]; | buffer description |
|
|
||||||
| CR | | |
|
|
||||||
| CN | | |
|
|
||||||
| CS | |
|
|
||||||
| CI | | |
|
|
||||||
| Ca | | |
|
|
||||||
@ -1,114 +0,0 @@
|
|||||||
|
|
||||||
# Deprecated!!
|
|
||||||
|
|
||||||
The following markers are defined:
|
|
||||||
|
|
||||||
1. CF (0x 43 46)
|
|
||||||
1. CK (0x 43 4b)
|
|
||||||
1. NO (0x 4e 4f)
|
|
||||||
1. CG (0x 43 47)
|
|
||||||
1. CD (0x 43 44)
|
|
||||||
1. NT (0x 4e 54)
|
|
||||||
1. CC (0x 43 43)
|
|
||||||
1. CP (0x 43 50)
|
|
||||||
1. CR (0x 43 52)
|
|
||||||
1. CN (0x 43 4e)
|
|
||||||
1. Cb (0x 43 62)
|
|
||||||
1. CS (0x 43 53)
|
|
||||||
|
|
||||||
Each of these markers are followed by multiple commata (0x 2c) separated parameters
|
|
||||||
and are terminated by a semicolon `;` = 0x 3b, except for the sequence following
|
|
||||||
the data marker CS, that may have any number of 0x3b occurencies, while still
|
|
||||||
terminated by a semicolon at the very end of the file (since CS is the last marker
|
|
||||||
section in the file). The markers have the following meaning:
|
|
||||||
|
|
||||||
- *CF* (3 parameters)
|
|
||||||
`|CF,2,1,1;`
|
|
||||||
specifies file format, key length and processor
|
|
||||||
- *CK* (4 parameters)
|
|
||||||
`|CK,1,3,1,1;`
|
|
||||||
start of group of keys
|
|
||||||
- *NO* (6 parameters)
|
|
||||||
`|NO,1,85,0,77,imc STUDIO 5.0 R3 (10.09.2015)@imc DEVICES 2.8R7 (26.8.2015)@imcDev__15190567,0,;`
|
|
||||||
origin of the file, provides some info about the software package/device
|
|
||||||
and its version
|
|
||||||
- *CB* (6 parameters)
|
|
||||||
group definition
|
|
||||||
- *CT* (8 parameters)
|
|
||||||
text definition
|
|
||||||
- *CG* (5 parameters)
|
|
||||||
`|CG,1,5,1,1,1;`
|
|
||||||
definition of a data field
|
|
||||||
|CG,1,KeyLang,AnzahlKomponenten,Feldtyp,Dimension;
|
|
||||||
- *CD* (mostly 11 parameters)
|
|
||||||
since we're dealing with measured entities from the lab this markers contains
|
|
||||||
info about the measurement frequency, i.e. sample rate. For instance
|
|
||||||
`|CD,2, 63, 5.0000000000000001E-03,1,1,s,0,0,0, 0.0000000000000000E+00,1;`
|
|
||||||
indicates a measured entity every 0.005 seconds, i.e. a sample rate = 200Hz
|
|
||||||
- *NT* (7 parameters)
|
|
||||||
`|NT,1,16,1,1,1980,0,0,0.0;`
|
|
||||||
|NT,1,KeyLang,Tag,Monat,Jahr,Stunden,Minuten,Sekunden;
|
|
||||||
triggerzeit
|
|
||||||
- *CC* (mostly 4 parameters)
|
|
||||||
`|CC,1,3,1,1;`
|
|
||||||
Start einer Komponente (component)
|
|
||||||
- *CP* (9 parameters)
|
|
||||||
`|CP,1,16,1,4,7,32,0,0,1,0;`
|
|
||||||
Pack-Information zu dieser Komponente
|
|
||||||
CP,1,KeyLang,BufferReferenz,Bytes,Zahlenformat,SignBits,Maske,Offset,DirekteFolgeAnzahl,AbstandBytes;
|
|
||||||
Bytes = 1...8
|
|
||||||
Zahlenformat : 1 = unsigned byte
|
|
||||||
2 = signed byte
|
|
||||||
3 = unsigned short
|
|
||||||
4 = signed short
|
|
||||||
5 = unsigned long
|
|
||||||
6 = signed long
|
|
||||||
7 = float
|
|
||||||
8 = double
|
|
||||||
9 = imc Devices
|
|
||||||
10 = timestamp ascii
|
|
||||||
11 =
|
|
||||||
12 =
|
|
||||||
13 =
|
|
||||||
|
|
||||||
- *CR* (7 parameters)
|
|
||||||
Wertebereich der Komponente, nur bei analogen, nicht bei digitalen Daten.
|
|
||||||
|CR,1,KeyLang,Transformieren,Faktor,Offset,Kalibriert,EinheitLang, Einheit;
|
|
||||||
provides the _physical unit_ of the measured entity, maybe shows the
|
|
||||||
minimum and maximum value during the measurment, e.g.
|
|
||||||
`|CR,1,60,0, 1.0000000000000000E+00, 0.0000000000000000E+00,1,4,mbar;`
|
|
||||||
Transformieren : 0 = nein
|
|
||||||
1 = ja, mit faktor und offset transformieren (für ganzzahlige Rohdaten)
|
|
||||||
Faktor,Offset: physikalischer Wert = Faktor * Rohdatenwerten + Offset
|
|
||||||
- *CN* (mostly 9 parameters)
|
|
||||||
gives the _name_ of the measured entity
|
|
||||||
|CN,1,KeyLang,IndexGruppe,0,IndexBit,NameLang,Name,KommLang,Kommentar;
|
|
||||||
`|CN,1,27,0,0,0,15,pressure_Vacuum,0,;`
|
|
||||||
- *Cb* (mostly 14 paramters) (optional?)
|
|
||||||
this one probably gives the minimum/maximum measured values!!
|
|
||||||
`|Cb,1,117,1,0,1,1,0,341288,0,341288,1,0.0000000000000000E+00,1.1781711390000000E+09,;`
|
|
||||||
- *CS* (mostly 4 parameters)
|
|
||||||
this markers announces the actual measurement data in binary format,
|
|
||||||
provide the number of values and the actual data,
|
|
||||||
e.g. `|CS,1, 341299, 1, ...data... ;`
|
|
||||||
|
|
||||||
### Open Issues and question?
|
|
||||||
|
|
||||||
- which parameter indicate(s) little vs. big endian?
|
|
||||||
|
|
||||||
## .parquet-file writer
|
|
||||||
|
|
||||||
The extracted and converted data originating from the *.raw file format may be efficiently grouped and
|
|
||||||
written as .parquet files
|
|
||||||
[parquet file writer example](https://github.com/apache/parquet-cpp/blob/master/examples/low-level-api/reader-writer.cc)
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- https://ch.mathworks.com/matlabcentral/fileexchange/30187-sequnce-to-read-famos-data-into-matlab-workspace
|
|
||||||
- https://community.ptc.com/t5/PTC-Mathcad/FAMOS-IMC-raw-data-in-MathCAD/td-p/130378
|
|
||||||
- http://marmatek.com/wp-content/uploads/2014/04/imc_STUDIO_Manual.pdf
|
|
||||||
|
|
||||||
### Parquet
|
|
||||||
|
|
||||||
- https://github.com/apache/parquet-cpp
|
|
||||||
- https://github.com/apache/parquet-cpp/tree/master/examples
|
|
||||||
Binary file not shown.
@ -1,217 +0,0 @@
|
|||||||
function [dataOut]=FAMOSimport(filename)
|
|
||||||
% Usage: data=FAMOSimport(filename);
|
|
||||||
%
|
|
||||||
% FAMOSimport() opens (MARC generated) FAMOS files and imports all signals.
|
|
||||||
%
|
|
||||||
% *************************************************************************
|
|
||||||
%
|
|
||||||
%
|
|
||||||
|
|
||||||
|
|
||||||
% Preset output to empty;
|
|
||||||
dataOut=[];
|
|
||||||
|
|
||||||
|
|
||||||
%% Check for valid input file
|
|
||||||
if exist('filename','var')~=1 ...
|
|
||||||
|| isempty(filename)
|
|
||||||
[filename, pathname] = uigetfile( ...
|
|
||||||
{'*.dat','FAMOS measurement files'; ...
|
|
||||||
'*.*','All files'},'Select FAMOS measurement file ...');
|
|
||||||
if isequal(filename,0)
|
|
||||||
disp('FAMOS-measurement file import cancelled.');
|
|
||||||
return;
|
|
||||||
end
|
|
||||||
filename=fullfile(pathname, filename);
|
|
||||||
clear pathname;
|
|
||||||
end
|
|
||||||
if exist(filename,'file')~=2
|
|
||||||
disp('Given file could not be found. Aborting import.');
|
|
||||||
return;
|
|
||||||
end
|
|
||||||
|
|
||||||
%% Load input file
|
|
||||||
fid=fopen(filename,'r','l');
|
|
||||||
data=fread(fid,inf,'uint8=>char','l')';
|
|
||||||
fclose(fid);
|
|
||||||
clear fid
|
|
||||||
|
|
||||||
|
|
||||||
%% Parse header information
|
|
||||||
dataOut.FileName=filename;
|
|
||||||
dataOut.TYPE='AFT MARC (FAMOS)';
|
|
||||||
header=strfind(data(1:200),[char(13) char(10) '|NO,']);
|
|
||||||
subIdx=strfind(data(header(1):header(1)+50),',');
|
|
||||||
dataOut.Device=strtrim(data(header(1)+subIdx(5):header(1)+subIdx(6)-2));
|
|
||||||
|
|
||||||
%% Parse measurement entries information
|
|
||||||
units=strfind(data,[char(13) char(10) '|CR,'])';
|
|
||||||
dataOut.Unit=cell(size(units));
|
|
||||||
dataOut.Factor=zeros(size(dataOut.Unit),'double');
|
|
||||||
dataOut.Offset=zeros(size(dataOut.Unit),'double');
|
|
||||||
for i=1:length(units)
|
|
||||||
subIdx=sort([strfind(data(units(i):units(i)+255),',') ...
|
|
||||||
strfind(data(units(i):units(i)+255),';')]);
|
|
||||||
dataOut.Factor(i)=str2double( ...
|
|
||||||
data(units(i)+subIdx(4):units(i)+subIdx(5)-2));
|
|
||||||
dataOut.Offset(i)=str2double( ...
|
|
||||||
data(units(i)+subIdx(5):units(i)+subIdx(6)-2));
|
|
||||||
dataOut.Unit{i}=data(units(i)+subIdx(8):units(i)+subIdx(9)-2);
|
|
||||||
|
|
||||||
end
|
|
||||||
clear units subIdx;
|
|
||||||
|
|
||||||
%Extract measurement variables names and corresponding time column
|
|
||||||
varName=strfind(data,[char(13) char(10) '|CN,'])';
|
|
||||||
dataOut.TimeIndex=zeros(size(varName),'uint32');
|
|
||||||
dataOut.Label=cellstr(char(zeros(size(dataOut.TimeIndex))));
|
|
||||||
dataOut.Data=zeros(size(dataOut.TimeIndex),'double');
|
|
||||||
for i=1:length(varName)
|
|
||||||
subIdx=strfind(data(varName(i):varName(i)+255),',');
|
|
||||||
dataOut.Label{i}=data(varName(i)+subIdx(7):varName(i)+subIdx(8)-2);
|
|
||||||
end
|
|
||||||
for i=1:length(varName)
|
|
||||||
subIdx=sort([strfind(data(varName(i):varName(i)+255),',') ...
|
|
||||||
strfind(data(varName(i):varName(i)+255),';')]);
|
|
||||||
TimeVarName=data(varName(i)+subIdx(9):varName(i)+subIdx(10)-2);
|
|
||||||
if i==1 || ~strcmp(dataOut.TimeIndex(i-1),TimeVarName)
|
|
||||||
idx=strmatch(TimeVarName,dataOut.Label,'exact');
|
|
||||||
if ~isempty(idx)
|
|
||||||
dataOut.TimeIndex(i)=idx(1);
|
|
||||||
else
|
|
||||||
warning('FAMOSconnect:invalidTimeLabel', ...
|
|
||||||
['Signal ''%s'' (%d) refers to non-existing ' ...
|
|
||||||
'time signal label ''%s''.'], ...
|
|
||||||
dataOut.Label{i},i,TimeVarName);
|
|
||||||
end
|
|
||||||
else
|
|
||||||
dataOut.TimeIndex(i)=dataOut.TimeIndex(i-1);
|
|
||||||
end
|
|
||||||
end
|
|
||||||
clear varName TimeVarName subIdx;
|
|
||||||
|
|
||||||
%Extract measurement value data type and bitlength
|
|
||||||
dataType=strfind(data,[char(13) char(10) '|CP,1,'])';
|
|
||||||
dataOut.DataType=cell(size(dataType));
|
|
||||||
dataOut.DataBits=zeros(size(dataOut.DataType),'uint8');
|
|
||||||
for i=1:length(dataType)
|
|
||||||
subIdx=strfind(data(dataType(i):dataType(i)+50),',');
|
|
||||||
switch (data(dataType(i)+subIdx(5):dataType(i)+subIdx(6)-2))
|
|
||||||
case '1' % uint8
|
|
||||||
dataOut.DataType{i}='uint8';
|
|
||||||
case '2' % int8
|
|
||||||
dataOut.DataType{i}='int8';
|
|
||||||
case {'3','9','11'} % uint16
|
|
||||||
dataOut.DataType{i}='uint16';
|
|
||||||
case '4' % int16
|
|
||||||
dataOut.DataType{i}='int16';
|
|
||||||
case '5' % uint32
|
|
||||||
dataOut.DataType{i}='uint32';
|
|
||||||
case '6' % int32
|
|
||||||
dataOut.DataType{i}='int32';
|
|
||||||
case '7' % float
|
|
||||||
dataOut.DataType{i}='single';
|
|
||||||
case {'8','10','13'} % double
|
|
||||||
dataOut.DataType{i}='double';
|
|
||||||
otherwise
|
|
||||||
dataOut.DataType{i}='UNKNOWN';
|
|
||||||
end
|
|
||||||
dataOut.DataBits(i)=str2double( ...
|
|
||||||
data(dataType(i)+subIdx(6):dataType(i)+subIdx(7)-2));
|
|
||||||
end
|
|
||||||
clear dataType subIdx;
|
|
||||||
|
|
||||||
%Extract measurement value data block and item number
|
|
||||||
dataInfo=strfind(data,[char(13) char(10) '|Cb,1,'])';
|
|
||||||
dataOut.DataBlock=zeros(size(dataInfo),'uint16');
|
|
||||||
dataOut.DataItem=zeros(size(dataOut.DataBlock),'uint16');
|
|
||||||
for i=1:length(dataInfo)
|
|
||||||
subIdx=strfind(data(dataInfo(i):dataInfo(i)+50),',');
|
|
||||||
dataOut.DataItem(i)=str2double( ...
|
|
||||||
data(dataInfo(i)+subIdx(5):dataInfo(i)+subIdx(6)-2));
|
|
||||||
dataOut.DataBlock(i)=str2double( ...
|
|
||||||
data(dataInfo(i)+subIdx(6):dataInfo(i)+subIdx(7)-2));
|
|
||||||
end
|
|
||||||
clear dataInfo subIdx;
|
|
||||||
|
|
||||||
|
|
||||||
%Extract measurement value binary data length and offset
|
|
||||||
dataBlock=strfind(data,[char(13) char(10) '|CS,'])';
|
|
||||||
dataOut.DataBlocks=length(dataBlock);
|
|
||||||
dataOut.DataBlocksLength=zeros(size(dataBlock),'uint32');
|
|
||||||
dataOut.DataBlocksItemLength=zeros(size(dataBlock),'uint32');
|
|
||||||
dataOut.DataBlocksOffset=zeros(size(dataBlock),'uint32');
|
|
||||||
for i=1:length(dataBlock)
|
|
||||||
subIdx=strfind(data(dataBlock(i):dataBlock(i)+50),',');
|
|
||||||
|
|
||||||
dataOut.DataBlocksOffset(i)=dataBlock(i)+subIdx(4)-1;
|
|
||||||
dataOut.DataBlocksLength(i)=str2double( ...
|
|
||||||
data(dataBlock(i)+subIdx(2):dataBlock(i)+subIdx(3)-2)) ...
|
|
||||||
-(subIdx(4)-subIdx(3)); %Fix offset
|
|
||||||
dataOut.DataBlocksItemLength(i)=dataOut.DataBlocksLength(i) ...
|
|
||||||
/(sum(dataOut.DataBits(dataOut.DataBlock==i)/8));
|
|
||||||
end
|
|
||||||
clear dataBlock subIdx;
|
|
||||||
|
|
||||||
|
|
||||||
%% Sort entries - note: DataItem-value continues over blocks.
|
|
||||||
[~, dataOrder]=sort(dataOut.DataItem);
|
|
||||||
dataOutField=fieldnames(dataOut);
|
|
||||||
for i=1:length(dataOutField)
|
|
||||||
if length(dataOut.(dataOutField{i}))==length(dataOut.DataItem) ...
|
|
||||||
&& ~strcmp(dataOutField{i},'DataBlocksLength')
|
|
||||||
dataOut.(dataOutField{i})=dataOut.(dataOutField{i})(dataOrder);
|
|
||||||
end
|
|
||||||
end
|
|
||||||
clear dataOrder dataOutField;
|
|
||||||
|
|
||||||
|
|
||||||
%% Extract measurement data, format: shots-aligned (not variables aligned)
|
|
||||||
data=cast(data,'uint8');
|
|
||||||
dataOut.Data=cell(length(dataOut.DataItem),1);
|
|
||||||
dataBlockId=1;
|
|
||||||
dataOffset=dataOut.DataBlocksOffset(dataBlockId);
|
|
||||||
dataVarIdx1=uint32(1: ...
|
|
||||||
dataOut.DataBlocksLength(dataBlockId) ...
|
|
||||||
/dataOut.DataBlocksItemLength(dataBlockId): ...
|
|
||||||
dataOut.DataBlocksLength(dataBlockId));
|
|
||||||
dataVarIdx2=reshape([dataVarIdx1; dataVarIdx1+1],1,[]);
|
|
||||||
dataVarIdx4=reshape([dataVarIdx1; dataVarIdx1+1; ...
|
|
||||||
dataVarIdx1+2; dataVarIdx1+3],1,[]);
|
|
||||||
for i=1:length(dataOut.Label)
|
|
||||||
if dataOut.DataBlock(i)>dataBlockId
|
|
||||||
dataBlockId=dataOut.DataBlock(i);
|
|
||||||
dataOffset=dataOut.DataBlocksOffset(dataBlockId);
|
|
||||||
dataVarIdx1=uint32(1: ...
|
|
||||||
dataOut.DataBlocksLength(dataBlockId) ...
|
|
||||||
/dataOut.DataBlocksItemLength(dataBlockId): ...
|
|
||||||
dataOut.DataBlocksLength(dataBlockId));
|
|
||||||
dataVarIdx2=reshape([dataVarIdx1; dataVarIdx1+1],1,[]);
|
|
||||||
dataVarIdx4=reshape([dataVarIdx1; dataVarIdx1+1; ...
|
|
||||||
dataVarIdx1+2; dataVarIdx1+3],1,[]);
|
|
||||||
end
|
|
||||||
switch dataOut.DataBits(i)
|
|
||||||
case 8
|
|
||||||
dataVal=cast(typecast(data(dataVarIdx1+dataOffset),...
|
|
||||||
dataOut.DataType{i}),'double');
|
|
||||||
dataOffset=dataOffset+1;
|
|
||||||
case 16
|
|
||||||
dataVal=cast(typecast(data(dataVarIdx2+dataOffset),...
|
|
||||||
dataOut.DataType{i}),'double');
|
|
||||||
dataOffset=dataOffset+2;
|
|
||||||
case 32
|
|
||||||
dataVal=cast(typecast(data(dataVarIdx4+dataOffset),...
|
|
||||||
dataOut.DataType{i}),'double');
|
|
||||||
dataOffset=dataOffset+4;
|
|
||||||
otherwise
|
|
||||||
fprintf(2,['Unsupported data width in item %d:' ...
|
|
||||||
'%d Bits - Skipping.\n'], ...
|
|
||||||
dataOut.DataItem(i),dataOut.DataBits(i));
|
|
||||||
dataOffset=dataOut.DataBits(i)/8;
|
|
||||||
continue;
|
|
||||||
end
|
|
||||||
dataVal=dataVal*dataOut.Factor(i)+dataOut.Offset(i);
|
|
||||||
dataOut.Data{i}=dataVal';
|
|
||||||
end
|
|
||||||
clear dataOffset dataBlockId dataVarIdx1 dataVarIdx2 dataVarIdx4 dataVal;
|
|
||||||
clear i data;
|
|
||||||
@ -1,222 +0,0 @@
|
|||||||
function Channel=importfamos(FullRawData)
|
|
||||||
%__________________________________________________________________________
|
|
||||||
% The sequnce importfamos.m was produced to convert imc raw data(*.raw;
|
|
||||||
% *.dat) to matlab data. Here, struct is used to manage the channel
|
|
||||||
% information.
|
|
||||||
%
|
|
||||||
% For more information of FAMOS file format, please see the
|
|
||||||
% manufacturer's website: http://www.imc-berlin.de
|
|
||||||
%
|
|
||||||
% Corresponding to Data Structure in Matlab, the channels are stored
|
|
||||||
% as struct struct: Channel + name (channel name)
|
|
||||||
% + comment
|
|
||||||
% + data (channel Value)
|
|
||||||
% + length
|
|
||||||
% + yUnit
|
|
||||||
% + t0
|
|
||||||
% + dt(sampling period)
|
|
||||||
% + xUnit
|
|
||||||
% Version history:
|
|
||||||
% Version 1.0 (2011.1.19); Current version only can deal with analog rawdata
|
|
||||||
% from imc devices. The digital and group function is ongoning and will be
|
|
||||||
% released in version 2.
|
|
||||||
% Version 1.1 (2013.12.31): In order to solve non-manually save data,
|
|
||||||
% regular pattern is introduced into sloving data structure without CR and
|
|
||||||
% LF.
|
|
||||||
% (only support data(Channel) from imc devices)
|
|
||||||
%
|
|
||||||
%%-------------------------------------------------------------------------
|
|
||||||
% Author: Liang
|
|
||||||
% Danke.Liang@gmail.com
|
|
||||||
% Started on Dec.14, 2010
|
|
||||||
%__________________________________________________________________________
|
|
||||||
if nargin == 0;
|
|
||||||
[RawData,FamosPath] = uigetfile({'*.raw';'*.dat'},'Select Famos Raw data');
|
|
||||||
FullRawData=strcat(FamosPath,RawData);
|
|
||||||
end
|
|
||||||
fid = fopen(FullRawData,'r');
|
|
||||||
if fid==-1,
|
|
||||||
disp('failed to read rawdata')
|
|
||||||
return
|
|
||||||
end
|
|
||||||
disp(['Info: Read data @' datestr(now) ' from' ])
|
|
||||||
disp([ ' ' FullRawData '...'])
|
|
||||||
Textall=fscanf(fid,'%c');
|
|
||||||
CSstring='\|CS,\d,\D*\d+,\D*\d+,';
|
|
||||||
CSend=regexp(Textall,CSstring,'end');
|
|
||||||
InfoSegment=Textall(1,1:CSend);
|
|
||||||
Cbstr='\|Cb.*?;';
|
|
||||||
CGstr='\|CG,.*?;';
|
|
||||||
CDstr='\|CD,.*?;';
|
|
||||||
NTstr='\|NT,.*?;';
|
|
||||||
CPstr='\|CP,.*?;';
|
|
||||||
CRstr='\|CR,.*?;';
|
|
||||||
CNstr='\|CN,.*?;';
|
|
||||||
ArrCG=char(regexp(InfoSegment,CGstr,'match'));
|
|
||||||
ChannelNum=size(ArrCG,1);
|
|
||||||
ArrCN=char(regexp(InfoSegment,CNstr,'match'));
|
|
||||||
ArrCD=char(regexp(InfoSegment,CDstr,'match'));
|
|
||||||
ArrNT=char(regexp(InfoSegment,NTstr,'match'));
|
|
||||||
ArrCP=char(regexp(InfoSegment,CPstr,'match'));
|
|
||||||
ArrCb=char(regexp(InfoSegment,Cbstr,'match'));
|
|
||||||
ArrCR=char(regexp(InfoSegment,CRstr,'match'));
|
|
||||||
%% CN
|
|
||||||
ChannelName=cell(ChannelNum,1);
|
|
||||||
ChannelComment=cell(ChannelNum,1);
|
|
||||||
%% CD,NT
|
|
||||||
CDsample=zeros(ChannelNum,1);
|
|
||||||
CDUnit=cell(ChannelNum,1);
|
|
||||||
TriggerTime=cell(ChannelNum,1);
|
|
||||||
%% CP
|
|
||||||
KeyBufferRef=zeros(ChannelNum,1);
|
|
||||||
KeyBytes=zeros(ChannelNum,1);
|
|
||||||
KeyNumberFormat=cell(ChannelNum,1);
|
|
||||||
KeySignBits=zeros(ChannelNum,1);
|
|
||||||
%% Cb
|
|
||||||
KeyBufferRefIndex=zeros(ChannelNum,1);
|
|
||||||
KeyBufferRefCb=zeros(ChannelNum,1);
|
|
||||||
KeyOffsetBufferInSamplesKey=zeros(ChannelNum,1);
|
|
||||||
KeyBufferFilledBytes=zeros(ChannelNum,1);
|
|
||||||
%% CR
|
|
||||||
KeyTransformation=zeros(ChannelNum,1);
|
|
||||||
KeyCRfactor=zeros(ChannelNum,1);
|
|
||||||
KeyCRoffset=zeros(ChannelNum,1);
|
|
||||||
KeyUnit=cell(ChannelNum,1);
|
|
||||||
%% Define Return object
|
|
||||||
Channel=struct('name','','comment','','data',[],'length',0,'yUnit','','t0','','dt','','xUnit','');
|
|
||||||
BinaryStart=CSend;
|
|
||||||
ChannelID=1;
|
|
||||||
while ChannelID <= ChannelNum
|
|
||||||
temptext=char(ArrCD(ChannelID,:));
|
|
||||||
[CDsample(ChannelID,1), CDUnit{ChannelID,1}]=ProcessCD(temptext);
|
|
||||||
[ChannelName{ChannelID,1},ChannelComment{ChannelID,1}]=ProcessCN(ArrCN(ChannelID,:));
|
|
||||||
Channel(ChannelID).name=ChannelName{ChannelID,1};
|
|
||||||
disp(strcat('Channel_',num2str(ChannelID),':',Channel(ChannelID).name))
|
|
||||||
Channel(ChannelID).comment=ChannelComment{ChannelID,1};
|
|
||||||
Channel(ChannelID).dt=strcat(num2str(CDsample(ChannelID,1)),CDUnit{ChannelID,1});
|
|
||||||
Channel(ChannelID).xUnit=CDUnit{ChannelID,1};
|
|
||||||
TriggerTime{ChannelID,1}=ProcessNT(ArrNT(ChannelID,:));
|
|
||||||
Channel(ChannelID).t0=TriggerTime{ChannelID,1};
|
|
||||||
[KeyBufferRef(ChannelID,1),KeyBytes(ChannelID,1),KeyNumberFormat{ChannelID,1},KeySignBits(ChannelID,1)]=ProcessCP(ArrCP(ChannelID,:));
|
|
||||||
[KeyBufferRefIndex(ChannelID,1),KeyBufferRefCb(ChannelID,1),KeyOffsetBufferInSamplesKey(ChannelID,1),KeyBufferFilledBytes(ChannelID,1)]=ProcessCblittle(ArrCb(ChannelID,:));
|
|
||||||
[KeyTransformation(ChannelID,1),KeyCRfactor(ChannelID,1),KeyCRoffset(ChannelID,1),KeyUnit{ChannelID,1}]=ProcessCR(ArrCR(ChannelID,:));
|
|
||||||
Channel(ChannelID).yUnit=KeyUnit{ChannelID,1};
|
|
||||||
BinaryRead= BinaryStart+KeyOffsetBufferInSamplesKey(ChannelID,1);
|
|
||||||
ChannelLength=KeyBufferFilledBytes(ChannelID,1)*8/KeySignBits(ChannelID,1);
|
|
||||||
Channel(ChannelID).data=ReadChannel(fid,BinaryRead,ChannelLength,KeyNumberFormat{ChannelID,1},KeyCRfactor(ChannelID,1),KeyCRoffset(ChannelID,1));
|
|
||||||
Channel(ChannelID).length=ChannelLength;
|
|
||||||
ChannelID=ChannelID+1;
|
|
||||||
end
|
|
||||||
fclose(fid);
|
|
||||||
end
|
|
||||||
%%
|
|
||||||
function [KeyDx, KeyUnit]= ProcessCD(TxtString)
|
|
||||||
% disp('Info: Processing key CD...');
|
|
||||||
CommaLocation=find(TxtString==',');
|
|
||||||
Txtemp=TxtString(CommaLocation(3)+1:CommaLocation(4)-1);
|
|
||||||
KeyDx=str2double(Txtemp);
|
|
||||||
KeyUnit=TxtString(CommaLocation(6)+1:CommaLocation(7)-1);
|
|
||||||
% disp('Info: Finished Process key CD!');
|
|
||||||
end
|
|
||||||
function TimeStart = ProcessNT(TxtString)
|
|
||||||
|
|
||||||
CommaLocation=find(TxtString==',');
|
|
||||||
Txtemp=TxtString(CommaLocation(3)+1:CommaLocation(4)-1);
|
|
||||||
KeyDay=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(4)+1:CommaLocation(5)-1);
|
|
||||||
KeyMonth=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(5)+1:CommaLocation(6)-1);
|
|
||||||
KeyYear=str2num(Txtemp);
|
|
||||||
|
|
||||||
Txtemp=TxtString(CommaLocation(6)+1:CommaLocation(7)-1);
|
|
||||||
KeyHours=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(7)+1:CommaLocation(8)-1);
|
|
||||||
KeyMinutes=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(8)+1:length(TxtString));
|
|
||||||
KeySeconds=str2num(Txtemp);
|
|
||||||
|
|
||||||
TimeStart=datestr(datenum([KeyYear, KeyMonth,KeyDay,KeyHours,KeyMinutes,KeySeconds]),'yyyy-mm-dd HH:MM:SS');
|
|
||||||
% disp('Info: Finished Processing key NT!');
|
|
||||||
end
|
|
||||||
function [KeyBufferRef,KeyBytes,KeyNumerFormat,KeySignBits]= ProcessCP(TxtString)
|
|
||||||
% disp('Info: Processing key CP...');
|
|
||||||
CommaLocation=find(TxtString==',');
|
|
||||||
Txtemp=TxtString(CommaLocation(3)+1:CommaLocation(4)-1);
|
|
||||||
KeyBufferRef=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(4)+1:CommaLocation(5)-1);
|
|
||||||
KeyBytes=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(5)+1:CommaLocation(6)-1);
|
|
||||||
NumberFormat=str2num(Txtemp);
|
|
||||||
switch (NumberFormat)
|
|
||||||
case 1
|
|
||||||
KeyNumerFormat='*uint';
|
|
||||||
case 2
|
|
||||||
KeyNumerFormat='*int';
|
|
||||||
case 3
|
|
||||||
KeyNumerFormat='*ushort';
|
|
||||||
case 4
|
|
||||||
KeyNumerFormat='*short';
|
|
||||||
case 5
|
|
||||||
KeyNumerFormat='*ulong';
|
|
||||||
case 6
|
|
||||||
KeyNumerFormat='*long';
|
|
||||||
case 7
|
|
||||||
KeyNumerFormat='*float';
|
|
||||||
case 8
|
|
||||||
KeyNumerFormat='*float32';
|
|
||||||
case 9
|
|
||||||
KeyNumerFormat='*'; % imc Device Transitional Recording
|
|
||||||
case 10
|
|
||||||
KeyNumerFormat='*TimeStampASII' ;% TimeStamp is famos type
|
|
||||||
case 11
|
|
||||||
KeyNumberFormat='*bit16'; %2-byte-word digital
|
|
||||||
case 13
|
|
||||||
KeyNumberFormat='*bit48';
|
|
||||||
end
|
|
||||||
Txtemp=TxtString(CommaLocation(6)+1:CommaLocation(7)-1);
|
|
||||||
KeySignBits=str2num(Txtemp);
|
|
||||||
end
|
|
||||||
function [KeyBufferRefIndex,KeyBufferRefCb,KeyOffsetBufferInSamplesKey,KeyBufferFilledBytes] = ProcessCblittle(TxtString)
|
|
||||||
% disp('Info: Processing key Cb...');
|
|
||||||
CommaLocation=find(TxtString==',');
|
|
||||||
Txtemp=TxtString(CommaLocation(3)+1:CommaLocation(4)-1);
|
|
||||||
KeyBufferRefIndex=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(5)+1:CommaLocation(6)-1);
|
|
||||||
KeyBufferRefCb=str2double(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(7)+1:CommaLocation(8)-1);
|
|
||||||
KeyOffsetBufferInSamplesKey=str2double(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(10)+1:CommaLocation(11)-1);
|
|
||||||
KeyBufferFilledBytes=str2double(Txtemp);
|
|
||||||
%disp('Info: Finished Processing key Cb!');
|
|
||||||
end
|
|
||||||
function [KeyTransformation,KeyCRfactor,KeyCRoffset,KeyUnit]= ProcessCR(TxtString)
|
|
||||||
% disp('Info: Processing key CR...')
|
|
||||||
%
|
|
||||||
CommaLocation=find(TxtString==',');
|
|
||||||
Txtemp=TxtString(CommaLocation(3)+1:CommaLocation(4)-1);
|
|
||||||
KeyTransformation=str2num(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(4)+1:CommaLocation(5)-1);
|
|
||||||
KeyCRfactor=str2double(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(5)+1:CommaLocation(6)-1);
|
|
||||||
KeyCRoffset=str2double(Txtemp);
|
|
||||||
Txtemp=TxtString(CommaLocation(7)+1:CommaLocation(8)-1);
|
|
||||||
KeyUnitLength=str2double(Txtemp);
|
|
||||||
KeyUnit=TxtString(CommaLocation(8)+1:CommaLocation(8)+KeyUnitLength);
|
|
||||||
% disp('Info: Finished Processing key CR!');
|
|
||||||
end
|
|
||||||
function [ChannelName,ChannelComment]= ProcessCN(TxtString)
|
|
||||||
CommaLocation=find(TxtString==',');
|
|
||||||
ChannelName=TxtString(CommaLocation(7)+1:CommaLocation(8)-1);
|
|
||||||
ChannelCommLength=TxtString(CommaLocation(8)+1:CommaLocation(9)-1);
|
|
||||||
if ChannelCommLength=='0';
|
|
||||||
ChannelComment='';
|
|
||||||
else
|
|
||||||
temp=str2double(ChannelCommLength);
|
|
||||||
ChannelComment=TxtString(CommaLocation(9)+1:CommaLocation(9)+temp);
|
|
||||||
end
|
|
||||||
end
|
|
||||||
function tempChannel=ReadChannel(FileID, ReadStart,ChannelLength, Datatype,factor,offset)
|
|
||||||
fseek(FileID,ReadStart,'bof');
|
|
||||||
tempChannel=double(fread(FileID,ChannelLength,Datatype))*factor+offset;
|
|
||||||
%disp('Info: a Channle was imported.... ');
|
|
||||||
end
|
|
||||||
Loading…
x
Reference in New Issue
Block a user