If you use this software in any of your work, please cite it using the "Cite this repository" button in the right sidebar or use this:
@software{leonbohmann_APReader,
author = {Bohmann, Leon},
doi = {10.5281/zenodo.8369804},
month = sep,
title = {{leonbohmann/APReader: v1.1.2}},
url = {https://github.com/leonbohmann/APReader},
version = {v1.1.2},
year = {2023}
}
Read binary files produced from catmanAP projects directly into python.
CatmanAP procudes .bin files after each measurement. While it is possible to export as a different format (i.e. txt or asc) it's not efficient because one has to change the export format after every measurement. Here comes the treat: Just export as binary and use this package to work with binary files directly.
After reading all channels from the binary file, the channels are analyzed and every measure-channel will receive a reference to a time channel, depending on the amount of entries in the channels and the fact, that the time-channel has to contain "time" or "zeit" in its name. What that means is, that a channel with x entries and the name "time - 1" will be regarded as the time-channel of any other channel with x Data Entries.
Here is an example plot, generated directly from a binary file:
Anywhere with python (note the uppercase U):
pip install -U apread
The workflow of the package is straight-forward. You supply a binary file created with CatmanAP and the script will read that into python.
First of all, the binary data is analyzed and packaged into seperate Channel objects. When all Channels are created, each Channel.Name will be checked against ([T|t]ime)|([Z|z]eit), which mark time channels which usually are the reference.
These Channels marked as istime are the basis for Groups. Inside a group you will find the ChannelX (time channel) and a bunch of other channels in ChannelsY, which are the channels containing data in that time domain. The corresponding channels inside a Group are found by analyzing their length. Since the total time measured is the same for all groups, it is assumed that Channels with the same data-length belong to the same group. Connecting the matching channels to the group give a structured representation of your measurement data.
Now that the Data is available in python you are free to do with that whatever you want. Until Version 1.0.x there were some features in which you can save the data but that feature has been removed.
Lets say you produced a file called measurements.bin and you put it in the directory of your python script, then you can create the APReader on that file. It's that simple. The Initialization may take some time depending on how large your .bin-File is.
from apread import APReader
reader = APReader('measurements.bin') # this will read in the file
Afterwards you can access the Channels by accessing the APReader.Channels Member. Channel and Group implement __str__ which will return the name and the length of data inside it.
for channel in reader.Channels:
print(channel)
# "Timechannel 1 - Standard" (120341 Entries)
# "T12_ref" (120341 Entries)
# "T33" (120341 Entries)
# "Timechannel 1 - Quick" (3022344 Entries)
# "F1" (3022344 Entries)
# "ast089" (3022344 Entries)
for group in reader.Groups:
print(group)
# "Timechannel 1 - Standard" (2 Data-channels, 120341 Entries)
# "Timechannel 1 - Quick" (2 Data-channels, 3022344 Entries)
To review your data on the fly, you can plot every entity in the data structure by calling .plot(). When plotting, every group will get its own figure window, in which all connected channels are plotted.
# plot the readers data
reader.plot()
# plot all groups
for group in reader.Groups:
group.plot()
# plot all channels
for channel in reader.Channels:
channel.plot()
As you can see, you can access the channels from the reader, which contains all channels (including time channels) or you can access them from the groups.
There are some more functions to plot specific data. When plotting multiple channels each channel gets its own y-axis.
group.plotChannel(0) # specific channel
group.plotChannels(0,3) # channel 1 to 3 (1,2,3)
group.plot([0, 2, 4]) # channel 1, 3 and 5
The same can be applied to the APReader. The only difference is that you can plot specific groups instead of channels.
reader.plotGroup(0) # specific group
reader.plotGroups(0,3) # group 1 to 3 (1,2,3)
reader.plot([0, 2, 4]) # group 1, 3 and 5
Thanks to (hakonbars PR13) you are now able to access external header information using channel.exthdr, a dicitionary containing all keys as described in this sheet.
['T0'] # ACQ timestamp info (NOW format)
['dt'] # ACQ delta t in ms
['SensorType'] # IDS code of sensor type
['SupplyVoltage'] # IDS code supply voltage
['FiltChar'] # IDS code of filter characteristics
['FiltFreq'] # IDS code of filter frequency
['TareVal'] # Current value in tare buffer
['ZeroVal'] # Current value in zero adjustment buffer
['MeasRange'] # IDS code of measuring range
['InChar'] # Input characteristics (0=x1,1=y1,2=x2,3=y2)
['SerNo'] # Amplifier serial number
['PhysUnit'] # Physical unit (if user scaling in effect, this is the user unit!)
['NativeUnit'] # Native unit
['Slot'] # Hardware slot number
['SubSlot'] # Sub-channel, 0 if single channel slot
['AmpType'] # IDS code of amplifier type
['APType'] # IDS code of AP connector type (MGCplus only)
['kFactor'] # Gage factor used in strain gage measurements
['bFactor'] # Bridge factor used in strain gage measurements
['MeasSig'] # IDS code of measurement signal (e.g. GROSS, NET) (MGCplus only)
['AmpInput'] # IDS code of amplifier input (ZERO,CAL,MEAS)
['HPFilt'] # IDS code of highpass filter
['OLImportInfo'] # Special information used in online export file headers
['ScaleType'] # 0=Engineering units, 1=Electrical
['SoftwareTareVal'] # Software tare (zero) for channels carrying a user scale
['WriteProtected'] # If true, write access is denied
['NominalRange'] # CAV value
['CLCFactor'] # Cable length compensation factor (CANHEAD only)
['ExportFormat'] # 0=8-Byte Double, 1=4-Byte Single, 2=2-Byte Integer (FOR CATMAN BINARY EXPORT ONLY!)
Only available from version
v1.1.1-alpha1and above
See test/testing.py for a full example. Modify the following around your APReader call:
import multiprocessing as mp
...
if __name__ == '__main__': # this line has to be included!
# without 'processes=...'!
pool = mp.Pool()
# pass the pool to the reader
reader = APReader(file, parallelPool=pool)
# make sure to close the pool after you are done with it
mp.close()
mp.join()
For the parallel loading to work, you have to define a parallel pool of processes in your top-level script. These processes will be accessed from within APReader-Functions. When passing no arguments to mp.Pool() it will automatically create as many processes as possible, according to the amount of threads your CPU allows (cores + virtual cores). It does not make sense to pass in more, since the APReader spawns the same amount of processes as there are CPU Threads. Increasing the amount of processes in your pool does not increase the amount of parallelism. It is fixed.
Keep in mind, that parallelisation is not always faster. Spawning of processes is expensive and can be wasteful for small files.
The results from APReader stay the same and you can continue your analysis.
Channel.date)
Channel.time will be deleted at some point in the future...Since these function change a lot based on current needs, I decided to remove the post-processing functionality completely. The user now needs to do the post-processing on his own, meaning the creation of plots using time and data channels...
UTF-8typing package (compatibility issues with python <3.9.x)getas to generate formatted string without savingLeon Bohmann – mail@leonbohmann.de
Distributed under the MIT license. See LICENSE for more information.
This software comes with no warranty, expressed or implied. Use at your own risk!
FAQs
Import data from CatmanAP binary files.
We found that apread demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.
Security News
Research
Socket researchers found a malicious Maven package impersonating the legitimate ‘XZ for Java’ library, introducing a backdoor for remote code execution.