Open source C++ Utility data Export/Import
CCEXP
You can clone CCEXP with MVECTOR:
1 2 | git clone --recursive https://github.com/terablade2001/CCEXP.git |
or use MVECTOR later if need:
1 2 3 | git submodule init git submodule update |
CCEXP Presentation
You can take a look at CCEPX presentation video at this vimeo link!
What is CCEXP ?
CCEXP (Class C EXPort) is a library in C++, based on templates and standard C/C++ libraries and functions, for organized data extraction and sharing between C++ code and other software (i.e. MatLab/Octave, Python etc). The sharing is currently done via files, which can store large amount of data.
Why should I use CCEXP ?
- Data Export / Analysis / Debugging
- Data Sharing in different platforms (C++, MatLab/Octave, Python)
Data Export / Analysis / Debugging
You make a piece of code and need to export different runtime values and data that are going to be analyzed in another software (i.e. MatLab/Octave, Python etc). Instead of writing down your own code using fopen/fstream etc, you can just include the CCEXP.hpp and CCEXP.cpp files in your project and use them, for exporting multiple different types of data in an organized form.
An example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | #<span class="pl-k">include</span> <span class="pl-s"><span class="pl-pds">"</span>CCEXP.hpp<span class="pl-pds">"</span></span> ... <span class="pl-c">// Initialize an CCEXP object and add your Tables…</span> CCEXP::CCEXP FileA; <span class="pl-en">CCEXP::Initialize</span>(FileA, <span class="pl-s"><span class="pl-pds">"</span>FileNameA.ccexp<span class="pl-pds">"</span></span>); CCEXP::AddTable <<span class="pl-c1">uint8_t</span>>(FileA,<span class="pl-s"><span class="pl-pds">"</span>Image<span class="pl-pds">"</span></span>,<span class="pl-s"><span class="pl-pds">"</span>uint8<span class="pl-pds">"</span></span>); CCEXP::AddTable <<span class="pl-k">float</span>>(FileA,<span class="pl-s"><span class="pl-pds">"</span>Resulted_Coeffs<span class="pl-pds">"</span></span>,<span class="pl-s"><span class="pl-pds">"</span>single<span class="pl-pds">"</span></span>); ... <span class="pl-c">// On a running environment, capture images and calc data..</span> <span class="pl-k">for</span> (<span class="pl-k">int</span> i=<span class="pl-c1">0</span>; i < NImages; i++) { <span class="pl-c1">uint8_t</span>* pImage = <span class="pl-c1">CaptureImage</span>(Camera.<span class="pl-c1">getImage</span>()); vector<<span class="pl-k">float</span>> Coeffs = <span class="pl-c1">CalcCoeffs</span>(pImage); <span class="pl-c">// Add the data fast and easy to CCEXP for export</span> CCEXP::AddRow<<span class="pl-c1">uint8_t</span>>(FileA, <span class="pl-s"><span class="pl-pds">"</span>Image<span class="pl-pds">"</span></span>, pImage, <span class="pl-c1">640</span>*<span class="pl-c1">480</span>); CCEXP::AddRow<<span class="pl-k">float</span>>(FileA, <span class="pl-s"><span class="pl-pds">"</span>Resulted_Coeffs<span class="pl-pds">"</span></span>, Coeffs.<span class="pl-c1">data</span>(), Coeffs.<span class="pl-c1">size</span>()); } ... <span class="pl-c">// During termination store the file with all the captures and data..</span> <span class="pl-en">CCEXP::StoreData</span>(FileA); <span class="pl-en">CCEXP::Reset</span>(FileA); |
At MatLab/Octave you can read and check if everything is ok, as:
1 2 3 4 5 6 | [~,d]=CCEXP(<span class="pl-s"><span class="pl-pds">'</span>FileNameA<span class="pl-pds">'</span></span>); N=<span class="pl-c1">1</span>; <span class="pl-c">% Get Nth Image</span> testImage = reshape(d.Image{N},[<span class="pl-c1">256</span> <span class="pl-c1">256</span>]}'; <span class="pl-c">% Get Nth Images' coeffs</span> testCoeffs = d.Resulted_Coeffs{N}; |
And what if you need to stop exporting the ‘Image’ table (because the camera’s data are ok, and you don’t need anymore those data)?… With CCEXP you can ignore or not easily each Table:
1 2 3 4 5 | <span class="pl-c">// Just add an 'I' (I-gnore) at the AddTable function, and the table will be</span> <span class="pl-c">// completely ignored. If need again to enable data of this table, remove the</span> <span class="pl-c">// 'I' from the name. An easy way to enable/disable export data without messing</span> <span class="pl-c">// up with a lot of the code!</span> CCEXP::AddTableI<<span class="pl-c1">uint8_t</span>>(FileA,<span class="pl-s"><span class="pl-pds">"</span>Image<span class="pl-pds">"</span></span>,<span class="pl-s"><span class="pl-pds">"</span>uint8<span class="pl-pds">"</span></span>); |
Data Sharing in different platforms (C++, MatLab/Octave, Python)
CCEXP is a very good tool, if you need to exchange exported data from C++ to another collaborator which works in any (supported) language. The CCEXP I/O system is hidden to clients, thus as far as all collaborators understand the data organizaton of CCEXP (Files, Tables(of different types), Rows and Colums) and the relative API in each language, the data can be shared easily between the different platforms.
Below an analytical example of data sharing from a MatLab/Octave script is given, following by an example of C++ data import to a program.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | <span class="pl-s"><span class="pl-smi">clear</span> all; close all; clc;</span> <span class="pl-c">% ################ PART #1 #####################################################</span> <span class="pl-c">%Suppose we have to share those data...</span> <span class="pl-c">% Table 1 Rows (float)</span> T1_1 = [<span class="pl-c1">1.1</span> <span class="pl-c1">2.2</span> <span class="pl-c1">3.3</span> <span class="pl-c1">4.4</span> <span class="pl-c1">5.5</span>]; T1_2 = []; T1_3 = [<span class="pl-c1">6.6</span> <span class="pl-c1">7.7</span>]; <span class="pl-c">% Table 2 Rows (uint8)</span> T2_1 = [<span class="pl-c1">1</span>]; T2_2 = [<span class="pl-c1">1</span> <span class="pl-c1">2</span>]; T2_3 = [<span class="pl-c1">1</span> <span class="pl-c1">0</span> <span class="pl-c1">3</span>]; T2_4 = [<span class="pl-c1">1</span> <span class="pl-c1">0</span> <span class="pl-c1">3</span> <span class="pl-c1">4</span>]; T2_5 = [<span class="pl-c1">1</span> <span class="pl-c1">0</span> <span class="pl-c1">3</span> <span class="pl-c1">4</span> <span class="pl-c1">5</span>]; T2_6 = [<span class="pl-c1">1</span> <span class="pl-c1">0</span> <span class="pl-c1">3</span> <span class="pl-c1">4</span> <span class="pl-c1">5</span> <span class="pl-c1">6</span>]; <span class="pl-c">% Then we have to convert the data to the following format:</span> <span class="pl-c">% Table 1</span> <span class="pl-c">% 1. Each row of a table is becoming a Cell</span> TVData1{<span class="pl-c1">1</span>} = T1_1; TVData1{<span class="pl-c1">2</span>} = T1_2; TVData1{<span class="pl-c1">3</span>} = T1_3; <span class="pl-c">% 2. The table is created as following:</span> <span class="pl-c">% Name, Type, Bytes per Element, The Table-data-cell, MaxRows (0=ALL)</span> Table1 = {<span class="pl-s"><span class="pl-pds">'</span>Table_1<span class="pl-pds">'</span></span>, <span class="pl-s"><span class="pl-pds">'</span>single<span class="pl-pds">'</span></span>, <span class="pl-c1">4</span>, TVData1, <span class="pl-c1">0</span> }; <span class="pl-c">% Table 2</span> TVData2{<span class="pl-c1">1</span>} = T2_1; TVData2{<span class="pl-c1">2</span>} = T2_2; TVData2{<span class="pl-c1">3</span>} = T2_3; TVData2{<span class="pl-c1">4</span>} = T2_4; TVData2{<span class="pl-c1">5</span>} = T2_5; TVData2{<span class="pl-c1">6</span>} = T2_6; Table2 = {<span class="pl-s"><span class="pl-pds">'</span>Table_2<span class="pl-pds">'</span></span>, <span class="pl-s"><span class="pl-pds">'</span>uint8<span class="pl-pds">'</span></span>, <span class="pl-c1">1</span>, TVData2, <span class="pl-c1">4</span>}; <span class="pl-c">% Store only 4 first rows!</span> <span class="pl-c">% Final CELL</span> <span class="pl-c">% Create the Cell with all the wanted Tables (using ';')</span> CCEXPData = {Table1; Table2}; <span class="pl-c">% Now use CCEXP_WRITE:</span> CCEXP_WRITE(<span class="pl-s"><span class="pl-pds">'</span>SharedData.ccexp<span class="pl-pds">'</span></span>,CCEXPData); |
The “SharedData.ccexp” file can now be shared with a C++ program and loaded as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | #<span class="pl-k">include</span> <span class="pl-s"><span class="pl-pds">"</span>CCEXP.hpp<span class="pl-pds">"</span></span> ... <span class="pl-c">// Define a CCEXP object, open the data file and load any wanted table.</span> CCEXP::CCEXP LD; <span class="pl-en">CCEXP::Open</span>(LD,<span class="pl-s"><span class="pl-pds">"</span>SharedData.ccexp<span class="pl-pds">"</span></span>); CCEXP::LoadTable<<span class="pl-k">float</span>>(LD,<span class="pl-s"><span class="pl-pds">"</span>Table_1<span class="pl-pds">"</span></span>,<span class="pl-s"><span class="pl-pds">"</span>single<span class="pl-pds">"</span></span>); CCEXP::LoadTable<<span class="pl-c1">uint8_t</span>>(LD,<span class="pl-s"><span class="pl-pds">"</span>Table_2<span class="pl-pds">"</span></span>,<span class="pl-s"><span class="pl-pds">"</span>uint8<span class="pl-pds">"</span></span>); <span class="pl-en">CCEXP::Close</span>(LD); <span class="pl-c">// Then you can access elements like this:</span> <span class="pl-k">for</span> (<span class="pl-c1">size_t</span> r = <span class="pl-c1">0</span>; r < CCEXP::Rows(LD, <span class="pl-s"><span class="pl-pds">"</span>Table_2<span class="pl-pds">"</span></span>); r++) { <span class="pl-k">for</span> (<span class="pl-c1">size_t</span> c = <span class="pl-c1">0</span>; c < <span class="pl-c1">CCEXP::Cols</span>(LD, <span class="pl-s"><span class="pl-pds">"</span>Table_2<span class="pl-pds">"</span></span>, r); c++) { <span class="pl-c1">uint8_t</span>* v1 = CCEXP::getVal<<span class="pl-c1">uint8_t</span>>(LD, <span class="pl-s"><span class="pl-pds">"</span>Table_2<span class="pl-pds">"</span></span>, r, c); <span class="pl-k">if</span> (v1 != <span class="pl-c1">NULL</span>) <span class="pl-c1">printf</span>(<span class="pl-s"><span class="pl-pds">"</span>%u <span class="pl-pds">"</span></span>, v1); } } |
CCEXP and MVECTOR
CCEXP is now based on the MVECTOR library, as std::vector library may or may not work the same in all libraries.
MVECTOR library also can track allocated memory. More info about MVECTOR can be found here:
https://github.com/terablade2001/MVECTOR.git
Build with CMAKE
You can clone CCEXP with MVECTOR:
1 2 | git clone --recursive https://github.com/terablade2001/CCEXP.git |
Then build like:
1 2 3 4 5 6 7 8 9 10 11 | mkdir build && cd build cmake -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=Release .. make # Test t00_Basics.cpp cls && CCEXP t00 # Test t01_Load.cpp cls && CCEXP t01 # Test t02_ExternalWriteLoad.cpp # (First create SharedData.ccexp with /MATLAB/test/t02_ExternalWriteLoad.m file) cls && CCEXP t02 |
CCEXPViewer
Using CCEXP Version 0.068, users can find at the folder /C++/test/CCEXPViewer/
a simple C++ demo example which creates an .exe to display .ccexp files data. The code is given here too.
The macro __CCEXP_PRINT_TABLES__
is just an example macro on how to use the new Analyze() functions of CCEXP.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | #<span class="pl-k">include</span> <span class="pl-s"><span class="pl-pds">"</span>../../src/include/CCEXP.hpp<span class="pl-pds">"</span></span> <span class="pl-k">int</span> <span class="pl-en">main</span>(<span class="pl-k">int</span> argc, <span class="pl-k">char</span>** argv) { <span class="pl-k">if</span> (argc != <span class="pl-c1">2</span>) { <span class="pl-c1">printf</span>(<span class="pl-s"><span class="pl-pds">"</span>Please provide a filename<span class="pl-cce">\n</span><span class="pl-pds">"</span></span>); <span class="pl-k">return</span> -<span class="pl-c1">1</span>; } <span class="pl-c1">printf</span>(<span class="pl-s"><span class="pl-pds">"</span><span class="pl-cce">\n</span>-------------------------------------------------------------<span class="pl-cce">\n</span><span class="pl-pds">"</span></span>); <span class="pl-c1">printf</span>(<span class="pl-s"><span class="pl-pds">"</span>CCEXP Library (%1.3f): <span class="pl-pds">"</span></span> <span class="pl-s"><span class="pl-pds">"</span>https://github.com/terablade2001/CCEXP<span class="pl-cce">\n</span><span class="pl-pds">"</span></span>, CCEXP_VERSION ); <span class="pl-c1">printf</span>(<span class="pl-s"><span class="pl-pds">"</span>------- CCEXP Viewer >> [%s] ------------<span class="pl-cce">\n</span><span class="pl-pds">"</span></span>, argv[<span class="pl-c1">1</span>]); <span class="pl-c1">__CCEXP_PRINT_TABLES__</span>(argv[<span class="pl-c1">1</span>]); <span class="pl-c1">printf</span>(<span class="pl-s"><span class="pl-pds">"</span>-------------------------------------------------------------<span class="pl-cce">\n\n</span><span class="pl-pds">"</span></span>); <span class="pl-k">return</span> <span class="pl-c1">0</span>; } |
CCEXPGet
CCEXPGet example is a tool, that can be modified/compiled and used directly to extract rows of CCEXP arrays from command-line.