Hi I am coming from a python background and a lot of my data is saved using to_pickle from the pandas libary in the case of dfs or pickle.dump in the case of python dictionaries. I have been googling around and all I can find is python related content. So is pickle available just for python or is there a way to read the data frame data using c++?
Asked
Active
Viewed 200 times
-2
-
How large is your data? Megabytes, or terabytes? How often is it changed? Every milliseconds, or once per week? What operating system do you code for? What will happen to you on software failure? – Basile Starynkevitch Jun 11 '22 at 13:09
-
And what did you try? Show some [mre] – Basile Starynkevitch Jun 11 '22 at 13:12
1 Answers
0
The pickle function is indeed specific to Python.
However, Python is open source, and coded in C (and Python).
You could read more about extending and embedding Python.
Then you could call C++ code (in particular, your C++ functions declared extern "C") from Python or (with efforts) Python code from a C++ program.
You probably could find open source libraries in C++ parsing CSV files.
See this question.
Another approach is (if possible) to choose a simpler data format (like JSON) for which both Python and C++ libraries exist.
Or use some sqlite database.
Or (on Linux at least) use inter-process communication facilities (e.g. unix(7) sockets) between a Python program/process and a C++ one.
Basile Starynkevitch
- 223,805
- 18
- 296
- 547
-
I can parse csv files its just that the vast majority maybe 95% of my data is stored in pickle. I think I will consider your json suggestion as it seems like a good option for my case thanks – JPWilson Jun 11 '22 at 13:56