I was working with ctypes a bit lately and noticed that the data transfer from Python to C, via ctypes, works with almost no overhead/delay, and I am passing huge arrays. This lets me assume that there is no data rearrangement occurring? How is this possible? Isn't the data structure in Python and C different, and wouldn't that mean that some form of restructuring is necessary for the arrays to be loaded without issues?
Asked
Active
Viewed 86 times
0
-
This depends on the type of arrays you are transferring, but usually ctypes passes a pointer and does not copy the array. For example, the `bytes` type in python contains a char array (hidden for the python user). You can try to modify it in c and see if the python object changed. Also even if ctypes would copy, this shouldn't take too long. – Tenobaal Aug 17 '22 at 12:28
-
I am passing like numpy arrays of size 10K x 10K, and it happens with no delay. if it would copy, that would take an infinite amount of time. And if it passes a pointer to an in-memory object, shouldn't it be structured differently in memory, because of Python != C? – dzudzudzu Aug 17 '22 at 12:38
-
Once the array is created, it doesn't have to be copied. https://stackoverflow.com/a/58735157/4788546, https://stackoverflow.com/a/58262388/4788546. But it's hard to guess without looking at the code. – CristiFati Aug 17 '22 at 13:14
-
@dzudzudzu numpy is a library written in C, so I assume it works with C arrays under the hood. Also C arrays are the computers native arrays, so most languages (even languages as abstract as python) use them. Java for example uses a C array in a struct to remember the size. – Tenobaal Aug 17 '22 at 13:16
-
Show specific example code. It depends on the call. – Mark Tolonen Aug 17 '22 at 15:48