Numpy arrays in C

Question

I've only just started C and am trying to make a backend for my python. At the moment I have a cumbersome system of writing huge input files (at least 10^6 floats) from the python then initialising the C (which has to load in lots of other data as well) running the C until termination, receiving its output only to have to reinitialize it again with a new input array.

I'm not sure if it is conceptually possible from looking around but I was hoping to just be able to direct the C to a memory address for a numpy array and use it as though it is an array made by the C. The aim is for the python to initialise the C, build its first array, run the C code until it is ready for the next array which will then be generated by python.

This is some dummy code that I wrote to try and get C to read numpy arrays. At the moment the C just has a segmentation fault and I'm not sure if its because I'm bad at C, it doesn't like reading the numpy arrays or a bit of both.

Front.py

array_interface was from here.

#! /usr/bin/python

import numpy as np
import subprocess as sp

ray = np.array([x*3.14 for x in range(10)])
addr = ray.__array_interface__['data'][0]

pro = sp.Popen(['./back', hex(addr)])
print pro.communicate()[0]
print ray[:5]
quit()

Back.c

#include <stdio.h>
#include <stdlib.h>

int main(char *argv[])
{
  float *addr;
  int n;
  float a[10];

  // Hopefully making a pointer to the first float in the np.ndarray
  addr = sscanf(argv[1], "%x");

  n = 0;
  while( n<10 )
  {
  // Hopefully stepping through the np.ndarray one float at a time.
    a[n] = *addr;
    addr++;
    n++;
  }
  // Return the first five values to compare with python.
  fprintf(stdout, '%f %f %f %f %f\n', a[0], a[1], a[2], a[3], a[4]);
  return 0;
}

Is it possible to have C read numpy arrays like this or is this approach fundamentally flawed? Is there a better way of doing it? if this does work, is it possible to use a Ctype bool, that both processes know the address of, to coordinate which process is working and which is waiting? e.g. python sets the value to 1 as it initialises the numpy array -the C is waiting stuck in a while(sp_bool==1) loop. Then when python is finished it changes the value to 0 and the C executes, finally changing the value back to 1 when it finishes.

I don't know Python enough to know whether there is a way, but there's one, it will be an in-process one, not an out-of-process solution like what you're doing now. Can't Python call C code from `.so` files? — Medinoc, Nov 28 '13 at 15:13
By the way, even if it could work, you're not using sscanf() correctly: It returns the number of values parsed, not the value itself. — Medinoc, Nov 28 '13 at 15:14
You can [embed Python in your C application](http://docs.python.org/3/c-api/intro.html#embedding-python). — Ramchandra Apte, Nov 28 '13 at 15:15
If you are writing the C code from scratch and want to run things from Python, the best would be to write a Python C extension module. The Numpy User Guide has [a good introduction to the subject](http://docs.scipy.org/doc/numpy/user/c-info.how-to-extend.html). — Jaime, Nov 28 '13 at 16:42

score 1 · Answer 1 · edited Dec 20 '21 at 22:24

It sounds like your best option might be to call C code from python. Not only is that very much possible, it's actually how NumPy is written. NumPy is mostly written in C so when you do np.add(array1, array2) Python passes array1 and array2 to NumPy's C code, the C code makes an output array for the results, adds the elements of the input arrays putting the result in the output array, and returns the output array.

I think the easiest way for you to make your C code available in Python is to wrap it using Cython. Here are two tutorials that hopefully will get you moving in the right direction. How to call existing C code from Cython and how to use NumPy arrays in Cython.

Also I should mention that you can access NumPy arrays from C code using the NumPy C API, this probably will be harder for you than using Cython because you're new to C, but it's another way to go if you chose that route.

Numpy arrays in C

1 Answers1