1

EDIT: This question is related to this: Python ctypes: how to pass row outputs from a C function into a pandas DataFrame?

My question is how to parse text printed to STDOUT via printf() into something useable by Python.

I'm working with a Python wrapper in Python3.x around a C library using ctypes. The C library currently does database queries. The function I'm accessing via ctypes is printing the tab-delimited rows of the database via printf():

void print_query(const char *input_file,
                 const char *index, const char *query_string);

whereby input_file is the path to the file, index is the index already generated, and query_string is the string used to query the database.

Here is the Python code I have now:

from ctypes import CDLL
import ctypes

lib = CDLL(".../path/libshared.so") 

lib.print_query.restype = ctypes.c_char
lib.print_query.argtypes = (ctypes.c_char_p, ctypes.c_char_p, ctypes.c_char_p)

According to the documentation, https://docs.python.org/3/library/ctypes.html

ctypes.c_char is a 1-character bytes object. When I try using the function lib.print_query, the function returns rows via printf() to STDOUT: 

foo = lib.print_query('../file', '.../index', 'January')

outputs

2012 februari 32.610
2012    maart 33.195
2012    april 51.891

and print(foo) returns b'\x02'

I cannot "access" what has been printed to STDOUT, and I'm not sure how to translate this into a simple string of text. I get the same results trying

foo = io.BytesIO(lib.print_query('../file', '.../index', 'January'))

How can I access this text? It would be useful to stream this into a pandas Dataframe, as it's tab-delimited data.

EDIT: I've tried piping with the following

import subprocess
proc = subprocess.Popen(lib.print_query('../file', '.../index', 'January'), stdout=subprocess.PIPE)

Here's the output:

FileNotFoundError: [Errno 2] No such file or directory: b'\x02': b'\x02'
ShanZhengYang
  • 16,511
  • 49
  • 132
  • 234
  • I'm not that knowledgeable about Python ctypes, but it seems to me that you tell Python that `print_query` returns a `char`, but the C function is declared to not return anything. – Some programmer dude Jun 09 '18 at 22:31
  • 1
    And no, if you want to capture what's written to `stdout` then that's not possible the way you attempt do it. Instead you either needs to refactor your C function to create a string which it returns, or by using pipe to redirect output to `stdout` and and capture it that way (possibly using child-processes). – Some programmer dude Jun 09 '18 at 22:33
  • @Someprogrammerdude "the C function is declared to not return anything" Do you mean the `void`? It does print via `printf()` within the function though – ShanZhengYang Jun 10 '18 at 02:16
  • @Someprogrammerdude Thanks for the help. "by using pipe to redirect output to stdout and and capture it that way (possibly using child-processes)." How would I pipe this to `stdout`? – ShanZhengYang Jun 10 '18 at 02:18
  • @Someprogrammerdude Also, if the context is helpful, this question is related to this: https://stackoverflow.com/questions/50777214/python-ctypes-how-to-pass-row-outputs-from-a-c-function-into-a-pandas-dataframe – ShanZhengYang Jun 10 '18 at 03:11
  • Writing with `printf()` writes to standard output. Standard output is a property of the process as a whole. Anything written to standard output cannot be read unless you've taken steps to arrange that standard output of the entire Python process is something special like a pipe or file that can also be read. This is unusual. Your C code, therefore, is not designed to convey information to Python. You will need to redesign it, or get its writers to redesign it, so that the information is available as a string by some more or less devious means, and then get that string to Python. – Jonathan Leffler Jun 10 '18 at 03:58
  • @JonathanLeffler Thanks for the help. This makes sense. If you check the other question, I'm still struggling with how I would pass the rows of data into something ctypes accepts, i.e. `struct **outputs`. https://stackoverflow.com/questions/50777214/python-ctypes-how-to-pass-row-outputs-from-a-c-function-into-a-pandas-dataframe – ShanZhengYang Jun 10 '18 at 08:55
  • @JonathanLeffler Is it clear to you how to pass in a very long string via ctypes? Using another data structure could get complicated---currently, the row outputs is a struct of a struct. – ShanZhengYang Jun 10 '18 at 09:20

0 Answers0