2

EDIT: Changed Title to ignore my assumption.

I have a library, written in c, which uses posix message queue to pass pointers to some runtime data between threads.

In cases where the data is coming from user application, everything seems to work fine, and I am able to access the data pointed by the pointer to a structure, coming from the message queue.

Now, I have a special case where the library itself will do a malloc of one instance of the structure, set a flag and send it to the same queue. On the receiving end, the structure is empty and the flag is zero. Calling free on the pointer causes a crash.

Here's code:

volatile s_thestruct * volatile data = malloc(sizeof(s_thestruct));
data->flags = THE_FLAG;
mq_send(handle, (char *)&data, sizeof(s_thestruct *), 1)

And on the receiving end:

ssize_t read = mq_receive(handle, (char*)&data, sizeof(s_thestruct*), NULL);
if(read != sizeof(s_thestruct *))
{
 // Error handling, no problems here
}
if(data->flags == THE_FLAG)
{
// Do something, never gets here
}
// Do something else, no it is not freed here

// Finally
free(data); // <--CRASH

And I will get: 

*** glibc detected *** /usr/bin/applicationthingy: free(): invalid pointer: 0x08053cf0 ***

Followed by dump. Within the memory map dump I find:

....
08048000-0804a000 r-xp 00000000 08:01 802680     /usr/bin/applicationthingy
0804a000-0804b000 r--p 00001000 08:01 802680     /usr/bin/applicationthingy
0804b000-0804c000 rw-p 00002000 08:01 802680     /usr/bin/applicationthingy
0804c000-0806f000 rw-p 00000000 00:00 0          [heap]
b6e00000-b6e21000 rw-p 00000000 00:00 0 
....

So, anyone got any suggestions of what's going on here?

Fizzl
  • 23
  • 6
  • You should compile your application with `gcc -Wall -g`, improve it till no warnings are given, then use http://valgrind.org/ and `gdb` to debug that issue. You should check if `malloc` has failed. – Basile Starynkevitch Oct 04 '12 at 10:22
  • 2
    Are you sure that this is correct: `sizeof(s_thestruct *))` in send and receive? – Kiril Kirov Oct 04 '12 at 10:23
  • @Kiril Yeah. I want to pass only pointer to the structure. I'm not passing whole structures through the message queue. This actually works fine in the normal more complex cases where other data is passed from a user application of this library. This only happens in this special case where the library allocates the structure just to pass a flag. I suspect the compiler or malloc optimize away something, but I canẗ figure what and why. – Fizzl Oct 04 '12 at 10:28
  • 1
    @Rohan's answer is correct. You should also not be using `volatile` (http://stackoverflow.com/questions/2484980/why-is-volatile-not-considered-useful-in-multithreaded-c-or-c-programming). The fact that you need an explicit cast is another sure sign you're doing something wrong. – jleahy Oct 04 '12 at 10:32
  • You bods seem to be missing the fact that it's the _pointer_ being sent, not the data that it points to. Hence, the `&data` is correct. – paxdiablo Oct 04 '12 at 10:33
  • Do you get the error with the minimal code you show, or only with your complete code? If you delete everything from your code, except the parts shown, do you still get the same result? If not, then the problem is somewhere in the code you omit. – ugoren Oct 04 '12 at 10:49

2 Answers2

4

That (passing pointers) should work okay, assuming they're from the same address space.

The only thing I can think of is if there's a chance the queue could be being written from a separate process? That would make pointers useless since what they're pointing to is totally different in different processes.

To be honest though, I wouldn't pass pointers around like that unless the structure itself was massive. If you pass the structure, you gain all the advantages of being able to do proper inter-process queues.

The other thing to check is is the library is somehow getting its allocations from a different memory arena than your main code. That would be a circumstance that would cause the free failure but probably not the incorrect flags since, even if they're using different arenas, they would both be in the same address space.

At a bare minimum, you should be printing out the value of data on the sending and receiving side, to ensure that it's coming through unscathed. It's entirely possible some other piece of code could be corrupting it (a la buffer overflows and so forth).

paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • Thank you! I think I figured what is happening here. The descriptors for the message queue are opened on program initialization. In this case, I am running a test program that on top of running pthreads, also forks new processes. I actually accounted for using unique queues with getpid(), but because of the early initialization something interesting is happening. – Fizzl Oct 04 '12 at 11:11
1

You should be passing data not &data (address of data), in mq_send(handle, (char *)&data, sizeof(s_thestruct *), 1).

Rohan
  • 52,392
  • 12
  • 90
  • 87
  • I originally thought that but I don't believe it's the case. OP is sending the pointer itself as a message and "the mq_send() function shall add the message **pointed to** by the argument msg_ptr". Using `data` will send the first four (/eight) bytes of the message structure rather than the address. – paxdiablo Oct 04 '12 at 10:31
  • 1
    Nope. I actually need to pass address of the pointer, because of the cast to (char *) for mq_send. Without this indirection it would send first four bytes from the actual content of the structure. – Fizzl Oct 04 '12 at 10:32
  • @paxdiablo, I don't think that is valid. A memory is allocated, initialized and address assigned in `data` that address needs to be passed. With `&data` its passing address where the allocated address is stored, and `&data` may not contain valid data (function returns etc.) when receiver is reading it. – Rohan Oct 04 '12 at 10:40
  • As @ugoren pointed out this might be problem due to size of the message passed. – Rohan Oct 04 '12 at 10:42
  • I'm pretty certain mq_send copies its data to the queue - in fact I've relied on it, sending local data to the queue then returning. It doesn't matter whether that's a structure or a pointer. That means it doesn't matter if the stack value disappears after mq_send(), the pointer on the queue is still valid because it's already been copied. – paxdiablo Oct 04 '12 at 10:44
  • @user1719636 Can you try printing `data` and `&data` at both sides and verify you are getting addresses correctly? – Rohan Oct 04 '12 at 10:52