Saving a size_t in a raw mem block and getting it back

Question

I'm saving a size_t type of data in a raw memory block in kernel land, then I send the entire memory block to user land and I try to get that size_t value back.

The type isn't guarantied to be equal-sized on both kernel and user land, so I'm wondering what's the best way to save the value and then restore it.

Edit:

Or, maybe just save another type of data than has the same size on both sides and that can the converted (or casted without data loss) to/from size_t

Edit 2:

I'm saving my data in the following format:

(size_of_data_chunk)(data_chunk)(size_of_data_chunk)(data_chunk)...

Common code:

Code in kernel land:

void add_chunk(membuffer *buffer, void *chunk, size_t size){

    if(buffer->data != NULL){
        buffer->data = krealloc(buffer->data, buffer->len + sizeof(size_t) + size, GFP_KERNEL);
        buffer->len += sizeof(size_t) + size;

        memcpy(buffer->data + buffer->len, &size, sizeof(size_t));
        memcpy(buffer->data + buffer->len + sizeof(size_t), chunk, size);
    }else{

        buffer->data = kmalloc(sizeof(size_t) + size, GFP_KERNEL);
        buffer->len = sizeof(size_t) + size;

        memcpy(buffer->data, &size, sizeof(size_t));
        memcpy(buffer->data + sizeof(size_t), chunk, size);

    }

}

Code in user land:

void *get_chunk(membuffer *buffer){

    size_t *size;
    void *new_buffer;
    void *chunk = NULL;

    size = malloc(sizeof(size_t));

    memcpy(size, buffer->data, sizeof(size_t));

    chunk = malloc(*size);
    memcpy(chunk, buffer->data + sizeof(size_t), *size);

    buffer->data = malloc(buffer->len - sizeof(size_t) - *size);

    memcpy(buffer->data, buffer->data + sizeof(size_t) + *size, buffer->len - sizeof(size_t) - *size);

    free(size);

    return chunk;
}

Note that I know what type of data will be contained on each chunk, so I don't need to save the type nor any other information, just the size of the chunk, and the chunk per-se.

Also note that this is my not-yet-finished (aka test) code. Maybe some free's are missing.

When you say a "raw memory block" exactly what mechanism are you using (on both sides)? — Benjamin Leinweber, Sep 25 '13 at 21:16
That's not really what I meant. How are you allocating the memory in userspace and then how are you attempting to access it in kernel space? What are the specific systems calls you are using? — Benjamin Leinweber, Sep 25 '13 at 21:19
Out of curiosity, where did you see `size_t` isn't guaranteed to be the same ? — cnicutar, Sep 25 '13 at 21:23
@cnicutar I was just told so by user named ```vad``` in ##kernel — alexandernst, Sep 25 '13 at 21:23
Have you considered just using a fixed size `uint64_t` or something ? — cnicutar, Sep 25 '13 at 21:24
@cnicutar Can I convert ```uint64_t``` to/from ```size_t``` without data loss? — alexandernst, Sep 25 '13 at 21:24
@cnicutar I'm guessing ```uint64_t``` will always be bigger than ```size_t``` (is that right?), and I'm sure that I won't have sizes bigger than ```size_t```. If those two assumprions are right, is there any problem doing it that way? — alexandernst, Sep 25 '13 at 21:29
Well, that's the thing I'm not sure about: whether `uint64_t` is always bigger than `size_t` - the standard doesn't guarantee it. — cnicutar, Sep 25 '13 at 21:30
`size_t` is defined as `__kernel_size_t` in the kernel which is indeed architecture dependent (http://lxr.linux.no/linux+v2.6.33/include/linux/types.h#L59). However, I would think that `size_t` should be consistent on an architecture basis between glibc and the kernel. — Benjamin Leinweber, Sep 25 '13 at 21:35
What really has me curious is how you are calling your kernel-space `add_chunk()` function from your user-space application. — Benjamin Leinweber, Sep 25 '13 at 21:36
@BenjaminLeinweber I'm not concerned at all about it not having the same size on different machines as I'm de/serializing data without exporting it from the machine where it was serialized. — alexandernst, Sep 25 '13 at 21:37
@BenjaminLeinweber no, ```add_chunk``` is called only by the kernel (in fact, by a kernel module of mine) — alexandernst, Sep 25 '13 at 21:37

score 1 · Answer 1 · edited May 23 '17 at 11:56

1

After looking at this for a while, it seem that size_t is a little weird. This post quotes Wikipedia (though it looks like the quoted text is not there anymore) as saying that size_t is defined in stdlib.h via stddef.h. I checked my Ubuntu 12.04 install and indeed:

/* Get size_t, wchar_t and NULL from <stddef.h>.  */
#define     __need_size_t
#ifndef __need_malloc_and_calloc
# define    __need_wchar_t
# define    __need_NULL
#endif

The stddef.h file is quite complicated, so I will not quote it here, but it appears to typedef __SIZE_TYPE__ size_t which is also defined external. I haven't traced it back much further, since it's getting sticky.

In the end, I believe that size_t should be consistent across a given architecture, be it the kernel or glibc.

On my 32-bit Ubuntu install, sizeof(size_t) consistently returns 4, whether from a user application or a hello world kernel loadable module.

edited May 23 '17 at 11:56

Community

1
1

answered Sep 25 '13 at 22:24

Benjamin Leinweber

2,774
1
24
41

Ok, I'll re-test my code, but the first several tests returned different values (not the size of ```size_t```, but the value I get when I serialize-deserialize). Do you see something wrong in my code? – alexandernst Sep 25 '13 at 22:27
Your code isn't showing how your userspace is passing data to the kernel space. I have an idea what might be going on, but I would need to see that first. – Benjamin Leinweber Sep 25 '13 at 22:29
I'm sending the data via netlink. Netlink received a length and a pointer to a block of memory. On the user side, the block is passed to the callback funciton. I'm assuming the mem block is passed correctly. – alexandernst Sep 25 '13 at 22:31
Hmm... I usually use IOCTLs, not netlink so I'm not so familiar. Isn't the size of the data you're sending part of the netlink interface? – Benjamin Leinweber Sep 25 '13 at 22:37
Yes, indeed. I must say to netlink "look, I'm giving you a pointer to a block of memory with a size of X, send it to userland" – alexandernst Sep 25 '13 at 22:38
http://www.linuxjournal.com/node/7356/print. It looks like this (admittedly old and possibly out of date) article defines a particular header you must use with a length field at a fixed 32-bits (which has nothing to do with size_t). Am I missing something? – Benjamin Leinweber Sep 25 '13 at 22:42

Saving a size_t in a raw mem block and getting it back

1 Answers1