1

I need some help converting a luajit pointer to a string and back.

First I define the ctype:

ffi.cdef[[
    typedef struct {
        unsigned char Bytes[16];
    } EncryptionKeys[100000000];

void* malloc(size_t);                   
void free(void*);
]]

Then use malloc to allocate some memory and then create the 'EncryptionKeys' variable.

local EncryptionKeyMemoryAddress = ffi.C.malloc(ffi.sizeof("EncryptionKeys"))

local EncryptionKeys = ffi.cast("EncryptionKeys(&)", EncryptionKeyMemoryAddress)

I first convert the variable into a lua string using:

ffi.string(EncryptionKeyMemoryAddress)

But I can't figure out how to convert it back! Can someone please help me?

FYI: I am passing the 'EncryptionKeyMemoryAddress' variable to one of the function parameters for a lua lane (https://lualanes.github.io/lanes/).

Edit: Here is the section of code that I am working on: This is for the client managers module of my server that manages a list of lua states that all have access to any clients connected to the server. They all use a shared section of memory that I want them to have access to using a pointer.



local ClientFFIString = [[
    
    typedef struct {
        unsigned char Bytes[16];
    } EncryptionKeys[100000000];

    void* malloc(size_t);                   
    void free(void*);
]]

ffi.cdef(Matchpools.FFIString)

local EncryptionKeyMemoryAddress = ffi.C.malloc(ffi.sizeof("EncryptionKeys"))

--------------------------------------------
function ClientManagers.CreateNewClientManager()

    local EncryptionKeys = ffi.cast("EncryptionKeys(&)", EncryptionKeyMemoryAddress)

    EncryptionKeys[0].Bytes[0] = 24

    print("___a", EncryptionKeys[0].Bytes[0])


    local NewIndex = #ClientManagers.List+1
    ClientManagers.List[NewIndex] = ClientManagerFunc(
        ClientFFIString, 
        ffi.string(EncryptionKeysMemoryAddress)
    )

end


--------------------------------------------
local ClientManagerFunc = Lanes.gen("*", function(ClientFFIString, EncryptionKeysMemoryAddress)

    ffi = require("ffi")

    ffi.cdef(ClientFFIString)


    local EncryptionKeys = ffi.cast("EncryptionKeys(&)", EncryptionKeyMemoryAddress)
    
    print("___a", EncryptionKeys[0].Bytes[0]) 
    -- I want this to be 24 just like it is in the function that created this lua state


    local ClientManagerRunning = true
    while ClientManagerRunning do

        --local dt = GetDt()

        --UpdateClientData(dt)

        --UpdateMatchmaking(dt)

    end

end)
Egor Skriptunoff
  • 23,359
  • 2
  • 34
  • 64
  • What do you mean by "convert it back"? Do you mean "copy it back"? Because my understanding is that `ffi.string` creates a copy of the binary data. And are you sure you want to copy 1.5GB of data? – Nicol Bolas Apr 16 '21 at 03:08
  • I have 2 different lua states that I want to use the same section of memory on my server. I want to give them both the pointer to that section of memory so they both can use it. I only want to pass the pointer to the memory as a lua string (not copy it). Does that make sense? – James Washington Apr 16 '21 at 04:36

1 Answers1

0

You can convert Lua string to your structure pointer (and later use it as an array):

ffi.cdef"typedef struct {unsigned char Bytes[16];} Key;"
ptr=ffi.cast("Key *", your_string)
print("First Byte of the First Key in your Array:", ptr[0].Bytes[0])

UPDATE:
Let's test how it works for an array containing three keys:

local ffi = require'ffi'
ffi.cdef"typedef struct {unsigned char Bytes[16];} Key;"
local your_string = string.char(11):rep(16)..string.char(22):rep(16)..string.char(33):rep(16)
local ptr=ffi.cast("Key *", your_string)
print("First Byte of the First Key in your Array:", ptr[0].Bytes[0])
print("First Byte of the Second Key in your Array:", ptr[1].Bytes[0])
print("First Byte of the Third Key in your Array:", ptr[2].Bytes[0])

It prints 11, 22, 33


UPDATE 2:
Pass the address of buffer instead of the content of buffer

In the main thread

-- allocate the buffer
local Keys = ffi.cast("EncryptionKeys&", ffi.C.malloc(ffi.sizeof("EncryptionKeys")))
-- write to the buffer
Keys[2].Bytes[5] = 42
-- create string containing 64-bit address
local string_to_send = tostring(ffi.cast("uint64_t", Keys))
-- Send string_to_send to a lane

Inside the lane

-- receive the string
local received_string = .....
-- restore the buffer pointer
local Keys = ffi.cast("EncryptionKeys&", loadstring("return "..received_string)())
-- read the data from the buffer
print(Keys[2].Bytes[5])

Egor Skriptunoff
  • 23,359
  • 2
  • 34
  • 64
  • When I try to apply this to my code I get the error message (ClientManagers.lua:147: 'struct 128 [100000000]' has no member named 'Bytes'). Maybe I am doing something wrong with pointers? – James Washington Apr 16 '21 at 04:55
  • Show your line #147. Please note that `EncryptionKeys` is incompatible with `Key`. You should use `EncryptionKeys` when converting to string and use `Key` when converting from string. – Egor Skriptunoff Apr 16 '21 at 12:35
  • Okay I understand now. This worked thank you! – James Washington Apr 16 '21 at 15:29
  • Please note that while you are working with `ptr[x].Bytes[y]` values the original Lua string `your_string` must stay anchored in Lua (for example, in a local variable) to avoid being garbage collected, because you are actually reading the content of this string. – Egor Skriptunoff Apr 16 '21 at 16:09
  • Understood. Thanks again. – James Washington Apr 17 '21 at 01:24
  • This solution actually only works for ptr[0]. Any indexes over that do not work. Do you know why and how I can fix that? – James Washington Apr 17 '21 at 07:23
  • Can't reproduce the problem. It works fine for me. The answer is updated with a code for testing. Can you post your code which fails? – Egor Skriptunoff Apr 17 '21 at 11:21
  • When I try your solution it works fine until I pass your_string into the lua lane and initialize the array inside the lane. Then the array does not access values above 0. Maybe I stated the question wrong. Basically I need to transfer the pointer of a memory location (EncryptionKeyMemoryAddress which is returned by malloc) to a lua lane so that I can access the data from inside the lua lane. I want to pass the 8 byte pointer not the actual data and use that. Think of multiple lua states having access to the same location of memory concurrently. – James Washington Apr 17 '21 at 16:13
  • Converting from address to array is almost the same. What cdata object does contain the address? How did you define it inside cdef? Is it `int64_t` / `void *` / something else ? – Egor Skriptunoff Apr 17 '21 at 17:24
  • In other words, what is the output of `print(type(your_string), ffi.typeof(your_string))` inside a lane? – Egor Skriptunoff Apr 17 '21 at 17:36
  • Printing inside of the lane throws an error, but printing outside of the lane results in: 'cdata, ctype' I am passing the result of malloc as the 'your_string' variable so I am pretty sure it is an 8 byte pointer. – James Washington Apr 17 '21 at 18:21
  • Yes the address is contained in 'void*' This is malloc in my cdef: ```void* malloc(size_t);``` – James Washington Apr 17 '21 at 18:23
  • Ok, outside a lane you have `void *` from malloc, and you pass it to a lane. But what value does the lane receive? `print(type(value))` – Egor Skriptunoff Apr 17 '21 at 21:22
  • type inside the lane is 'string' – James Washington Apr 17 '21 at 21:32
  • So, you should ask a new question "How to pass a string to lua-lane" (and set corresponding tag to call for users with lua-lanes experience). Or (probably) you can simply read the lua-lanes docs. – Egor Skriptunoff Apr 17 '21 at 21:38
  • Okay I am going to ask a new question. I just tried to get the length of the string outside of the lane and it was zero also. Could it have to do with using the (&) operator for malloc instead of * ? ```ffi.C.malloc(ffi.sizeof("EncryptionKeys(&)"))``` – James Washington Apr 17 '21 at 21:40
  • See update 2. It contains an example how to pass address of buffer. – Egor Skriptunoff Apr 17 '21 at 22:37
  • Thanks alot Egor :). – James Washington Apr 17 '21 at 23:10