I am generating strings in Julia to use in Python. I would like to use Shared Memory (InterProcessCommunication.jl and Multiprocessing in Python). Currently, Julia generates strings then sends them to Python, which then reads the first number (so determine string length) before converting the rest into an encoded string.
I thought that shared memory would be much faster, but my method of timing (see below) seems to give 60-65 micros to:
- Send the string and string length
- Detect change in python, read the message and convert to bytes.
- Send back an indication for julia to detect.
I am using Ubuntu. Comparatively, using TCP sockets gives 200 micros (so only a 3x speedup).
GSTiming comes from here: #How can I get millisecond and microsecond-resolution timestamps in Python?
Julia:
using InterProcessCommunication
using Random
function copy_string_to_shared_memory(Aptr::Ptr{UInt8}, p::Vector{UInt8})
for i = 1:length(p)
unsafe_store!(Aptr, p[i], i + 4)
end
end
function main()
A = SharedMemory("myid"; readonly=false)
Aptr = convert(Ptr{UInt8}, pointer(A))
Bptr = convert(Ptr{UInt32}, pointer(A))
u = []
for i = 1:105
# p = Vector{UInt8}(randstring(rand(50:511)))
p = Vector{UInt8}("Hello Stack Exchange" * randstring(rand(1:430)))
a = time_ns()
copy_string_to_shared_memory(Aptr, p)
unsafe_store!(Bptr, length(p), 1)
# Make sure we can write to it
while unsafe_load(Aptr, 1) != 1
nanosleep(10e-7)
end
b = time_ns()
println((b - a) / 1000) # How i get times
sleep(0.01)
end
end
main()
Python Code:
from multiprocessing import shared_memory
import array
import time
import GSTiming
def main():
shm_a = shared_memory.SharedMemory("myid", create=True, size=512)
shm_a.buf[0] = 1 # Modify single byte at a time
u = []
for i in range(105):
while shm_a.buf[0] == 1:
GSTiming.delayMicroseconds(1)
s = bytes(shm_a.buf[4:shm_a.buf[0]+4])
shm_a.buf[0] = 1
shm_a.close()
shm_a.unlink() # Call unlink only once to release the shared memory
main()