I need to send a huge data bytes approx 1000 MB from Module A to Module B in Azure IoT Edge. (Module A and Module B are in same edge device) Message Routes allows max 262143 bytes and Direct method allows approx 160 KB. Is there any other options to send huge data bytes between modules, Performance is a primary concern and I do not want to split the data and send it as a chunks.
Thanks
Edge routing is not the only way for modules to communicate. Afterall these are docker containers and there is nothing preventing you from implementing your own TCP link between two of them. Since docker provides a virtual network with a DNS server you will also be able to avoid any variation in IP addresses of your containers. You can look up the container's address by name and, by default, the container will use docker's DNS service to do that thus you will be able to resolve it to an IP address. All Azure IoT Edge modules except edgeAgent run on a docker network called azure-iot-edge. You would need to implement the code to move the data. You could do this from scratch or perhaps you could use a predefined protocol such as ftp or scp. You also need to consider that, unless you take steps to secure this transfer, it will not be encrypted (something Edge does do for you when routing).
I'm not suggesting this is going to give you the best performance. I am simply offering you other avenues for your research.
What are these data used for? Another option I can think of is to share data using docker volume, https://docs.docker.com/storage/volumes/ or bind mounts, https://docs.docker.com/storage/bind-mounts/.
Having a shared folder helped (https://learn.microsoft.com/en-us/azure/iot-edge/how-to-access-host-storage-from-module#link-module-storage-to-device-storage) and used google protocol buffer that helped in reducing our file 1/3 1000MB file reduced to 300MB.