How to allocate memory in a DriverKit system extension and map it to another process?

Question

I have allocated memory in my application and passed its pointer and size to IOConnectCallStructMethod. Using IOMemoryDescriptor::CreateMapping I have then mapped this memory to the DriverKit system extension process, and it is possible to write to this mapped memory location and read the data from my application.

I would now like to do something similar for memory that is allocated in the system extension, and then map it to the application that is using the system extension. I would like to create a set of memory buffers in the system extension, and then write to it from the application and then signal to the system extension with IOConnectCallScalarMethod that a given buffer should be sent to the USB device, using IOUSBHostPipe::AsyncIO. When the CompleteAsyncIO callback then comes as a result of the sending completing, I would notify back to the application that it is now possible to copy data to the first buffer that was sent. The mechanism for this could probably be done using IOConnectCallAsyncStructMethod, and the OSAction object that is created in the system extension. What I don't understand is how to map memory allocated in the system extension to the application.

I have answered your question but note that if you retain the `structureOutputDescriptor` and/or `structureInputDescriptor` of your external method's `IOUserClientMethodArguments`, you can continue to use that descriptor in the driver as you wish. It really depends which approach is more convenient for your use case. There is one downside to the structure descriptor approach: structure inputs and outputs only turn up in the dext/kext as a memory descriptor if they're more than 4096 bytes long. Otherwise they're copied by value into/from the `OSData*` typed field instead. — pmdj, Jun 05 '20 at 11:43
All of the IOMemoryDescriptor that are accessible in the IOUserClientMethodArguments are allocated by the application, right? Can I pass an `IOMemoryDescriptor` that points to memory allocated by the application to for example `IOUSBHostPipe::AsyncIO`? Or do I have to allocate any descriptor that is used with `IOUSBHostPipe::AsyncIO` with `IOUSBHostInterface::CreateIOBuffer`? — tuple_cat, Jun 05 '20 at 13:01
You can safely pass the memory descriptors from `IOUserClientMethodArguments` to `IOUSBHostPipe::AsyncIO`. In most cases, this will be a zero-copy operation. The downsides are the awkward size threshold and the fact that you can't subdivide or concatenate the `IOMemoryDescriptor` in any way. (I think you can drop trailing bytes by passing a smaller length to `AsyncIO` though.) — pmdj, Jun 05 '20 at 13:39
`IOUSBHostInterface::CreateIOBuffer` should be used for buffers allocated for USB I/O in preference to `IOBufferMemoryDescriptor::Create` - i.e. if you were already going to be allocating your buffer memory inside the dext anyway. — pmdj, Jun 05 '20 at 13:39
Incidentally, in kexts, you have `IOSubMemoryDescriptor` and `IOMultiMemoryDescriptor` which you can use to splice memory descriptors from near enough any source before using them for I/O. (In fact you can define your own subclasses which can apply whatever splicing logic you like.) I've reported their absence in DriverKit as a bug to Apple, if those would be helpful to you I recommend you do the same. — pmdj, Jun 05 '20 at 13:43
ok, then I will go with the solution of allocating in the application and pass the buffer with for example `IOConnectCallAsyncStructMethod`. The maximum buffer size I want to send is larger than 4096 so the buffer should appear as an `IOMemoryDescriptor`. Yes, I am planning to pass the length of the data written to that buffer as the `dataBufferLength` parameter of `AsyncIO`. I did not have any need for `IOSubMemoryDescriptor` or `IOMultiMemoryDescriptor` yet, but I will keep them in mind and submit a request for them to be added if I find a use case for them. — tuple_cat, Jun 05 '20 at 13:58
Just to be clear: Note that it's not about the *maximum* buffer size, but the *actual* size passed to `IOConnectCallAsyncMethod`'s struct argument(s). So if your external method can legally be called with a buffer of 4096 bytes or less, you have to special case it and copy to an `IOBufferMemoryDescriptor`. It's super annoying. For output structs, this means you can't actually asynchronously fill the buffer if it's 4096 bytes or fewer, because the `structureOutput` field is an `OSData*` and synchronously returned. — pmdj, Jun 05 '20 at 14:03
The 4096 byte limit is defined via the `io_struct_inband_t` type by the way. (I'd rather use `sizeof(io_struct_inband_t)` than a magic 4096 in my special-casing code.) — pmdj, Jun 05 '20 at 14:04
ok, sound good to use the `sizeof(io_struct_inband_t)`. I just realised that `IOSubMemoryDescriptor` would be handy. The max packet size of the USB device that I am communicating with is much smaller that the maximum size of the buffers that I want to send so somehow the buffers needs to be split. What would be the best way to ask Apple to add something similar to DriverKit? — tuple_cat, Jun 05 '20 at 16:30

score 4 · Accepted Answer · answered Jun 05 '20 at 11:37

This is what IOUserClient::CopyClientMemoryForType in DriverKit is for, which gets invoked when your user process calls IOConnectMapMemory64 from IOKit.framework. The kext equivalent, incidentally, is IOUserClient::clientMemoryForType and essentially works exactly the same.

To make it work, you need to override the CopyClientMemoryForType virtual function in your user client subclass.

In the class definition in .iig:

virtual kern_return_t CopyClientMemoryForType(
    uint64_t type, uint64_t *options, IOMemoryDescriptor **memory) override;

In the implementation .cpp, something along these lines:

kern_return_t IMPL(MyUserClient, CopyClientMemoryForType) //(uint64_t type, uint64_t *options, IOMemoryDescriptor **memory)
{
    kern_return_t res;
    if (type == 0)
    {
        IOBufferMemoryDescriptor* buffer = nullptr;
        res = IOBufferMemoryDescriptor::Create(kIOMemoryDirectionInOut, 128 /* capacity */, 8 /* alignment */, &buffer);
        if (res != kIOReturnSuccess)
        {
            os_log(OS_LOG_DEFAULT, "MyUserClient::CopyClientMemoryForType(): IOBufferMemoryDescriptor::Create failed: 0x%x", res);
        }
        else
        {
            *memory = buffer; // returned with refcount 1
        }
    }
    else
    {
        res = this->CopyClientMemoryForType(type, options, memory, SUPERDISPATCH);
    }
    return res;
}

In user space, you would call:

    mach_vm_address_t address = 0;
    mach_vm_size_t size = 0;
    IOReturn res = IOConnectMapMemory64(connection, 0 /*memoryType*/, mach_task_self(), &address, &size, kIOMapAnywhere);

Some notes on this:

The value in the type parameter comes from the memoryType parameter to the IOConnectMapMemory64 call that caused this function to be called. Your driver therefore can have some kind of numbering convention; in the simplest case you can treat it similarly to the selector in external methods.
memory is effectively an output parameter and this is where you're expected to return the memory descriptor you want to map into user space when your function returns kIOReturnSuccess. The function has copy semantics, i.e. the caller expects to take ownership of the memory descriptor, i.e. it will eventually drop the reference count by 1 when it is no longer needed. The returned memory descriptor need not be an IOBufferMemoryDescriptor as I've used in the example, it can also be a PCI BAR or whatever.
The kIOMapAnywhere option in the IOConnectMapMemory64 call is important and normally what you want: if you don't specify this, the atAddress parameter becomes an in-out parameter, and the caller is expected to select a location in the address space where the driver memory should be mapped. Normally you don't care where this is, and indeed specifying an explicit location can be dangerous if there's already something mapped there.
If user space must not write to the mapped memory, set the options parameter to CopyClientMemoryForType accordingly: *options = kIOUserClientMemoryReadOnly;

To destroy the mapping, the user space process must call IOConnectUnmapMemory64().

How to allocate memory in a DriverKit system extension and map it to another process?

1 Answers1

Linked