Why does calling XChangeProperty from Rust and Glium generate a segfault?

Question

I'm trying to make an xlib call from Rust and I can't figure out why this code is giving me a segfault.

main.rs:

extern crate glium;
extern crate x11;

fn main() {
    let mut events_loop = glium::glutin::EventsLoop::new();
    let context = glium::glutin::ContextBuilder::new();
    let window = glium::glutin::WindowBuilder::new()
        .with_dimensions(800, 600)
        .with_title("Backr");
    let display = glium::Display::new(window, context, &events_loop).unwrap();

    unsafe {
        use x11::xlib;

        let x_display = display.gl_window().platform_display() as *mut xlib::Display;
        let x_window = display.gl_window().platform_window() as u64;
        let x_type = CStr::from_bytes_with_nul(b"_NET_WM_WINDOW_TYPE\0").unwrap();
        let x_value = CStr::from_bytes_with_nul(b"_NET_WM_WINDOW_TYPE_DESKTOP\0").unwrap();
        xlib::XChangeProperty(
            x_display,
            x_window,
            xlib::XInternAtom(x_display, x_type.as_ptr(), xlib::False),
            xlib::XA_ATOM,
            32,
            xlib::PropModeReplace,
            xlib::XInternAtom(x_display, x_type.as_ptr(), xlib::False) as *const u8,
            1,
        );
    }
}

Cargo.toml:

[package]
name = "rust-xlib"
version = "0.1.0"
authors = ["Me <me@me.me>"]

[dependencies]
glium = "*" 
x11 = { version = "*", features = ["xlib"] }

The segmentation fault, as reported by Valgrind, is:

Invalid read of size 8
   at 0x4E7DC40: _XData32 (in /usr/lib/libX11.so.6.3.0)
   by 0x4E58682: XChangeProperty (in /usr/lib/libX11.so.6.3.0)
   by 0x126898: rust-xlib::main (main.rs:17)
   by 0x362397: __rust_maybe_catch_panic (in /home/me/dev/rust-xlib/target/debug/rust-xlib)      
   by 0x34F847: std::panicking::try (in /home/me/dev/rust-xlib/target/debug/rust-xlib)
   by 0x34C325: std::rt::lang_start (in /home/me/dev/rust-xlib/target/debug/rust-xlib)
   by 0x1269F2: main (in /home/me/dev/rust-xlib/target/debug/rust-xlib)
 Address 0x188 is not stack'd, malloc'd or (recently) free'd


Process terminating with default action of signal 11 (SIGSEGV): dumping core
 Access not within mapped region at address 0x18A
   at 0x4E7DC40: _XData32 (in /usr/lib/libX11.so.6.3.0)
   by 0x4E58682: XChangeProperty (in /usr/lib/libX11.so.6.3.0)
   by 0x126898: rust-xlib::main (main.rs:17) 
   by 0x362397: __rust_maybe_catch_panic (in /home/me/dev/rust-xlib/target/debug/rust-xlib)      
   by 0x34F847: std::panicking::try (in /home/me/dev/rust-xlib/target/debug/rust-xlib)
   by 0x34C325: std::rt::lang_start (in /home/me/dev/rust-xlib/target/debug/rust-xlib)
   by 0x1269F2: main (in /home/me/dev/rust-xlib/target/debug/rust-xlib)

From the error message it seems like there is something wrong with the data parameter to XChangeProperty, but I don't really know what might be wrong with it.

The [second-to-last argument of `XChangeProperty`](https://docs.rs/x11/2.16.0/x11/xlib/fn.XChangeProperty.html) is a `char *` — why are you creating an `Atom` for it and then casting it? — Shepmaster, Oct 29 '17 at 15:40
@Shepmaster perhaps because the XChangeProperty man page very explicitly says you have to do just that (one needs to cast not the Atom itself but its *address* to `char*`, I'm not sure what exactly Rust's `as *const u8` does). — n. m. could be an AI, Oct 29 '17 at 21:23
Thanks! I was able to figure it out. As for why an atom for the second-to-last argument, I was going by an example I found here: https://stackoverflow.com/a/31362328/8852117 — Aylmoa, Oct 29 '17 at 21:35
When I try to pass the second-to-last argument as above, I still get a segfault though. — Aylmoa, Oct 29 '17 at 21:39
I see that you say that you figured it out and then edited your question. You also say it still segfaults. What is the current behavior of the posted code? — Shepmaster, Oct 30 '17 at 12:18
Yeah sorry, I was unclear. I figured out how to pass the C strings from Rust (I think), but I still get the same segfault. When I pass the data argument as you said to do (so just pass the atom by value), it doesn't segfault, but doesn't seem to produce the desired behaviour either. — Aylmoa, Oct 30 '17 at 15:36
I think you need to pass its address. In general you will have an array of those. The last argument is the length of the array (1 if passing a single element, but you still need to pass its address). — n. m. could be an AI, Oct 30 '17 at 18:49
It looks like `XInternAtom` would return a small-ish integer (like `0x188` or `0x18A`). If you are indeed casting that to a raw pointer, then the pointer's value will point to the memory address `0x188` or `0x18A`, leading to the errors. Try `let x_foo = xlib::XInternAtom(x_display, x_type.as_ptr(), xlib::False); xlib::XChangeProperty(..., &x_foo, 1);` as suggested by @n.m. (if that gives a type error, try `&x_foo as const u8`). — Shepmaster, Oct 31 '17 at 02:39
Alright, looks like it works now, but unfortunately the call doesn't have the effect I was hoping for (making the OpenGL window desktop background), so I can't be sure. Probably need to do some other thing I don't know about yet. I will post an answer with the code that I have. Thanks a lot in any case :) — Aylmoa, Oct 31 '17 at 10:59

score -1 · Answer 1 · edited Oct 31 '17 at 11:58

This should be the correct solution:

unsafe {
    use x11::xlib;

    let x_display = display.gl_window().platform_display() as *mut xlib::Display;
    let x_window = display.gl_window().platform_window() as u64;
    let x_type = CStr::from_bytes_with_nul(b"_NET_WM_WINDOW_TYPE\0").unwrap();
    let x_value = CStr::from_bytes_with_nul(b"_NET_WM_WINDOW_TYPE_DESKTOP\0").unwrap();
    let x_data = xlib::XInternAtom(x_display, x_value.as_ptr(), xlib::False);
    xlib::XChangeProperty(
        x_display,
        x_window,
        xlib::XInternAtom(x_display, x_type.as_ptr(), xlib::False),
        xlib::XA_ATOM,
        32,
        xlib::PropModeReplace,
        std::mem::transmute(&x_data),
        1,
    );
}

Please **do not** use `transmute`; there's almost never a reason to do so. Simple casts (maybe even 2 in a row) are always better. Your answer is very lacking in details — please describe what the changes are (so future readers don't have to diff the question and answer) and why they are needed. — Shepmaster, Oct 31 '17 at 11:59

Why does calling XChangeProperty from Rust and Glium generate a segfault?

1 Answers1