Returning non-trivial gstreamer "pad callbacks" as boxed closures

Question

I'm trying to write a factory function for the creation of closures for use as 'pad callbacks' in gstreamer. I've provided a stripped down example that should compile with the gstreamer crate and gstreamer binaries/plugins installed.

Through my research, I've gotten the factory function to work by using the 'impl trait' method, instead of boxing. I'd like to figure out the boxed method though, as it seems more appropriate in some situations.

This is as close as I've gotten. The problem can be seen by uncommenting the portion labeled Closure function using 'Box<>'. I've tried specifying the Fn portion as a type parameter with a where clause, as well as many other attempts. In this attempt, it looks like the issue is that I can't unbox the closure function to use as assignment to a local variable, or as use in the add_probe callback due to requiring compile-time size, which is the whole reason for the box in the first place...

Ctrl+C or 'exit\n' from stdin should close the program.

extern crate gstreamer as gst;

use gst::prelude::*;
use std::io;

fn create_impl_probe_fn(
    x: i32,
) -> impl Fn(&gst::Pad, &mut gst::PadProbeInfo) -> gst::PadProbeReturn + Send + Sync + 'static {
    move |_, _| {
        println!("Idle... {}", x);

        gst::PadProbeReturn::Pass
    }
}

fn create_boxed_probe_fn(
    x: i32,
) -> Box<Fn(&gst::Pad, &mut gst::PadProbeInfo) -> gst::PadProbeReturn + Send + Sync + 'static> {
    Box::new(move |_, _| {
        println!("Idle... {}", x);

        gst::PadProbeReturn::Pass
    })
}

fn main() {
    println!("Starting...");
    //TODO Pass args to gst?
    gst::init().unwrap();

    //GStreamer
    let parse_line = "videotestsrc ! autovideosink name=mysink";

    let pipeline = gst::parse_launch(parse_line).unwrap();
    let ret = pipeline.set_state(gst::State::Playing);
    assert_ne!(ret, gst::StateChangeReturn::Failure);

    //Inline closure
    let mut x = 1;
    pipeline
        .clone()
        .dynamic_cast::<gst::Bin>()
        .unwrap()
        .get_by_name("mysink")
        .unwrap()
        .get_static_pad("sink")
        .unwrap()
        .add_probe(gst::PadProbeType::BLOCK, move |_, _| {
            println!("Idle... {}", x);

            gst::PadProbeReturn::Pass
        });

    //Closure function using 'impl'
    x = 20;
    let impl_probe_fn = create_impl_probe_fn(x);
    //TEMP Test
    pipeline
        .clone()
        .dynamic_cast::<gst::Bin>()
        .unwrap()
        .get_by_name("mysink")
        .unwrap()
        .get_static_pad("sink")
        .unwrap()
        .add_probe(gst::PadProbeType::BLOCK, impl_probe_fn);

    /*
    //Closure function using 'Box<>'
    x = 300;
    let boxed_probe_fn = create_boxed_probe_fn(x);
    //TEMP Test
    pipeline
        .clone()
        .dynamic_cast::<gst::Bin>()
        .unwrap()
        .get_by_name("mysink")
        .unwrap()
        .get_static_pad("sink")
        .unwrap()
        .add_probe(gst::PadProbeType::BLOCK, *boxed_probe_fn);
    */

    //Input Loop
    loop {
        let mut input = String::new();
        io::stdin().read_line(&mut input).unwrap();

        match input.trim() {
            "exit" => break,
            "info" => {
                let (state_change_return, cur_state, old_state) =
                    pipeline.get_state(gst::CLOCK_TIME_NONE);
                println!(
                    "Pipeline Info: {:?} {:?} {:?}",
                    state_change_return, cur_state, old_state
                );
            }
            "pause" => {
                let _ = pipeline.set_state(gst::State::Paused);
                println!("Pausing");
            }
            "resume" => {
                let _ = pipeline.set_state(gst::State::Playing);
                println!("Resuming");
            }
            _ => println!("Unrecognized command: '{}'", input.trim()),
        }

        println!("You've entered: {}", input.trim());
    }

    //Shutdown
    let ret = pipeline.set_state(gst::State::Null);
    assert_ne!(ret, gst::StateChangeReturn::Failure);
    println!("Shutting down streamer");
}

I know several similar questions exist on the net and here at SO, but I can't seem to figure how to apply any of the solutions to this specific function. I've included "non-trivial" and "gstreamer" in the title to differentiate.

[EDIT] Sorry, here's a bit more info... just didn't want to muddy the water or complicate the issue...

I can't post everything I've tried. It's well over 10+ hours of small changes/attempts and many tabs. I can reproduce a few attempts that seemed close, or that I expected to work.

The above Box attempt is how I figured it would work based on the information here: https://doc.rust-lang.org/1.4.0/book/closures.html

https://doc.rust-lang.org/book/second-edition/ch19-05-advanced-functions-and-closures.html (This one doesn't close over any stack values, and so doesn't have the 'move'.)

Rust closures from factory functions (More that makes me feel like what I have should work...)

Box<> section of the rust book: https://doc.rust-lang.org/book/second-edition/ch15-01-box.html

Here's the add_probe signature: https://sdroege.github.io/rustdoc/gstreamer/gstreamer/trait.PadExtManual.html#tymethod.add_probe

Here's the error from the above (with offending add_probe call uncommented):

error[E0277]: the trait bound `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync: std::marker::Sized` is not satisfied
  --> src/main.rs:63:14
   |
63 |             .add_probe(gst::PadProbeType::BLOCK, *boxed_probe_fn);
   |              ^^^^^^^^^ `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync` does not have a constant size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync`

So, I guess since the size of the closure isn't known at compile-time, I can't pass it as an argument?

Changing the dereference to be on the assignement line above the '.add_probe' gives a similar error:

error[E0277]: the trait bound `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync: std::marker::Sized` is not satisfied
  --> src/main.rs:57:13
   |
57 |         let boxed_probe_fn = *create_boxed_probe_fn(x);
   |             ^^^^^^^^^^^^^^ `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync` does not have a constant size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync`
   = note: all local variables must have a statically known size

error[E0277]: the trait bound `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync: std::marker::Sized` is not satisfied
  --> src/main.rs:63:14
   |
63 |             .add_probe(gst::PadProbeType::BLOCK, boxed_probe_fn);
   |              ^^^^^^^^^ `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync` does not have a constant size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `for<'r, 's, 't0> std::ops::Fn(&'r gst::Pad, &'s mut gst::PadProbeInfo<'t0>) -> gst::PadProbeReturn + std::marker::Send + std::marker::Sync`

I understand the need for a stack based binding to need a compile-time size.... so this almost feels imposible to do unless the add_probe function itself took a Boxed<> arguement?

Onto some more attempts. Several places, including the add_probe function signature itself use a Type parameter and the 'where' clause to specify the Fn trait.

add_probe declaration: https://github.com/sdroege/gstreamer-rs/blob/db3fe694154c697afdaf3efb6ec65332546942e0/gstreamer/src/pad.rs

Post recomending using the 'where' clause: Sized is not implemented for the type Fn

So, let's try that, changing the create_boxed_probe_fn to:

fn create_boxed_probe_fn<F>(x: i32) -> Box<F>
    where F: Fn(&gst::Pad, &mut gst::PadProbeInfo) -> gst::PadProbeReturn + Send + Sync + 'static {
    Box::new(move |_, _| {
        println!("Idle... {}", x);

        gst::PadProbeReturn::Pass
    })
}

Error:

error[E0308]: mismatched types
  --> src/main.rs:15:18
   |
15 |           Box::new(move |_, _| {
   |  __________________^
16 | |             println!("Idle... {}", x);
17 | |
18 | |             gst::PadProbeReturn::Pass
19 | |         })
   | |_________^ expected type parameter, found closure
   |
   = note: expected type `F`
              found type `[closure@src/main.rs:15:18: 19:10 x:_]`

This seems to be because we've specified the type above, but a closure ofcourse is it's own type. Trying the following doesn't work, as it's a trait, and can't be cast with 'as':

fn create_boxed_probe_fn<F>(x: i32) -> Box<F>
    where F: Fn(&gst::Pad, &mut gst::PadProbeInfo) -> gst::PadProbeReturn + Send + Sync + 'static {
    Box::new(move |_, _| {
        println!("Idle... {}", x);

        gst::PadProbeReturn::Pass
    } as F)
}

Error:

error[E0308]: mismatched types
  --> src/main.rs:15:18
   |
15 |           Box::new(move |_, _| {
   |  __________________^
16 | |             println!("Idle... {}", x);
17 | |
18 | |             gst::PadProbeReturn::Pass
19 | |         } as F)
   | |______________^ expected type parameter, found closure
   |
   = note: expected type `F`
              found type `[closure@src/main.rs:15:18: 19:15 x:_]`

error[E0605]: non-primitive cast: `gst::PadProbeReturn` as `F`
  --> src/main.rs:15:30
   |
15 |           Box::new(move |_, _| {
   |  ______________________________^
16 | |             println!("Idle... {}", x);
17 | |
18 | |             gst::PadProbeReturn::Pass
19 | |         } as F)
   | |______________^
   |
   = note: an `as` expression can only be used to convert between primitive types. Consider using the `From` trait

It mentions the 'From' trait. I've not looked into that, because implimenting traits for a closure doesn't seem right. I'm not even sure it's possible?

I also tried what they seem to call type ascription (Instead of 'as F' using ':F'), but this seem unsupported at this time: https://github.com/rust-lang/rust/issues/23416

This guy had the same issue, but it seems like his solution was to not use a type parameter and instead specify the Fn portion without the where clause. (Which is my top non-working attempt.) Not entirely sure, since he doesn't post what he did to fix it. https://github.com/rust-lang/rust/issues/51154

No amount of adding the impl keyword to any of the box versions seems to help. The syntax for using it like I do in the unboxed "working" version seems new and I haven't found great documentation on it. Here's some info on it: https://github.com/rust-lang/rfcs/blob/master/text/1522-conservative-impl-trait.md

More links that are related:

How do I store a closure in Rust?

Closure in the return type for a Rust function

expected trait core::ops::FnMut, found type parameter

https://doc.rust-lang.org/std/boxed/trait.FnBox.html

Answer 1

Since Rust 1.35

Box<dyn Fn(...)> implements Fn(...), so you can simply pass the Box directly to add_probe:

        .add_probe(gst::PadProbeType::BLOCK, boxed_probe_fn);

Original answer

This problem can be reduced to a surprisingly concise example:

fn call<F: Fn()>(f: F) {
    f();
}

fn main() {
    let g = || ();                            // closure that takes nothing and does nothing
    let h = Box::new(|| ()) as Box<dyn Fn()>; // that but as a Fn() trait object
    call(g); // works
    call(h); // fails
}

The heart of the problem is that Box<dyn Fn()> does not implement Fn(). There's no good reason this doesn't work, but there are a collection of factors that make it awkward to fix:

1. It's not possible to call a method that takes self by value on a trait object. This makes it not possible to call a Box<dyn FnOnce()>. The current workaround is to use Box<dyn FnBox>, which does implement FnOnce() (but this does not directly apply to your situation or the example above, since you want to use Fn).
2. Despite this, it may one day become possible to call a Box<dyn FnOnce()>, so FnBox is in a sort of Limbo where people don't want to fix or stabilize it to work around a temporary issue.
3. Adding an impl to core to make Fn() work may nevertheless conflict with FnBox in ways that I don't quite grasp. There are several comments about this on issue #28796.

It's possible that implementing Fn() for Box<dyn Fn()> just can't be done in the language as is. It's also possible that it could be done, but is a bad idea for forwards compatibility reasons; and it's also possible that it could be done and is a good idea, but nobody has done it yet. That said, with things as they are now, you have a couple of mostly unpleasant options.

As someone suggested in the question comments, you could make your own wrapper struct that wraps a closure, and implement Fn() for Box<Wrapper<F>>.

You could make your own trait ProbeFn, which is implemented for any closure of the correct type, and implement Fn() for Box<dyn ProbeFn>.

In some cases, you may be able to use a &dyn Fn() instead of a Box<dyn Fn()>. This works in the above example:

    call(&*h);

Unlike Box<dyn Fn()>, &dyn Fn() does implement Fn(). It's not as general, though, because obviously it doesn't have ownership. However, it does work on the stable compiler -- implementing Fn() yourself requires unstable.