How to iterate over one of two iterators from a generic function pointer (without a Box)

Question

This question is very similar to this question and this question but I hope is different (and, if not, sorry for wasting your time!).

I (think) I would like to write some code like this:

type IterCreator = fn(n: usize) -> /* some usize iterator */;

struct Walker {
  creator: IterCreator,
}
impl Walker {
    fn walk(&self, ns: &[usize]) {
        ns.map(|&n| (self.creator)(n)).for_each(|iter| {
            // ...
        });
    }
}

fn do_walks(use_iter_2: bool, ns: &[usize]) {
    let creator = if use_iter_2 {
        create_iter_2
    } else {
        create_iter_1
    };
    let walker = Walker { creator };
    walker.walk(ns);
}

I can get this to work with Box:

fn create_iter_1(n: usize) -> Box<dyn Iterator<Item = usize>> {
    Box::new(Iter1(n))
}
fn create_iter_2(n: usize) -> Box<dyn Iterator<Item = usize>> {
    Box::new(Iter2(n))
}

type IterCreator = fn(n: usize) -> Box<dyn Iterator<Item = usize>>;

struct Walker {
  creator: IterCreator,
}

But the dynamic dispatch is "slow". In small benchmarks it's 100x slower than a solution with generics:

fn create_iter_1(n: usize) -> Iter1 {
    Iter1(n)
}
fn create_iter_2(n: usize) -> Iter2 {
    Iter2(n)
}
type IterCreator<I> = fn(n: usize) -> I;

struct Walker<I: Iterator<Item = usize>> {
  creator: IterCreator<I>,
}

However I now have to duplicate the logic in the conditional!

fn do_walks(use_iter_2: bool, ns: &[usize]) {
    if use_iter_2 {
        let walker = Walker { creator: create_iter_2 };
        walker.walk(ns);
    } else {
        // ...
    }
}

If I don't, I get:

// ...
    let creator = if use_iter_2 {
        create_iter_2
    } else {
        create_iter_1
    };
    // `if` and `else` have incompatible types

Trait objects don't seem to work (although I assume even if they did, it's still dynamic dispatch?):

type StrategyCreator = fn(n: usize) -> dyn Iterator<Item = usize>;

// ...
    let creator = if use_iter_2 {
        create_iter_2 as IterCreator
    } else {
        create_iter_1 as IterCreator
    };
    // non-primitive cast!

And having the type alias return an impl is unstable. I feel like this "can't be done" because the compiler doesn't know how big creator is unless

1. it can nail a specific type to it (which it can't in the generic case)
2. it's a pointer to a dynamic object that implements a trait (which introduces the performance hit)

But I feel like when I think something "can't be done" I'm usually wrong :-) - is there a way to do this?