Can I cast shared_ptr & to shared_ptr & without changing use_count?

Question

I have a program that uses boost::shared_ptrs and, in particular, relies on the accuracy of the use_count to perform optimizations.

For instance, imagine an addition operation with two argument pointers called lhs and rhs. Say they both have the type shared_ptr<Node>. When it comes time to perform the addition, I'll check the use_count, and if I find that one of the arguments has a reference count of exactly one, then I'll reuse it to perform the operation in place. If neither argument can be reused, I must allocate a new data buffer and perform the operation out-of-place. I'm dealing with enormous data structures, so the in-place optimization is very beneficial.

Because of this, I can never copy the shared_ptrs without reason, i.e., every function takes the shared_ptrs by reference or const reference to avoid distorting use_count.

My question is this: I sometimes have a shared_ptr<T> & that I want to cast to shared_ptr<T const> &, but how can I do it without distorting the use count? static_pointer_cast returns a new object rather than a reference. I'd be inclined to think that it would work to just cast the whole shared_ptr, as in:

void f(shared_ptr<T> & x)
{
  shared_ptr<T const> & x_ = *reinterpret_cast<shared_ptr<T const> *>(&x);
}

I highly doubt this complies with the standard, but, as I said, it will probably work. Is there a way to do this that's guaranteed safe and correct?

Updating to Focus the Question

Critiquing the design does not help answer this post. There are two interesting questions to consider:

1. Is there any guarantee (by the writer of boost::shared_ptr, or by the standard, in the case of std::tr1::shared_ptr) that shared_ptr<T> and shared_ptr<T const> have identical layouts and behavior?

2. If (1) is true, then is the above a legal use of reinterpret_cast? I think you would be hard-pressed to find a compiler that generates failing code for the above example, but that doesn't mean it's legal. Whatever your answer, can you find support for it in the C++ standard?

Answer 1

I sometimes have a shared_ptr<T> & that I want to cast to shared_ptr<T const> &, but how can I do it without distorting the use count?

You don't. The very concept is wrong. Consider what happens with a naked pointer T* and const T*. When you cast your T* into a const T*, you now have two pointers. You don't have two references to the same pointer; you have two pointers.

Why should this be different for smart pointers? You have two pointers: one to a T, and one to a const T. They're both sharing ownership of the same object, so you are using two of them. Your use_count therefore ought to be 2, not 1.

Your problem is your attempt to overload the meaning of use_count, co-opting its functionality for some other purpose. In short: you're doing it wrong.

Your description of what you do with shared_ptrs who's use_count is one is... frightening. You're basically saying that certain functions co-opt one of its arguments, which the caller is clearly using (since the caller obviously is still using it). And the caller doesn't know which one was claimed (if any), so the caller has no idea what the state of the arguments is after the function. Modifying the arguments for operations like that is usually not a good idea.

Plus, what you're doing can only work if you pass shared_ptr<T> by reference, which itself isn't a good idea (like regular pointers, smart pointers should almost always be taken by value).

In short, you're taking a very commonly used object with well-defined idioms and semantics, then requiring that it be used in a way that they are almost never used, with specialized semantics that work counter to the way everyone actually uses them. That's not a good thing.

You have effectively created the concept of co-optable pointer, a shared pointer that can be in 3 use states: empty, in use by the person who gave it to you only and thus you can steal from it, and in use by more than one person so you can't have it. It's not the semantics that shared_ptr exists to support. So you should write your own smart pointer that provides these semantics in a much more natural way.

Something that recognizes the difference between how many instances of a pointer you have around and how many actual users of it you have. That way, you can pass it around by value properly, but you have some way of saying that you are currently using it and don't want one of these other functions to claim it. It could use shared_ptr internally, but it should provide its own semantics.

Answer 2

static_pointer_cast is the right tool for the job — you've already identified that.

The problem with it isn't that it returns a new object, but rather that it leaves the old object unchanged. You want to get rid of the non-const pointer and move on with the const pointer. What you really want is static_pointer_cast< T const >( std::move( old_ptr ) ). But there isn't an overload for rvalue references.

The workaround is simple: manually invalidate the old pointer just as std::move would.

auto my_const_pointer = static_pointer_cast< T const >( modifiable_pointer );
modifiable_pointer = nullptr;

It might be slightly slower than reinterpret_cast, but it's a lot more likely to work. Don't underestimate how complex the library implementation is, and how it can fail when abused.

An aside: use pointer.unique() instead of use_count() == 1. Some implementations might use a linked list with no cached use count, making use_count() O(N) whereas the unique test remains O(1). The Standard recommends unique for copy on write optimization.

EDIT: Now I see you mention

I can never copy the shared_ptrs without reason, i.e., every function takes the shared_ptrs by reference or const reference to avoid distorting use_count.

This is Doing It Wrong. You've added another layer of ownership semantics atop what shared_ptr already does. They should be passed by value, with std::move used where the caller no longer desires ownership. If the profiler says you're spending time adjusting reference counts, then you might add some references-to-pointer in the inner loops. But as a general rule, if you can't set a pointer to nullptr because you're no longer using it, but someone else might be, then you've really lost track of ownership.

Answer 3

If you cast a shared_ptr to a different type, without changing the reference count, this implies that you'll now have two pointers to the same data. Hence, unless you erase the old pointer, you can't do this with shared_ptrs without "distorting the reference count".

I would suggest that you use raw pointers here instead, rather than going out of your way to not use the features of shared_ptrs. If you need to sometimes create new references, use enable_shared_from_this to derive a new shared_ptr to an existing raw pointer.

Answer 4

When it comes time to perform the addition, I'll check the use_count, and if I find that one of the arguments has a reference count of exactly one, then I'll reuse it to perform the operation in place.

This isn't necessarily valid unless you're applying some other rules across the whole program to make it so. Consider:

shared_ptr<Node> add(shared_ptr<Node> const &lhs,shared_ptr<Node> const &rhs) {
    if(lhs.use_count()==1) {
        // do whatever, reusing lhs
        return lhs;
    }
    if(rhs.use_count()==1) {
        // do whatever, reusing rhs
        return rhs;
    }
    shared_ptr<Node> new_node = ... // do whatever without reusing lhs or rhs
    return new_node;
}

void foo() {
    shared_ptr<Node> a,b;
    shared_ptr<Node> c = add(a,b);

    // error, we still have a and b, and expect that they're unchanged! they could have been modified!
}

Instead if you take the smart pointers by value:

shared_ptr<Node> add(shared_ptr<Node> lhs,shared_ptr<Node> rhs) {

And the use_count()==1 then you know that your copy is the only one and it should be safe to reuse it.

However, there's a problem in using this as an optimization, because copying a shared_ptr requires synchronization. It could well be that doing all this synchronization all over the place costs far more than you save by reusing existing shared_ptrs. All this synchronization is the reason it's recommended that code that does not take ownership of a shared_ptr should take the shared_ptr by reference instead of by value.