3

My code is allocating memory and never freeing it, even though it should (at least in my opinion).

The header looks like this:

typedef boost::asio::ssl::stream<boost::asio::ip::tcp::socket> sslSocket_t;

class Object {
    boost::asio::io_service ioService_;
    boost::asio::ip::tcp::acceptor acceptor_;
    boost::asio::ssl::context context_;

    void functionOne();
    void functionTwo(shared_ptr<sslSocket_t>& sslSocket, const boost::system::error_code& error)
}

And my source like this:

void Object::functionOne() {
    for (int i = 0; i < 10; i++) {
        shared_ptr<sslSocket_t> sslSocket(new sslSocket_t(ioService_, context_));
        acceptor_.async_accept(sslSocket->lowest_layer(),
                       boost::bind(&Object::functionTwo, this, sslSocket, boost::asio::placeholders::error));
    }
    acceptor_.cancel();

    boost::asio::io_service::work work(ioService_);
    ioService_.run();
}

void functionTwo(shared_ptr<sslSocket_t>& sslSocket, const boost::system::error_code& err) {
    // Do nothing
}

So when i call Object.functionOne(), memory is getting allocated to the Object.ioService_ object, in order to be able to call the bound asynchronous method. Then after the loop, all pending asynchronous actions on the acceptor are getting canceled. The appropriate handler is getting invoked as soon as Object.ioService_.run() is called (i've been testing that). BUT for some reason, the allocated memory does not get freed. So can someone please explain, why the memory is not getting deallocated and give me a hint how to free it?

Btw.: I'm working on debian and looking at /proc/self/status -> VmRSS to whatch the used memory.

@Vinnie Falco

#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/bind.hpp>

#include <iostream>
#include <memory>

typedef boost::asio::ssl::stream<boost::asio::ip::tcp::socket> sslSocket_t;

using namespace std;

struct T  {

    boost::asio::io_service ioService_;
    boost::asio::ip::tcp::acceptor acceptor_;
    boost::asio::ssl::context context_;

    void functionOne() {
        for (int i = 0; i < 10; i++) {
            shared_ptr<sslSocket_t> sslSocket(new sslSocket_t(ioService_, context_));
            acceptor_.async_accept(sslSocket->lowest_layer(),
                               boost::bind(&T::functionTwo, this, sslSocket, boost::asio::placeholders::error));
        }
        acceptor_.cancel();

        boost::asio::io_service::work work(ioService_);
        ioService_.run();
    }

    void functionTwo(shared_ptr<sslSocket_t>& sslSocket, const boost::system::error_code& err) {
        // Do nothing
    }

    T() : acceptor_(ioService_,
                    boost::asio::ip::tcp::endpoint(boost::asio::ip::tcp::v4(), 443)),
          context_(boost::asio::ssl::context::sslv23_server)  {

    }

    ~T()  {

    }
};

int main()  {

    try  {
        T t;

        t.functionOne();
    } catch (std::exception& e) {
        cout << "Exception: " << e.what() << endl;
  }
}

My question is not, if and why the destructor of T is called, this works as supposed to. But the behaviour concerning the used memory is strange. So if you increase the limit in the for loop, you will observe, that a lot of memory is getting reserved by the program, even though it should be released after all asnchronous handlers have been invoked. But the sslSocket objects are not getting deallocated, which is what my question is about: Why is the memory (specifically the memory allocated for the sslSocket), bound to the functor functionTwo, not deallocated, even after the asynchronous method fucntionTwo has been invoked and no reference to the sslSocket is left?

My final approach to explain my concern (edit 28 April)

Alright, i made a runnable example, that shows my concern: My Problem in an example

Output:

Before leaking call:     6984 kB
Asynchronous calls of functionTwo: 10000
Memory while ioService is still running:   460244 kB
Memory after ioService is stopped:   460244 kB

What's even more crazy is, that in my own local implementation I get the following output:

Memory leaking call:     8352 kB
Asynchronous calls of functionTwo: 10000
Memory while ioService is still running:   471932 kB
Memory after ioService is stopped:     8436 kB

So it can clearly be seen: the memory is not freed, even after all asynchronous operations have been invoked.

Summary and understood(?) behaviour (last edit)

As some of you might have missunderstood, i'm not thinking that there is some kind of a leak in my code. I named the structure in my code example Leak, which might have confused you, but my question is not if and where a memory leak occurs in my example. It's about the memory allocation in combination with the ioService object. First I thought, that the claimed memory is increasing infinitly. I made a last approach to understand this behaviour and came to the conclusion, that the memory management is fine. Memory is not reclaimed by the OS, but the program's memory allocation is converging to a limit, which is fine with me. So this problem is out of my way.

Example: Converging memory consumption

What disturbed me most was that my local implemention showed a slightly different behaviour. There the memory was reclaimed by the OS, when the ioService object was done with its jobs and reset, which satisfied my expectations.

So to sum up all observations:

The allocated memory is managed by the C++ Runtime and the OS. It's quite difficult (if not even impossible?) to directly observe the allocation procedure, since it's optimized to reduce the amount of requests for new memory pages, which means that allocated and freed memory might not be immidiatly reallocated by the OS.

To point out the critical point to me with this behaviour, i want to describe the usage of my program: I'm developing a server application, which means, that the program should run an infinit amount of time. If the program claims a lot of peak memory at some time, it's totaly fine, but it needs to release the claimed memory at some point in runtime, not after the runtime. So to me, there is only one question remaining:

Will the claimed (but unused) memory be reclaimed by the OS at some time? Or do I have to manage the memory on my own (with new and delete) in runtime?

Norman
  • 73
  • 7
  • how many connections are accepted by this server? – Nim Apr 28 '16 at 17:18
  • @Nim None. It's just about the consumed memory, which is simply done by constructing *sslSocket*'s and adding asnchronous operations. – Norman Apr 28 '16 at 17:24
  • Well - the handler is only executed once there is a connection, else it will simply hold on to the sockets until there is an accept - anyway - see my answer - your approach with asio and async operations is broken.. – Nim Apr 28 '16 at 17:26
  • @Nim The handler is executed since i call *acceptor.cancel()*, so all pending operations will get aborted as soon as *ioService_.run()* is getting called. And yes, i also implemented the typical aproach, my example was only ment to highlight my problem. Thank you already for your patience :-) – Norman Apr 28 '16 at 17:51
  • ..ahh - missed that, yes - it should get aborted. Can you wrap the ssl socket in a simple class and see if the destructor of that class is triggered. I would be extremely surprised if there is a dangling reference somewhere to that shared pointer... – Nim Apr 28 '16 at 17:57
  • @Nim Here is that program http://melpon.org/wandbox/permlink/k3vmCNBvNPxiM2h6 – Vinnie Falco Apr 28 '16 at 17:59
  • Well - that's clear then, the socket is destroyed as expected on the callback, further more, I'd be extremely surprised if any allocations for the handler weren't also cleaned up. I think you need to specify clearly how you are looking at the memory leak (If it is that - for example run it through valgrind to confirm for yourself..) as I don't think this is the case here.. – Nim Apr 28 '16 at 18:04
  • @VinnieFalco I'm sorry, it took me a while, but now i have a code example, that shows my problem:http://melpon.org/wandbox/permlink/S56tYueJySIHdsWO – Norman Apr 28 '16 at 18:43
  • @Nim The Code that Vinnie Falco posted is not mine, but his approach to reproduce my problem. Still appriciate his effort :-) – Norman Apr 28 '16 at 18:45
  • .. modify your code, add a line to print the memory usage *after* the try..catch block. You will see that the resident set size does not change even though the io_service has been destroyed. This has nothing to do with the asio - it's simple memory allocation in linux.. Run this program through valgrind to assure yourself that there is no leak... You should read about how memory allocation in linux works.. – Nim Apr 28 '16 at 18:52
  • @Nim According to http://stackoverflow.com/a/18555384/5000012 , the objects should at least get freed, when the ioService is destroyed. – Norman Apr 28 '16 at 19:31
  • there is a difference between objects being freed and the os reclaiming memory (which is what you are measuring) and the latter does not happen (not unless you do your own memory management with mapped files.) Memory will only be reclaimed by the os once the process is done.. – Nim Apr 28 '16 at 20:00

4 Answers4

1

I'm not sure what the problem is but I think you're doing something wrong. Could you provide a self-contained example that exhibits the problem? This sample program compiles and runs, and the destructor is called:

#include <boost/asio.hpp>
#include <functional>
#include <iostream>
#include <memory>

struct T
{
    T()
    {
        std::cerr << "T::T()\n";
    }

    ~T()
    {
        std::cerr << "T::~T()\n";
    }
};

void f(std::shared_ptr<T>&)
{
}

int main()
{
    using namespace boost::asio;
    io_service ios;
    ios.post(std::bind(&f, std::make_shared<T>()));
    ios.run();
}

You can see the output here: http://melpon.org/wandbox/permlink/0fkIAnoMXDOeedx7

The output:

T::T()
T::~T()
Vinnie Falco
  • 5,173
  • 28
  • 43
1

It took me a while, but i finally managed to work it out by myself.

So to clearify things, let's make sure, that the root of my problem is understood: I'm developing a server application, which is meant to run for an infinit amount of time. This application must be able to handle a lot of concurrent incomming connections. At some point in time, there may be a peak in load, leading to a lot of claimed memory to my application. Then after a while, most incomming requests have been processed, causing a lot of objects to be freed in runtime. Since the OS is in no need for memory (My application is the only huge memory consumer on the server), all freed memory stays with my application and can be reused at another point in time. This would be absolutely fine with me, but some customers and administrators might misinterpret the greater amount of constantly claimed memory as a memory leaking application. To avoid this, i wanted to hand some of the unused memory back to the OS manually. In my example, the bound handlers to the ioService (e.g. accepting a new connection) would be freed in runtime, but the appropriate memory won't be reclaimed by the OS. So to do this manually, i found the following solution:

Release unused heap memory under Linux in C/C++: int malloc_trim(size_t pad)

The documentation can be found here. Simplified explained, this method releases unused memory from the heap to the OS, which is exactly what i've been searching for. I'm aware that under memory optimization aspects, the manual use of this function maybe dangerous, but since i only want to release the memory every few minutes, this performance issue is acceptable to me.

Thank you all for your efforts and patience!

Norman
  • 73
  • 7
0

Taking your selfcontained example and running it under valgrind shows that exactly nothing is leaked

==14098== 
==14098== HEAP SUMMARY:
==14098==     in use at exit: 73,696 bytes in 7 blocks
==14098==   total heap usage: 163,524 allocs, 163,517 frees, 733,133,505 bytes allocated
==14098== 
==14098== LEAK SUMMARY:
==14098==    definitely lost: 0 bytes in 0 blocks
==14098==    indirectly lost: 0 bytes in 0 blocks
==14098==      possibly lost: 0 bytes in 0 blocks
==14098==    still reachable: 73,696 bytes in 7 blocks
==14098==         suppressed: 0 bytes in 0 blocks
==14098== Rerun with --leak-check=full to see details of leaked memory

If you supply valgrind --show-leak-kinds=all --leak-check=full ./test you'll find that the "leaked" (left-over reachables) are the usual static stuff allocated from libssl/libcrypto. They get allocated even if you only do 1 iteration. No change for 10000 iterations.

==14214== Memcheck, a memory error detector
==14214== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==14214== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==14214== Command: ./test 10000
==14214== 
Before leaking call:    50056 kB
Asynchronous calls of functionTwo: 10000
Memory while ioService is still running:   265592 kB
Memory after ioService is stopped:   265592 kB
==14214== 
==14214== HEAP SUMMARY:
==14214==     in use at exit: 73,696 bytes in 7 blocks
==14214==   total heap usage: 163,524 allocs, 163,517 frees, 733,133,505 bytes allocated
==14214== 
==14214== 24 bytes in 1 blocks are still reachable in loss record 1 of 7
==14214==    at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214==    by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53BF315: lh_insert (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C1863: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C245D: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x40C9CA: context (context.ipp:70)
==14214==    by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214==    by 0x403E13: main (test.cpp:86)
==14214== 
==14214== 32 bytes in 1 blocks are still reachable in loss record 2 of 7
==14214==    at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214==    by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53BE7AE: sk_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x507FD69: ??? (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214==    by 0x5081E68: SSL_COMP_get_compression_methods (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214==    by 0x5087532: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214==    by 0x40B9A8: do_init (openssl_init.ipp:39)
==14214==    by 0x40B9A8: boost::asio::ssl::detail::openssl_init_base::instance() (openssl_init.ipp:131)
==14214==    by 0x403C3C: openssl_init (openssl_init.hpp:60)
==14214==    by 0x403C3C: __static_initialization_and_destruction_0 (openssl_init.hpp:90)
==14214==    by 0x403C3C: _GLOBAL__sub_I_count (test.cpp:96)
==14214==    by 0x40FE1C: __libc_csu_init (in /home/sehe/Projects/stackoverflow/test)
==14214==    by 0x5EC09CE: (below main) (libc-start.c:245)
==14214== 
==14214== 32 bytes in 1 blocks are still reachable in loss record 3 of 7
==14214==    at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214==    by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53BE7CC: sk_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x507FD69: ??? (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214==    by 0x5081E68: SSL_COMP_get_compression_methods (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214==    by 0x5087532: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214==    by 0x40B9A8: do_init (openssl_init.ipp:39)
==14214==    by 0x40B9A8: boost::asio::ssl::detail::openssl_init_base::instance() (openssl_init.ipp:131)
==14214==    by 0x403C3C: openssl_init (openssl_init.hpp:60)
==14214==    by 0x403C3C: __static_initialization_and_destruction_0 (openssl_init.hpp:90)
==14214==    by 0x403C3C: _GLOBAL__sub_I_count (test.cpp:96)
==14214==    by 0x40FE1C: __libc_csu_init (in /home/sehe/Projects/stackoverflow/test)
==14214==    by 0x5EC09CE: (below main) (libc-start.c:245)
==14214== 
==14214== 128 bytes in 1 blocks are still reachable in loss record 4 of 7
==14214==    at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214==    by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53BEFE1: lh_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C1512: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C182F: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C245D: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x40C9CA: context (context.ipp:70)
==14214==    by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214==    by 0x403E13: main (test.cpp:86)
==14214== 
==14214== 176 bytes in 1 blocks are still reachable in loss record 5 of 7
==14214==    at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214==    by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53BEFBF: lh_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C1512: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C182F: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C245D: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x40C9CA: context (context.ipp:70)
==14214==    by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214==    by 0x403E13: main (test.cpp:86)
==14214== 
==14214== 600 bytes in 1 blocks are still reachable in loss record 6 of 7
==14214==    at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214==    by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C23F5: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214==    by 0x40C9CA: context (context.ipp:70)
==14214==    by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214==    by 0x403E13: main (test.cpp:86)
==14214== 
==14214== 72,704 bytes in 1 blocks are still reachable in loss record 7 of 7
==14214==    at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214==    by 0x57731FF: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14214==    by 0x4010609: call_init.part.0 (dl-init.c:72)
==14214==    by 0x401071A: call_init (dl-init.c:30)
==14214==    by 0x401071A: _dl_init (dl-init.c:120)
==14214==    by 0x4000D09: ??? (in /lib/x86_64-linux-gnu/ld-2.21.so)
==14214==    by 0x1: ???
==14214==    by 0xFFEFFFF76: ???
==14214==    by 0xFFEFFFF7D: ???
==14214== 
==14214== LEAK SUMMARY:
==14214==    definitely lost: 0 bytes in 0 blocks
==14214==    indirectly lost: 0 bytes in 0 blocks
==14214==      possibly lost: 0 bytes in 0 blocks
==14214==    still reachable: 73,696 bytes in 7 blocks
==14214==         suppressed: 0 bytes in 0 blocks
==14214== 
==14214== For counts of detected and suppressed errors, rerun with: -v
==14214== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Your way of measuring memory usage is not sound.

sehe
  • 374,641
  • 47
  • 450
  • 633
  • Thanks for your effort, but as I mention in my final summary, i'm not looking for some kind of leak, but i'm trying to understand how the memory allocation and deallocation in combination with the *ioService* object works. – Norman May 10 '16 at 11:47
-1

I think your approach is broken. You should never interleave async operations with asio. If your do all sorts of undefined crap will happen. The way you normally implement async accept is as follows:

void do_accept() {
  shared_ptr<sslSocket_t> socket(new sslSocket_t(ioService_, context_));
  // Queue an async accept operation
  acceptor_.async_accept(socket->lowest_layer(), [this, socket](auto ec) {
    if (!ec) {
      // Handle the socket
    }
    // If not shutting down
    this->do_accept(); // next accept
  });
}
Nim
  • 33,299
  • 2
  • 62
  • 101
  • I wasn't the one that -1'd you, but I'd say your bold statement that you should never interleave async operations is not correct. You can interleave them all you like; it's just that the order in which they are executed is not guaranteed (and, indeed, could be concurrent). In some designs, this might be perfectly fine. Others, not so much. – Kaz Dragon May 10 '16 at 11:49
  • @KazDragon - on one socket, I'm not sure I agree with you when the documentation says the following (for `async_read` for example): "This operation is implemented in terms of zero or more calls to the stream's async_read_some function, and is known as a composed operation. The program must ensure that the stream performs no other read operations (such as async_read, the stream's async_read_some function, or any other composed operations that perform reads) until this operation completes. " – Nim May 10 '16 at 14:11
  • @KazDragon, I'd say for `accept` it's not clear from the documentation whether it's safe to interleave, but for other operations (e.g. write, same principle applies.) You can however mix read/write operations - but that's a given.. – Nim May 10 '16 at 14:16