6

I have a Linux service (daemon) that has multiple-threads and uses boost io_service listening on a TCP socket. When I receive a certain message on that socket I want to start another service with e.g. /etc/init.d/corosync start.

The problem is that after starting the service, when I quit my own service, the other service has inherited the sockets from my own service and it remains in a strange status where I cannot stop it the usual way.

Before exiting my process "MonitorSipServer" the open socket shows like this:

netstat -anop |grep 144
tcp        0      0 0.0.0.0:20144           0.0.0.0:*               LISTEN          4480/MonitorSipServ off (0.00/0/0)
tcp        0      0 140.0.24.181:20144      140.0.101.75:47036      ESTABLISHED 4480/MonitorSipServ off (0.00/0/0)

After exiting my process "MonitorSipServer" the open socket shows like this:

netstat -anop |grep 144
tcp        0      0 0.0.0.0:20144           0.0.0.0:*               LISTEN      4502/corosync    off (0.00/0/0)
tcp        0      0 140.0.24.181:20144      140.0.101.75:47036      ESTABLISHED 4502/corosync    off (0.00/0/0)

I have already tried with system, popen and with fork + execv or execve with null environment. It's always the same result or worse. My last hope was the Linux setsid command, but it has not worked either.

Any help would be appreciated. Regards, Jan

Tanner Sansbury
  • 51,153
  • 9
  • 112
  • 169
JanCG
  • 105
  • 1
  • 6

3 Answers3

7

If you're referring to the socket descriptors themselves being inherited by exec'd child processes, and this being undesirable, then you could pass SOCK_CLOEXEC when creating the sockets with socket(2) to make sure that they are closed when you execute other programs. (This won't close the connection by the way, since your program still has a reference to the socket.)

If you are using some higher-level library, then check if there is some way to get it to pass this flag, or do fcntl(sock_fd, F_SETFD, fcntl(sock_fd, F_GETFD) | FD_CLOEXEC) to set the close-on-exec flag on the descriptor after it is created if you can get to it. (The fcntl(2) approach can be racy in a multi-threaded environment though, since some thread could exec(3) a program between the point where the socket is created and when FD_CLOEXEC is set on it.)

If the above won't work, then you could manually fork(2) and then close(2) the socket descriptors before executing a service. The advantage of SOCK_CLOEXEC is that the sockets will only be closed if the exec*() is actually successful, which sometimes makes it easier to recover from errors. In addition, SOCK_CLOEXEC can avoid some races and makes it harder to forget to close descriptors.

Ulfalizer
  • 4,664
  • 1
  • 21
  • 30
  • 2
    Note also that in many cases you don't have control over all file descriptors and whether they're created with SOCK_CLOEXEC, in which case you can run a loop that close() all descriptors up to sysconf(_SC_OPEN_MAX), after fork() (possibly start at fd 3 if you want to leave stdin/out/err alone, or redirect those to /dev/null) – nos Mar 06 '15 at 14:10
  • 1
    Currently, not all file descriptors can be created atomically with `FD_CLOEXEC` set. In a multi-threaded application another thread may fork a process after a file descriptor was created but before `FD_CLOEXEC` was set. Hence, the child process may accidentally inherit file descriptors. – Maxim Egorushkin Mar 06 '15 at 14:26
  • 1
    http://stackoverflow.com/questions/899038/getting-the-highest-allocated-file-descriptor/ provides some additional insight re. closing descriptors to avoid inheritance. – Ulfalizer Mar 06 '15 at 14:56
  • Thanks. This helped a lot. For `boost::asio::ip::tcp::acceptor` I had to use the `native_handler()` as file descriptor for `fcntl`. As you mentioned, the established connections keep open inspite of setting `CLOEXEC` to `boost::asio::ip::tcp::socket`. So I have to close them manually. I still have a problem with some blocked signals in the child process, but I think I will open another thread for this. – JanCG Mar 09 '15 at 15:22
  • @jgaida seems you failed to create that question, but the answer is likely http://www.boost.org/doc/libs/1_63_0/doc/html/boost_asio.html#boost_asio.overview.posix.fork – sehe Mar 28 '17 at 12:10
3

Boost.Asio supports the fork() system call:

  • It requires the program to prepare and notify the io_service of the fork with io_service::notify_fork():

    io_service_.notify_fork(boost::asio::io_service::fork_prepare);
if (fork() == 0)
{
  io_service_.notify_fork(boost::asio::io_service::fork_child);
  ...
}
else
{
  io_service_.notify_fork(boost::asio::io_service::fork_parent);
  ...
}

    Failing to use this pattern results in unspecified behavior. For some configurations, the parent will fail to receive event notifications, as they are consumed by the child.

  • It is the program's responsibility to handle any file descriptors accessible through Boost.Asio's public API. For example, if the parent has an open acceptor, then the child would need to explicitly invoke acceptor::close() before spawning its own children processes or replacing the process image. The fork support documentation states:

    Note that any file descriptors accessible via Boost.Asio's public API (e.g. the descriptors underlying basic_socket<>, posix::stream_descriptor, etc.) are not altered during a fork. It is the program's responsibility to manage these as required.

  • Boost.Asio's fork support is not safe for multi-threaded processes. For multi-threaded processes, fork() states that the child may only invoke async-signal-safe operations between fork() and one of the exec() functions. io_service::notify_fork() does not make this guarantee, and the current implementation invokes non async-signal-safe operations.

With that said, I have seen applications not observe undesirable behavior when using Boost.Asio's fork() support in a multi-threaded process. However, if one wishes to meet the requirements of both fork() and Boost.Asio, then one solution is to fork the daemon process when it is still single threaded. The child process will remain single threaded and perform fork() and exec() when it is told to do so by the parent process via inter-process communication. This solution is suggested and demonstrated in this answer.

Community
  • 1
  • 1
Tanner Sansbury
  • 51,153
  • 9
  • 112
  • 169
  • 1
    Thank you @tanner-sansbury, I used the `notify_fork` as you said because I think it is the correct way. Although, for the sockets there is no difference. I think the best would be to fork a child process before starting the multi-threading, but it seems a bit too complicated for the "simple" thing I want to do. – JanCG Mar 09 '15 at 15:30
0

I have a Linux service (daemon) that has multiple-threads and uses boost io_service listening on a TCP socket. When I receive a certain message on that socket I want to start another service with e.g. /etc/init.d/corosync start

For modern Linux'es with systemd you may like to look into socket activation. That is, your main service daemon will just send a datagram to a unix socket (that datagram may contain file descriptors you would like to explicitly pass to the other service). systemd would start that other service for you if has not started yet.

The benefit of using systemd is that your service does not need to run as root in order to be able to start another service and the service processes are completely isolated from each other (no inheritance of anything as when forking).

Maxim Egorushkin
  • 131,725
  • 17
  • 180
  • 271