0

I'm trying to run a series of commands through execv() and forking a new process in C to run each one, and yet for some reason they aren't running in parallel. The following code is run for each process, with "full" being the filepath and "args" being the arguments. I know that the execv() part isn't the issue, it has to do with the way I'm forking and waiting.

            int status;
            pid_t pid = fork();

            if (pid == 0) {
                execv(full, args);
                //perror("execv");
            } else if (pid < 0) {
                printf("%s\n", "Failed to fork");
                status = -1;
            } else {
                if (waitpid(pid, &status, 0) != pid) {
                    status = -1;
                    return status;
                }
                
            }

When running this code, the forked commands simply run one after the other. I don't know how this could be happening.

Fathom820
  • 1
  • 3
  • The scheduling of processes is managed entirely by the OS. Try running commands that run for a few seconds and print something as they go, that way there's a greater chance the output will interleave. – dbush Mar 29 '22 at 21:53
  • You're calling `waitpid` immediately after each `fork`, with default options (no use of `WNOHANG` to do a non-blocking poll), so it's blocking immediately in the parent until the child process exits. How do you expect them to run in parallel if the parent always blocks for each process? It's conceivably possible there is some parallelism in your actual launching code (though mixing threads with `fork` gets... weird), but without a [MCVE] we have no way to know. – ShadowRanger Mar 29 '22 at 21:54
  • Hi, I'm not running any more parallelism than this, which is why I didn't include much more. How would I make it not wait after each fork, and just let them run in parallel? – Fathom820 Mar 29 '22 at 21:57
  • @Fathom820: Don't call `waitpid`? Just store off the `pid` somewhere and wait for all the children in bulk at some point (or even more lazily, store none of the PIDs, and just increment a counter for each successful launch, and eventually call `wait` that many times to wait for/clean up after all the children, or to ignore them entirely, set a SIGCHLD handler to ignore the notifications from the children). – ShadowRanger Mar 29 '22 at 22:00

1 Answers1

1

If you don't want to wait for each child process, don't call waitpid immediately; as written, you fork a child, then immediately stop all processing in the parent until the child process exits, preventing you from forking any further children. If you want to launch multiple children without leaving zombie processes lying around (and possibly monitoring them all at some point to figure out their exit status), you can do one of:

  1. Store off the pids from each fork in an array, and call waitpid on them one by one after you've launched all the processes you need to launch
  2. Store a count of successfully launched child processes and call wait that many times to wait on them in whatever order they complete.
  3. Ignore the SIGCHLD from the child processes entirely, for when you don't care when they exit, don't need to know their status, etc.
ShadowRanger
  • 143,180
  • 12
  • 188
  • 271
  • My issue is this: I don't know how many child processes can be run. It could be 0, it could be 250. So how would I store how many processes have been launched? – Fathom820 Mar 30 '22 at 00:55
  • @Fathom820: The simplest solution if you need to track that is case #2. Declare a variable `static int running_processes = 0;` (at global scope if multiple functions need to use it). Change your `else` case to just `++running_processes;`. When you call `wait` and it reaps a process, `--running_processes;`. – ShadowRanger Mar 30 '22 at 01:01
  • If the `else` case is just `++running_processes`, then where would I call `wait`? Outside the `else` statement? – Fathom820 Mar 30 '22 at 01:15
  • @Fathom820: That would come at some much later point, once you'd launched all the processes you care to launch. Without knowing when you launch processes and why, and when/why you need to wait on them, or not, I can't be more specific. – ShadowRanger Mar 30 '22 at 01:19
  • This function is part of a shell I'm writing, and the function I showed above is run in a while() loop whenever a new command is issued, as it uses a new process to run each one (as long as it isn't built-in). I'm attempting to run this outside of my main loop, after all of the child processes have been created, but I'm still getting the same result. – Fathom820 Mar 30 '22 at 01:44
  • @Fathom820: Under normal conditions (when no `&` used, no interprocess piping with `|` involved, and no subshells are involved), you should be waiting on the process immediately. I'm not clear why you think it should be parallelizing them. You need to think through the *intent*. When you launch a process backgrounded, sure, don't wait on it, maybe keep a linked list or dynamically expanding array of pids to intermittently check (with `WNOHANG`). When you launch it in the foreground, wait immediately. When you launch a pipeline, it's more complicated, but get the basics working first. – ShadowRanger Mar 30 '22 at 03:10
  • @Fathom820: The point here is that in a real shell, you're responding to user input, and whether you wait now or later depends on the commands provided. I can't give greater help than that; your existing code is too barebones to provide useful feedback on. – ShadowRanger Mar 30 '22 at 03:11
  • The intent is to have them in parallel because this is an assignment. It's not something I chose for myself. What I'm looking for is to have them run in parallel, and then resume the main process once they've all completed. – Fathom820 Mar 30 '22 at 15:57
  • @Fathom820: I get that. I've given the advice that applies to the code you provided (defer the use of `wait`/`waitpid` until all processes are `fork`ed). If you don't `wait`/`waitpid` until all child processes are forked, and all the child processes `exec` in short order (e.g. macOS is known to have issues with the high-level system framework libraries that make `fork`ing without a prompt `exec` unstable), which the code is doing, they should run in parallel (to the limits of your hardware/OS parallelism support). I can't psychically debug all the code you left out, or your new code. – ShadowRanger Mar 30 '22 at 22:08