Run commands in Emacs asynchronously, but display output incrementally

Question

I have a utility function:

(defun execute-in-buffer (command-with-args buffer)
  "Execute a string COMMAND-WITH-ARGS representing a shell command with arguments,
inserting the results in BUFFER."
  (switch-to-buffer buffer)
  (insert (format ">>> %s\n" command-with-args))
  (let* ((command-args-list (s-split " " command-with-args))
         (command (car command-args-list))
         (args (cdr command-args-list)))
    (apply 'call-process command nil buffer t args)))

This allows me to do things like (execute-in-buffer "ls /" (get-buffer-create "*my-output*"). However, it's not well suited for slow commands. If I call a series of slow commands, I don't get any output until the very end:

(let ((buf (get-buffer-create "*my-output")))
  (execute-in-buffer "sleep 10" buf)
  (execute-in-buffer "ls /" buf))

I want to be able to call synchronously, so the next command only runs after the previous one finishes. However, I want to see the output from my commands as they run. How would I do this?

(Example code is just for show, I'm happy to drop it in favour of something else.)

shouldn't you be using `with-current-buffer` or `set-buffer` instead of `switch-to-buffer`? — sds, May 29 '13 at 15:01
This is by design -- I want my output buffer to be visible. In other circumstances, I'd definitely use `with-current-buffer` or `set-buffer`. — Wilfred Hughes, May 30 '13 at 09:56
in that case you need to use `set-buffer` and then `display-buffer`. — sds, May 30 '13 at 12:52

score 7 · Accepted Answer · edited May 23 '17 at 12:10

Using synchronous processes

If you want to stick with synchronous processes (e.g. using call-process), you need to call (redisplay) after each call to execute-in-buffer for the display to be updated and the output to be visible (also see this question for more details). However, the output of each command will not be visible until the process terminates, and emacs will hang while the external processes are running.

Using asynchronous processes

Using asynchronous processes is a bit more complicated, but avoids hanging Emacs while the commands are running, which also solves the redisplay issue. The tricky part here is to sequentially chain all commands. Here is a bit of elisp which should do the trick:

(defun execute-commands (buffer &rest commands)
  "Execute a list of shell commands sequentially"
  (with-current-buffer buffer
    (set (make-local-variable 'commands-list) commands)
    (start-next-command)))

(defun start-next-command ()
  "Run the first command in the list"
  (if (null commands-list)
      (insert "\nDone.")
    (let ((command  (car commands-list)))
      (setq commands-list (cdr commands-list))
      (insert (format ">>> %s\n" command))
      (let ((process (start-process-shell-command command (current-buffer) command)))
        (set-process-sentinel process 'sentinel)))))

(defun sentinel (p e)
  "After a process exited, call `start-next-command' again"
  (let ((buffer (process-buffer p)))
    (when (not (null buffer))
      (with-current-buffer buffer
        ;(insert (format "Command `%s' %s" p e) )
        (start-next-command)))))

;; Example use
(with-current-buffer (get-buffer-create "*output*") (erase-buffer))
(execute-commands "*output*"
                  "echo 1"
                  "sleep 1"
                  "echo 2; sleep 1; echo 3"
                  "ls /")

Thanks, that improves matters considerably. However, ideally I'd like to see the output of a command live, rather than waiting for the command to terminate. — Wilfred Hughes, May 29 '13 at 14:42
I added a solution based on asynchronous processes, which should solve this issue. Please see my edited answer — François Févotte, May 29 '13 at 19:53
And thank you too: I think I'm going to use this technique in [a project of mine](http://github.com/ffevotte/slurm.el) — François Févotte, May 30 '13 at 17:47

score 7 · Answer 2 · answered May 29 '13 at 14:47

7

This works for me:

(async-shell-command "echo 1; sleep 10; echo 2; sleep 10; ls /" "*abcd*")

This can be adapted to do what you need?

answered May 29 '13 at 14:47

Miserable Variable

28,432
15
72
133

Ooh, this is very close. In my original `execute-in-buffer`, I also display the commands I'm running before I run them. I can't see any neat way of doing this short of `(async-shell-command "echo \"echo 1\"; echo 1;")` etc. – Wilfred Hughes May 29 '13 at 14:59
1

If this is for personal use -- as against a productized functionality -- `(async-shell-command "set -xv; echo 1; sleep 10; echo 2; sleep 10; ls /" "*abcd*")` might work for you – Miserable Variable May 29 '13 at 15:06

Run commands in Emacs asynchronously, but display output incrementally

2 Answers2

Linked