Make Clojure's println "thread-safe" in the same way as in Java

Question

While playing around with concurrent calls to println in Clojure I found that its behaviour is different from Java's System.out.println.

What in Java I would write

class Pcalls {
    public static void main(String[] args) {
        Runnable[] fns = new Runnable[3];
        for (int i = 0; i < 3; i++) {
            fns[i] = new Runnable() {
                @Override public void run() {
                    for (int i = 1; i <= 5; i++) {
                        System.out.println("Hello iteration " + i);
                    }
                }
            };
        }
        for (Runnable fn : fns) new Thread(fn).start();
    }
}

I paraphrased in Clojure as:

(doall (apply pcalls
              (repeat 3 #(dotimes [i 5] (println "Hello iteration" (inc i))))))

Unfortunately, in the Clojure version the output lines often appear interleaved:

Hello iterationHello iteration  1
Hello iteration Hello iteration 2
Hello iteration 3
1
Hello iteration 4
1
Hello iteration Hello iteration5
 Hello iteration 2
Hello iteration 23

Hello iteration Hello iteration 4
3Hello iteration 
5
Hello iteration 4
Hello iteration 5
(nil nil nil)

In Java this never happens, every message is printed on its own line.

Can you explain how and why Clojure's println differs from Java's, and how to arrive at a similar kind of "thread-safe" behaviour with println in Clojure?

score 6 · Answer 1 · answered Sep 06 '13 at 16:56

A convention in clojure is to lock *out*, which refers to the location printed to.

user> (doall (apply pcalls
            (repeat 3 #(dotimes [i 5]
                             (locking *out*
                               (println "Hello iteration" (inc i)))))))
Hello iteration 1
Hello iteration 1
Hello iteration 2
Hello iteration 3
Hello iteration 4
Hello iteration 5
Hello iteration 1
Hello iteration 2
Hello iteration 3
Hello iteration 4
Hello iteration 5
Hello iteration 2
Hello iteration 3
Hello iteration 4
Hello iteration 5
(nil nil nil)

score 3 · Accepted Answer · answered Sep 06 '13 at 16:56

3

Internally, println sends output to the writer that is the currently-bound value for *out*. There are a couple of reasons that calls to this are not atomic:

The println function is multiple arity. If handed multiple objects, it makes multiple writes to *out*.
Calls to println are delegated to an internal multimethod called print-method (which can be extended to add print support for custom types). The print-method implementation for non-string objects, especially collection types, can make multiple writes to *out*. This is in contrast to Java's println which will call .toString on the object and make a single write.

If you want atomic println's, you'll probably have to explicitly synchronize your calls, e.g.:

(let [lock (Object.)]
  (defn sync-println [& args]
    (locking lock (apply println args))))

answered Sep 06 '13 at 16:56

Alex

13,811
1
37
50

Thanks for the explanation of the *how*! But now that I know about `locking` I prefer @noisesmith's solution that locks on `*out*`. Or do you have a specific reason for using the lock object? – glts Sep 06 '13 at 17:59
1

In general I prefer for my code not to lock on objects that it doesn't own. You never know what else might be locking on `*out*`, or why. Chances are it's OK, but it only takes one misbehaved writer to hog the lock and screw things up for everybody. – Alex Sep 06 '13 at 18:42
This explains why it is so, but not why it was made so? It could have been that println delegates a str-method and performs concatenation after going through all arguments and performs a single write to the *out* stream for example. – Didier A. Dec 27 '19 at 05:48
I think in this case it makes more sense to lock on `*out*`, because the goal is to synchronize writes to *out*, so if something else you don't know about has a hold on *out*, you probably want to respect that as well and wait before writing to it. – Didier A. Dec 27 '19 at 06:36

score 3 · Answer 3 · answered Dec 27 '19 at 06:17

New in Clojure 1.10, one can also make use of tap> to synchronize println, as such:

(add-tap println)
(tap> [1 2 3 4])
;> [1 2 3 4]

Now you can send to tap> to print in the order tap receives in a thread safe manner:

(doall (apply pcalls
              (repeat 3 #(dotimes [i 5] (tap> (str "Hello iteration" " " (inc i)))))))
Hello iteration 1
Hello iteration 1
Hello iteration 2
Hello iteration 3
Hello iteration 4
Hello iteration 5
Hello iteration 1
Hello iteration 2
Hello iteration 3
Hello iteration 4
Hello iteration 5
Hello iteration 2
Hello iteration 3
Hello iteration 4
Hello iteration 5
(nil nil nil)

Just note that tap> is arity-1, so you can't pass it more than one thing, which means in this case you have to use str first to concatenate what you want printed.

With `tap>`, you can also have it do synchronized pretty printing:

(add-tap (bound-fn* clojure.pprint/pprint))
(tap> {:a 100 :b 200 :c 300 :d 200 :f 400 :g 400000000 :h 3992 :l {:k 10203 :f 39945 :o 29394}})

{:a 100,
 :b 200,
 :c 300,
 :d 200,
 :f 400,
 :g 400000000,
 :h 3992,
 :l {:k 10203, :f 39945, :o 29394}}

Under the hood, tap> uses a java.util.concurrent.ArrayBlockingQueue to synchronize the calls to it.

Also note that tap> is asynchronous. So while things are being printed, it won't block. That means that if you quit the app before it is done printing, it won't finish:

(doall (apply pcalls
              (repeat 3 #(dotimes [i 5] (tap> (str "Hello iteration" " " (inc i)))))))
(System/exit 0)

"Hello iteration 1"
"Hello iteration 2"
"Hello iteration 3"
"Hello iteration 4"

score 0 · Answer 4 · edited May 23 '17 at 11:51

For completeness, an alternative to using Clojure's locking is to rely on the synchronisation of System.out (to which *out* is bound by default) provided by the Java host.

(doall (apply pcalls
              (repeat 3 #(dotimes [i 5]
                           (.println *out* (str "Hello iteration " (inc i)))))))

(defn out-println [& args]
  (.println *out* (apply str (interpose \space args))))

But note that the answers to Synchronization and System.out.println suggest that technically the Java API does not guarantee synchronisation for System.out.println. And of course *out* can be rebound in Clojure.

score 0 · Answer 5 · answered Nov 21 '13 at 18:58

You can also solve this problem with core.async:

(def print-chan (chan 10))

(defn aprintln [& message]
  (>!! print-chan message))

(defn start-printer! [] (thread (while true
                                  (apply println (<!! print-chan)))))

(defn do-a-thing [] (aprintln "Doing a thing"))

(defn do-another-thing [] (aprintln "Doing another thing"))

(defn -main []
  (start-printer!)
  (future (do-a-thing))
  (do-another-thing))

This will ensure that your outputs don't interleave, no matter how many threads call aprintln at once.

Make Clojure's println "thread-safe" in the same way as in Java

5 Answers5

Linked