Asynchronous programming and concurrency

Different platforms have different programming models. For instance, JavaScript applications are single-threaded and have an event loop. When making a network call, it is common to register a callback that will be invoked at a later stage, when that network call completes either successfully or with an error.

In contrast, when we're on a JVM, we can take full advantage of multithreading to achieve concurrency. It is simple to spawn new threads via one of the many concurrency primitives provided by Clojure, such as futures.

However, asynchronous programming becomes cumbersome. Clojure futures don't provide a native way for us to be notified of their completion at a later stage. In addition, retrieving values from a not-yet-completed future is a blocking operation. This can be seen clearly in the following snippet:

(defn do-something-important [] 
  (let [f (future (do (prn "Calculating...") 
                      (Thread/sleep 10000)))] 
    (prn "Perhaps the future has done its job?") 
    (prn @f) 
    (prn "You will only see this in about 10 seconds..."))) 
 
(do-something-important) 

The second call to print dereferences future, causing the main thread to block since it hasn't finished yet. This is why you only see that the last print after the thread in which future is running has finished. Callbacks can, of course, be simulated by spawning a separate thread to monitor the first one, but this solution is clunky at best.

An exception to the lack of callbacks is GUI programming in Clojure. Much like JavaScript, Clojure Swing applications also possess an event loop and can respond to user input and invoke listeners (callbacks) to handle them.

Another option is rewriting the previous example with a custom callback that is passed into future:

(defn do-something-important [callback] 
  (let [f (future (let [answer 42] 
                    (Thread/sleep 10000) 
                    (callback answer)))] 
    (prn "Perhaps the future has done its job?") 
    (prn "You should see this almost immediately and then in 10 secs...") 
     f)) 
 
(do-something-important (fn [answer] 
                          (prn "Future is done. Answer is " answer))) 

This time, the order of the outputs should make more sense. However, if we return future from this function, we have no way to give it another callback. We have lost the ability to perform an action when future ends and are back to having to dereference it, thus blocking the main thread again—exactly what we wanted to avoid.

As you might have realized, CES is closely related to asynchronous programming: the stock market application we built in the previous chapter is an example of such a program. The main—or UI—thread is never blocked by the observables fetching data from the network. Additionally, we were also able to register callbacks when subscribing to them.

In many asynchronous applications, however, callbacks are not the best way to go. Heavy use of callbacks can lead to what is known as callback hell. Clojure provides a more powerful and elegant solution.

In the next few sections, we will explore core.async, a Clojure library for asynchronous programming, and how it relates to Reactive Programming.