Crypto me a tea

Forklift - a pattern for performing monadic actions in a worker thread

Shae Erisson wants to build a browser-based GHC interpreter for his Google Summer of Code project. This involves writing a web application to control a GHCi session. For the web server part, he’s using the pleasingly minimal scotty package, whereas the Haskell interpreter functionality is provided by the hint package.

However, there is a problem: to keep track of various internals, each package uses a monad. But the two monads are incompatible! It is almost impossible to use one of them as a monad transformer on top of the other. How can we combine them anyway?

Andrew Farmer suggested instead to put the interpreter session into a different thread, the worker thread, and to use MVars for communication between the web server and the worker thread. And indeed, that is how similar projects like active-hs solve this problem as well.

However, I didn’t want to create a new communication protocal just for this situation, so I thought of a generic way to do this, which I want to call “the forklift pattern” and which I will now describe.

Your feedback is appreciated! It’s probably a good idea to release this on hackage, but I would like to iron out a few kinks beforehand. The preliminary source code can be found as a gist. Leave your comments below!

The forklift pattern

So, the situation is this: we have a monad m that, for example, represents an interpreter for Haskell expressions. We want to make its computations available in the IO monad and we can do so by running the monad m in a separate worker thread that communicates via MVars. For instance, in the case of the Haskell interpreter, we want to send Haskell expressions to the worker thread for evaluation.

The simple but key insight is this: what is the most general possible communication protocal for talking to the worker thread? Why, we can simply ask the worker thread to perform an arbitrary action of type m a! In other words, we simply put a complete monadic action into the MVar and have the worker thread execute it for us.

I will call this pattern the “forklift” pattern because we are forking a worker thread to lift arbitrary monadic actions. Nifty.

Here we go. The worker thread is represented by the data type

data ForkLift m = ForkLift { requests :: Chan (m ()) }

which just keeps track of a channel were you can send your monadic actions to. To create a new ForkLift, we fork a worker thread that will forever read values from the channel and execute them

newForkLift :: MonadIO m
            => (m () -> IO ()) -> IO (ForkLift m)
newForkLift unlift = do
    channel <- newChan
    let loop = forever . join . liftIO $ readChan channel
    forkIO $ unlift loop
    return $ ForkLift channel

When we want to execute a monadic action, we can use the forklift to “carry it over to the IO monad”. Getting the result value back is a bit tricky due to type safety, but we can simply use another MVar and encode the sending back in the action itself.

carry :: MonadIO m => ForkLift m -> m a -> IO a
carry forklift act = do
    ref <- newEmptyMVar
    writeChan (requests forklift) $ do
        liftIO . putMVar ref =<< act
    takeMVar ref

And here a quick test with the state monad transformer

test :: IO ()
test = do
    state <- newForkLift (flip evalStateT (0::Int))
    carry state $ modify (+10)
    carry state $ modify (+10)
    print =<< carry state get

-- > test
-- 20

Of course, the evalStateT function was always able to map the state monad to the IO monad, but the point here is that the state is preserved between different invocations of the carry function.

Before I can put this on hackage, there are a few rough edges that need to be smoothed out.

1. What happens when the worker thread dies because of an exception? At the moment, all calls to carry will deadlock. They should probably throw an exception instead, WorkerGone?
2. What about garbage collection? When the ForkLift goes out of scope, then the worker thread should also be killed. See my StackOverflow question.

Comments