17

Monad transformers are known for all standard monads (Reader, Writer, State, Cont, List, etc.), but each of these monad transformers works in a slightly different way. There is no general method or formula for constructing a monad transformer given a definition of a type constructor with a monad instance. So, it is not assured that a monad data type designed according to some arbitrary business requirements will have a monad transformer. Is there such an explicit example?

Related work

Another question explains that a functor composition of two monads is not necessarily a monad. See also this question. Those examples do not answer the present question - they merely illustrate the problem of having no general method for constructing a monad transformer. Those examples show that, given two monads M and N, we will sometimes find that M (N a) is a monad, sometimes that N (M a) is a monad, and sometimes neither will be a monad. But this shows neither how to construct a monad transformer for M or N, nor whether it exists at all.

An answer to another question argues that the IO monad cannot have a monad transformer because if it had one IOT, we could apply IOT to List, and then lifting an empty list (lift []) into the resulting monad would have to undo the side effects performed "earlier" by the IO monad. This argument is based on the idea that the IO monad "actually performs" side effects that, presumably, cannot be undone. However, the IO monad is not an explicit type constructor.

Discussion

In every example where a monad type is given explicitly, a monad transformer could be found somehow, - sometimes with certain ingenuity required. For example, the ListT transformer that existed in the Haskell library was relatively recently found to be incorrect in a subtle way, but the problem was eventually fixed by changing the definition of ListT.

Standard examples of monads without transformers are monads such as IO, which is actually not defined by an explicit type constructor - IO is an opaque "magic" type defined by the library somehow, at low level. It seems clear that one cannot define IO as an explicit type constructor, with a monad instance given by pure functions. The IO example shows that a monad transformer may fail to exist if we allow the monad instance to contain hidden low-level code with impure side effects. So, let us limit our attention to monads implemented using pure functions.

There does not seem to be an algorithm that would derive monad transformers automatically from the monad's source code. Do we even know that this is always possible?

To make it more clear what I mean by an "explicit example" of a monad: Suppose I claim that

 type Q u v a = ((u -> (a, Maybe a)) -> v) -> u -> (a, Maybe a)

can have a lawful Monad instance with respect to the type parameter a, and I produce the source code for an implementation of the Monad instance for Q u v as pure functions return and join. Do we then know that Q u v has a monad transformer QT u v such that QT u v Id is equivalent to Q u v, and the laws of the monad transformers hold? Do we then know how to construct QT explicitly? I don't.

To decide this question, we need either

  • to demonstrate an algorithm that would find a monad transformer from an arbitrary given type constructor and a given implementation of a monad instance; e.g. given the code type F a = r -> Either (a, a) (a, a, Maybe a) and an implementation of a monad instance for this, to find the code for the monad transformer; let's limit ourselves to type constructors made out of any combination of ->, tuple, and Either for simplicity; or
  • to demonstrate a counterexample: an explicit monad type constructor, given by an explicit code definition, e.g. type F a = r -> Either (a, a, a) (a, a, Maybe a) or whatever, such that this is a lawful Monad, with a Monad instance given by pure functions, but we can prove that F has no monad transformer.
winitzki
  • 3,179
  • 24
  • 32
  • 3
    `IO` itself is not magic, just the state that it wraps. `newtype IO a = IO (magic -> (magic, a))`. – chepner May 09 '18 at 19:36
  • 6
    @chepner it is though, because the state-passing "semantics" can not model some of its features, specifically `forkIO`. (You can of course make the argument that the magic of *magic* is what handles threading too, but we are beginning to reach pretty far) – luqui May 09 '18 at 20:03
  • 1
    The monadness of the monad you are after must rely on some property of pure code that is not guaranteed also by a monad law. One example of such a property would be `const x y == x` (in monad land that is `const <$> x <*> y == x` which is not true). Maybe that's a place to start. – luqui May 09 '18 at 20:15
  • @chepner: Isn’t the only magic thing about `RealWorld` that it’s neither unlifted (because it has kind `*`) nor lifted (because it has no runtime representation and so doesn’t contain ⊥)? Or does GHC treat `RealWorld` values in some other special way that doesn’t follow from those properties? I can’t seem to find any documentation on what’s *really* magical about it other than that. – Jon Purdy May 09 '18 at 20:39
  • 1
    If we throw away the magic effectful `IO`, and concentrate on pure functions, is there a reason why a monad transformer may fail to exist? That is, can we take such a pair of wel-behaved, pure-functional monads that producing a transformer cannot logically be constructed? I frankly don't see why that could happen. What law of monad transformers could we potentially even break? – 9000 May 09 '18 at 21:13
  • @9000 I updated my question to add a qualification that, indeed, we are interested only in pure functions. Let me rephrase the question more, to answer your comment. Basically, while you say you don't see how we could violate the laws of monad transformers, you seem to assume that it's obvious _how to write_ a monad transformer, given any type constructor of some monad and an implementation of its monad instance. But the point of my question is that we have no procedure that will accomplish that in the general case. – winitzki May 09 '18 at 21:58
  • @JonPurdy, that remark is really about how the `RealWorld` type is manufactured by GHC. It's of absolutely no relevance to the model of `IO` or anything even low-level `IO` hacking libraries need to care one whit about. But there is *plenty* of disgusting magic in the optimizer relating to things that pass `State#` values around, which is why the extreme brokenness of the `IO` type turns out to be a problem in practice. The biggest problem is strictness analysis. – dfeuer May 10 '18 at 01:39
  • 3
    "`ListT` done right" is from 2007. The original (somewhat invalid) `ListT` seems to have shown up in GHC 5, in 2002. So I'm not sure I'd say even "relatively recently". – dfeuer May 10 '18 at 02:18
  • 1
    I don't quite understand what do you mean by "a monad transformer of a monad". A monad transformer is a wrapper over another type, which is, given that the underlying type is a monad, also a monad. But, if you, instead, start with a concrete type F, what do you mean by "monad transformer for F" – max630 May 10 '18 at 02:54
  • If you are looking for a regular way to build monad transformers, you probably should take a look at [Freer monad](http://okmij.org/ftp/Computation/free-monad.html#freer) – max630 May 10 '18 at 03:08
  • 4
    @max630 Presumably `T` is a monad transformer for `F` if `T` is a monad transformer and there is a monad isomorphism between `T Identity` and `F` (that is, a standard isomorphism that respects bind and return). – Daniel Wagner May 10 '18 at 03:49
  • @dfeuer: Thanks, I was grepping for `RealWorld` in the GHC sources when I should’ve been looking at `State#`. Time to read some code! – Jon Purdy May 10 '18 at 07:31
  • @max630 I call `ReaderT` the "monad transformer of the `Reader` monad", `WriterT` the "monad transformer of the `Writer` monad` etc. What would you call that? For example, how you would fill the blank in "The monad transformer `ListT` is _____ of / for / from / to / ... the `List` monad"? – winitzki May 10 '18 at 21:13
  • @max630 I am familiar with free monads and the "freer" paper. The free monad has a monad transformer, which is defined in a way different from the way we define `ReaderT`, `WriterT`, `StateT`, `ListT`, or `ContT`. So, the construction of the free monad's monad transformer does not really help me to understand how to construct monad transformers for monads in a systematic way. I have updated my question to reflect this comment. – winitzki May 10 '18 at 21:19
  • There are at least two other possibilities you don't mention. It may be possible to prove that such a transformer exists (according to some well-accepted mathematical theory) even if the algorithm you seek does not exist. Indeed, I'd place my bet on that possibility or a counterexample. It's also possible that this is independent of the rest of mathematics, although I doubt it. – dfeuer May 10 '18 at 22:15
  • Is this a duplicate of https://stackoverflow.com/questions/24515876/is-there-a-monad-that-doesnt-have-a-corresponding-monad-transformer-except-io ? – Benjamin Hodgson May 11 '18 at 12:14
  • @BenjaminHodgson Yes, thank you. A very similar question indeed. I don't see a satisfactory (constructive) answer there. In the accepted answer, it is claimed that every monad has a monad transformer, but no constructive procedure is given for deriving a monad transformer given the source code of a monad. – winitzki May 11 '18 at 16:12
  • 1
    The answer there describes how to go from an adjunction to a corresponding monad transformer, so all that remains is to go from the monad to the underlying adjunction. [Bartosz Milewski's blog](https://bartoszmilewski.com/2017/03/14/algebras-for-monads/) delivers, as always, with a Haskell-oriented description of that decomposition process. Have a Google for "monadic adjunction" for more info – Benjamin Hodgson May 11 '18 at 16:23
  • @BenjaminHodgson Thank you, this is useful. Although Bartosz also says that there is a whole family of adjunctions, not a unique one. I will try to see if his blog really answers the question. Can't wait to see how to construct monad transformers for the examples `Q u v a` and `F a` in my question. Hopefully it won't be just a statement about the "existence" of some functor. – winitzki May 11 '18 at 17:07
  • @BenjaminHodgson No, unfortunately Bartosz Milewski's blog gives only a purely theoretical construction: the EM and the Kleisli adjunctions. There is no indication about how to find the specific program code that correspond to these functors and categories... It also says that there is a whole category of adjunctions, so the EM and the K adjunctions are perhaps not the only two. So, even if we can find the code that represents them, we will not know which one actually yields correct monad transformers. Disappointed. – winitzki May 15 '18 at 02:29
  • Found an example of a monad that probably doesn't have an explicit monad transformer. `L a = Either a (r -> a)` where `r` is a fixed type. My efforts to guess its monad transformer failed so far. See also my new answer in https://stackoverflow.com/a/54177543/593934 – winitzki Jan 18 '19 at 02:23
  • My claim was wrong: the monad `L` defined by `type L a = Either a (r -> a)` has a lawful monad transformer. The type is `type LT m a = m (Either a (r -> m a))`. This was not obvious to me 2 years ago. Details are in the book "The Science of Functional Programming" chapter 14. https://github.com/winitzki/sofp – winitzki Mar 13 '21 at 20:55

1 Answers1

1

This isn't an answer, but it's way too big for a comment. We can write

{-# LANGUAGE GeneralizedNewtypeDeriving
  , DeriveFunctor #-}

import Control.Monad.Free

-- All the IO primops you could ever need
data IOF a = PutStrLn String a
           | GetLine (String -> a)
           deriving Functor

newtype MyIO a = MyIO {unMyIO :: Free IOF a}
  deriving (Functor, Applicative, Monad)

But we can actually make a monad transformer out of this:

import Control.Monad.Trans.Free

newtype IOT m a = IOT {unIOT :: FreeT IOF m a}
  deriving (Functor, Applicative, Monad, MonadTrans)

So I don't really think even IO is excluded, although the isomorphism in that case is not "internal".

dfeuer
  • 48,079
  • 5
  • 63
  • 167
  • You seem to be making a free monad out of a given monad first. This produces a different monad. A free monad built out of a monad `F` has a monad transformer, of course, but it will not be the monad transformer of `F`. – winitzki May 10 '18 at 21:15
  • @winitzki, `IOF` isn't a monad. It's the base functor of `IO`. You can use Churchy versions of `Free` and `FreeT` to get somewhat more precise isomorphisms, but I think this should already do if you watch your abstraction barrier. – dfeuer May 10 '18 at 21:27
  • Like I said, though, this isn't a proper answer because (unlike `IO`), not every monad is a free monad. – dfeuer May 10 '18 at 21:29
  • How is `IOF` defined, what is the type expression for `IOF a` such that `Free IOF` (i.e. the free monad built on IOF) is equivalent to the `IO` monad? I think, either this can't be done, or `IOF` must be an opaque type, not definable in the pure lambda calculus, just like `IO` is. In the example you gave, `IOF a` is defined as the functor equivalent to `Either (String, a) (String -> a)`. The free monad based on this will be certainly a monad, but it will not be the same as the `IO` monad. – winitzki May 10 '18 at 21:53
  • 1
    @winitzki, that type represents a hypothetical type with far more constructors. And I'm assuming that `IO` and `IOT` are hidden (by the module system) so they only expose certain operations (and can't be inspected generally). It would actually be possible for a Haskell implementation to implement `IO` in a similar fashion. Obviously, there's no way to map `IO a -> MyIO a` in Glasgow Haskell, but that's not a very interesting fact. – dfeuer May 10 '18 at 22:06