How to understand `MonadUnliftIO`'s requirement of "no stateful monads"?

Question

I've looked over https://www.fpcomplete.com/blog/2017/06/tale-of-two-brackets, though skimming some parts, and I still don't quite understand the core issue "StateT is bad, IO is OK", other than vaguely getting the sense that Haskell allows one to write bad StateT monads (or in the ultimate example in the article, MonadBaseControl instead of StateT, I think).

In the haddocks, the following law must be satisfied:

askUnliftIO >>= (\u -> liftIO (unliftIO u m)) = m

So this appears to be saying that state is not mutated in the monad m when using askUnliftIO. But to my mind, in IO, the entire world can be the state. I could be reading and writing to a text file on disk, for instance.

To quote another article by Michael,

False purity We say WriterT and StateT are pure, and technically they are. But let's be honest: if you have an application which is entirely living within a StateT, you're not getting the benefits of restrained mutation that you want from pure code. May as well call a spade a spade, and accept that you have a mutable variable.

This makes me think this is indeed the case: with IO we are being honest, with StateT, we are not being honest about mutability ... but that seems another issue than what the law above is trying to show; after all, MonadUnliftIO is assuming IO. I'm having trouble understanding conceptually how IO is more restrictive than something else.

Update 1

After sleeping (some), I am still confused but am gradually getting less so as the day wears on. I worked out the law proof for IO. I realized the presence of id in the README. In particular,

instance MonadUnliftIO IO where
  askUnliftIO = return (UnliftIO id)

So askUnliftIO would appear to return an IO (IO a) on an UnliftIO m.

Prelude> fooIO = print 5
Prelude> :t fooIO
fooIO :: IO ()
Prelude> let barIO :: IO(IO ()); barIO = return fooIO
Prelude> :t barIO
barIO :: IO (IO ())

Back to the law, it really appears to be saying that state is not mutated in the monad m when doing a round trip on the transformed monad (askUnliftIO), where the round trip is unLiftIO -> liftIO.

Resuming the example above, barIO :: IO (), so if we do barIO >>= (u -> liftIO (unliftIO u m)), then u :: IO () and unliftIO u == IO (), then liftIO (IO ()) == IO (). **So since everything has basically been applications of id under the hood, we can see that no state was changed, even though we are using IO. Crucially, I think, what is important is that the value in a is never run, nor is any other state modified, as a result of using askUnliftIO. If it did, then like in the case of randomIO :: IO a, we would not be able to get the same value had we not run askUnliftIO on it. (Verification attempt 1 below)

But, it still seems like we could do the same for other Monads, even if they do maintain state. But I also see how, for some monads, we may not be able to do so. Thinking of a contrived example: each time we access the value of type a contained in the stateful monad, some internal state is changed.

Verification attempt 1

> fooIO >> askUnliftIO
5
> fooIOunlift = fooIO >> askUnliftIO
> :t fooIOunlift
fooIOunlift :: IO (UnliftIO IO)
> fooIOunlift
5

Good so far, but confused about why the following occurs:

 > fooIOunlift >>= (\u -> unliftIO u)

<interactive>:50:24: error:
    * Couldn't match expected type `IO b'
                  with actual type `IO a0 -> IO a0'
    * Probable cause: `unliftIO' is applied to too few arguments
      In the expression: unliftIO u
      In the second argument of `(>>=)', namely `(\ u -> unliftIO u)'
      In the expression: fooIOunlift >>= (\ u -> unliftIO u)
    * Relevant bindings include
        it :: IO b (bound at <interactive>:50:1)

Answer 1

"StateT is bad, IO is OK"

That's not really the point of the article. The idea is that MonadBaseControl permits some confusing (and often undesirable) behaviors with stateful monad transformers in the presence of concurrency and exceptions.

finally :: StateT s IO a -> StateT s IO a -> StateT s IO a is a great example. If you use the "StateT is attaching a mutable variable of type s onto a monad m" metaphor, then you might expect that the finalizer action gets access to the most recent s value when an exception was thrown.

forkState :: StateT s IO a -> StateT s IO ThreadId is another one. You might expect that the state modifications from the input would be reflected in the original thread.

lol :: StateT Int IO [ThreadId]
lol = do
  for [1..10] $ \i -> do
    forkState $ modify (+i)

You might expect that lol could be rewritten (modulo performance) as modify (+ sum [1..10]). But that's not right. The implementation of forkState just passes the initial state to the forked thread, and then can never retrieve any state modifications. The easy/common understanding of StateT fails you here.

Instead, you have to adopt a more nuanced view of StateT s m a as "a transformer that provides a thread-local immutable variable of type s which is implicitly threaded through a computation, and it is possible to replace that local variable with a new value of the same type for future steps of the computation." (more or less a verbose english retelling of the s -> m (a, s)) With this understanding, the behavior of finally becomes a bit more clear: it's a local variable, so it does not survive exceptions. Likewise, forkState becomes more clear: it's a thread-local variable, so obviously a change to a different thread won't affect any others.

This is sometimes what you want. But it's usually not how people write code IRL and it often confuses people.

For a long time, the default choice in the ecosystem to do this "lowering" operation was MonadBaseControl, and this had a bunch of downsides: hella confusing types, difficult to implement instances, impossible to derive instances, sometimes confusing behavior. Not a great situation.

MonadUnliftIO restricts things to a simpler set of monad transformers, and is able to provide relatively simple types, derivable instances, and always predictable behavior. The cost is that ExceptT, StateT, etc transformers can't use it.

The underlying principle is: by restricting what is possible, we make it easier to understand what might happen. MonadBaseControl is extremely powerful and general, and quite difficult to use and confusing as a result. MonadUnliftIO is less powerful and general, but it's much easier to use.

So this appears to be saying that state is not mutated in the monad m when using askUnliftIO.

This isn't true - the law is stating that unliftIO shouldn't do anything with the monad transformer aside from lowering it into IO. Here's something that breaks that law:

newtype WithInt a = WithInt (ReaderT Int IO a)
  deriving newtype (Functor, Applicative, Monad, MonadIO, MonadReader Int)

instance MonadUnliftIO WithInt where
  askUnliftIO = pure (UnliftIO (\(WithInt readerAction) -> runReaderT 0 readerAction))

Let's verify that this breaks the law given: askUnliftIO >>= (\u -> liftIO (unliftIO u m)) = m.

test :: WithInt Int
test = do
  int <- ask
  print int
  pure int

checkLaw :: WithInt ()
checkLaw = do
  first <- test
  second <- askUnliftIO >>= (\u -> liftIO (unliftIO u test))
  when (first /= second) $
    putStrLn "Law violation!!!"

The value returned by test and the askUnliftIO ... lowering/lifting are different, so the law is broken. Furthermore, the observed effects are different, which isn't great either.