Haskell-way of modeling a type with dynamic JSON fields?

Question

I am new to Haskell, coming from an imperative programming background. I would like to be able to serialize an object to JSON in the "Haskell way", but not quite sure how to do that yet.

I have read Chapter 5 of RealWorldHaskell which talks about JSON a bit, and played around with Aeson. I have also looked at a few JSON API libraries written in Haskell, such as:

• Facebook API in Haskell
• Stripe API in Haskell

That got me to the point of being able to create very basic JSON strings from objects (also thanks to this blog post):

{-# LANGUAGE OverloadedStrings, DeriveGeneric #-}

import Data.Aeson
import GHC.Generics

data User = User {
  email :: String,
  name :: String
} deriving (Show, Generic)

instance ToJSON User

main = do
  let user = User "foo@example.com" "Hello World"
  let json = encode user
  putStrLn $ show json

That will print out:

"{\"email\":\"foo@example.com",\"name\":\"Hello World\"}"

Now the goal is, add another field to a User instance that can have arbitrary fields. The Facebook Graph API has a field called data, which is a JSON object with any property you want. For example, you may make requests like this to Facebook's API (pseudocode, not familiar with the Facebook API exactly):

POST api.facebook.com/actions
{
  "name": "read",
  "object": "book",
  "data": {
    "favoriteChapter": 10,
    "hardcover": true
  }
}

The first two fields, name, and object are type String, while the data field is a map of arbitrary properties.

The question is, what is the "Haskell way" to accomplish that on the User model above?

I can understand how to do the simple case:

data User = User {
  email :: String,
  name :: String,
  data :: CustomData
} deriving (Show, Generic)

data CustomData = CustomData {
  favoriteColor :: String
}

But that isn't quite what I'm looking for. That means the User type, when serialized to JSON, will always look like this:

{
  "email": "",
  "name": "",
  "data": {
    "favoriteColor": ""
  }
}

The question is, how do you make it so you only have to define that User type once, and then can have arbitrary fields attached to that data property, while still benefitting from static typing (or whatever is close to that, not super familiar with the details of types yet).

Answer 1

It depends on what you mean by arbitrary data. I'm going to extract what I think is a reasonable and non-trivial definition of "data contains an arbitrary document type" and show you a couple of possibilities.

First I'll point to a past blog post of mine. This demonstrates how to parse documents that vary in structure or nature. Existing example here: http://bitemyapp.com/posts/2014-04-17-parsing-nondeterministic-data-with-aeson-and-sum-types.html

As applied to your data type, this could look something like:

data CustomData = NotesData Text | UserAge Int deriving (Show, Generic)
newtype Email = Email Text deriving (Show, Generic)
newtype Name  = Name  Text deriving (Show, Generic)

data User = User {
  email :: Email,
  name  :: Name,
  data  :: CustomData
} deriving (Show, Generic)

Next I'll show you to define parameterizable structure with the use of a higher kinded type. Existing example here: http://bitemyapp.com/posts/2014-04-11-aeson-and-user-created-types.html

newtype Email = Email Text deriving (Show, Generic)
newtype Name  = Name  Text deriving (Show, Generic)

-- 'a' needs to implement ToJSON/FromJSON as appropriate
data User a = User {
  email :: Email,
  name  :: Name,
  data  :: a
} deriving (Show, Generic)

With the above code we've parameterized data and made User a higher kinded type. Now User has structured parameterized by the types of its type arguments. The data field can now be a document such as with User CustomData, a string User Text or a number User Int. You probably want a semantically meaningful type, not Int/String. Use newtype as necessary to accomplish this.

For a rather worked up example of how to lend structure and meaning to a data type that many would otherwise encode as (Double, Double), see https://github.com/NICTA/coordinate.

You can combine these approaches if you think it appropriate. It depends partly on whether you want your type to be able to express a particular, single, possibility in the type argument to the enclosing document or not.

I have a ton of JSON processing code and examples of how to structure data in my library at https://github.com/bitemyapp/bloodhound

The guiding principle is to make invalid data unrepresentable via the types to the extent possible. Consider using "smart constructors" when types alone can't validate your data.

See more about smart constructors here: https://www.haskell.org/haskellwiki/Smart_constructors

Answer 2

If you really wanted to accept a fully arbitrary JSON substructure with Aeson's FromJSON class, I'd advise that you create a field user :: Value, which is Aeson's generic type for any JSON value. If you find possible types of this JSON value later, you may convert it using FromJSON again, but initially it will hold anything that is there.