How to pattern match structures returned by mochijson2?

Question

I've just started tinkering with Erlang and am building a very simple test web application which is just intended to show my twitter timeline.

I'm using webmachine to write the app and erlyDTL to render the templates.

My question is related to the structures returned by mochiweb's mochijson2:decode/1 function.

I can successfully fetch and deocde my timeline as in the following example:

1> Url = "http://api.twitter.com/1/statuses/user_timeline.json?screen_name=<TWITTER_SCREEN_NAME_HERE>".
2> inets:start().
3> {ok, {_, _, Response}} = httpc:request(Url).
4> DecodedJson = mochijson2:decode(Response).

The mochijson2:decode/1 function returns a list of tuples of the format:

[{struct, proplist()}, {struct, proplist()}, ...]

However, to pass the timeline into erlyDTL I need to get rid of the struct atom tag and simply pass a list of proplists to the webmachine resource (rendered by erlyDTL). Being quite new to pattern matching, I figured that the following list comprehension would achieve this:

Timeline = [Tweet || {struct, Tweet} <- DecodedJson].

Indeed, this works perfectly for all of the items in each Tweet proplist except for one, <<"user">>, the value of which is itself a {struct, proplist()} tuple. I can't for the life of me figure out how to nuke the struct atom from this nested tuple and was wondering if anyone could provide an example of Erlang code which would pattern match both the outer Tweet in {struct, Tweet} and the User {struct, User} contained within each Tweet.

The end goal is to be able to access each tweet in Django template language as in the following example:

{{ tweet.text }}  <- works
{{ tweet.created_at }}  <- works
{{ tweet.user.profile_image_url }}  <- ???

Any help would be greatly appreciated!

Could you post an actual example timeline that shows your problem? — DefLog, Jun 10 '11 at 18:20

score 2 · Answer 1 · answered Jun 11 '11 at 17:29

2

Here's what we use internally for a similar purpose:

%% @doc Flatten {struct, [term()]} to [term()] recursively.
destruct({struct, L}) ->
    destruct(L);
destruct([H | T]) ->
    [destruct(H) | destruct(T)];
destruct({K, V}) ->
    {K, destruct(V)};
destruct(Term) ->
    Term.

For other uses of mochijson2 terms, KVC might be useful to you: https://github.com/etrepum/kvc

answered Jun 11 '11 at 17:29

YOUR ARGUMENT IS VALID

2,029
12
11

I shoved this into a module as helpers:destruct() and it works perfectly. Awesome. Thanks very much - also for the KVC tip. – peeb Jun 12 '11 at 10:27

Muzaaya Joshua · Answer 2 · 2011-06-13T08:42:56.637

In a recent project ive worked on, we were dealing with big JSON data structures coming from EXT JS front end apps. One example of the JSON object is here below (this is just the skeleton of the JSON ):

{
        "presence_token":"734737328233HDHSBSHSYEYEYWYWGWE",
        "presence_time":"HH:Mins:Secs",
        "friend_requests":
        [
            {
                "from":"Username",
                "type":"buddy",
                "date":"DD/MM/YY",
                "time":"HH:Mins:Secs",
                "name":"Your Full name",
                "email":"user@example.com"
            }
        ],
        "group_status":
        [
            {
                "group_name":"ecampus",
                "status":"running",
                "members":["phil","josh","shazz"],
                "start_date":"DD/MM/YY",
                "start_time":"HH:Mins:Secs"             
            },
            {
                "group_name":"buganda",
                "status":"off"
            }
        ],
        "friend_status":
        [
            {
                "friend":"Friend_username",
                "status":"online",
                "log_on_time":"HH:Mins:Secs",
                "state":"available",
                "name":"Friend_Fullname",
                "email":"user@example.com"              
            },
            {
                "friend":"Friend_username",
                "status":"offline",                 
                "name":"Friend_Fullname",
                "email":"user@example.com"              
            }           
        ]           
    }

After mochijson2:decode/1, the struct object i had appears like this:

        {struct,[{<<"presence_token">>,
                  <<"734737328233HDHSBSHSYEYEYWYWGWE">>},
                 {<<"presence_time">>,<<"HH:Mins:Secs">>},
                 {<<"friend_requests">>,
                  [{struct,[{<<"from">>,<<"Username">>},
                            {<<"type">>,<<"buddy">>},
                            {<<"date">>,<<"DD/MM/YY">>},
                            {<<"time">>,<<"HH:Mins:Secs">>},
                            {<<"name">>,<<"Your Full name">>},
                            {<<"email">>,<<"user@example.com">>}]}]},
                 {<<"group_status">>,
                  [{struct,[{<<"group_name">>,<<"ecampus">>},
                            {<<"status">>,<<"running">>},
                            {<<"members">>,[<<"phil">>,<<"josh">>,<<"shazz">>]},
                            {<<"start_date">>,<<"DD/MM/YY">>},
                            {<<"start_time">>,<<"HH:Mins:Secs">>}]},
                   {struct,[{<<"group_name">>,<<"buganda">>},
                            {<<"status">>,<<"off">>}]}]},
                 {<<"friend_status">>,
                  [{struct,[{<<"friend">>,<<"Friend_username">>},
                            {<<"status">>,<<"online">>},
                            {<<"log_on_time">>,<<"HH:Mins:Secs">>},
                            {<<"state">>,<<"available">>},
                            {<<"name">>,<<"Friend_Fullname">>},
                            {<<"email">>,<<"user@example.com">>}]},
                   {struct,[{<<"friend">>,<<"Friend_username">>},
                            {<<"status">>,<<"offline">>},
                            {<<"name">>,<<"Friend_Fullname">>},
                            {<<"email">>,<<"user@example.com">>}]}]}]}

Now i decided to create a module which will convert this struct into a "deep" proplist, this module would contain a function struct:all_keys/1 which if i feed it with the struct object it generates lists and tuples in an organised way. Here is the code:

-module(struct).
-export([all_keys/1]).

is_struct({struct,_}) -> true;
is_struct(_) -> false.

to_binary(S) when is_list(S)-> list_to_binary(S);
to_binary(S) when is_integer(S)-> S;
to_binary(S) when is_atom(S)-> to_binary(atom_to_list(S));
to_binary(S) -> S.

to_value(V) when is_binary(V)-> binary_to_list(V);
to_value(V) when is_integer(V)-> V;
to_value(V) when is_list(V)-> 
    try list_to_integer(V) of
        PP -> PP
    catch
        _:_ -> 
            try list_to_float(V) of
                PP2 -> PP2
            catch
                _:_ -> V
            end
    end;
to_value(A)-> A.

to_value2({struct,L})->
    all_keys({struct,L});
to_value2([{struct,_L}|_Rest] = LL)->
    [all_keys(XX) || XX <- LL];
to_value2(D) when is_binary(D)-> to_value(D);
to_value2(D) when is_list(D)-> 
    [to_value2(Any) || Any <- D].    

all_keys({struct,L})-> 
    [{to_value(Key),to_value2(Val)} || {Key,Val} <- L];
all_keys(List)-> [all_keys(X) || X <- List].

Now, calling struct:all_keys(Struct_object) will give this output:

[{"presence_token",P_token},
 {"presence_time",P_time},
 {"friend_requests",
  [[{"from","Username"},
    {"type","buddy"},
    {"date","DD/MM/YY"},
    {"time","HH:Mins:Secs"},
    {"name","Your Full name"},
    {"email","user@example.com"}]]},
 {"group_status",
  [[{"group_name","ecampus"},
    {"status","running"},
    {"members",["phil","josh","shazz"]},
    {"start_date","DD/MM/YY"},
    {"start_time","HH:Mins:Secs"}],
   [{"group_name","buganda"},{"status","off"}]]},
 {"friend_status",
  [[{"friend","Friend_username"},
    {"status","online"},
    {"log_on_time","HH:Mins:Secs"},
    {"state","available"},
    {"name","Friend_Fullname"},
    {"email","user@example.com"}],
   [{"friend","Friend_username"},
    {"status","offline"},
    {"name","Friend_Fullname"},
    {"email","user@example.com"}]]}]

The above proplist is easier to work with than the struct object. However, you may find another version of the struct module especially in a famous mochiweb example called Sticky Notes whose link i do not have right now.The struct module i have pasted above should be able to help you with using mochijson2. success

I have tried to use your code, and found one mistake. There are two "[[" before "{"from","Username"}. But the code is very helpful for further modification. — Chen Yu, Mar 19 '12 at 13:13

David Weldon · Answer 3 · 2011-06-10T22:06:20.953

0

Based on the structure you described, you could try:

timeline(List) -> timeline(List, []).

timeline([], Result) -> lists:reverse(Result);
timeline([{struct, S}|T], Result) -> timeline(T, [S|Result]);
timeline([{<<"user">>, {struct, S}}|T], Result) -> timeline(T, [S|Result]);
timeline([_|T], Result) -> timeline(T, Result).

I put that code in a module called twitter:

> twitter:timeline([{struct, 1}, {struct, 2}, {<<"user">>, {struct, 3}}, 5]).
[1,2,3]

You may want to replace <<"user">> with _ depending on your specific needs. You probably also want to introduce some kind of exception handling since you are dealing with input from the outside world.

edited Jun 10 '11 at 22:06

answered Jun 10 '11 at 21:56

David Weldon

63,632
11
148
146

Thanks for this David - since I'm very new to Erlang, I'll be iterating exception handling into the application as I go along. Right now, I'm still getting used to not thinking in Python paradigms :-) – peeb Jun 12 '11 at 10:28

How to pattern match structures returned by mochijson2?

3 Answers3