reading json objects in hadoop map reduce for processing data

Question

iam a beginner in hadoop,can any one help me in reading json in mapreduce job.

i have googled and found jaql is suitable for reading json.but i didnot find any documentaion on how it could be implemented in our map reduce job.

is there any other framework which supports reading json in map reduce?

any suggestions on this?

Thanks in Advance

I am now facing the same confusion. Can you please tell me how you achieved this? — MChirukuri, Jun 26 '15 at 14:42

score 0 · Answer 1 · edited May 23 '17 at 11:58

0

I would rather trust the MapReduce framework itself to handle this. MapReduce allows us to write custom Inout/Output Formats to handle data which is not supported by it OOTB, like JSON. See this question for an example. I would prefer this as I won't require any third party stuff for this. It's just a matter of extending the MapReduce API(But it's just my choice. Other's may find something else more suitable).

But, the easiest way, IMHO, would be to use Hive or Pig to handle JSON data. You don't have to do much in order to make it work, as both these project have OOTB JSON support. See this for Hive-JSON SerDe and this for Pig's JsonLoader and JsonStorage.

HTH

edited May 23 '17 at 11:58

Community

1
1

answered Sep 02 '13 at 08:19

Tariq

34,076
8
57
79

i have googled and read about custom inputformat,but i am totally in confusion on implementing it for parsing my data – user1585111 Sep 02 '13 at 12:09
What's the confusion? – Tariq Sep 02 '13 at 12:13
my input data looks like { "values":[ {},{},{}]},how can i make the record reader know the start and end lines of my json object inside my json array?any idea? – user1585111 Sep 02 '13 at 13:12

reading json objects in hadoop map reduce for processing data

1 Answers1