23

I want to open a parquet file and view the contents of the table in Intellij. Is there a way to do this currently or with a plugin?

Nikhil Suthar
  • 2,289
  • 1
  • 6
  • 24
nobody
  • 7,803
  • 11
  • 56
  • 91
  • Why does it have to be Intellij? – OneCricketeer Feb 08 '17 at 01:17
  • Like, you could just write a simple java project http://stackoverflow.com/questions/28615511/how-to-read-a-parquet-file-in-a-standalone-java-code – OneCricketeer Feb 08 '17 at 01:32
  • 1
    A simple answer for your question is no, there is no plugin nor a way to do this in IntelliJ. On the other hands, there is the parquet-tools CLI that can help you manipulate parquet files from your shell. That said, I'm voting to close this question as being off topic for asking us to recommend or find a tool. Sorry Ben. – eliasah Feb 08 '17 at 08:28
  • 2
    I would also like to know how to view it in intellij. for those who use intellij to develop scala projects and want to test things locally, it will be convenient to check the parquet files. – yang Jul 30 '18 at 23:06
  • Relevant: https://youtrack.jetbrains.com/issue/BDIDE-2 – breandan Nov 04 '19 at 02:57
  • https://blog.jetbrains.com/scala/2019/10/16/meet-big-data-tools-spark-integration-and-zeppelin-notebooks-in-intellij-idea/ ```January 2020: Integration with HDFS (file explorer) and the Parquet viewer ``` so probably we will have something soon – Oleksandr Lykhonosov Nov 13 '19 at 13:36
  • It does work for me - but only some of the time :\ – jtlz2 Jan 30 '23 at 13:20

4 Answers4

10

You need to install: Avro and Parquet Viewer plugin in order to view this kind of file: https://plugins.jetbrains.com/plugin/12281-avro-and-parquet-viewer

pari
  • 788
  • 8
  • 12
  • 2
    A link to a solution is welcome, but please ensure your answer is useful without it: [add context around the link](//meta.stackexchange.com/a/8259) so your fellow users will have some idea what it is and why it’s there, then quote the most relevant part of the page you're linking to in case the target page is unavailable. [Answers that are little more than a link may be deleted.](//stackoverflow.com/help/deleted-answers) – dippas Jun 08 '20 at 09:54
5

If you just want to open a Parquet file, it is part of the Big Data Tools plugin (JetBrain's official). Just install it and then double click the file it will open in the editor as a table.

enter image description here

bluesummers
  • 11,365
  • 8
  • 72
  • 108
1

The answer for you is no, at least now.
But if the reason you want to view Parquet tables on Intellij is because you want to view Parquet file with GUI tool, I suggest you use tools Bigdata File Viewer.

It's a desktop application to view Parquet and also other binary format data like ORC and AVRO. It's pure Java application so that can be run at Linux, Mac and also Windows.

It supports complex data type like array, map, etc.

enter image description here

Eugene
  • 10,627
  • 5
  • 49
  • 67
1

As @Pari said in a previous answer, you can download the Avro and Parquet Viewer plugin. Even for the community edition.

But in order to see view the files, you need to go to the left-botton part of the IDE. Then click on Avro and Parquet Viewer and you can drag a file form the IDE navigator.

So, you will view your file. Click on 3 for visualizing the file as a table.

enter image description here

I mention this, since double clicking a file in the IDE navigator will show you the UFT-8 representation of a binary file.

FYI: You need to restart the IDE after installing.

A.Casanova
  • 555
  • 4
  • 16