0

I currently am trying to modify the block size of the file splits in hadoop through the java file (no I don't want to modify it through the xml file) and the java file that I need to modify is called DFSOutputStream.java (in the org.apache.hadoop.hdfs package). In hadoop 2.7.1 this java file is located in a jar file called hadoop-hdfs-2.7.1.jar and when I extract it, it contains a class file called DFSOutputStream.class. Once I decompile DFSOutputStream.class I retrieve the DFSOutputStream.java file and that is where I can modify the block sizes in the java code.

Because of using the eclipse maven plugin, the dependencies/imports for DFSOutputStream.java works with no errors, all imports are detected and working. However, DFSOutputStream.java can not resolve any of the variables with a variable declaration of a DataStreamer type. Is there a class file I am missing so that this class variable (DataStreamer) work?

Once again, I don't think there is anything wrong with my dependencies because all my imports work without error. it's just that one variable, if I'm wrong, please do enlighten me.

IFH
  • 161
  • 1
  • 2
  • 14

1 Answers1

1

It is probably an issue (or limit or setting) of the decompiler you used, which didn't show the inner class DataStreamer.

Instead of using a decompiler, you can directly look at the sources of the library, the sources classifier jar is available from the Maven repository here.

Downloading it and getting to the concerned class, you will find the correct declaration of the DataStreamer class at line 227:

  class DataStreamer extends Daemon {
     ...

In general, most of the Maven dependencies will always provide their sources (and javadoc) classifier, no need to use a decompiler. Note that Maven doesn't know about sources or javadoc, they are just classifier for the same library and version, so it is not necessarily true that they will always be there, but most probably they will.

If you want to get the sources from Maven without browsing and downloading them manually, you can check another SO question and answer on the topic.

Community
  • 1
  • 1
A_Di-Matteo
  • 26,902
  • 7
  • 94
  • 128