0

Create a directory on the remote HDFS from localhost

Written code but don't understand what and where to write the core-site.xml and the hdfs-site.xml.

public class HadoopCall {

    public void demomkdir(String dir) throws IOException
    {
        Configuration obj = new Configuration();

        obj.set("fs.defaultFS", "http://datlpdsnn01.pds.in.****.com:50070/");
        FileSystem fs = FileSystem.get(obj);

        Path pth = new Path(dir);
        fs.mkdirs(pth);
        System.out.println("created");
        fs.close(); 
    }

    public static void main(String[] args) throws IOException {
        HadoopCall oj = new HadoopCall();
        oj.demomkdir("user/*******/javacodemkdir");
    }

}
Malathi
  • 2,119
  • 15
  • 40
Abhishek Dutta
  • 75
  • 4
  • 12

1 Answers1

0

Reference : Hadoop Configuration class

As given in the above link:

core-default.xml: Read-only defaults for hadoop.

core-site.xml: Site-specific configuration for a given hadoop installation.

Both are files, with configuration specific to the installation and the default ones. You can add the path of any of the above xml with the method addResource(Path path). If the paths are not given, the xml file paths are inferred from classpath variables.

Malathi
  • 2,119
  • 15
  • 40
  • actually my project is in Eclipse which is in my local machine. I want to know whether I have to create manually core-site.xml, etc. or it will take automatically from the server! Please be in touch because I am not finding anyone with the answer – Abhishek Dutta May 30 '19 at 09:40
  • also could you please tell what are the dependencies I should include in build.gradle I have Hadoop 2.6.0-cdh5.14.2. Also Eclipse Oxygen, Java is 1.8 on the local machine. Hadoop is one some another server. I have two active namenodes. I can use ssh command to login the hdfs. But I am not the admin – Abhishek Dutta May 30 '19 at 09:46
  • 1
    You can use this link to get started: https://www.quickprogrammingtips.com/java/how-to-read-hdfs-file-in-java.html – Malathi May 30 '19 at 10:33
  • My hdfs is not in my local computer. It is on remote server. Do I have to enter the password. Because it is in the remote server I don't know where the core-site.xml is? – Abhishek Dutta May 31 '19 at 04:45
  • check this: https://stackoverflow.com/questions/15941108/hdfs-access-from-remote-host-through-java-api-user-authentication – Malathi May 31 '19 at 05:02
  • Thanks a lot, now it worked. Thank you very much! Keep helping. Thanks – Abhishek Dutta May 31 '19 at 06:02