0

I have a set of files which resides in local machine (Machine A) and I wanted to load these files to remote machine's HDFS (Machine B). Both machine's IP address are different. Hadoop is installed only on Machine B. How do I write batch script for this. So that running batch script in my local machine will load the files that are in local machine to remote HDFS machine.

NorthCat
  • 9,643
  • 16
  • 47
  • 50
Akshay
  • 3
  • 1
  • 4
  • Can you access Machine B via a mapped network drive? And why does it matter if Hadoop is installed? – SomethingDark Mar 20 '15 at 04:36
  • Yes I can access Machine B via network. Actually the situation is that we have many machines in customers sites and the customer does not want to install hadoop. All that he wants to do is just run the batch file and the batch file is supposed to do loading of files from customer machine to hdfs (which is in different machine). – Akshay Mar 20 '15 at 09:05
  • You should be more specific as to what software/configuration is (dis)allowed on the client machine. Having only "bash" installed on the machine is unrealistic - you have to have other software. What is it about the Hadoop client that's not acceptable? Can you run a service, like ftpd that allows pulling data _from_ the client? – Nickolay Mar 21 '15 at 09:09

1 Answers1

1

You should have install hadoop on your local machine too.But you dont need the local mechine to be part of your hadoop cluster.You just make the local machine as a client to B.Then you can run cmd hadoop fs -put SOME_LOCAL_FILE /HDFS_TARGET_PATH to put your file on local machine to hadoop cluster B

amow
  • 2,203
  • 11
  • 19
  • Thanks amow for the response. Actually the situation is that we have many machines in customers sites and the customer does not want to install hadoop. All that he wants to do is just run the batch file and the batch file is supposed to do loading of files from customer machine to hdfs (which is in different machine). – Akshay Mar 20 '15 at 09:03
  • configuring the client: http://stackoverflow.com/questions/22252349/hadoop-client-node-configuration – Nickolay Mar 21 '15 at 09:10