0
hadoop fs -put popularNames.txt /user/hadoop/dir1/popularNames.txt

My folders are very huge, about 3 TB. I don't want to copy the folder, I want to set the hdfs to the location. How to make it?

Ani Menon
  • 27,209
  • 16
  • 105
  • 126
Gank
  • 4,507
  • 4
  • 49
  • 45
  • 1
    Set hdfs to that location? What is that supposed to mean? – Ani Menon Jun 25 '16 at 04:04
  • The old big folder `D:\folder1 and E:\folder1 ` just make hdfs save on `D:\folder1 and E:\folder1 ` – Gank Jun 25 '16 at 04:29
  • 1
    No it has to be on the hdfs to use hadoop features. Otherwise you are just having the file in you local and hadoop can't run a map/reduce job or maintain replication, etc.. – Ani Menon Jun 25 '16 at 04:34
  • Thanks! My files are all images resource. I don't need map reduce, I only want to visit,save and delete them. Can you have better idea to make it? @AniMenon – Gank Jun 25 '16 at 04:46
  • Use a database. Not hdfs. [Check this](http://stackoverflow.com/questions/2278186/storing-images-in-nosql-stores) – Ani Menon Jun 25 '16 at 05:32
  • You don´t need MapReduce, You don´t need to move your files, then What are you really asking for ? – RojoSam Jun 25 '16 at 15:01

1 Answers1

1

HDFS: Hadoop distributed file system.

You can't add a link to point to a location, because the data must be present in the HDFS(not on local). The whole point of using hadoop is distributed computation, which is possible when your data is distributed on a cluster.

hadoop fs -put had to be used to move the file from your local to the hdfs in order to use hadoop framework.

Ani Menon
  • 27,209
  • 16
  • 105
  • 126
  • But I don't want to move or copy, because the disk capacity is too small to copy. And the older folder is used by old software system. Can I just `hadoop fs -put` and make hdfs save on the folder's old location ? – Gank Jun 25 '16 at 04:29
  • 1
    @Gank no the file had to reside on the hdfs for you to utilize the hadoop framework which includes the map reduce and all the other features. – Ani Menon Jun 25 '16 at 04:32
  • Thanks! My files are all images resource. I don't need map reduce, I only want to visit,save and delete them. Can I make it? – Gank Jun 25 '16 at 04:39
  • You need a database, not hdfs. – Ani Menon Jun 25 '16 at 05:32