0

I have a file named as market_20140101.csv.

I want the bash script to extract the date from the filename, that date should become directory and the "market_20140101.csv" file is copied inside that directory.

I have around 500 files of the same name with change in date. date is from 20140101 - 20160125.

I need separate date directory and that respective file to be copied inside that directory.

Manish
  • 33
  • 1
  • 8

3 Answers3

1

If you files are in the same directory :

for f in *.csv; do
  dirname=$(sed -n 's/.*\([0-9]\{8\}\)\..*/\1/p' <<< "$f")
  [[ ! -d "$dirname" ]] && mkdir "$dirname" 
  cp "$f" "$dirname"
done

For each file, the date is validated as a 8 characters string before extension.

SLePort
  • 15,211
  • 3
  • 34
  • 44
  • The above code, when I ran, says mkdir: cannot create directory ‘’: No such file or directory – Manish Mar 11 '16 at 11:08
  • the filename are of the following structure - market_20140101.csv, market_20140102.csv, market_20140103.csv ... and so on. all of these files are under single directory. – Manish Mar 11 '16 at 11:13
  • I updated my answer (removed the grep version that was not so safe). It processes files in current directory so script and files must be in the same dir. – SLePort Mar 11 '16 at 11:21
  • 1
    You can avoid the external `sed` call with `${var#market_}` and `${var%.csv}`. This will require slightly more code, but will typically perform a lot better. – tripleee Mar 11 '16 at 11:37
  • 1
    You're right, but it won't make much difference on 500 files. – SLePort Mar 11 '16 at 12:42
  • Hello, I need the script to be much more robust for errors and exceptions – Manish Mar 18 '16 at 05:38
0

A quick solution to extract the date would be: echo "$var" | cut -d'_' -f2 | cut -d'.' -f1

The first cut will split the string based on the _ character and take the second part, which is cut again with respect to the . character and then we take the first part out.

Then you can store the returned name in a variable and make a directory.

I will recommend you to store all the file names in a file with each filename in a separate line. Then cat the line into cut pipe like above and get the list of directory names. After that give it a sort and uniq to get unique directory names.

For example: cut -d'_' -f2 file_with_filenames | cut -d'.' -f1 | sort -u

EDIT: Changes based on @triplee 's comment.

phoxis
  • 60,131
  • 14
  • 81
  • 117
  • I have used below command, due to that I am able to extract date and make directory, but I am not getting how to copy that file in that directory. echo $file | awk -F '[_.]' '{system("mkdir -p "$2"")}' – Manish Mar 11 '16 at 10:09
  • Make source and destination strings and then use `cp` with those strings. – phoxis Mar 11 '16 at 10:13
  • But, I want to copy the file from which date was extracted to the date directory. how to use "cp" in awk. – Manish Mar 11 '16 at 10:25
  • A 20k+ user should know better than post probably the two most common beginner errors. The [useless use of `cat`](http://www.iki.fi/era/unix/award.html) is merely harmless, but the absence of proper quoting is a recipe for disaster. http://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-variable – tripleee Mar 11 '16 at 11:36
  • @tripleee: Yes, in this case the `cat` is useless, but for the given task it is just an indication. Though thanks for highllighting the quote. For the current example in hand, it does not make any different, but, yes proper quoting is necessary. Personally, I would not use bash and maybe resort to Perl for more intense work of this type. – phoxis Mar 11 '16 at 14:48
  • @Manish: Well, I think if you check the awk man page, you can find it out, as the idea will remain same. I cannot comment on awk without referring to the manual, but as the question had a bash tag, I tried to answer it from a bash perspective. I recommend you to see Kenavoz 's answer. – phoxis Mar 11 '16 at 14:51
0

You can use a compound command to create and verify that your directory is created and provide an error if something goes awry:

for i in *.csv; do
    dt=$(sed 's/^.*_\(.*\)[.].*$/\1/'<<<"$i")
    mkdir "$dt" && mv "$i" "$dt" || echo "error creating '$dt'"
done

That just provides a minimum indication beyond the default error from mv in the event something didn't work as expected. You can tailor the message to suit your needs.

note: above the file is moved to the new directory instead of being copied. If you meant copy instead of move, then substitute cp for mv.

David C. Rankin
  • 81,885
  • 6
  • 58
  • 85