Shell script to rename files by removing substring from initial name

Question

Need a Bash shell script that would rename multiple files by eliminating from the file name the variable length substring contained between two dots, including the first dot. First example: initial file name is "AAAAAAAA.BBBBB-BBB.jpg" and the final file name is "AAAAAAAA.jpg" -- substring ".BBBBB-BBB" was eliminated. Second example: initial file name is "mmmmmmmmmm.vvv-yyyyy.txt" and the final file name is "mmmmmmmmmm.txt" -- substring ".vvv-yyyyy" was eliminated.

My script keeps appending the file extension to the initial name. I am using CYGWIN on a Windows Surface laptop.

#!/bin/bash
for file in *.jpg *.txt; do
  if [[ $file == *.*-*.* ]]; then
    new_file=${file%%.*.*-*}.${file##*.}
    mv "$file" "$new_file"
  fi
done

Gilles Quénot · Answer 1 · 2023-04-20T12:43:57.673

1

Using Perl's rename:

rename -n 's/\.[^-]+-[^-]+(?=\.)//' AAAAAAAA.BBBBB-BBB.jpg 
rename(AAAAAAAA.BBBBB-BBB.jpg, AAAAAAAA.jpg)

Remove -n switch, aka dry-run when your attempts are satisfactory to rename for real.

The regular expression matches as follows:

Node	Explanation
`\.`	.
`[^-]+`	any character except: `-` (1 or more times (matching the most amount possible))
`-`	-
`[^-]+`	any character except: `-` (1 or more times (matching the most amount possible))
`(?=`	look ahead to see if there is:
`\.`	.
`)`	end of look-ahead

edited Apr 20 '23 at 12:43

answered Apr 20 '23 at 12:25

Gilles Quénot

173,512
41
224
223

Your script now appends the name twice. From AAAAAAA.BBBBB-BBB.txt we get AAAAAAA.BBBBB-BBB.txt.AAAAAAA.BBBBB-BBB.txt -- it does not remove the first dot and the following substring up to the file extension. – COLIN GHERGHE Apr 20 '23 at 12:40
The Perl version is usually the way to go, the sed syntax is a killer feature – Gilles Quénot Apr 20 '23 at 12:44

score 1 · Answer 2 · answered Apr 20 '23 at 12:29

The problem is with the first pattern

Try a simpler version of the pattern to cut the prefix part

#!/bin/bash
for file in *.jpg *.txt; do
  if [[ $file == *.*-*.* ]]; then
    new_file=${file%%.*-*.*}.${file##*.}
    mv "$file" "$new_file"    
    #                             
  fi
done

Nikolaj Š. · Answer 3 · 2023-04-22T19:42:57.433

1

for file in *.*-*.{jpg,txt}}; do
    new_file=$(sed 's/\.[^.]\+\././' <<< "$file")
    mv "$file" "$new_file"
done

edited Apr 22 '23 at 19:42

answered Apr 20 '23 at 12:52

Nikolaj Š.

1,457
1
10
17

1

`echo $file` runs you into the problems described in [I just assigned a variable, but `echo $variable` shows something else!](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else). Always `echo "$file"` -- or just use `<<<"$file"` instead of echo at all. – Charles Duffy Apr 22 '23 at 02:17
@CharlesDuffy, thanks, fixed. Missed those – Nikolaj Š. Apr 22 '23 at 07:27

score 0 · Answer 4 · answered Apr 22 '23 at 09:51

A simple version with only bash syntax:

for f in *.*-*.*; do
  [[ -e "$f" ]] && mv "$f" "${f%%.*}.${f##*.}"
done

If you want to work only on some extensions (jpg, txt...):

for f in *.*-*.{jpg,txt}; do
  [[ -e "$f" ]] && mv "$f" "${f%%.*}.${f##*.}"
done

The test [[ -e ... ]] is here just to protect against the absence of file.

RARE Kpop Manifesto · Answer 5 · 2023-04-22T08:27:25.900

UPDATE 1 for Mr. Pedantic :

    # gawk profile, created Sat Apr 22 04:14:46 2023

    # BEGIN rule(s)

    BEGIN {
     1      FS = RS
     1      OFS = "\f\r\t\t\t"
     1      CONVFMT = "%.250g"
    }
  1  ($++NF = escQ(__ = $!_))^_ + \
     ($++NF =    shellQ(__))^_ + \
     ($++NF =  printfQ(__))^_

     3  function escQ(__,_)
    {
     3      return substr("", _ = "\47", 
                    -gsub(_, (_)  "\\" (_)_,__))__)
    }

     2  function shellQ(__, _)
    {
     2      return (_ = "\47") escQ(__)_
    }

     1  function printfQ(__, _)
    {
     1      return " printf " (_ = "\47") "%s"(_)" " shellQ(__)
    }

       nawk :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

      mawk1 :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

      mawk2 :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

   gawk -Se :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

   gawk -ce :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

   gawk -Pe :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

   gawk -Me :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

   gawk -re :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

 gawk -p- -be :: ain't got nothin'
            ain'\''t got nothin'\''
            'ain'\''t got nothin'\'''
             printf '%s' 'ain'\''t got nothin'\'''

echo $'AAAAAAAA.BBBBB-BBB.jpg\nDDDDDDDDD.EEEEEEE-EEEEEE.TXT' | gtee >( gcat -n >&2 ) | 

     1  AAAAAAAA.BBBBB-BBB.jpg
     2  DDDDDDDDD.EEEEEEE-EEEEEE.TXT

nawk '(__=$-_) sub(".+", "mv -v \47"(__)"\47 \47&\47", 
                   $!(NF = NF))^_' FS='[.].+[.]' OFS=.

{m,g}awk      'sub(".+", "mv -v \47"($_)"\47 \47&\47", 
                   $!(NF = NF))^_' FS='[.].+[.]' OFS=.

mv -v 'AAAAAAAA.BBBBB-BBB.jpg' 'AAAAAAAA.jpg'
mv -v 'DDDDDDDDD.EEEEEEE-EEEEEE.TXT' 'DDDDDDDDD.TXT'

...then pipe the output to any light weight shell like dash for execution.

(nawk has a race condition so the original $0 must first be saved, but neither mawk nor gawk has this issue)

How does this handle filenames containing literal quote characters within their text? — Charles Duffy, Apr 22 '23 at 02:18
@CharlesDuffy : that wasn't part of `OP`'s requirements. ::::::::::::::::::::: Go add another `gsub(/\47/, "&\\\\&&")` at the front if you're pedantic about it. And updated main post for mr. pedantic — RARE Kpop Manifesto, Apr 22 '23 at 08:28
"Doesn't break my system's security when the filenames are weird" is an _implicit_ requirement. If someone has to state it out loud in order for their system to not be insecure, then _almost everyone's_ systems will be insecure. Thus -- thank you for the update. — Charles Duffy, Apr 22 '23 at 19:50
@CharlesDuffy : if `posix` weren't so brain dead enough to permit `\n` in filenames life would be so much easier. As for filename security, it's not my job either to do someone else's homework. — RARE Kpop Manifesto, Apr 23 '23 at 13:46
If you consider making sure your advice is fit-to-purpose (and security _is_ a part of fit-to-purpose in the modern world) as "someone else's homework", are you sure it's appropriate to be offering that advice in a public forum where people are expected to be able to professionally apply the things they learn? — Charles Duffy, Apr 23 '23 at 20:43
@CharlesDuffy : and rather ironic coming from the guy who actually believes `POSIX` is fit for anyone else's purpose other than the committee members themselves — RARE Kpop Manifesto, Apr 25 '23 at 02:36

score -2 · Answer 6 · answered Apr 21 '23 at 11:58

-2

Check the below code.

Sample filenames taken from your question.

AAAAAAAA.BBBBB-BBB.jpg
mmmmmmmmmm.vvv-yyyyy.txt

Code:

#!/bin/bash

for file in `ls *.*.{txt,jpg}`
    do
        mv $file $(echo $file | awk -F "." '{print $1"."$NF}')
    done

Output:

# ls -rt | tail -2
mmmmmmmmmm.txt
AAAAAAAA.jpg

answered Apr 21 '23 at 11:58

mrrobot.viewsource

223
6

`for file in *.*.{txt,jpg}` – KamilCuk Apr 21 '23 at 12:51
1

[Why you shouldn't parse the output of `ls`](https://mywiki.wooledge.org/ParsingLs) – Charles Duffy Apr 22 '23 at 02:17

Shell script to rename files by removing substring from initial name

6 Answers6

The regular expression matches as follows: