Recursively replacing non-standard charecters from folders and files on win10

Question

I have many folders with sub-folders of files I get from colleagues with special and even hidden characters like these:

Non standard chars: {µ, 市, ', &, 「 , invisible char, Ü, é, ... }

I am looking for a script or a windows tool that would rename all sub-folders and files in one go by replacing any non-standard character to a standard one according to characters in List X. Bonus: even better would be, if the tool would check List R, and use a replacement rule, if one is defined. If not, it should just replace the non-standard char with "_".

List X: {A-Z, a-z, 0-9, (, ), [,],-, _}
List R: {é->e , ü->u, ä->a, @->at,...} (Replacement rules)

Appreciate any hint to a tool or script.

For the diacritic characters, I would use this: [Converting Unicode string to ASCII](https://stackoverflow.com/a/46660695/1701026) — iRon, Feb 07 '21 at 08:28

js2010 · Accepted Answer · 2021-02-20T14:47:47.737

4

How's this? Replacing specific characters, then anything not between space and tilde or tab, like the unicode snowman.

echo hi | set-content filä.txt,filé.txt,fil☃.txt # in script or ise
$pairs = ('ä','a'), ('é','e'), ('[^ -~\t]','_')
dir -r | where name -match '[^ -~\t]' | rename-item -newname {
  $name = $_.name
  foreach ($pair in $pairs) {
    $name = $name -replace $pair
  }
  $name
} -whatif


What if: Performing the operation "Rename File" on target "Item:
C:\users\admin\foo\filä.txt Destination: C:\users\admin\foo\fila.txt".
What if: Performing the operation "Rename File" on target "Item:
C:\users\admin\foo\filé.txt Destination: C:\users\admin\foo\file.txt".
What if: Performing the operation "Rename File" on target "Item:
C:\users\admin\foo\fil☃.txt Destination: C:\users\admin\foo\fil_.txt".

edited Feb 20 '21 at 14:47

answered Feb 06 '21 at 21:13

js2010

23,033
6
64
66

1

The -replace with $pair. Wow, mind blown. Thanks for that! – Daniel Feb 06 '21 at 21:35
That is very cool. But is this script recursive, so that it goes through folders and fixes all files? – A.T. Feb 20 '21 at 06:36
@A.T. Sure, with get-childitem's -recurse option. – js2010 Feb 20 '21 at 14:35

score 1 · Answer 2 · answered Feb 06 '21 at 20:35

I would parse the file/directory names and if each character didn't fall within an ascii range, replace it with an arbitrary character. You can use the following and build upon it using a hashtable for your List X and List R items.

filter ConvertTo-FileSafe {
    $userString = [char[]]$_
    $charToReplaceWith = '.'
    for ($currentChar = 0; $currentChar -lt $userString.Length; $currentChar++) {
        if ([System.IO.Path]::GetInvalidFileNameChars().Contains($userString[$currentChar])){
            $userString[$currentChar] = $charToReplaceWith
        }
        $previousChar = $userString[$currentChar]
    }
    return ($userString -join '')
}

The question is about transliterating characters, not about replacing invalid filename characters. — Theo, Feb 06 '21 at 20:43

Recursively replacing non-standard charecters from folders and files on win10

2 Answers2