(grep) Regex to match non-ASCII characters? FROM WINDOWS

Question

I'm developing a pre-commit hook to avoid committing files with non-ascii chars, it works as well from unix system, using the below REGEX:

grep -P -n '[\x80-\xFF]' /tmp/app.txt

Now the issue that is giving me a lot of pain is that when i commit from windows, the result of the grep change, giving me a lot of char more than non ascii chars...

Does someone know how to fix this? I really try a lot of different things..

Can you invert the test and use something like `strings /tmp/app.txt`? — Walter A, May 30 '16 at 17:38
Sounds clever, can you go more in deep, it's not so clear to me what you meant.. — ivoruJavaBoy, May 31 '16 at 08:33
The basic problem is that the range doesn't work for UTF-8. This has been discussed before. — Thomas Dickey, Jun 01 '16 at 08:39

score 0 · Answer 1 · answered May 31 '16 at 21:51

strings -n 1 filename will show the normal characters, but what if you only want to see the kind of file? file filename will show the kind of file but I am afraid it won't work for you.
You might try something like

cat /tmp/app.txt | tr -d "[:print:]\r\n" | wc -c

or avoiding the cat

tr -d "[:print:]\r\n" < /tmp/app.txt | wc -c

(grep) Regex to match non-ASCII characters? FROM WINDOWS

1 Answers1