0

So I have created thousands of files using python and now they all have this character "¶" at the end when I use cat to read them. This character however breaks my file as my file is an OpenSSH private key and it doesn't like this character. I can remove it using editors like vi or nano but there has to be an automated way because editing thousands of files manually is pretty much impossible. I have already tried several things like removing the last character/byte of a file but it doesn't work.

image

Does anyone know how to fix that?

schachexpertat
  • 1
  • 2
    It is very unlikely you have the `¶` character; but rather that you have a CR character that shows up in your editor as `¶`, which typically gets put in by Windows tools, and gets hated on by Unix tools. If so, `dos2unix` can do it, as can `vim`, as can many other tools. For example, [How to convert DOS/Windows newline (CRLF) to Unix newline (LF) in a Bash script](https://stackoverflow.com/questions/2613800/how-to-convert-dos-windows-newline-crlf-to-unix-newline-lf-in-a-bash-script). – Amadan Aug 22 '21 at 00:20
  • I have a similar guess to the above, I'd say that's just your shell indicating that the command's output did not end with a newline (specifically, `cat` did not print a trailing newline because the file didn't have one). You can test this hypothesis running this command: `echo -n aaaa`. If it outputs `aaaa¶`, then it is the case. – Leonardo Dagnino Aug 22 '21 at 01:41
  • Ok I have tried this and it seems to be true that it's only my shell and I have just tried the same using the normal bash shell and the "¶" is not there but I still get an invalid format error. Nevertheless, that's another story I assume and was not really part of the question. – schachexpertat Aug 22 '21 at 01:59
  • You should try to find out what bytes that character really is instead of how your specific editor displays it. `hexdump -C` may help – Evert Aug 22 '21 at 02:13
  • Your problem is the `CRLF` (carriage-return line feed) line endings that were created as a result of the python operations. `CR` (`'\r'`) is (`0xd`, e.g. ASCII 13) and `NL` (`'\n'`) is (`0xa`, e.g. ASCII 10) See [ASCII Table and Description](https://www.asciitable.com/). DOS/windows uses CRLF and Linux uses LF for line endings. In WSL, you can run `dos2unix filename` to convert the line endings. (you can use `unix2dox` to go the other way). In WSL you can install the `util-linux` package that provides `hexdump` and do `hexdump -Cv file` to see all characters in hex and ASCII – David C. Rankin Aug 22 '21 at 02:24
  • 1
    Alright guys, thanks for all the help. I have figured out why I get an invalid format error. I just forgot the `0xa` aka. `\n` at the end. Now everything works perfectly fine. – schachexpertat Aug 22 '21 at 02:41

1 Answers1

1

If I were you I will think a little bit out of box ,

you can use notepad ++ editor

1 - Press ctrl + F from your keyboard

2 - Go to (Find In Files Tab)

3 - In the Find What box Enter "¶"

4 - Let (Replace With box ) empty

5 - In the (Directory) Filed , select your code folder path

6 - Click on Replace in files

It will remove unwanted character from all files

Mohammad
  • 36
  • 7
  • 1
    Thanks for showing me this neet trick but in this case it doesn't fix my problem as the CR at the end doesn't seem to be a real character that you can replace with notepad++. It always says that there is no such character in the file. dos2unix does also not remove it unfortunately. – schachexpertat Aug 22 '21 at 01:36