1

Say I have two files:

1.json

{"foo":"bar"}\n

2.json

{"foo":"bar"}

when using checksum routines, is there a way to ignore the trailing whitespace?

maybe something like this:

md5sum < <(cat file | trim_somehow)
Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
  • 1
    Trailing newlines (and only newlines) are removed by shell command substitution (if the data fits in shell memory): `printf %s "$(cat file)" | md5sum` – dave_thompson_085 Jul 09 '19 at 06:05

1 Answers1

2

You can use sed or xargs.

xargs is much simpler, but be careful with it. I'm not sure if it is safe to use in this context. Read the comments below this answer https://stackoverflow.com/a/12973694/4330274. (There are a lot of answers to your question in that post).

md5sum < <(cat file | xargs) will delete trailing/leading whitespaces (Also, as stated by dave_thompson_085 on the comments down below, it will compress each whiltespace sequence to one whitespace and it will remove quotation marks and backslashes) from the file before passing it to the md5sum utility.

Note: xargs appends a new line to the end of the input.

I recommend using sed for this purpose. It is much safer. Read this answer https://stackoverflow.com/a/3232433/4330274

3li
  • 598
  • 3
  • 13
  • sure how about all beginning/trailing whitespace? – Alexander Mills Jul 09 '19 at 05:05
  • Oh sorry I misunderstood the question, I will edit the answer. – 3li Jul 09 '19 at 05:06
  • 1
    `xargs` with the default `echo` also compresses any embedded whitespace sequence to a single SP, which isn't usually a problem for JSON, and removes all quotes and many backslashes, which is. – dave_thompson_085 Jul 09 '19 at 06:05
  • what do you mean "with the default echo"? – Alexander Mills Jul 09 '19 at 20:13
  • @AlexanderMills He is referring to the `echo` utility. Try `echo "Alexander 's' / Mills" | xargs`. All the whitespace sequences will be replaced with one whitespace. The quotation marks and the backslash will be removed, which is really a problem in case you use it with JSON since JSON uses quotation marks in its syntax. – 3li Jul 10 '19 at 05:25