Reading txt file returns string instead of int in bash script. Invalid arithmetic operator

Question

I'm trying to compare the read out number with an int but I get an error message

: integer expression expected: [: 10
: integer expression expected: [: 69
: integer expression expected: [: 1
: integer expression expected: [: 23
: integer expression expected: [: 24
: integer expression expected: [: 69
: integer expression expected: [: 120
: integer expression expected: [: 96
: integer expression expected: [: 21
: integer expression expected: [: 21

The homerseklettest3.txt contains (in each row) an 'x' and 'y' coordinates, a date, a time, and a heat number. I need to write out what the highest heat number is and where is that ('x,y' coordinates).

I've tried removing the hidden characters that maybe there but then I got even more error messages.

#! /bin/bash
highestTemp=-10000
highestTempX=0
highestTempY=0
while read -r line
do
    x=$(echo "$line" | cut -d "," -f 1)
    y=$(echo "$line" | cut -d "," -f 2)
    temp=$(echo "$line" | cut -d "," -f 5) 

    if [ "$temp" -gt $highestTemp ]
    then
        highestTemp=$temp
        highestTempX=$x
        highestTempY=$y
    fi
done < homerseklettest3.txt

echo "Highest Temperature: $highestTemp"
echo "Coordinates: $highestTempX $highestTempY"

The output should be:

Highest Temperature: 120 Coordinates: 92.7418529 99.9999999

Here's the text file from which I read the data.

47.6498634,19.1404118,2003.3.22,19:20,10
23.1231234,69.9651548,2000.11.13,7:42,69
69.6969696,11.1111111,1985.8.25,1:1,1
11.2222222,22.3333333,6969.10.1,18:12,23
47.6498634,43.2312457,2120.2.30,14:14,24
92.7418529,99.9999999,1500.10.9,9:20,69
92.7418529,99.9999999,1760.5.10,5:20,120
23.1231234,69.9651548,2010.8.20,16:36,96
92.7418529,99.9999999,1761.5.10,5:20,21
92.7418529,99.9999999,1760.5.10,5:20,21

`$highestTemp = $temp` and the like should have been `highestTemp=$temp` — M. Nejat Aydin, Nov 12 '21 at 17:45
The whole script could be reduced into a one liner: `IFS=, read highestTempX HighestTempY _ _ highestTemp < <(sort -t, -k5nr homerseklettest3.txt | head -n1)` — M. Nejat Aydin, Nov 12 '21 at 17:54
cut-n-paste your script into [shellcheck.net](https://www.shellcheck.net/) to get a remport of syntax issues, make the suggested changes, then run your script; at that point if you're still having problems ... update the question with a copy of your fixed code and the issues you're experiencing; also, update the question with the expected output from your script — markp-fuso, Nov 12 '21 at 18:05
In the text description, you call the file "testhomerseklet3.txt". In your code, it's "homerseklettest3.txt". Which is it? If it's not the one that's in the code, that's at least part of the problem. — A. R., Nov 12 '21 at 18:15
I've tried all that I could, I appriciate the one liner code effort but that won't work since this is a school project and I'm nowhere near that level. I've updated the post with the latest develpoments. — SnowballAvocado, Nov 12 '21 at 18:20
Remove the pipeline to `tr`. without getting into all the details, that is causing the `temp` variable to be unset. — glenn jackman, Nov 12 '21 at 18:31
Fixed the typo and removed the pipeline, now I've got the same error message I started with. — SnowballAvocado, Nov 12 '21 at 18:40
`done < <(tr -d '\r' < homerseklettest3.txt)` fixes the CRLF problem without introducing a new pipeline-driven problem. This is described in detail in [BashFAQ #24](http://mywiki.wooledge.org/BashFAQ/024). Or just strip the carriage returns from the data after reading it. — Charles Duffy, Nov 12 '21 at 18:50
...wait, your old `tr` was messing with `\n`, not `\r`. It's `\r`s that cause your problem; `\n` is the native/normal/correct way to write a newline on UNIXy platforms, whereas Windows-y platforms use `\r\n` instead -- so to correctly read a file made on Windows from UNIX tools, you want to make it delete or ignore the `\r`s, aka CRs. — Charles Duffy, Nov 12 '21 at 18:53
Also, `while read -r line; do x=$(echo "$line" | cut -d, -f 1)` is silly. Just use `while IFS=$',\r\n' read -r x y _ _ temp; do` and then `read` itself puts the data into the variables you need. — Charles Duffy, Nov 12 '21 at 18:54

David C. Rankin · Answer 1 · 2021-11-12T18:57:11.050

Bash builtin parameter-expansions for substring removal are all you need. The forms available in bash and that are POSIX are:

${var#pattern}      # trim shortest match of pattern from front of $var
${var##pattern}     # trim longest match of pattern from front of $var
${var%pattern}      # trim shortest match of pattern from back of $var
${var%%pattern}     # trim longest match of pattern from back of $var

(note: pattern can contain the normal globbing characters such as '*')

In your case that is simply:

  x="${line%%,*}"       ## trim from back through 1st ,
  y="${line#$x,}"       ## trim $x from front
  y="${y%%,*}"          ## trim from bach through 1st ,
  temp="${line##*,}"    ## trim from front through last ,

Now your script can be written as:

#! /bin/bash

highestTemp=-10000
highestTempX=0
highestTempY=0

fname="${1:-/dev/stdin}"      ## read from file (or stdin if no file)

## set IFS to separate on \r as well, test for non-POSIX eof
while IFS=$' \t\r\n' read -r line || [ -n "$line" ]
do
  x="${line%%,*}"       ## trim from back through 1st ,
  y="${line#$x,}"       ## trim $x from front
  y="${y%%,*}"          ## trim from bach through 1st ,
  temp="${line##*,}"    ## trim from front through last ,
  
  if [ "$temp" -gt "$highestTemp" ]   ## only integer values allowed
  then
    highestTemp=$temp
    highestTempX=$x
    highestTempY=$y
  fi
  
done < "$fname"

printf "Highest Temperature: %s\nCoordinates:  %s %s\n" \
       $highestTemp $highestTempX $highestTempY

Note: don't use command-substitions with echo and a pipe | to invoke utilities like x=$(echo "$line" | cut -d "," -f 1). That is an anti-pattern (bad juju). Bash provides a here string to redirect variables on stdin, e.g. cut -d "," -f 1 <<< "$line" or a here doc for multi-line input. Using parameter expansions is always preferred to invoking utilities for string processing.

Example Use/Output

With your data in file dat/xytemps you would have:

$ bash xytemps.sh dat/xytemps
Highest Temperature: 120
Coordinates:  92.7418529 99.9999999

Note this is trivially done in awk or with sort and tail, but is a good exercise for learning bash. Look things over an let me know if you have further questions.

Doesn't it look like the OP's data contains CRLFs? I don't see anything here that tries to strip trailing CRs. — Charles Duffy, Nov 12 '21 at 18:48
OOF... good catch. Updated to include `\r` in IFS (...fricking DOS files ...) — David C. Rankin, Nov 12 '21 at 18:49
Also, why keep the original `read -r line` at all, instead of using `read` to split on commas directly into the `x`, `y` and `line` variables? — Charles Duffy, Nov 12 '21 at 18:55
It depends. If this exercise was about `IFS`, then sure, split on `','` into separate variables (5 of them). But if this exercise was on string processing (which I assumed) then parameter expansions would be the needed approach. But I get your point, it would be much easier to split into 5 variables and be done with it and avoid the 4 lines of string processing. — David C. Rankin, Nov 12 '21 at 18:58
Thank you so much for the voodoo magic and the explanation but I just needed a quick fix for an university class, I'm in my first year and we have never used these stuffs before, so I'm just gonna go with @CharlesDuffy -s quick solution — SnowballAvocado, Nov 12 '21 at 19:03
Nothing wrong with that. Though I wouldn't consider this much voodoo. It's just basic string processing in bash. Good luck with your course. — David C. Rankin, Nov 12 '21 at 19:04

Reading txt file returns string instead of int in bash script. Invalid arithmetic operator

1 Answers1