1

So I have a file and each line has some info and a date (birthday). And I want to print the lines with dates after a given date. I use this awk command 

awk -F '|' 'FNR>1 $dateA<=$5 {print $1" "$2" "$3" "$4" "$5" "$6" "$7" "$8}' $FILE

But it doesnt work properly (all file lines are printed). The dates are in YYYY-MM-DD format so alphabetical order is also chronological.

EDIT: Some lines from the input file

1099511628908|Chen|Wei|female|1985-08-02|2010-05-24T20:52:26.582+0000|27.98.244.108|Firefox
1099511633435|Smith|Jack|male|1981-04-19|2010-05-26T03:45:11.772+0000|50.72.193.218|Internet Explorer
1099511635042|Kiss|Gyorgy|male|1984-09-14|2010-05-16T22:57:41.808+0000|91.137.244.86|Chrome
1099511635218|Law-Yone|Eric|male|1987-01-20|2010-05-26T20:10:22.515+0000|203.81.95.235|Chrome
1099511638444|Jasani|Chris|female|1981-05-22|2010-04-29T20:50:40.375+0000|196.223.11.62|Firefox
2199023256615|Arbelaez|Gustavo|male|1986-11-02|2010-07-17T18:53:47.633+0000|190.96.218.101|Chrome
Jean-François Fabre
  • 137,073
  • 23
  • 153
  • 219
JimS
  • 349
  • 1
  • 4
  • 19
  • 1
    Is it the exact command you are typing? You are using single quotes in this example, so note that your `$dateA` variable might not be expanded. – Qeole Oct 09 '16 at 19:52
  • expanded? what do you mean? – JimS Oct 09 '16 at 19:53
  • I mean in this example `$dateA` will not be replaced by the shell by, e.g., `2016-10-09`, but instead provided as the string `$dateA` to awk. Since awk does not know this variable, it will probably replace it by an empty string. – Qeole Oct 09 '16 at 19:55
  • dateA is a variable initialized earlier in the code I am writing. How can awk perceive dateA as empty string? – JimS Oct 09 '16 at 19:58
  • 2
    If it is a shell variable, embedding it between single quotes will prevent it from being replaced by its value. I mean, try `echo $HOME` vs `echo '$HOME'`. With single quotes, you provide the string `$dateA` to awk instead of the vallue of this variable. – Qeole Oct 09 '16 at 20:02
  • I am confused about how it should be written in order to work – JimS Oct 09 '16 at 20:07
  • Did you get an answer to [your previous question](http://stackoverflow.com/q/39945363/1745001)? If so please mark whichever answer you accept as such. – Ed Morton Oct 09 '16 at 23:06

3 Answers3

6

As it was said by others, a variable in single quotes will not be expanded by the shell. Awk will see the name of the variable, not its value.

One possible solution is to do this (assuming comparing strings is correct):

dateA='1985-01-01'
infile='file to read values from'
awk -F '|' -v dateA="$dateA" '{if (FNR>1 && dateA<=$5) {print}}' "$infile"

A more idiomatic solution (a bit less clear is):

awk 'FNR>1 && dateA<=$5' FS='|' dateA="$dateA" "$infile"

Or (yes, all the quoting is needed):

awk 'FNR>1 && "'"$dateA"'"<=$5' FS='|' "$infile"

But before even thinking of using this option read this about code injection.

Community
  • 1
  • 1
  • Well, someone before you wrote something similar except this works and that didn't. Don't know the reason but thank you very much! :) – JimS Oct 09 '16 at 20:44
  • Awk is not C. What you wrote `{if (FNR>1 && dateA<=$5) {print}}` is C-style syntax, it would be written in awk as just `FNR>1 && dateA<=$5`. – Ed Morton Oct 09 '16 at 23:07
  • Thanks @EdMorton , I thought that this user needed a more clear description of intent, but yes, you are absolutely correct. –  Oct 10 '16 at 01:39
  • @EdMorton On second look, even the print is also optional: `awk 'FNR>1 && dateA<=$5' FS='|' dateA="$dateA" "$infile"` –  Oct 10 '16 at 01:48
  • Right that's why I left it out. Also, do not do `"'"$dateA"'"`. Google it. – Ed Morton Oct 10 '16 at 10:12
  • Arrrgh. @EdMorton You do know that it is almost impossible to "google it" about some odd quotes: ```"'"```. That's not helpful at all, just cryptic. ... ... If you mean to say [your comment here](https://stackoverflow.com/questions/39947847/comparing-dates-using-awk-in-bash/39948321?noredirect=1#comment67179119_39948047), then say that please. –  Oct 10 '16 at 18:42
  • I googled it for you: https://www.google.com/search?q=pass+shell+variables+to+awk&ie=utf-8&oe=utf-8 – Ed Morton Oct 10 '16 at 19:15
1

For this to work : 

awk -vdateA=1985-01-01 -F '|' 'FNR>1 dateA<=$5' $FILE
    ~~~~~~~~~~~~~~~~~~
Gilles Quénot
  • 173,512
  • 41
  • 224
  • 223
0

Your variable $dateA is a shell variable. If you embed it between single quotes, it will not be interpreted by the shell (it will not be replaced by its value) and will be forwarded “as is” to awk.

Awk receives string $dateA. It believes it is an awk variable, but does not have any value for it, so it uses an empty string instead. All your dates are considered “bigger” to an empty string, so all lines match and are printed.

Several solutions to this. Beside Gilles Quenot's one, where you manually indicate to awk that dateA is a shell variable, you can also take it “out of the quotes” like this:

awk -F '|' 'FNR>1 '$dateA'<=$5 {print $1" "$2" "$3" "$4" "$5" "$6" "$7" "$8}' $FILE
                  ^      ^

This works when the date is an integer value, but not when it is on YYYY-MM-DD format. In the latter case, we need to… escape the variable with quotes to tell awk this is a string.

We want to do something that looks like "$dateA". Since we already added single quotes not to pass $dateA to awk as is, we must add double quotes around it: "'$dateA'". This way, the shell first replaces dateA with its value, then removes the single quotes, and gives "1985-08-23" to awk.

So the line is:

awk -F '|' 'FNR>1 && "'$dateA'"<=$5 {print $1" "$2" "$3" "$4" "$5" "$6" "$7" "$8}' $FILE

Edit: Not sure whether this depends on awk version, but I got a syntax error whith this example and also had to replace FNR>1 '$date'<$5 with FNR>1 && '$date'<$5 (additional &&). GNU awk here.

Qeole
  • 8,284
  • 1
  • 24
  • 52
  • if I double quote it still no change – JimS Oct 09 '16 at 20:18
  • With the additional `&&`, this line does work for me. I set `dateA` to `1985` and it only prints two results. So I don't really know :-/ Did you check your `dateA` variable contains the value you want? – Qeole Oct 09 '16 at 20:20
  • i checked before posting and yes dateA does take the value I give it as an argument. if you put a full date like 1985-08-23 does it still work? edit i checked with 1985 and yes it works but it doesnt work with full dates :/ – JimS Oct 09 '16 at 20:25
  • Do not do this. Never let a shell variable expand to become part of the text of an awk script, it's error-prone and leads to all sorts of cryptic errors. Just set an awk variable from the value of a shell variable and then use the awk variable within the script. – Ed Morton Oct 09 '16 at 23:09