Return line number with Ruby readline

Question

searching for a simple ruby/bash solution to investigate a logfile, for example an apache access log.

my log contains lines with beginning string "authorization:"

goal of the script is to return the whole next but one line after this match, which contains the string "x-forwarded-for".

host: 10.127.5.12:8088^M
accept: */*^M
date: Wed, 19 Apr 2019 22:12:36 GMT^M
authorization: FOP ASC-amsterdam-b2c-v7:fkj9234f$t34g34rf=^M
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0)
x-forwarded-for: 195.99.33.222, 10.127.72.254^M
x-forwarded-host: my.luckyhost.com^M
x-forwarded-server: server.luckyhost.de^M
connection: Keep-Alive^M
^M

My question relates to the if condition. How can I get the line number/caller from readline and in second step return the whole next line with x-forwarded-for.

file = File.open(args[:apache_access_log], "r")
log_snapshot = file.readlines
file.close

log_snapshot.reverse_each do |line|
  if line.include? "authorization:"
    puts line

  end
end

https://stackoverflow.com/q/3024372/23915 There are alternatives in this thread that can efficiently read a file backwards, as loading an entire log file in memory, then reversing it, and iterating to get the last few lines can be pretty hard on your system. — Unixmonkey, Apr 25 '19 at 19:27
I am not really sure if that is what you want, but probably its computationally and maintainability-cheaper to run `grep --line-number --after-context 1 logfile.txt` (for *xoid systems) and consume its output with your ruby script. — Felix, Apr 25 '19 at 21:44
I don't understand your question. In your example, do you wish to retrieve the three lines, `"authorization: FOP..."`, `"user-agent: Mozilla/5.0..."` and `"x-forwarded-for:..."` and the line number of the first of these? You said your file contains (sic) 'lines beginning `"authorization:"'`, not `'a line" beginning...'`. What if there are more than one? Do you wish the last one? If so, do you know if it will be near the end of the file? Lastly, in general terms, how large is the log file? Is it OK to read the entire file or, for efficiency, do you wish to read only the end of it? — Cary Swoveland, May 07 '19 at 00:06

score 1 · Accepted Answer · answered Apr 25 '19 at 19:14

Maybe something along these lines:

log_snapshot.each_with_index.reverse_each do |line, n|
  case (line)
  when /authorization:/
    puts '%d: %s' % [ n + 1, line ]
  end
end

Where each_with_index is used to generate 0-indexed line numbers. I've switched to a case style so you can have more flexibility in matching different conditions. For example, you can add the /i flag to do a case-insensitive match really easily or add \A at the beginning to anchor it at the beginning of the string.

Another thing to consider using the block method for File.open, like this:

File.open(args[:apache_access_log], "r") do |f|
  f.readlines.each_with_index.reverse_each do |line, n|
    # ...
  end
end

Where that eliminates the need for an explicit close call. The end of the block closes it for you automatically.

Return line number with Ruby readline

1 Answers1