3

today I was working with very large log files and I want to display them via lighttpd on my RPi, but they grow bigger every day and thus soon will take a very long time to load.

To prevent this problem I figured I would be able to have a button to read, say the last 500 lines from this log file. Of course I have not much experience with javascript, but I think it is doable, right?

Anyway, I couldn't find any good tutorial describing how to do this in javascript, although I found this in PHP, but since I'm not working in PHP I cannot use it. So, a example of how to read the last 500 lines in javascript or maybe a button to progressively read 500 lines every time you click on a button starting from the end of the file would be extremely handful for me.

Thanks in advance, guys.

(P.S: I've read it's not possible to read a file with javascript, but with AJAX calls it is possible.)

Community
  • 1
  • 1
Linus
  • 1,516
  • 17
  • 35
  • I don't think this is possible. Why can't you use PHP? When you ask for (request) a file from the server, the server must send the entire file eventually. – soktinpk Jun 30 '14 at 15:35
  • @soktinpk It makes for alot of more configuration with my RPi, I want to keep it as simple as possible, and if that's not possible, I'll probably have to go with PHP anyway.. – Linus Jun 30 '14 at 15:36
  • The only way I could think of, then, is to manually split the large file into separate chunks of 500 lines. – soktinpk Jun 30 '14 at 15:38
  • I agree with soktinpk. Just use AJAX to make a request to the server and then use a server-side language to output only the last 50 lines of the file. If you use PHP, then you might be able to use PHP to save data in a database and to get only the last 50 items in your database in your output file. – Noble Mushtak Jun 30 '14 at 15:38
  • Where is the logfile located? I guess you could simply create a webservice that reads the last *x* lines and return that. So, do the "filtering" on the server and let the client handle the presentation only. An example of how to read the last *x* lines could be, `tail -n 500 `. – Lasse Christiansen Jun 30 '14 at 15:39
  • @LasseChristiansen-sw_lasse Well, I'm using [lighttpd](http://www.lighttpd.net/) and Arch Linux for my RPi and therefor it's located in the default directory (i.e /srv/http/) – Linus Jun 30 '14 at 15:41
  • Okay, then I suggest that you simply create a web service (using nodejs, php, or whatever server side language you can get to work with lighttpd) and then simply do the filtering there - i.e. using the `tail` command. – Lasse Christiansen Jun 30 '14 at 15:45
  • @LasseChristiansen-sw_lasse Thanks, is there anyway to make a call from javascript to PHP when a button is pressed? I need to process these lines indivudally in javascript.. – Linus Jun 30 '14 at 15:47
  • 2
    You have a lot of options here - if you want a simple example of how to invoke a server side CGI written in Python (hosted using lighttpd on an RPi) I actually made a project based on that a year ago: https://github.com/swlasse/RPiDeskController. But yeah, basically, you need to do the AJAX request using for instance jQuery, have an endpoint listening, handle the request, and parse the response you get back on the client. – Lasse Christiansen Jun 30 '14 at 15:51

3 Answers3

4

No, it's not possible with JavaScript. The client makes a request for a resource at a uniform resource locator - it's up to your server to serve up the last 500 lines if that's the desired response.

Simple unix command tail -500 mylog can be interpreted by a number of server-side scripting languages if PHP isn't available to you.

AlienWebguy
  • 76,997
  • 17
  • 122
  • 145
  • Thanks for the suggestion, it seems like that extra bit of configuration needed is probably worth it anyway to get PHP working, how would I easily invoke the unix command from PHP? – Linus Jun 30 '14 at 15:46
2

Strictly speaking, you might be able to do this by making two HTTP requests:

  1. Make a HEAD request, which should return a Content-Length header, which tells you the size of the file.
  2. Do a GET request with a Content-Range header to request the last N bytes of the file.

Of course, this only works if the lines in the file tend to stay close to a particular average length. So if you know the lines in the file tend to have around 100 bytes, and you want the last 20 lines, you could request the last, say, 2,400 bytes (20 lines ✕ 100 bytes + 20% for good measure). Then you could parse the response to get the last 20 lines.

This is untested (since I don't have a server handy to test against) and doesn't have any error handling, but something like this ought to work, assuming lighttpd is configured to return the Content-Range header with responses to HEAD requests:

var url = "http://example.com/some-file.txt",
    averageLineLength = 100,
    numLinesToFetch   = 20,
    marginForError    = 0.2,
    bytesToFetch      = averageLineLength * numLinesToFetch * (1 + marginForError)
;

$.ajax({ url: url, type: "HEAD" })
  .then( function(_, _, xhr) {
    var fileSize = parseInt(xhr.getResponseHeader('Content-Length')),
        contentRangeStart = fileSize - bytesToFetch,
        contentRangeEnd   = fileSize - 1,
        contentRange      = contentRangeStart + "-" + contentRangeEnd + "/" + fileSize,
        headers           = { "Content-Length": bytesToFetch,
                              "Content-Range":  "bytes " + contentRange }
    ;

    return $.ajax({ url: url, type: "GET", headers: headers });
  } )
  .then( function(data, statusText, xhr) {
    var lines = data.split("\n");
    lines = lines.slice(numLinesToFetch * -1);
    console.log( "Last " + numLinesToFetch + " lines:\n",
                 lines );
  } )
;

It's kind of convoluted, and requires requesting extra data every time (and if you're getting a lot of data there are memory implications, especially with the naive data.split), and I'm not sure if lighthttp is equipped to do this out of the box, but it's an idea!

Jordan Running
  • 102,619
  • 17
  • 182
  • 182
  • Thank you for the suggestion, but it'll probably take more time to implement something like that then it would setting up PHP for lighttpd and configuration it etc. – Linus Jun 30 '14 at 16:12
  • @Linus That's understandable, although I did have fun writing the example code. – Jordan Running Jun 30 '14 at 17:31
0

If you have Ruby handy, here's a quick CGI script that does what's discussed in the comments above:

#!/usr/bin/env ruby
require "cgi"

NUM_LINES    = 20
FILE_TO_TAIL = "/var/log/system.log"
COMMAND      = "| tail -#{NUM_LINES} #{FILE_TO_TAIL}"

cgi = CGI.new
open(COMMAND) {|io| cgi.out("text/plain", &io.method(:read)) }

And here's a lighttpd configuration (not mine) that shows how to point lighttpd at the script.

Jordan Running
  • 102,619
  • 17
  • 182
  • 182