BufferedReader doesn't read all lines from file

Question

I am trying to read /proc/net/xt_qtaguid/stats in Android 6.
Using cat command, I get this:

2 a0 0 0 123456 311 48329 737 48 1
3 b0 0 0 0 0 0 0 0
4 c0 123456 311 48329 737 48 1
5 d0 111111 111 22222 222 33 1

My java code tries to read the file line by line:

File sysDataFile = new File(PATH_TO_FILE);
BufferedReader bufReader = null;
FileReader fileReader = null;
try {
   fileReader = new FileReader(sysDataFile);
   bufReader = new BufferedReader(fileReader);

   String line = null;
   while ((line = bufferedReader.readLine()) != null) {
       // print to console each line
       System.out.println("Line: " + line);
   }
 } catch (IOException e) {
    System.out.println("IOException thrown!");           
 } finally {
   bufReader.close();
   fileReader.close();
 }

When I run the above code, it only print out the first two lines in console:

Line: 2 a0 0 0 123456 311 48329 737 48 1
Line: 3 b0 0 0 0 0 0 0 0

Why?

Answer 1

I am trying to read /proc/net/xt_qtaguid/stats in Android 6.

In Linux, files under /proc/ are not actually files: they are handled by procfs, a special filesystem that executes code each time the file entry is read or written (code being a callback function defined in a kernel module). So those pseudo-files are not static like regular files, but completely dynamic. The size gives an interesting clue: (most of) those files have a length of 0 (as can be seen with ls -l), but when read they show some content.

In short, it is to be expected that reading that same file from 2 different contexts yields 2 different results.

In this instance, the "file" callbacks are handled by xt_qtaguid module for Android, which manages "per-application/delegated data usage monitoring".

This answer says:

this virtual file's read_proc function limit the uid, every application can only read the header and its own line.

The first part is a bit vague but seems to indicate the difference is based on user id, and the module will only "print" 2 lines of data when a regular application reads this file (please note Android assigns a unique user ID to each application and runs it as that user in a separate process).

You don't give enough details but I have to assume that when you print it from adb using cat, you probably don't have the same user id and permissions as when you try to read it from your application. I did not track the exact implementation details (if you want to, the source for this module can be read here), but other variables might come into play.

The doc says:

In the case of applications that provide network data transfer as a service, such as the download manager, media streaming service, etc, it is possible to attribute the ownership of the network data transfer to the UID of the requesting application using the TrafficStats.setThreadStatsUid() function call. The caller must hold the android.permission.MODIFY_NETWORK_ACCOUNTING permission to re-assign the ownership of the network traffic.

So a process/application can use TrafficStats.setThreadStatsUid() in order to get more lines from that file, but that requires MODIFY_NETWORK_ACCOUNTING permission.

Answer 2

Since performance is not an issue, the easiest and safest option would be to launch cat in a process from java redirect the standard output to an stream and read the contents in a reader. Using this as guidance: Run cat command in Android

No need to modify permissions to provide a backtick

Answer 3

The readLine() method reads until the next '\n' character. Hypothetically, if you have two consecutive '\n' characters, then readLine() could return null prematurely. However, if this was the case, then cat should have put an extra line in between those two lines.

At this point you can try two things:

1. check the charset your system file and your program use are the same

2. change this:

   String line = null;
   while ((line = bufferedReader.readLine()) != null) {
       // print to console each line
       System.out.println("Line: " + line);
   }
   

   to this:

   char c = null;
   while ((c = (char)bufferedReader.read()) != null) {
       // print to console each char
       System.out.println("Line: " + c);
   }
   

and see what happens.