awk trouble outputting a certain column in dumpsys

Question

I am having trouble with awk outputting a certain column from my log file.

Input: a dumpsys meminfo log with multiple snapshots in the file Attempted output: the PSS, Heap Size, Heap Alloc, and Heap Free totals.
Small sidenote: I am formatting the output in js array syntax. I'll handle the closing bracket later

sample input below (several of these snapshots are in the actual file):

Applications Memory Usage (kB):
Uptime: 232535528 Realtime: 544103118

** MEMINFO in pid 28141 [com.test.helloworld] **
                   Pss  Private  Private  Swapped     Heap     Heap     Heap
                 Total    Dirty    Clean    Dirty     Size    Alloc     Free
                ------   ------   ------   ------   ------   ------   ------
  Native Heap        0        0        0        0   246840   217671      648
  Dalvik Heap     8327     8208        0      556    11892    10861     1031
 Dalvik Other     2639     2288      124     1012                           
        Stack      528      528        0        0                           
       Ashmem      138      128        0        0                           
    Other dev    89770    11540    69612        0                           
     .so mmap    10497     2204     7836     8448                           
    .apk mmap      280        0      140        0                           
    .dex mmap     1219       64      444      244                           
   Other mmap      950      576      128      736                           
      Unknown   223484   223468        0       96                           
        TOTAL   337832   249004    78284    11092   258732   228532     1679

 Objects
               Views:        9         ViewRootImpl:        1
         AppContexts:        4           Activities:        1
              Assets:        3        AssetManagers:        3
       Local Binders:       14        Proxy Binders:       24
    Death Recipients:        1
     OpenSSL Sockets:        4

 SQL
         MEMORY_USED:        0
  PAGECACHE_OVERFLOW:        0          MALLOC_SIZE:        0

It is an odd problem. My awk command works at outputting all the columns except the 8th column.

Working example - Without trying to output the 8th column, I output $1 instead and it works as intended. Note - I broke up the awk command into several lines for stackoverflow readability, I actually input it as all one line:

awk -v line_counter="$COUNTER" -v row_operand=0 '       
  BEGIN{row_operand=NR%19;
  if (row_operand == 0){
   print "[[\x27 frame \x27,\x27 Total_PSS_RAM_Usage \x27,\x27 Heap_Size \x27,\x27 Heap_Alloc \x27,\x27 Heap_Free \x27],"
  }
}
{row_operand=NR-19;row_operand=row_operand%32;
   if (row_operand == 0){
      print "["line_counter","$2","$6","$7","$1" ],"
   }
}' logfile.txt

And the output:

[[' frame ',' Total_PSS_RAM_Usage ',' Heap_Size ',' Heap_Alloc ',' Heap_Free '],
[0,337832,258732,228532,228532 ],
[0,338591,258732,227821,227821 ],
[0,338431,214996,202579,202579 ],

Problem example: When trying to output the 8th column, $8, the whole output format breaks and the row_operand value is totally omitted. I am very confused since I am only changing one digit (ie. $1 to $8) to get these different results.

awk -v line_counter="$COUNTER" -v row_operand=0 '
BEGIN{row_operand=NR%19;
  if (row_operand == 0){
     print "[[\x27 frame \x27,\x27 Total_PSS_RAM_Usage \x27,\x27 Heap_Size \x27,\x27 Heap_Alloc \x27,\x27 Heap_Free \x27],"
  }
}
{row_operand=NR-19;row_operand=row_operand%32;
  if (row_operand == 0){print "["line_counter","$2","$6","$7","$8" ],"
  }
}' logfile.txt

And the ouput:

[[' frame ',' Total_PSS_RAM_Usage ',' Heap_Size ',' Heap_Alloc ',' Heap_Free '],
],337832,258732,228532,1679
],338591,258732,227821,2390
],338431,214996,202579,5864

Any help would be appreciated.

score 4 · Accepted Answer · edited May 23 '17 at 11:56

Firstly, I don't think "row_operand value is totally omitted"; both your examples show 3 lines of output - the first with field values "337832,258732,228532". Also, sure enough, the 8th field value for that line does display 1679, which is correct for the sample you've shown.

So, all the data IS there, but the square brackets have been shifted to the start of the output lines.

So, I copied your sample text onto my linux system, and got :

[[' frame ',' Total_PSS_RAM_Usage ',' Heap_Size ',' Heap_Alloc ',' Heap_Free '],
[,337832,258732,228532,1679 ],

However, when I changed the input file from Unix to DOS format, I got :

[[' frame ',' Total_PSS_RAM_Usage ',' Heap_Size ',' Heap_Alloc ',' Heap_Free '],
 ],37832,258732,228532,1679

So that's the problem - you must have saved your file at some stage in Windows.

There are different tools on different systems to do the conversion (eg. dos2unix ) - have a look at How to convert DOS/Windows newline (CRLF) to Unix newline (\n) in a Bash script? - or do an internet search for "dos to unix"

Great sleuthing; a simple way to remove the CRs with standard tools in bash is `sed $'s/\r$//'`. — mklement0, Feb 28 '15 at 06:04
This is the answer. I just needed to run the sed command in your provided link and mklement's comment, "sed $'s/\r$//" . This whole situation is odd since I never saved the file in a windows environment. The file was as-generated by the dumpsys command in an android shell. Either way, great sleuthing indeed. Thank you racraman — Hud Sent, Mar 02 '15 at 19:43

awk trouble outputting a certain column in dumpsys

1 Answers1