How can I select part of txt file with a bash script?

Question

I have a logcat.txt file and I have to select only some field from it. I try to explain it better with my specific case:

file.txt

I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412946569,"result":"","class":"android.os.SystemProperties","method":"get","type":"content","args":["debug.second-display.pkg"]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412946637,"result":"","class":"android.os.SystemProperties","method":"get","type":"content","args":["gsm.sim.operator.iso-country"]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412946637,"result":"","class":"android.telephony.TelephonyManager","method":"getSimCountryIso","type":"fingerprint","args":[]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949364,"result":"","class":"android.os.SystemProperties","method":"get","type":"content","args":["gsm.sim.operator.iso-country"]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949364,"result":"","class":"android.telephony.TelephonyManager","method":"getSimCountryIso","type":"fingerprint","args":[]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949365,"class":"android.app.ContextImpl","method":"registerReceiver","type":"binder","args":["horoscope.android.LicenseActivity$3@52828f54",{"mPriority":0,"mActions":["SMS_SENT"],"mHasPartialTypes":false}]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949373,"class":"android.app.ContextImpl","method":"registerReceiver","type":"binder","args":["horoscope.android.LicenseActivity$3@52828f54",{"mPriority":0,"mActions":["SMS_SENT"],"mHasPartialTypes":false}]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949380,"class":"android.app.ContextImpl","method":"registerReceiver","type":"binder","args":["horoscope.android.LicenseActivity$4@52931658",{"mPriority":0,"mActions":["SMS_DELIVERED"],"mHasPartialTypes":false}]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949384,"class":"android.app.ContextImpl","method":"registerReceiver","type":"binder","args":["horoscope.android.LicenseActivity$4@52931658",{"mPriority":0,"mActions":["SMS_DELIVERED"],"mHasPartialTypes":false}]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949404,"class":"javax.crypto.spec.SecretKeySpec","method":"javax.crypto.spec.SecretKeySpec","type":"crypto","args":["\n0x00000000 0A F9 E4 5D BB DB CE 8B 57 27 4D 5A 1C 2A 37 7D ...]....W'MZ.*7}","AES"]}
I/Xposed  ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free:{"timestamp":1463412949404,"class":"javax.crypto.spec.SecretKeySpec","method":"javax.crypto.spec.SecretKeySpec","type":"crypto","args":["\n0x00000000 0A F9 E4 5D BB DB CE 8B 57 27 4D 5A 1C 2A 37 7D ...]....W'MZ.*7}","AES"]}

So from this file I have to select, for each line, only "class":"","method":"". The result must be like this:

{"class":"android.os.SystemProperties","method":"get"}
{"class":"android.os.SystemProperties","method":"get"} {"class":"android.telephony.TelephonyManager","method":"getSimCountryIso"} {"class":"android.os.SystemProperties","method":"get"} {"class":"android.telephony.TelephonyManager","method":"getSimCountryIso"} {"class":"javax.crypto.spec.SecretKeySpec","method":"javax.crypto.spec.SecretKeySpec"} {"class":"javax.crypto.spec.SecretKeySpec","method":"javax.crypto.spec.SecretKeySpec"}

I'm trying to do this with a sed command but I can't.

Please post the sed command you have tried and the output you get — Gary, Apr 04 '17 at 15:16
it is useful because a) it shows you have tried to solve the problem and b) allows someone to solve it and show you what you did wrong — Matthias, Apr 05 '17 at 10:12

score 0 · Answer 1 · answered Apr 04 '17 at 15:30

0

You can try with sed:

sed 's/^.*"class":"([^"]*)".*"method":"([^"]*)".*$/{"class":"\1","method":"\2"}/gp' file.txt -nr

answered Apr 04 '17 at 15:30

Gerry

349
4
8

Gary · Accepted Answer · 2017-04-04T19:17:49.760

0

How about this:

sed 's/.*\("class":"[a-zA-Z\.]*","method":"[a-zA-Z\.]*"\).*/\{\1\}/' logcat.txt

This is basically matching based on one big capturing group. I am assuming that class fields and method fields only contain alpha characters and '.'. You may have to adjust.

edited Apr 04 '17 at 19:17

answered Apr 04 '17 at 15:30

Gary

6,357
5
30
36

Your assumption is correct, but this command returns a void output. – danieledaquale Apr 04 '17 at 18:08
This regex works as long as you use sed with an input file. Answer updated – Gary Apr 04 '17 at 19:21

score 0 · Answer 3 · answered Apr 04 '17 at 15:30

Parsing JSON directly in bash is going to be difficult, and using sed could be hard to maintain (if you want to expand the results-set, or if the JSON key ordering is dynamic).

However, if you're not averse to using inline python, this could be done in a single line:

cut -d: -f3- file.txt | while read line; do \
    echo $line |\
    python -c "import sys, json; obj = json.load(sys.stdin); trimmed = {'class': obj['class'], 'method': obj['method']}; print(json.dumps(trimmed));"; \
done

It's also not the friendliest of solutions, but it will take the JSON-only portion of file.txt, pass it line-by-line to the python code which will select, and output, only the portion(s) you're interested in.

To expand the output, you'll only need to add more to the portion inside the python script.

score 0 · Answer 4 · edited May 23 '17 at 11:46

0

I would always use something like jq when parsing json in bash. Try this

grep -oE '\{.*\}' file.txt | jq -r '{"class":.class,"method":.method}'

The -o flag selects only what was matched, so this grep's everything between the brackets (that's assuming no brackets except in the json) and sends it to jq.

If you know the pattern great.little.war.game.free will always be there, you could also use regex assertions and a lookbehind:

grep -oP '(?<=great.little.war.game.free:).*' file.txt | jq -r '{"class":.class,"method":.method}'

From here.

edited May 23 '17 at 11:46

Community

1
1

answered Apr 04 '17 at 15:34

Matthias

3,160
2
24
38

How can I remove all text before braces { }, in this case I/Xposed ( 2559): Droidmon-apimonitor-com.astrolog.great.little.war.game.free: ? In this way I can use jq to extract the wanted fields. – danieledaquale Apr 04 '17 at 17:53
I was assuming there would always be that line "great.little.war.game.free", and the grep command then does remove everything before the braces. If that's not the case, something else need be done to filter it out – Matthias Apr 05 '17 at 10:09
@danieledaquale I have updated my answer to (I think) solve your problem – Matthias Apr 05 '17 at 10:20

How can I select part of txt file with a bash script?

4 Answers4