How to parse HAR file to extract text content?

Question

I saved my network data in a har file. Now I want to extract the whole dictionary of content that contains specific word as an indicator to save that dictionary to an array. There are multiple similar dicts in the har file that contain that value and I want to create an array of all the responses.

I am fairly new to python(and coding in general), explainlikeimfive kind explanation will greatly help me.

https://pypi.org/project/haralyzer/ looks like it may be a good fit. It can parse and analyze a har file. — Brian Minton, Jun 10 '20 at 18:10

Anwarvic · Answer 1 · 2021-07-23T09:30:02.710

12

You can use haralyzer module. You can install it easily using pip like so:

pip install haralyzer

The following code uses this sample har file:

>>> import json
>>> from haralyzer import HarParser, HarPage
>>>
>>> with open('sample.har', 'r') as f:
...     har_parser = HarParser(json.loads(f.read()))
>>>
>>> data = har_parser.har_data
>>> type(data)
<class 'dict'>
>>>
>>> data.keys()
dict_keys(['version', 'creator', 'pages', 'entries'])
>>>
>>> har_parser.har_data["pages"]
[{'startedDateTime': '2013-08-24T20:16:16.997Z', 'id': 'page_1', 'title': 'http://ericduran.github.io/chromeHAR/', 'pageTimings': {'onContentLoad': 317, 'onLoad': 406}}]

For more info, check the official GitHub repository.

edited Jul 23 '21 at 09:30

answered Jun 10 '20 at 18:13

Anwarvic

12,156
4
49
69

How can I access specific responses (text that the site was showing to the user) that contain a word? The har file is huge around 200 mb. To clarify the data is shown as a table in the site. So there are multiple table like data in form of dict in the har file – alias micheal Jun 10 '20 at 18:48
could you provide this har file? – Anwarvic Jun 10 '20 at 18:49
How can i provide you the har? It contains alot of data that Im not sure if I can share – alias micheal Jun 10 '20 at 18:58
Then, you can use my code as a starting point and if you found any problem with it, I'm happy to help – Anwarvic Jun 10 '20 at 18:59
I have created a smaller har and changed as much as I could to make it okay How can I send it to you? – alias micheal Jun 10 '20 at 19:13
upload it to a cloud service like GoogleDrive and provide the link in the question. – Anwarvic Jun 10 '20 at 19:17
Hi Anwarvic, would you mind checking your linked in? – alias micheal Jun 10 '20 at 19:28
Heads up people. The "har file" link is not a link but a download. Be careful! – X_Trust Jul 22 '21 at 21:44
1

@X_Trust, I have changed the link. Now, you can view the file before downloading. – Anwarvic Jul 23 '21 at 09:31
1

haralyzer seems to be broken. Unable to read har files with it – Alex Mar 22 '22 at 07:47
@Alex same problem here. Structure issues, i think. – Alessandro Togni May 24 '22 at 07:33

score 1 · Answer 2 · answered Sep 05 '20 at 16:32

Tacking on to the answer from Anwarvic, entries in the HAR file that have a text-based content type contain the actual content in the key entry -> response -> content -> text. So, here is an example printing the content of all such entries.

.... initialize har parser as per documentation ....

for page in har_parser.pages:
    for entry in page.entries:
        # Need to be careful accessing the text property, it will not exist for non text-based responses.
        print(entry['response']['content'].get('text', ''))

From there you can use in or a regex to see if the response text of the entry matches the text you are looking for.

How to parse HAR file to extract text content?

2 Answers2