-1

I am creating a watcher for my Videos section in youtube which will keep monitoring the stats of latest video uploaded. I am not using Selenium because it will keep the browser engaged and interrupt. But using requests_html to load the /videos page and return me the stats like how many view are now and send me a message on Telegram app.

So when I did requests_html call i retrieve a small json which has all the videos & stats but the command json.load() actually coverts it into dict of dict. The intention is to use the result of json.load() as JSON so that I can perform json parsing.

Attached the snapshot of JSON structure and highlighted the desired keys

I couldnt find any better example that tell me how to convert it into just JSON instead of dict of dict. because there are multiple levels approx 20+ to retrieve the desired key value. I have seen very simple example that do the following, but it is not possible to write those many nodes in the below statement.

aKeyValue = dictName['parentnode']['childNode']

Secondly if it is just JSON then I believe jsonpath_ng and its parse methods can be used to retrieve a desired key with having to provide the complete path from root. If this is tnot the right way, please suggest any other Py Module.

QA Rampage
  • 3
  • 4
  • Just how do you intend to "convert it into just JSON"? Python does not support a native JSON type. It seems that you need to look up how to traverse a dict tree in Python, as that is essentially equivalent to a JSON tree. – Prune Nov 03 '20 at 06:49
  • Well, *of course*, what did you *expect* `json.load` to do? It *deserializes the text-based serializaiton format, i.e. the JSON* into a Python data structure. You perhaps can just use the raw text in whatever json path library you intend to use. To reiterate, *JSON is a text-based serialization format*. Even in JS, it isn't really correct to say "I have a JSON" when you are talking about an actual materialized javascript Object. – juanpa.arrivillaga Nov 03 '20 at 06:51
  • If you find accessing the desired node through a dictionary too cumbersome, you could use a jsonpath library of your choice. – Wups Nov 03 '20 at 06:59
  • If you want search a value through all nested dictionary, refer this article: https://stackoverflow.com/questions/10756427/loop-through-all-nested-dictionary-values – 정도유 Nov 03 '20 at 08:20

1 Answers1

0

the recursive functions didnt work properly. I tried may be 10+ different functions, none worked on the JSON. Finally I found jsonpath-ng to work after I read thru its entire documentation

It was pretty simple, I dont know why I didnt figure this out earlier.

json.loads(jsonString)
parse($..gridRenderer.items[0].gridVideoRenderer.viewCountText.simpleText)

But at the same time the statement below doesnt work. Although statement below wasnt required, but i just tried. It worked on jsonpath evaluator online but not in the python code.

$..gridRenderer.items[0].gridVideoRenderer.viewCountText[?(simpleText="5927 views")]

If someone could point out why the above statement wouldnt work in python code ?

QA Rampage
  • 3
  • 4