How to scrape Hindi Content only that is wrapped in DIV elements with out any class or id using Python, Beautifulsoup?

Question

I have to scrape all the webpages of Hindi Song Lyrics website. Each page contains song lyrics in Hindi and English.

Target Site: Lyrics Hindi Song

For scraping I am using Python, Beautifulsoup.

I am successfully able to scrape data from all the pages except one task and that is to fetch Hindi Lyrics from each webpage.

Following is the my code.

import pymysql
import requests
from bs4 import BeautifulSoup

r=requests.get("https://www.lyricshindisong.in/2020/02/khuda-bhi-asamaan-se-jab-jmin-par.html")
soup=BeautifulSoup(r.content,'html5lib')
pagettitle=soup.find('h1').text
songcontentwrapper=soup.find('div',{'class':'post-outer'})
targetcontents=songcontentwrapper.find_all('div')
for targetcontent in targetcontents:
    print(targetcontent.text)

The above code gives me following result.

Khuda Bhi Asamaan Se Jab Jmin Par Dekhata Hoga Lyrics from the Hindi Bollywood cinema Dharti(1970). Singer(s) of this song: Mohammed Rafi. Music Composed By Shankar Jaikishan. SongKhuda bhi asamaan se jab jmin par dekhata hogaSinger(s)Mohammed RafiMusic(s)Shankar JaikishanLyrics(s)Rajendra Krishna(राजेंद्र कृष्ण) (adsbygoogle = window.adsbygoogle || []).push({}); खुदा भी आसमाँ से जब ज़मीं पर देखता होगामेरे मेहबूब को किसने बनाया सोचता होगा मुसव्विर खुद परेशां है के ये तस्वीर किसकी हैबनोगी जिसकी तुम ऐसी हसीं तक़दीर किसकी हैकभी वो जल रहा होगा, कभी खुश हो रहा होगा ज़माने भर की मस्ती को निगाहों में समेटा हैकली से जिस्म को कितनी बहारों में लपेटा हैहुआ तुमसा कोई पहले न कोई दूसरा होगा फ़रिश्ते भी यहाँ रातों को आकर घूमते होंगेजहाँ रखती हो तुम पाँव, जगह वो चूमते होंगेकिसी के दिल पे क्या गुज़री, ये वो ही जानता होगा अब आप आपका मनचाहा गाने के बोल अपनी ऊँगली की नोक पर पाएं। डाउनलोड करें हमारा एंड्राइड ऐप्प। Link: Lyrics Hindi Song Android App Khuda bhi asamaan se jab jmin par dekhata hogaMere mehabub ko kisane banaaya sochata hoga Musawwir khud pareshaan hai ke ye taswir kisaki haiBanogi jisaki tum aisi hasin takdir kisaki haiKabhi wo jal raha hoga, kabhi khush ho raha hoga Jmaane bhar ki masti ko nigaahon men sameta haiKali se jism ko kitani bahaaron men lapeta haiHua tumasa koi pahale n koi dusara hoga Frishte bhi yahaan raaton ko akar ghumate hongeJahaan rakhati ho tum paanw, jagah wo chumate hongeKisi ke dil pe kya gujri, ye wo hi jaanata hoga You can access your favourite song lyrics at your finger tips. Download our android app available at Google Play Store. Link: Lyrics Hindi Song Android App Play

You might be interested in the following all Hindi Song Lyrics from the Bollywood Feature Film "Dharti" . Ishq ki main bimar ki walla Teer e nazar hai paar Ki walla tujhse hua hai pyar huyi alla Dil hai bekarar ki walla Aankho mein khumar Ki walla tujhpe dil nishar huyi alla....(Read More) Ye albeli pyar ki rahe Ye jane pahchane rashte Kal bhi the ye nikhre nikhre Aaj bhi hai ye haste haste Ye albeli pyar ki rahe Ye jane pahchane rashte Kal bhi the ye nikhre nikhre Aaj bhi hai ye haste haste Ye albeli pyar ki rahe....(Read More) Khuda bhi aasma se jab Zameen par dekhata hoga Khuda bhi aasma se jab Zameen par dekhata hoga Mere mehboob ko kisane Banaya sochata hoga Khuda bhi aasma se jab Zameen par dekhata hoga Mere mehboob ko kisane Banaya sochata hoga Khuda bhi aasma se....(Read More) Jab se aankhe ho gayi Tumse char is dharti par Jab se aankhe ho gayi Tumse char is dharti par Kadam kadam par machal Raha hai pyaar is dharti par Jab se aankhe ho gayi Tumse char is dharti par....(Read More) Shu shu shu shu shu shu Dhire dheere bolo ji shu shu Bhed mat kholo ji shu shu Dhire dheere bolo ji shu shu Bhed mat kholo ji shu shu Deewaro ke bhi kan hai Khatre mein apni jaaan hai Deewaro ke bhi kan hai Khatre mein apni jaaan hai Shu shu dheere dheere bolo ji shu shu....(Read More) Meri gali mein aaya chor Tan ka chor mann ka chor Dil ko chura ker bhaga Mere dil ko chura ker bhaga Meri gali mein aaya chor Tan ka chor mann ka chor Dil ko chura ker bhaga Mere dil ko chura ker bhaga Meri gali mein aaya chor Tan ka chor mann ka chor....(Read More) Ye mausam bheega bheega hai Hawa bhi jyada jyada hai Kyun na machlega dil mera Tumko pane ka irada hai Ye mausam bheega bheega hai Hawa bhi jyada jyada hai Kyun na machlega dil mera Tumko pane ka irada hai Ye mausam bheega bheega hai....(Read More) You are viewing android version of our website: Lyrics Hindi SongTo visit our website: Click Here

Required task is only to get Hindi Lyrics.

Problem: The song lyrics are wrapped around div tag and it has no class or id.

So please guide me how can I get Hindi Lyrics only using Python and Beautifulsoup.

Answer 1

The simplest approach would probably be to just split your text on each space, remove every word that's not in English, and join the words back together with spaces in between.

See this post Detect strings with non English characters in Python

Using the isEnglish() function from the accepted answer, you could do something like

hindi_only = " ".join([i for i in scraped_text.split(" ") if not isEnglish(i)])