I am trying to write the output to a .txt file but it is only returning the last link that is scraped.
this is the code
from bs4 import BeautifulSoup
import requests
base_url = 'https://www.youtube.com/'
page = requests.get(base_url)
soup = BeautifulSoup(page.content, 'html.parser')
for tag in soup.find_all('a'):
link = tag.get('href')
if str(link).startswith("/"):
print(base_url + link)
if str(link).startswith("http"):
print(link)
f = open("queue.txt", "w")
f.write(link)
this is the output to the txt file:
/news://www.youtube.com/howyoutubeworks?utm_campaign=ytgen&utm_source=ythp&utm_medium=LeftNav&utm_content=txt&u=https%3A%2F%2Fwww.youtube.com%2Fhowyoutubeworks%3Futm_source%3Dythp%26utm_medium%3DLeftNav%26utm_campaign%3Dytgen