Getting Video Links from Youtube Channel in Python Selenium

Question

I am using Selenium in Python to scrape the videos from Youtube channels' websites. Below is a set of code. The line videos = driver.find_elements(By.CLASS_NAME, 'style-scope ytd-grid-video-renderer') repeatedly returns no links to the videos (a.k.a. the print(videos) after it outputs an empty list). How would you modify it to find all the videos on the loaded page?

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome()
driver.get('https://www.youtube.com/wendoverproductions/videos')

videos = driver.find_elements(By.CLASS_NAME, 'style-scope ytd-grid-video-renderer')
print(videos)

urls = []
titles = []
dates = []

for video in videos:
    video_url = video.find_element(by=By.XPATH, value='.//*[@id="video-title"]').get_attribute('href')
    urls.append(video_url)
    video_title = video.find_element(by=By.XPATH, value='.//*[@id="video-title"]').text
    titles.append(video_title)
    video_date = video.find_element(by=By.XPATH, value='.//*[@id="metadata-line"]/span[2]').text
    dates.append(video_date)

score 1 · Accepted Answer · answered Nov 26 '22 at 22:09

Implementation using Selenium:

First of all, I wanna solve the problem meaning wanted to pull data with the help of YouTube API and I'm about to reach the goal but for some API's restrictions like API KEY's rquests restrict and some other complexities, I couldn't grab complete data that's why I go with super powerful Selenium engine as my last resort and it works like a charm.

Full working code as an example:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import time
import pandas as pd
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions()
#All are optional
options.add_experimental_option("detach", True)
options.add_argument("--disable-extensions")
options.add_argument("--disable-notifications")
options.add_argument("--disable-Advertisement")
options.add_argument("--disable-popup-blocking")
options.add_argument("start-maximized")

s=Service('./chromedriver')
driver= webdriver.Chrome(service=s,options=options)

driver.get('https://www.youtube.com/wendoverproductions/videos')
time.sleep(3)

item = []
SCROLL_PAUSE_TIME = 1
last_height = driver.execute_script("return document.documentElement.scrollHeight")

item_count = 180

while item_count > len(item):
    driver.execute_script("window.scrollTo(0,document.documentElement.scrollHeight);")
    time.sleep(SCROLL_PAUSE_TIME)
    new_height = driver.execute_script("return document.documentElement.scrollHeight")

    if new_height == last_height:
        break
    last_height = new_height
    

data = []
try:
    for e in WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, 'div#details'))):
        title = e.find_element(By.CSS_SELECTOR,'a#video-title-link').get_attribute('title')
        vurl = e.find_element(By.CSS_SELECTOR,'a#video-title-link').get_attribute('href')
        views= e.find_element(By.XPATH,'.//*[@id="metadata"]//span[@class="inline-metadata-item style-scope ytd-video-meta-block"][1]').text
        date_time = e.find_element(By.XPATH,'.//*[@id="metadata"]//span[@class="inline-metadata-item style-scope ytd-video-meta-block"][2]').text
        data.append({
            'video_url':vurl,
            'title':title,
            'date_time':date_time,
            'views':views
            })
except:
    pass
    
item = data
print(item)
print(len(item))
# df = pd.DataFrame(item)
# print(df)

OUTPUT:

[{'video_url': 'https://www.youtube.com/watch?v=oL0umpPPe-8', 'title': 'Samsung’s Dangerous Dominance over South Korea', 'date_time': '10 days ago', 'views': '2.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=GBp_NgrrtPM', 'title': 'China’s Electricity Problem', 'date_time': '3 weeks ago', 'views': '1.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=YBNcYxHJPLE', 'title': 'How the World’s Wealthiest People Travel', 'date_time': '1 month ago', 'views': '2.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=iIpPuJ_r8Xg', 'title': 'The US Military’s Massive Global Transportation System', 'date_time': '1 month ago', 'views': '1.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=MY8AB1wYOtg', 'title': 'The Absurd Logistics of Concert Tours', 'date_time': '2 months ago', 'views': '1.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=8xzINLykprA', 'title': 'Money’s Mostly Digital, So Why Is Moving It So Hard?', 'date_time': '2 months ago', 'views': '1.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=f66GfsKPTUg', 'title': 'How This Central African City Became the World’s Most Expensive', 'date_time': '3 months ago', 'views': '3.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=IDLkOWW0_xg', 'title': 'The Simple Genius of NYC’s Water Supply System', 'date_time': '3 months ago', 'views': '1.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=U9jirFqex6g', 'title': 'Europe’s Experiment: Treating Trains Like Planes', 'date_time': '3 months ago', 'views': '2M views'}, {'video_url': 'https://www.youtube.com/watch?v=eoWcQUjNM8o', 'title': 'How the YouTube Creator Economy Works', 'date_time': '4 months ago', 'views': '1M views'}, {'video_url': 'https://www.youtube.com/watch?v=V0Xx0E8cs7U', 'title': 'The Incredible Logistics Behind Weather Forecasting', 'date_time': '4 months ago', 'views': '1.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=v0aGGOK4kAM', 'title': 'Australia Had a Mass-Shooting Problem. Here’s How it Stopped', 'date_time': '5 months ago', 'views': '947K views'}, {'video_url': 'https://www.youtube.com/watch?v=AW3gaelBypY', 'title': 'The Carbon Offset Problem', 'date_time': '5 months ago', 'views': '1.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=xhYl7Jjefo8', 'title': 'Jet Lag: The Game - A New Channel by Wendover Productions', 'date_time': '6 months ago', 'views': '328K views'}, {'video_url': 'https://www.youtube.com/watch?v=oESoI6XxZTg', 'title': 'How to Design a Theme Park (To Take Tons of Your Money)', 'date_time': '6 months ago', 'views': '1.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=AQbmpecxS2w', 'title': 'Why Gas Got So Expensive (It’s Not the War)', 'date_time': '6 months ago', 'views': '3.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=U_7CGl6VWaQ', 'title': 'How Cyberwarfare Actually Works', 'date_time': '7 months ago', 'views': '1.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=R9pxFgJwxFE', 'title': 'The Incredible Logistics Behind Corn Farming', 'date_time': '7 months ago', 'views': '1.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=SrTrpwzVt4g', 'title': 'The Sanction-Fueled Destruction of 
the Russian Aviation Industry', 'date_time': '8 months ago', 'views': '4.3M 'https://www.youtube.com/watch?v=b1JlYZQG3lI', 'title': "Why There are Now So Many Shortages (It's Not COVID)", 'date_time': '1 year ago', 'views': '7.7M 
'https://www.youtube.com/watch?v=N4dOCfWlgBw', 'title': 'The Insane Logistics of Shutting Down the Cruise Industry', 'date_time': '1 year ago', 'views': '2M views'}, {'video_url': 'https://www.youtube.com/watch?v=3CuPqeIJr3U', 'title': "China's Vaccine Diplomacy", 'date_time': '1 year ago', 'views': '916K views'}, {'video_url': 'https://www.youtube.com/watch?v=DlTq8DbRs4k', 'title': "The UK's Failed Experiment in Rail Privatization", 'date_time': '1 year ago', 'views': '1.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=VjiH3mpxyrQ', 'title': 'How to Start an Airline', 'date_time': '1 year ago', 'views': '1.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=pLcqJ2DclEg', 'title': 'The Electric Vehicle Charging Problem', 'date_time': '1 year ago', 'views': '4.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=3gdCH1XUIlE', 'title': "How Air Ambulances (Don't) Work", 'date_time': '1 year ago', 'views': '1M views'}, {'video_url': 'https://www.youtube.com/watch?v=2qanMpnYsjk', 'title': "How Amazon's Super-Complex Shipping System 
Works", 'date_time': '1 year ago', 'views': '2.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=7R7jNWHp0D0', 'title': 'The News You Missed in 2020, From Every Country in the World (Part 2)', 'date_time': '1 year ago', 'views': '1.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=GIFV_Z7Y9_w', 'title': 'The News You Missed in 2020, From Every Country in the World (Part 1)', 'date_time': '1 year ago', 'views': '2.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=KXRtNwUju5g', 'title': "How China Broke the World's Recycling", 'date_time': '1 year ago', 'views': '3.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=fTyUE162lrw', 'title': 
'Why Long-Haul Low-Cost Airlines Always Go Bankrupt', 'date_time': '1 year ago', 'views': '1.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=ZAEydOjNWyQ', 'title': 'How Living at the 
South Pole Works', 'date_time': '2 years ago', 'views': '2.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=_BCY0SPOFpE', 'title': "Egypt's Dam Problem: The Geopolitics of the Nile", 'date_time': '2 years ago', 'views': '1.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=v_rXhuaI0W8', 'title': 'The 8 Flights That Show How COVID-19 Reinvented Aviation', 'date_time': 
'2 years ago', 'views': '1.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=Ongqf93rAcM', 'title': "How to Beat the Casino, and How They'll Stop You", 'date_time': '2 years ago', 'views': '3.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=byW1GExQB84', 'title': 'Distributing the COVID Vaccine: The Greatest Logistics Challenge Ever', 'date_time': '2 years ago', 'views': '911K views'}, {'video_url': 'https://www.youtube.com/watch?v=DTIDCA7mjZs', 'title': 'How to Illegally Cross the Mexico-US Border', 'date_time': '2 years ago', 'views': '1.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=H_akzwzghWQ', 'title': 'How COVID-19 Broke the Airline 
Pricing Model', 'date_time': '2 years ago', 'views': '2.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=7C1fPocIFgU', 'title': 'The Broken Economics of Organ Transplants', 'date_time': '2 years ago', 'views': '600K views'}, {'video_url': 'https://www.youtube.com/watch?v=3J06af5xHD0', 'title': 'The Final Years of Majuro [Documentary]', 'date_time': '2 years ago', 'views': '2M views'}, {'video_url': 'https://www.youtube.com/watch?v=YgiMqePRp0Y', 'title': 'The Logistics of Covid-19 Testing', 'date_time': '2 years ago', 'views': '533K views'}, {'video_url': 'https://www.youtube.com/watch?v=Rtmhv5qEBg0', 'title': "Airlines' Protocol for After a Plane Crash", 'date_time': '2 years ago', 'views': '1.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=6GMoUmvw8kU', 'title': 'Why Taiwan and China are Battling over Tiny Island Countries', 'date_time': '2 years ago', 'views': '855K views'}, {'video_url': 'https://www.youtube.com/watch?v=QlPrAKtegFQ', 'title': 'How Long-Haul Trucking Works', 'date_time': '2 years ago', 'views': '2.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=uAG4zCsiA_w', 'title': 'Why Helicopter Airlines Failed', 'date_time': '2 years ago', 'views': '1.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=NtX-Ibi21tU', 'title': 'The Five Rules of Risk', 'date_time': '2 years ago', 'views': '1M 
views'}, {'video_url': 'https://www.youtube.com/watch?v=r2oPk20OHBE', 'title': "Air Cargo's Coronavirus Problem", 'date_time': '2 years ago', 'views': '1.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=ABIkWS_YavM', 'title': 'How Offshore Oil Rigs Work', 'date_time': '2 years ago', 'views': '2.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=-pNBAxx4IRo', 'title': 
"How the US' Hospital Ships Work", 'date_time': '2 years ago', 'views': '1.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=VX2e2iEg_pM', 'title': 'COVID-19: How Aviation is Fighting for Survival', 'date_time': '2 years ago', 'views': '1.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=Ppjv0H-Yt5Q', 'title': 'The Logistics of the US Census', 'date_time': '2 years ago', 'views': '887K views'}, {'video_url': 'https://www.youtube.com/watch?v=QvUpSFGRqEo', 'title': 'How Boeing Will Get the 737 MAX Flying Again', 'date_time': '2 years ago', 'views': '1.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=3Sh7hghljuQ', 'title': 'How China Built a Hospital in 10 Days', 'date_time': '2 years ago', 'views': '2.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=vpcUVOjUrKk', 'title': 'The Business of Ski Resorts', 'date_time': '2 years ago', 'views': '2.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=FCEwPio2bkg', 'title': "American Sports' Battle for China", 'date_time': '2 years ago', 'views': '494K views'}, {'video_url': 'https://www.youtube.com/watch?v=5-QejUTDCWw', 'title': "The World's Most Useful Airport 
[Documentary]", 'date_time': '2 years ago', 'views': '9M views'}, {'video_url': 'https://www.youtube.com/watch?v=RyG7nzteG64', 'title': 'The Logistics of the US Election', 'date_time': '2 years 
ago', 'views': '838K views'}, {'video_url': 'https://www.youtube.com/watch?v=dSw7fWCrDk0', 'title': 'Amtrak’s Grand Plan for Profitability', 'date_time': '2 years ago', 'views': '1.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=5SDUm1bx7Zc', 'title': "Australia's China Problem", 'date_time': '3 years ago', 'views': '6.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=U1a73gdNs0M', 'title': 'The US Government Program That Pays For Your Flights', 'date_time': '3 years ago', 'views': '886K views'}, {'video_url': 'https://www.youtube.com/watch?v=tLS9IK693KI', 'title': 'The Logistics of Disaster Response', 'date_time': '3 years ago', 'views': '938K views'}, {'video_url': 'https://www.youtube.com/watch?v=cnfoTAxhpzQ', 'title': 'Why So Many Airlines are Going Bankrupt', 'date_time': '3 years ago', 'views': '2.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=erS2YMYcZO8', 'title': 'The Logistics of Filming Avengers', 'date_time': '3 
years ago', 'views': '1.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=XjbYloKJX7c', 
'title': "Boeing's China Problem", 'date_time': '3 years ago', 'views': '3.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=A0qt0hdCQtg', 'title': "The US' Overseas Military Base Strategy", 'date_time': '3 years ago', 'views': '3.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=I9ttpHvK6yw', 'title': 'How to Stop an Epidemic', 'date_time': '3 years ago', 'views': '2.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=pJ_LUFBSoqM', 'title': "The NFL's Logistics Problem", 'date_time': '3 years ago', 'views': '2.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=jYPrH4xANpU', 'title': 'The Economics of Private Jets', 'date_time': '3 years 
ago', 'views': '4.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=KgsxapE27NU', 'title': "The World's Shortcut: How the Panama Canal Works", 'date_time': '3 years ago', 'views': '4.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=C1f2GwWLB3k', 'title': 'How Air Traffic 
Control Works', 'date_time': '3 years ago', 'views': '4M views'}, {'video_url': 'https://www.youtube.com/watch?v=u2-ehDQM6TM', 'title': 'Extremities: A New Scripted Podcast from Wendover', 'date_time': '3 years ago', 'views': '193K views'}, {'video_url': 'https://www.youtube.com/watch?v=17oZPYcpPnQ', 'title': "Iceland's Tourism Revolution", 'date_time': '3 years ago', 'views': '1.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=SUsqnD9-42g', 'title': 'Mini Countries Abroad: How Embassies Work', 'date_time': '3 years ago', 'views': '5.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=BfNEOfEGe3I', 'title': 'The Economics That Made Boeing Build the 737 Max', 'date_time': '3 years ago', 'views': '2.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=EkRRo5DN9lI', 'title': 'The Logistics of the International Space Station', 'date_time': '3 years ago', 'views': '3.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=E3jfvncofiA', 'title': 'How Airlines Decide Where to Fly', 'date_time': '3 years ago', 'views': '2.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=xX0ozxrZlEQ', 'title': 'How Rwanda is Becoming the Singapore of Africa', 'date_time': '3 years ago', 'views': '6.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=9poImReDFeY', 'title': 'How Freight Trains Connect the World', 'date_time': '3 years ago', 'views': '2.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=69EVxLLhciQ', 'title': 'How Hong Kong Changed Countries', 'date_time': '3 years ago', 'views': '2.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=gdy0gBVWAzE', 'title': 'Living Underwater: 
How Submarines Work', 'date_time': '3 years ago', 'views': '10M views'}, {'video_url': 'https://www.youtube.com/watch?v=bnoUBfLxZz0', 'title': 'The Super-Fast Logistics of Delivering Blood By Drone', 'date_time': '3 years ago', 'views': '1M views'}, {'video_url': 'https://www.youtube.com/watch?v=msjuRoZ0Vu8', 'title': 'The New Economy of the Warming Arctic', 'date_time': '3 years ago', 'views': '844K views'}, {'video_url': 'https://www.youtube.com/watch?v=c0pS3Zx7Fc8', 'title': 'Cities at Sea: How Aircraft Carriers Work', 'date_time': '3 years ago', 'views': '13M views'}, {'video_url': 'https://www.youtube.com/watch?v=TNUomfuWuA8', 'title': 'The Rise of 20-Hour Long Flights', 'date_time': '3 years ago', 'views': '3.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=0JDoll8OEFE', 'title': 'Why China Is so Good at Building Railways', 'date_time': '4 years ago', 'views': '8.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=7cjIWMUgPtY', 'title': 'The Magic Economics of Gambling', 'date_time': '4 years ago', 'views': '2.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=cognzTud3Wg', 'title': 'Why the World is Running Out of 
Pilots', 'date_time': '4 years ago', 'views': '4.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=EodxubsO8EI', 'title': 'How Fighting Wildfires Works', 'date_time': '4 years ago', 'views': '1.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=30XpSozOZII', 'title': 'How to Build a $100 Million Satellite', 'date_time': '4 years ago', 'views': '1.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=zQV_DKQkT8o', 'title': "How Africa is Becoming China's China", 'date_time': '4 years ago', 'views': '8.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=wdU1WTBJMl0', 'title': 'How Airports Make Money', 'date_time': '4 years ago', 'views': '4.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=6OLVFa8YRfM', 'title': 'The Insane Logistics of Formula 1', 'date_time': '4 years ago', 'views': '6.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=jdNDYBt9e_U', 'title': 'The Most Valuable Airspace in the World', 'date_time': '4 years ago', 'views': '5.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=5r90DYjZ76g', 'title': "Guam: Why America's Most Isolated Territory Exists", 'date_time': '4 years ago', 'views': '6M views'}, {'video_url': 'https://www.youtube.com/watch?v=1Y1kJpHBn50', 'title': 'How to Design Impenetrable Airport Security', 'date_time': '4 years ago', 'views': '4.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=hiRBQxHrxNw', 'title': 'Space: The Next Trillion Dollar Industry', 'date_time': '4 years ago', 'views': '3.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=-s3j-ptJD10', 'title': 'The Logistics of Living in Antarctica', 'date_time': '4 
years ago', 'views': '5M views'}, {'video_url': 'https://www.youtube.com/watch?v=y3qfeoqErtY', 'title': 'How Overnight Shipping Works', 'date_time': '4 years ago', 'views': '7.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=IvAvHjYoLUU', 'title': 'Why Cities Exist', 'date_time': 
'4 years ago', 'views': '3.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=72hlr-E7KA0', 'title': 'How Airlines Price Flights', 'date_time': '4 years ago', 'views': '3.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=voozHXadYYE', 'title': 'The Gene Patent Question', 'date_time': '4 years ago', 'views': '587K views'}, {'video_url': 'https://www.youtube.com/watch?v=uU3kLBo_ruo', 'title': 'The Nuclear Waste Problem', 'date_time': '5 years ago', 'views': '3.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=V1YMPk3XhCc', 'title': 'The Little Plane War', 'date_time': '5 years ago', 'views': '4.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=h97fXhDN5qE', 'title': "Elon Musk's Basic Economics", 'date_time': '5 years ago', 'views': 
'5.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=GiBF6v5UAAE', 'title': "China's Geography Problem", 'date_time': '5 years ago', 'views': '12M views'}, {'video_url': 'https://www.youtube.com/watch?v=-cjfTG8DbwA', 'title': 'Why Public Transportation Sucks in the US', 'date_time': '5 years ago', 'views': '3.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=ql0Op1VcELw', 'title': "What's Actually the Plane of the Future", 'date_time': '5 years ago', 'views': '5M views'}, {'video_url': 'https://www.youtube.com/watch?v=RaGG50laHgI', 'title': 'TWL is back! (But not here...)', 'date_time': '5 years ago', 'views': '303K views'}, {'video_url': 'https://www.youtube.com/watch?v=yT9bit2-1pg', 'title': 'How to Stop a Riot', 'date_time': '5 years ago', 'views': '5.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=e-WO-c9xHms', 'title': 'How Geography Gave the US Power', 'date_time': '5 years ago', 'views': '3.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=E7Jfrzkmzyc', 'title': 'Why Chinese Manufacturing Wins', 'date_time': '5 years ago', 'views': '6.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=dGXahSnA_oA', 'title': 'How Airlines Schedule Flights', 'date_time': '5 years ago', 'views': '4M views'}, 
{'video_url': 'https://www.youtube.com/watch?v=N4PW66_g6XA', 'title': 'How to Fix Traffic Forever', 'date_time': '5 years ago', 'views': '3.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=j48Z3W35FI0', 'title': 'How the US Government Will Survive Doomsday', 'date_time': '5 years 
ago', 'views': '4.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=MP1OAm7Pzps', 'title': 'Why the Northernmost Town in America Exists', 'date_time': '5 years ago', 'views': '8.9M views'}, {'video_url': 'https://www.youtube.com/watch?v=lkCeKc1GTMs', 'title': 'Which Country Are International Airports In?', 'date_time': '5 years ago', 'views': '3.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=Nu2WOxXxsHw', 'title': 'How the Post Office Made America', 'date_time': '5 years ago', 'views': '1.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=HSxSgbNQi-g', 'title': 'Small Planes Over Big Oceans (ETOPS Explained)', 'date_time': '5 years ago', 'views': '3.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=ZcDwtO4RWmo', 'title': "Canada's New Shipping Shortcut", 'date_time': '5 years ago', 'views': '3.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=EqWksuyry5w', 'title': 'Why Airlines Sell More Seats Than They Have', 'date_time': '5 years ago', 'views': '2.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=fwjwePe-HmA', 'title': 'Why Trains are so Expensive', 'date_time': '5 years ago', 'views': '3.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=v3C_5bsdQWg', 'title': "Russia's Geography Problem", 'date_time': '5 years ago', 'views': '8.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=BzB5xtGGsTc', 'title': 'The Economics of Airline Class', 'date_time': '5 years 
ago', 'views': '10M views'}, {'video_url': 'https://www.youtube.com/watch?v=n1QEj09Pe6k', 'title': "Why Planes Don't Fly Faster", 'date_time': '5 years ago', 'views': '7.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=YJRqB1xtIxg', 'title': "The US President's $2,614 Per Minute Transport System", 'date_time': '5 years ago', 'views': '11M views'}, {'video_url': 'https://www.youtube.com/watch?v=ancuYECRGN8', 'title': 'Every State in the US', 'date_time': '5 years ago', 'views': '6.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=bL2WPDtLYNU', 'title': 'How 
to Make First Contact', 'date_time': '5 years ago', 'views': '3.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=3PWWtqfwacQ', 'title': 'Why Cities Are Where They Are', 'date_time': '5 years ago', 'views': '5.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=_lj127TKu4Q', 'title': 'Is the European Union a Country?', 'date_time': '5 years ago', 'views': '2M views'}, {'video_url': 'https://www.youtube.com/watch?v=d1CVVoAihBc', 'title': 'Every Country in the World 
(Part 2)', 'date_time': '5 years ago', 'views': '3.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=P-b4SUOfn_4', 'title': 'Every Country in the World (Part 1)', 'date_time': '5 years 
ago', 'views': '8.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=thqbjA2DC-E', 'title': 'The Five Freedoms of Aviation', 'date_time': '6 years ago', 'views': '3.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=_rk2hPrEnk8', 'title': "Why Chicken Sandwiches Don't Cost $1500", 'date_time': '6 years ago', 'views': '3M views'}, {'video_url': 'https://www.youtube.com/watch?v=aQSxPzafO_k', 'title': 'Urban Geography: Why We Live Where We Do', 'date_time': '6 years ago', 'views': '3.3M views'}, {'video_url': 'https://www.youtube.com/watch?v=NlIdzF1_b5M', 'title': 'Big Plane vs Little Plane (The Economics of Long-Haul Flights)', 'date_time': '6 years ago', 'views': '4.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=HpfvOc8HJdg', 'title': 'TWL #10.5: Argleton (+ Patreon)', 'date_time': '6 years ago', 'views': '202K views'}, {'video_url': 
'https://www.youtube.com/watch?v=7ouiTMXuDAQ', 'title': 'How Much Would it Cost to Live on the Moon?', 'date_time': '6 years ago', 'views': '1M views'}, {'video_url': 'https://www.youtube.com/watch?v=-aQ2E0mlRQI', 'title': 'The Plane Highway in the Sky', 'date_time': '6 years ago', 'views': '4.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=mbEfzuCLoAQ', 'title': 'Why Trains Suck in America', 'date_time': '6 years ago', 'views': '5.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=N7CvEt51fz4', 'title': 'How Maritime Law Works', 'date_time': '6 years ago', 'views': '3.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=7PsmkAxVHdM', 'title': 'Why College is so Expensive', 'date_time': '6 years ago', 'views': '1.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=JoYNhX15w4k', 'title': 'TWL #10: The Day Sweden Switched Driving Directions', 'date_time': '6 years ago', 'views': '1.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=1c-jBfZPVv4', 'title': 'TWL #9: The Secret Anti-Counterfeit Symbol', 'date_time': '6 years ago', 'views': '1.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=069y1MpOkQY', 'title': 'How Budget Airlines Work', 'date_time': '6 years ago', 'views': '9.2M views'}, {'video_url': 'https://www.youtube.com/watch?v=n7RHv_MIIT0', 'title': 'TWL #8: Immortality Through Quantum Suicide', 'date_time': '6 years ago', 'views': '1.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=6Oe8T3AvydU', 'title': 'Why Flying is So Expensive', 'date_time': '6 years ago', 
'views': '4.8M views'}, {'video_url': 'https://www.youtube.com/watch?v=LnEyjwdoj7g', 'title': 'TWL #7: This Number is Illegal', 'date_time': '6 years ago', 'views': '3.7M views'}, {'video_url': 
'https://www.youtube.com/watch?v=5XdYbmova_s', 'title': 'TWL #6: Big Mac Economics', 'date_time': '6 years ago', 'views': '1.6M views'}, {'video_url': 'https://www.youtube.com/watch?v=F53TA37Mqck', 'title': 'TWL #5: Timekeeping on Mars', 'date_time': '6 years ago', 'views': '810K views'}, {'video_url': 'https://www.youtube.com/watch?v=O6o5C-i02c8', 'title': 'TWL #4: Which Way Should Toilet Paper Face?', 'date_time': '6 years ago', 'views': '739K views'}, {'video_url': 'https://www.youtube.com/watch?v=JllpzZgAAl8', 'title': 'TWL #3: Paper Towns- Fake Places Made to Catch Copyright Thieves', 'date_time': '6 years ago', 'views': '1.1M views'}, {'video_url': 'https://www.youtube.com/watch?v=v_iaurPRhCs', 'title': 'TWL #2: Bir Tawil- The Land Without a Country', 'date_time': '6 years ago', 'views': '852K views'}, {'video_url': 'https://www.youtube.com/watch?v=_ugvJi2pIck', 'title': 'TWL #1: Presidents Are 4x More Likely to be Lefties', 'date_time': '6 years ago', 'views': '546K views'}, {'video_url': 'https://www.youtube.com/watch?v=RRWggYusSds', 'title': 
"Why Time is One of Humanity's Greatest Inventions", 'date_time': '6 years ago', 'views': '1.4M views'}, {'video_url': 'https://www.youtube.com/watch?v=h6AWAoc_Lr0', 'title': 'Space Law-What Laws are There in Space?', 'date_time': '6 years ago', 'views': '1.5M views'}, {'video_url': 'https://www.youtube.com/watch?v=2H9KEcb74aA', 'title': "A Map Geek's Tour of the World #2", 'date_time': '6 years ago', 'views': '248K views'}, {'video_url': 'https://www.youtube.com/watch?v=ThNeIT7aceI', 'title': 'How the Layouts of Grocery Stores are Secretly Designed to Make You Spend More Money', 'date_time': '6 years ago', 'views': '953K views'}, {'video_url': 'https://www.youtube.com/watch?v=bg2VZIPfX0U', 'title': 'How to Create a Country', 'date_time': '6 years ago', 'views': '3.7M views'}, {'video_url': 'https://www.youtube.com/watch?v=pVB4TEeMcgA', 'title': "A Map Geek's Tour of the World", 'date_time': '6 years ago', 'views': '405K views'}, {'video_url': 'https://www.youtube.com/watch?v=avh7ez858xM', 'title': 'The Messy Ethics of Self Driving Cars', 'date_time': '6 years ago', 'views': '461K views'}, {'video_url': 'https://www.youtube.com/watch?v=Y3jlFtBg0Y8', 'title': 'The Surprising Easternmost Point in the US', 'date_time': '6 years ago', 'views': '213K views'}, {'video_url': 'https://www.youtube.com/watch?v=F-ZskaqBshs', 'title': "Containerization: The Most Influential Invention That You've Never Heard Of", 'date_time': '6 years ago', 'views': '684K views'}, {'video_url': 'https://www.youtube.com/watch?v=Nn-ym8y1_kw', 'title': 'The World is Shrinking', 'date_time': '7 years ago', 'views': '183K views'}, {'video_url': 'https://www.youtube.com/watch?v=8LqqVfPduTs', 'title': 'How Marketers Manipulate Us: Psychological Manipulation in Advertising', 'date_time': '7 years ago', 'views': '458K views'}]

Total: 189

Benjamin Loison · Answer 2 · 2022-12-12T01:10:17.533

If you don't have a YouTube Data API v3 developer key:

The following procedure requires you to have a Google account.

Go to: https://console.cloud.google.com/projectcreate

Click on the CREATE button.

Go to: https://console.cloud.google.com/marketplace/product/google/youtube.googleapis.com

Click on the ENABLE button.

Click on the CREATE CREDENTIALS button.

Choose the Public data option.

Click on the NEXT button.

Note the displayed API Key and continue reading.

If you have a YouTube Data API v3 developer key:

To get the videos of a given YouTube channel id (see this answer if you don't know how to get the channel id of a given YouTube channel), replace its second character (C) to U to obtain its uploads playlist id and provide it as a playlistId to YouTube Data API v3 PlaylistItems: list endpoint.

This is a Python sample code listing videos of a given channel id (don't forget to replace API_KEY with your YouTube Data API v3 developer key):

import requests, json

CHANNEL_ID = 'UC07-dOwgza1IguKA86jqxNA'
PLAYLIST_ID = 'UU' + CHANNEL_ID[2:]
API_KEY = 'AIzaSy...'

URL = f'https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&playlistId={PLAYLIST_ID}&maxResults=50&key={API_KEY}'

pageToken = ''

while True:
    pageUrl = URL
    if pageToken != '':
        pageUrl += f'&pageToken={pageToken}'
    response = json.loads(requests.get(pageUrl).text)
    print(response['items'])
    if 'nextPageToken' in response:
        pageToken = response['nextPageToken']
    else:
        break

For documentation concerning the pagination, see this webpage.

Shreyan Nandanwar · Answer 3 · 2023-04-14T19:50:54.740

This process doesn't need any kind of use of api or channel id. Getting video links with selenium python:

from selenium import webdriver
import os
import re
import json
import pyautogui

from selenium.webdriver.common.by import By

os.environ['PATH'] += r"PATH_chromedriver"


driver = webdriver.Chrome()
driver.get("CHANNEL_URL")
driver.implicitly_wait(1)

video = driver.find_element(By.XPATH, '/html/body/ytdapp/div[1]/ytd-page-manager/ytd-browse/div[3]/ytd-c4-tabbed-header-renderer/tp-yt-app-header-layout/div/tp-yt-app-header/div[2]/tp-yt-app-toolbar/div/div/tp-yt-paper-tabs/div/div/tp-yt-paper-tab[2]')
video.click()

driver.implicitly_wait(10)

driver.execute_script("window.scrollTo(0,document.body.scrollHeight);")
pyautogui.vscroll(-10)
driver.implicitly_wait(5)

html = driver.page_source
links = re.findall(r'href=\"\/watch\?v=(.{11})', html)

video_links = []
for link in links:
    video_links.append("https://www.youtube.com/watch?v=" + link)

video_links = [*set(video_links)]
print(video_links)
print(len(video_links))

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). — Community, Apr 18 '23 at 19:10

Getting Video Links from Youtube Channel in Python Selenium

3 Answers3

Implementation using Selenium:

If you don't have a YouTube Data API v3 developer key:

If you have a YouTube Data API v3 developer key:

Linked