Using an API key Why can I max retrieve 20k videos? Also why only public videos?

Question

On YouTube API to fetch all videos on a channel

found compact sample PHP under heading "Here is the code that will return all video ids under your channel".

Program shown below.

I expanded the program to fetch various attributes of each video, including ACCESS.

I have a channel with over 20,000 videos and large quote.

The program ran nicely and produced a .csv with video attributes.

It ran for about 2 hours and 10 minutes and stopped at 20,000 videos. In addition it only picked up PUBLIC videos.

How can the above two issues be remedied?

<?php 

// FROM https://stackoverflow.com/questions/18953499/youtube-api-to-fetch-all-videos-on-a-channel/70071113#70071113

    $baseUrl = 'https://www.googleapis.com/youtube/v3/';
    // https://developers.google.com/youtube/v3/getting-started
    $apiKey = 'API_KEY';
    // If you don't know the channel ID see below
    $channelId = 'CHANNEL_ID';

    $params = [
        'id'=> $channelId,
        'part'=> 'contentDetails',
        'key'=> $apiKey
    ];
    $url = $baseUrl . 'channels?' . http_build_query($params);
    $json = json_decode(file_get_contents($url), true);

    $playlist = $json['items'][0]['contentDetails']['relatedPlaylists']['uploads'];

    $params = [
        'part'=> 'snippet',
        'playlistId' => $playlist,
        'maxResults'=> '50',
        'key'=> $apiKey
    ];
    $url = $baseUrl . 'playlistItems?' . http_build_query($params);
    $json = json_decode(file_get_contents($url), true);

    $videos = [];
    foreach($json['items'] as $video)
        $videos[] = $video['snippet']['resourceId']['videoId'];

    while(isset($json['nextPageToken'])){
        $nextUrl = $url . '&pageToken=' . $json['nextPageToken'];
        $json = json_decode(file_get_contents($nextUrl), true);
        foreach($json['items'] as $video)
            $videos[] = $video['snippet']['resourceId']['videoId'];
    }
    print_r($videos);
?>

Answer 1

In addition it only picked up PUBLIC videos.

The code you are currently using uses an API key. API keys are used to access public data only.

If you want to access private data then you will need to be authorized using Oauth2 as a user who has access to the private videos.

It ran for about 2 hours and 10 minutes and stopped at 20,000 videos.

This question is a little harder for me to answer as i cant test it i don't have a YouTube Channel with 20k videos.

I can guess that as you are using an api key there is a limit to the number of videos they will let you download with an api key. They probably dont want people downloading all public videos on YouTube.

I suggest that you try and authorize it with Oauth2 and see if the limit is still there.

php example with Oauth2.

• Create installed app credentials see video
• make sure to enable the YouTube data api under libary.
• change the channelId in this code to your own

Code

<?php

/**
 * Sample PHP code for youtube.search.list
 * See instructions for running these code samples locally:
 * https://developers.google.com/explorer-help/guides/code_samples#php
 */

if (!file_exists(__DIR__ . '/vendor/autoload.php')) {
  throw new Exception(sprintf('Please run "composer require google/apiclient:~2.0" in "%s"', __DIR__));
}
require_once __DIR__ . '/vendor/autoload.php';

$client = new Google_Client();
$client->setApplicationName('API code samples');
$client->setScopes([
    'https://www.googleapis.com/auth/youtube.force-ssl',
]);

// TODO: For this request to work, you must replace
//       "YOUR_CLIENT_SECRET_FILE.json" with a pointer to your
//       client_secret.json file. For more information, see
//       https://cloud.google.com/iam/docs/creating-managing-service-account-keys
$client->setAuthConfig('YOUR_CLIENT_SECRET_FILE.json');
$client->setAccessType('offline');

// Request authorization from the user.
$authUrl = $client->createAuthUrl();
printf("Open this link in your browser:\n%s\n", $authUrl);
print('Enter verification code: ');
$authCode = trim(fgets(STDIN));

// Exchange authorization code for an access token.
$accessToken = $client->fetchAccessTokenWithAuthCode($authCode);
$client->setAccessToken($accessToken);

// Define service object for making API requests.
$service = new Google_Service_YouTube($client);

$queryParams = [
    'channelId' => 'UCyqzvMN8newXIxyYIkFzPvA',
    'forMine' => false
];

$response = $service->search->listSearch('snippet', $queryParams);
print_r($response);

Note: I dont have php installed on this machine so i cant test it but this should be close. Let me know if you have any issues.