16

I wanted to upload the files to my Google Drive using a Google service account credentials.

I downloaded credential as a JSON file from Google Developer Console, and got credential from it.

Here is my code snippet.

google_drive_service = discovery.build('drive', 'v3',
          credentials=ServiceAccountCredentials.from_json_keyfile_name
                           os.path.join(settings.CLIENT_PATH, settings.CLIENT_SECRET_FILE),
                           scopes=settings.SCOPES))


media = MediaFileUpload(tmp_file_path, mimetype=tmp_file.content_type, resumable=True)
google_drive_service.files().create(body=file_metadata, media_body=media, fields='id').execute()

The code runs and there is no error, however I can't find out the files uploaded to my Google Drive account. I am not sure why files are not uploaded. Would you like to help me to fix this problem?

Linda Lawton - DaImTo
  • 106,405
  • 32
  • 180
  • 449
Snipper03
  • 1,484
  • 4
  • 15
  • 29

2 Answers2

26

The issue you are having is that a service account is not you. you have uploaded a file to the service accounts Google drive account not your personal drive account. Try doing a file list you should see the file.

Suggestion. Take the service account email address and share a directory on your personal Google drive account with it like you would share with any other user. The Service account will then be able to upload to this directory. Just make sure to set the permissions on the file after you upload it granting your personal drive account access to the file. When the file is uploaded it will be owned by the service account.

Linda Lawton - DaImTo
  • 106,405
  • 32
  • 180
  • 449
  • 1
    @DalmTo Thanks! This really fixed my problem. You are really a life saver. Thanks again! – Snipper03 Apr 05 '18 at 10:29
  • 2
    Remember what i said about permissions. You dont want files on your personal drive account that you cant access. I have also never figured out who is charged for the storage space of a file stored by a service account on someones account. Its a mystery. – Linda Lawton - DaImTo Apr 05 '18 at 10:55
  • Hello @DalmTo I'd shared a directory with service account mail address, but file isn't uploaded to shared directory yet. And I can see the file list uploaded in service account google drive. And, I am getting
    mail from Mail delivery system.
    – Snipper03 Apr 05 '18 at 12:20
  • 1
    You need to set the directory in your code. file_metadata should include the parent thats the id of the directory to upload to default is going to be root. Which is the service accounts directory. – Linda Lawton - DaImTo Apr 05 '18 at 12:32
  • Thanks @DalmTo. I have some more question. Is there any way to create folder if not exists, and return existing one if exists? – Snipper03 Apr 05 '18 at 16:12
  • not automatically that would be too easy. you need to check first – Linda Lawton - DaImTo Apr 05 '18 at 17:43
  • its the service account that is going to be charged for the storage space. – Spacemonkey Aug 14 '18 at 20:07
  • 1
    How to access the service account? it's seems like it is a Robot account so it is not possible to access regular apps, the only way I found is by sharing the Service account folder where I uploaded the files with my personal account, but this is hacky, is there any better way? – zakaria amine Jan 16 '19 at 14:25
1

Try this Python terminal client for Google Drive for easy uploading, deleting, listing, sharing files or folders.

client_secret.json

{"installed":{"client_id":"698477346386-5kbs1fh3c6eu46op4qvf30ehp6md8o56.apps.googleusercontent.com","project_id":"proven-dryad-122714","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://accounts.google.com/o/oauth2/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"9j4oMk4HI0ZyPvQrz0jFFA4q","redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]}}

GDrive.py

from __future__ import print_function
import sys
import io
import pip
import httplib2
import os
from mimetypes import MimeTypes



try:
    from googleapiclient.errors import HttpError
    from apiclient import discovery
    import oauth2client
    from googleapiclient.http import MediaFileUpload, MediaIoBaseDownload
    from oauth2client import client
    from oauth2client import tools
except ImportError:
    print('goole-api-python-client is not installed. Try:')
    print('sudo pip install --upgrade google-api-python-client')
    sys.exit(1)
import sys


class Flag:
    auth_host_name = 'localhost'
    noauth_local_webserver = False
    auth_host_port = [8080, 8090]
    logging_level = 'ERROR'


try:
    import argparse

    # flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
    flags = Flag()
except ImportError:
    flags = None

# If modifying these scopes, delete your previously saved credentials
# at ~/.credentials/drive-python-quickstart.json
SCOPES = 'https://www.googleapis.com/auth/drive'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'GDrive'



def get_credentials():

    home_dir = os.path.expanduser('~')
    credential_dir = os.path.join(home_dir, '.credentials')
    if not os.path.exists(credential_dir):
        os.makedirs(credential_dir)
    credential_path = os.path.join(credential_dir,
                                   'drive-python-quickstart.json')

    store = oauth2client.file.Storage(credential_path)
    credentials = store.get()
    if not credentials or credentials.invalid:
        flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
        flow.user_agent = APPLICATION_NAME
        # if flags:
        credentials = tools.run_flow(flow, store, flags)
        # else:  # Needed only for compatibility with Python 2.6
        #     credentials = tools.run(flow, store)
        print('Storing credentials to ' + credential_path)
    return credentials


def upload(path, parent_id=None):
    mime = MimeTypes()
    credentials = get_credentials()
    http = credentials.authorize(httplib2.Http())
    service = discovery.build('drive', 'v3', http=http)

    file_metadata = {
        'name': os.path.basename(path),
        # 'mimeType' : 'application/vnd.google-apps.spreadsheet'
    }
    if parent_id:
        file_metadata['parents'] = [parent_id]

    media = MediaFileUpload(path,
                            mimetype=mime.guess_type(os.path.basename(path))[0],
                            resumable=True)
    try:
        file = service.files().create(body=file_metadata,
                                  media_body=media,
                                  fields='id').execute()
    except HttpError:
        print('corrupted file')
        pass
    print(file.get('id'))


def share(file_id, email):
    def callback(request_id, response, exception):
        if exception:
            # Handle error
            print(exception)
        else:
            print(response.get('id'))

    credentials = get_credentials()
    http = credentials.authorize(httplib2.Http())
    service = discovery.build('drive', 'v3', http=http)
    batch = service.new_batch_http_request(callback=callback)
    user_permission = {
        'type': 'user',
        'role': 'reader',
        'emailAddress': email
    }
    batch.add(service.permissions().create(
        fileId=file_id,
        body=user_permission,
        fields='id',
    ))
    batch.execute()


def listfiles():
    results = service.files().list(fields="nextPageToken, files(id, name,mimeType)").execute()
    items = results.get('files', [])
    if not items:
        print('No files found.')
    else:
        print('Files:')
        print('Filename (File ID)')
        for item in items:
            print('{0} ({1})'.format(item['name'].encode('utf-8'), item['id']))
        print('Total=', len(items))

def delete(fileid):
    service.files().delete(fileId=fileid).execute()


def download(file_id, path=os.getcwd()):
    request = service.files().get_media(fileId=file_id)
    name = service.files().get(fileId=file_id).execute()['name']
    fh = io.BytesIO()
    downloader = MediaIoBaseDownload(fh, request)
    done = False
    while done is False:
        status, done = downloader.next_chunk()
        print(int(status.progress() * 100))
    f = open(path + '/' + name, 'wb')
    f.write(fh.getvalue())
    print('File downloaded at', path)
    f.close()


def createfolder(folder, recursive=False):
    if recursive:
        print('recursive ON')
        ids = {}
        for root, sub, files in os.walk(folder):
            par = os.path.dirname(root)

            file_metadata = {
                'name': os.path.basename(root),
                'mimeType': 'application/vnd.google-apps.folder'
            }
            if par in ids.keys():
                file_metadata['parents'] = [ids[par]]
            print(root)
            file = service.files().create(body=file_metadata,
                                          fields='id').execute()
            id = file.get('id')
            print(id)
            ids[root] = id
            for f in files:
                print(root+'/'+f)
                upload(root + '/' + f, id)
    else:
        print('recursive OFF')
        file_metadata = {
                'name': os.path.basename(folder),
                'mimeType': 'application/vnd.google-apps.folder'
            }
        file = service.files().create(body=file_metadata,
                                          fields='id').execute()
        print(file.get('id'))

if __name__ == '__main__':
    credentials = get_credentials()
    http = credentials.authorize(httplib2.Http())
    service = discovery.build('drive', 'v3', http=http)

    method = sys.argv[1]
    if method == 'upload':
        if os.path.isdir(sys.argv[2]):
            if len(sys.argv) == 4 and sys.argv[3] == 'R':
                createfolder(sys.argv[2], True)
            else:
                createfolder(os.path.basename(sys.argv[2]))

        else:
            upload(sys.argv[2])
    elif method == 'list':
        listfiles()
    elif method == 'delete':
        delete(sys.argv[2])
    elif method == 'download':
        download(sys.argv[2], sys.argv[3])
    elif method == 'share':
        share(sys.argv[2], sys.argv[3])
    elif method == 'folder':
        createfolder(sys.argv[2])
    elif method == 'debug':
        print(os.getcwd())
Yogi
  • 609
  • 1
  • 8
  • 21
  • 1
    Thanks. I am building web back-end API and cannot use terminal. And, this scenario is using Oauth2 to authenticate, which is for common user - not for Google service account. – Snipper03 Apr 05 '18 at 02:55