0

I want to convert my docx to pdf using onedrive, so i uploaded my docx in onedrive and download it on same function. I am using python django webserver.

def myfunctionname(token,filecontent):
    headers = {"Content-Type": "text/plain"}
    txt = fileContent

    graph_client = OAuth2Session(token=token)
    drive_url = "mywholeurl"
    upload = graph_client.put(drive_url, data=txt, headers=headers)
    download = graph_client.get(drive_url + '?format=pdf')
    return download.url

It took me 5 seconds to upload and download for one request but when i do 20 requests at same time to complete all requests it took around 40 seconds, for 50 concurrent requests it took me around 80 seconds.

I was expecting to get all results in same 5 seconds for any number of requests. Can you explain where i am doing wrong?

Jayesh
  • 46
  • 6

1 Answers1

0

Few points you can consider while implementing functionality like this

1) Do not download the file immediately after upload.

2) Firstly have an operation for uploading files and utilize queue to add the url for the uploaded file like below

import sys
import os
import urllib
import threading
from Queue import Queue

class DownloadThread(threading.Thread):
    def __init__(self, queue, destfolder):
        super(DownloadThread, self).__init__()
        self.queue = queue
        self.destfolder = destfolder
        self.daemon = True

    def run(self):
        while True:
            url = self.queue.get()
            try:
                self.download_url(url)
            except Exception,e:
                print "   Error: %s"%e
            self.queue.task_done()

    def download_url(self, url):
        # change it to a different way if you require
        name = url.split('/')[-1]
        dest = os.path.join(self.destfolder, name)
        print "[%s] Downloading %s -> %s"%(self.ident, url, dest)
        urllib.urlretrieve(url, dest)

def download(urls, destfolder, numthreads=4):
    queue = Queue()
    for url in urls:
        queue.put(url)

    for i in range(numthreads):
        t = DownloadThread(queue, destfolder)
        t.start()

    queue.join()

if __name__ == "__main__":
    download(sys.argv[1:], "/tmp")

3) Last and most importantly, implement Multi-threading while downloading files. Multi threading needs to be implemented while uploading files too.

Check this link for multi-threading in python.

Alternatively try this.

Reference:

http://dag.wiee.rs/home-made/unoconv/

Hope this helps.

Mohit Verma
  • 5,140
  • 2
  • 12
  • 27