0

I'm trying to write a web parser script using requests module. Here is my current code:

import requests
import subprocess
import json
import sys
import threading
import time
from Queue import Queue

numberOfViewers = int(sys.argv[1])
builderThreads = int(sys.argv[2])
startTime = time.time()
numberOfSockets = 0
concurrent = 25
urls = []
urlsUsed = []

def getURL(): # Get tokens
  output = subprocess.Popen(["livestreamer", "twitch.tv/CHANNEL_NAME", "-j"],
  stdout=subprocess.PIPE).communicate()[0]
  return json.loads(output)['streams']['worst']['url'] # Parse json and return the URL parameter

def build(): # Builds a set of tokens, aka viewers
  global numberOfSockets
  global numberOfViewers
  while True:
    if numberOfSockets < numberOfViewers:
      numberOfSockets += 1
      print ("Building viewers " + str(numberOfSockets) + "/" + str(numberOfViewers))
      urls.append(getURL())

def view(): # Opens connections to send views
  global numberOfSockets
  while True:
    url=q.get()
    requests.head(url)
    if (url in urlsUsed):
      urls.remove(url)
      urlsUsed.remove(url)
      numberOfSockets -= 1
    else:
      urlsUsed.append(url)
      q.task_done()

      if __name__ == '__main__':
        for i in range(0, builderThreads):
          threading.Thread(target = build).start()

          while True:
            while (numberOfViewers != numberOfSockets): # Wait until sockets are built
              time.sleep(1)

              q=Queue(concurrent*2)
              for i in range(concurrent):
                try:
                  t=threading.Thread(target=view)
                  t.daemon=True
                  t.start()
                except:
                  print ('thread error')
                  try:
                    for url in urls:
                      print (url)
                      q.put(url.strip())
                      q.join()
                  except KeyboardInterrupt:
                    sys.exit(1)

But when I run the code, it says:

Traceback (most recent call last):
  File "C:\Users\flamelier\Desktop\Twitch.py", line 1, in <module>
    import requests
ImportError: No module named 'requests'

Why am I getting this error? How do I install this module? Will this error keep repeating for all the scripts henceforth? How can I prevent such similar errors in the future?

Harshith Thota
  • 856
  • 8
  • 20
flamelier
  • 13
  • 3

3 Answers3

1

Requests is a 3rd party module. You should first install it to Python using PIP or easy_install.

Harshith Thota
  • 856
  • 8
  • 20
0

You have to run pip3 install requests as requests doesn't come with Python by default, as it is a third party library.

TomServo
  • 7,248
  • 5
  • 30
  • 47
ByteSize
  • 5
  • 3
0

Even after you have pip3-installed requests, the code shown won't do anything. The

    if __name__ == "__main__"

test and everything after it is part of an else block in the view function. Back this line and the block that follows out to the left margin.

verisimilidude
  • 475
  • 6
  • 9