3

I've written a script in Python using Selenium to log in to Instagram and then search for some hashtag, as in #NewYorkbarbers, and get the link of that hashtag. My script can successfully log in, click on the Not Now button if Turn on Notifications box shows up, and then put that hashtag in the search box, but I can't make my script initiate that search to produce result against that hashtag.

I've tried so far:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


class InstagramScraper:

    login_url = 'https://www.instagram.com/accounts/login/?source=auth_switcher'

    def __init__(self,username,password):
        self.driver = webdriver.Chrome()
        self.wait = WebDriverWait(self.driver,10)
        self.login(username,password)

    def login(self,username,password):
        self.driver.get(self.login_url)
        self.wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'input[name="username"]'))).send_keys(username)
        self.wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'input[name="password"]'))).send_keys(password)
        self.wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'button[type="submit"]'))).click()
        try:
            self.wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'button.HoLwm'))).click()
        except Exception:pass

    def use_hashtag(self):
        self.wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'input[placeholder="Search"]'))).send_keys("#NewYorkbarbers",Keys.ENTER)

if __name__ == '__main__':
    scraper = InstagramScraper('username','password')
    scraper.use_hashtag()

  • How can I use return within login() method as any ideal method should have return statement within it?

  • How can I produce the result of that hashtag search?

ejderuby
  • 710
  • 5
  • 21
MITHU
  • 113
  • 3
  • 12
  • 41
  • Try this to solve the clicking issue `self.wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,'.drKGC a[href]'))).click()`. However, as this class `drKGC` doesn't look static, it may break in near future. The more robust approach would be to go for xpath, as in `self.wait.until(EC.element_to_be_clickable((By.XPATH,'//*[./input[@placeholder="Search"]]//a[@href]'))).click()` – SIM Jul 15 '19 at 11:54
  • you sure that "not now" click is working? I had to turn off notifications to get that out of the way. – pcalkins Jul 15 '19 at 19:27
  • Yes, I'm sure. I've tested it several times @pcalkins. – MITHU Jul 15 '19 at 20:02
  • What is your expected output?do you want to select the 1st list item `#NewYorkbarbers` and click right(27,673) post , after send_keys("#NewYorkbarbers") ? or you are after something else? – KunduK Jul 17 '19 at 16:04
  • My second question has already been answered in the comment which is working flawlessly. However, I'm after any answer against my first question which is ***How can I use return within login() method as any ideal method should have return statement within it?*** @KunduK. – MITHU Jul 17 '19 at 17:09
  • I couldn't test because my browser update and I didn't update the chromedrive, but try to separate the send_key and the first time you send the value to search, the second time instead to send key.ENTER do: .send_keys(u'\ue007') it's the ascii code for ENTER, some page doesn't accept key.ENTER – Carlo 1585 Jul 18 '19 at 15:49

3 Answers3

1

General Idea of having return statement is to go back. When you have return in a statement it goes back to the place from where it is got called. when u write return value it means go back and take this value. If no return statement is given it means return None

thedudecodes
  • 1,479
  • 1
  • 16
  • 37
  • Okay, I got what you meant. Now for the sake of real-life implementation, how might this `login()` method look like when I comply with your above suggestion @pawan? – MITHU Jul 18 '19 at 11:17
  • You don't have to return anything it's fine. you return anything from a method when you need the return value for any further process. you are logging in when constructor is called. and in your case logging in is important and everything after happens after that. so in your case its okay to not return anything. But if you want you can return value for success and failure of login and if its success then only call the use_hashtag method. – thedudecodes Jul 18 '19 at 15:48
0

You are using Enter key to initiate the search, but if you go and open Instagram in the browser you will know that even manually that is not possible.

Instagram does not initiate the search simply on a single key hit of enter, when you press enter for the first time it will focus on the very first search suggestion which takes a little bit of time to load (which is not guaranteed to be the #tag that you type so it is better to also check text and the click on particular #tag) and when you press the enter second time it will initiate the search for hashtag that is right now focused.

Even this is not enough, in fact, you need to wait for the suggestion dropdown of the search box to open and then find the desired #tag from the suggested item and press enter.

Basically, the script is too fast to give time for search suggestions to appear (Even manually you can hit Enter two times faster then dropdown to appear and search won't be initiated.) and also it requires two times hit of Enter, first will focus the very first suggestion and second will initiate the search.

Eternal
  • 928
  • 9
  • 22
0

In python, not all functions need to return something, and it's not even a convention to have all functions end with a return statement (Unlike other languages such as C). In fact, according to this question, it would be best practice not to have a return statement. But a simple return at the end of the function will be the best option if you really insist on having an explicit return statement.

As for your other problem, a new div appears when the search suggestions have loaded, so you have to wait for that div, then send enter twice.

Here is the new use_hashtag method: (sorry for using xpath and not css selector, but I'm not familiar with css and the chrome developer tools kept giving things dependent on class names):

def use_hashtag(self):
    search_input = self.wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'input[placeholder="Search"]')))
    search_input.send_keys("#NewYorkbarbers")
    self.wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="react-root"]/section/nav/div[2]/div/div/div[2]/div[2]/div[2]')))
    search_input.send_keys(Keys.ENTER, Keys.ENTER)
lol cubes
  • 125
  • 7
  • I tried you solution but it always gives me the wrong results, meaning it picks another result from that option dropdown other than `NewYorkbarbers`. – MITHU Jul 22 '19 at 18:18