Seleniumbase checkout bot script isn't running fast enough

Question

I need help getting this checkout bot script written in seleniumbase to run faster. I want it to buy anything new posted to a website where things sell out in seconds, which basically means there is a bot faster than mine because I keep getting beat. I have it refreshing the page a few times a second looking for newly posted items. Maybe there is a better way to do this than check for new href links on the main page than writing to file? Or maybe seleniumbase isn't the fastest way to do this?

from seleniumbase import BaseCase
import time

class MyTestClass(BaseCase):
    def save_urls_to_file(self, url_list, file_path):
        with open(file_path, 'a') as file:  # Use 'a' (append) mode instead of 'w' (write) mode
            for url in url_list:
                file.write(url + '\n')

    def read_urls_from_file(self, file_path):
        with open(file_path, 'r') as file:
            return [line.strip() for line in file]

    def get_all_href_links(self):
        elements = self.find_elements('a')
        return [element.get_attribute('href') for element in elements if element.get_attribute('href')]

    def search_for_new_urls(self, main_url, file_path, processed_file_path):
        self.open(main_url)
        all_urls = self.get_all_href_links()

        # Filter out already processed URLs
        processed_urls = self.read_urls_from_file(processed_file_path)
        new_urls = list(set(all_urls) - set(processed_urls))

        if new_urls:
            self.save_urls_to_file(new_urls, file_path)
            with open(processed_file_path, 'a') as processed_file:
                for new_url in new_urls:
                    processed_file.write(new_url + '\n')

            for new_url in new_urls:
                self.run_custom_code(new_url)

    def run_custom_code(self, url):
        self.open(url)
        self.click("div.sqs-add-to-cart-button-inner")
        self.click('div[data-test="continue-to-cart"]')
        self.click('button[data-test="cart-button"]')
        self.type('input[name="email"]', "me@gmail.com", timeout=2)
        self.click('button[data-test="continue-button"]')
        self.type('input[name="fname"]', "John")
        self.type('input[name="lname"]', "Doe")
        self.type('input[data-test="line1"]', "123 Easy")
        self.click('span:contains("Street, Any City, FL, USA")'
        time.sleep(.5)
        self.click('button[data-test="continue-button"]')
        self.click('button[data-test="continue-button"]')
        self.switch_to_frame('//iframe[contains(@name, "privateStripeFrame")]', timeout=2)
        self.find_element('input[name="cardnumber"]').send_keys('4242 4242 4242 4242')
        self.find_element('input[name="exp-date"]').send_keys('00 01')
        self.find_element('input[name="cvc"]').send_keys('346')
        self.switch_to_default_content()
        self.click('button[data-test="continue-button"]')
        self.click('button[data-test="purchase-button"]')        
        time.sleep(2)
        self.save_screenshot("new.png")

    def process_new_urls(self, main_url, file_path, processed_file_path):
        while True:
            self.search_for_new_urls(main_url, file_path, processed_file_path)
            time.sleep(.2)  

    def test_run_script(self):
        main_url = "http://example.com/shop"  
        file_path = "/home/urls.txt"  
        processed_file_path = "/home/processed_urls.txt"  

        self.process_new_urls(main_url, file_path, processed_file_path)

if __name__ == "__main__":
    MyTestClass().test_run_script()

I tried removing time.sleep() but the .5 needs to be in there or it hangs up. I also tried waiting on the element to load but that doesn't work. It's just a limitation of the site.

score 0 · Answer 1 · answered Sep 01 '23 at 15:22

With SeleniumBase, there are some ways you may be able to speed up your tests.

Here are some pytest command-line options that you should know about:

--pls=none --> Set pageLoadStrategy to "none": This strategy causes Selenium to return immediately after the initial HTML content is fully received by the browser.
--sjw --> Skip JS Waits, such as wait_for_ready_state_complete().

Those may help, but it may lead to test flakiness if you use raw selenium commands mixed in.

To remove possible flakiness from that speed-up, swap lines like this:

self.find_element('input[name="cvc"]').send_keys('346')

with this instead:

self.type('input[name="cvc"]', '346')

But, you may also find that using the built-in JS methods are a little faster. So instead of using this:

self.type(selector, text)

you may use this instead:

self.js_type(selector, text)

And there's also a js_click() method:

self.js_click(selector)

There's also --headless mode, which can speed up tests:

pytest --headless

After combining all those changes, you should expect a nice speed boost on your script if done properly.

Seleniumbase checkout bot script isn't running fast enough

1 Answers1