1

I need to be able to web scrape information daily on a website that requires login credentials. I have wrote a program in C# that uses Selenium and Chrome, which works well when run normally, or through the Windows Task Scheduler when selecting "run only when user is logged on". However, Chromedriver.exe stalls when selecting "run whether user is logged on or not", presumably because this option forces chrome to run in the background and thus my program is not able to enter in any login credentials. I need to be able to run this program whether the user is logged on or not, because it is on a server where multiple users may be accessing it at a time.

I have tried wrapping the execution of the .exe within a batch file and using different web browsers (firefox/Chrome).

I am looking for

  1. any solutions to make a Selenium app work as a background process with the Task Scheduler option "run whether user is logged on or not"

or

  1. a framework other than Selenium with which I can automate entering text and clicking, and which can work in the non-interactive mode that is launched through the windows Task Scheduler.

Thanks for reading my question.

Vincent Lewis
  • 43
  • 3
  • 1
    Have you tried using WebClient to send and receive HTTP requests programmatically? – mr.coffee Dec 01 '20 at 21:40
  • I haven't, I didn't know that WebClient would be able to navigate and log in to websites. I'll look into it – Vincent Lewis Dec 01 '20 at 21:50
  • 1
    WebClient does not present a UI. You instead send raw HTTP requests. To login you would (making assumptions here) send a POST request to the login URL e.g. /account/login and ensure the correct form fields are present in the request body. – mr.coffee Dec 01 '20 at 23:01
  • 1
    Big thanks mr.coffee, I sent HTTP request programmatically and now I have a program that is functioning as I need it! – Vincent Lewis Dec 03 '20 at 17:44

1 Answers1

2

It seems that you are looking to use "headless chrome". According to Getting Started with Headless Chrome:

A headless browser is a great tool for automated testing and server environments where you don't need a visible UI shell.

Look at the following chrome switches (options):

  • --headless (or "headless")
  • --disable-gpu

Here's are a couple of resources for chrome switches (options):

According to the ChromeOptions Class documentation, you should be able to do the following:

ChromeOptions options = new ChromeOptions();

For use with ChromeDriver:

ChromeDriver driver = new ChromeDriver(options);

The following post may be helpful: How to start ChromeDriver in headless mode

Tu deschizi eu inchid
  • 4,117
  • 3
  • 13
  • 24
  • Unfortunately this does not help - while running headless does eliminate a visual UI, it still does not work with windows Task scheduler. – Vincent Lewis Dec 02 '20 at 19:38