1

I've got a crunchy scraping problem in my hands with a lot of javascript that creates session-dependent cookies and I am trying to bypass this question using selenium. I'm using the python driver (python-selenium, version 2.2.0-1 on debian). Without the Remote Driver obtained from selenium-server-standalone-2.39.0.jar the browser was starting but not working (it reported some profile issues). Using the Remote Driver everything is fine except that set_preference is not working (I need to define a browser_profile in order to be able to automatically save some files):

profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2) # custom location 
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', '/tmp')
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/pdf')
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/pdf')
#browser = webdriver.Firefox()
browser = webdriver.Remote(
   command_executor='http://127.0.0.1:4444/wd/hub',
   desired_capabilities=DesiredCapabilities.FIREFOX, browser_profile=profile)

In this way not only the dialog with the save/open choice is opened but examining about:config I do not find the confs to have been set. On this debian I've got Iceweasel 24.2.0 but I've also tried on a ubuntu with plain firefox with no luck.

Any idea about what could be my problem?

vodka
  • 498
  • 2
  • 9

3 Answers3

1

I realize this is a bit late, but I found what I believe is the actual answer to this question.

Turns out that profile.set_preference("browser.download.manager.showWhenStarting",False)

attempts to set a preference that FF doesn't recognize. If you look in about:config at a normal profile, that setting isn't there. At least not as of FF 35. Maybe it was sometime in the past, but no more.

Anyhow, that is apparently enough to make the revised profile invalid, causing FF to reject the whole thing (or possibly causes the Selenium library to crash when passing the profile to FF) at:

browser = webdriver.Remote(..., browser_profile=profile)

Related issue:

https://code.google.com/p/selenium/issues/detail?id=7017

I found that removing that call to set_preference setting allowed the modified profile to take effect.

My tentative conclusion is that the modified profile must have only modifications that FF understands.

gwideman
  • 2,705
  • 1
  • 24
  • 43
0

Could you just make the settings you need and permanently save the profile? You could then load this profile without the need for set_preference.

edit: Also, see Python - Remote Webdriver with Extension installed in it

Community
  • 1
  • 1
sam-6174
  • 3,104
  • 1
  • 33
  • 34
  • Using my Iceweasel profile indeed worked, thanks. It would be interesting to find out why set_preference does not work but right now for me it's ok :) – vodka Feb 18 '14 at 16:42
0

Your solution for this case here This cause of the profile has not be adopted. We need to notify the firefox to update the new profile.

Community
  • 1
  • 1
nđq
  • 388
  • 5
  • 14