browser_switcher_service.cc(238)] XXX Init() error with Python Selenium Script with Chrome for Web Scraping

Question

I was using a small python script along with chrome drivers to download manga from Mangafox. It used to run fine until a few days ago when I updated the Chrome Browser. The following error is shown every time I try it now:

[14664:14280:0420/202509.245:ERROR:browser_switcher_service.cc(238)] XXX Init()
[14664:14280:0420/202509.678:ERROR:device_event_log_impl.cc(162)] [20:25:09.679] Bluetooth: bluetooth_adapter_winrt.cc:1186 Getting Radio failed. Chrome will be unable to change the power state by itself.
[14664:14280:0420/202509.695:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1264 OnPoweredRadioAdded(), Number of Powered Radios: 1
[14664:14280:0420/202509.696:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1283 OnPoweredRadiosEnumerated(), Number of Powered Radios: 1

I have used the selenium module with chrome drivers. I have tried updating my web drivers, tried making the code loop or sleep until the web-page loads completely.

My Code is as follows:

from selenium import webdriver
import os
import urllib.request


Main = 'https://ww3.mangafox.online/'
Name = str(input('Enter Name of Manga as on \'ww3.Mangafox.online\':  '))
Name = Name.replace(' ', '-')
Name = Name.replace('\'', '-')

driver = webdriver.Chrome(r'C:\Users\freak\Assignments\SDP\Drivers\chromedriver.exe')
driver.get(Main + Name)

Tags = driver.find_elements_by_tag_name('a')
List = [] 
for Tag in Tags:
    List.append(str(Tag.get_attribute('href')))

dir = os.path.join('C:\\','Users', 'freak', 'Assignments', 'SDP', 'Test', Name.replace('-', '_'))
print('Checking the existence of folder; %s' % dir)
if not os.path.exists(dir):
    print('Folder not found. Attempting to create folder: %s' % dir)
    os.mkdir(dir)
    print('Folder successfully created')

Index = []
for i, element in enumerate(List):
    if (str(Name) + '/chapter') in element:
        Index.append(i)

Chapters = []

After this part is just a loop I used to download the images. But for some reason, an error is shown and the list made for Tags remains empty. The driver.find_elements_by_tag_name('a') fails completely.

score 5 · Answer 1 · answered Aug 06 '20 at 16:49

These error messages...

[14664:14280:0420/202509.245:ERROR:browser_switcher_service.cc(238)] XXX Init()
[14664:14280:0420/202509.678:ERROR:device_event_log_impl.cc(162)] [20:25:09.679] Bluetooth: bluetooth_adapter_winrt.cc:1186 Getting Radio failed. Chrome will be unable to change the power state by itself.
[14664:14280:0420/202509.695:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1264 OnPoweredRadioAdded(), Number of Powered Radios: 1
[14664:14280:0420/202509.696:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1283 OnPoweredRadiosEnumerated(), Number of Powered Radios: 1

...implies that the on_init_ method failed in std::make_unique<base::ScopedClosureRunner>(std::move(on_init)).

Analysis

These errors are defined in bluetooth_adapter_winrt.cc as follows:

Getting Radio failed. Chrome will be unable to change the power state by itself:

void BluetoothAdapterWinrt::OnGetRadio(base::ScopedClosureRunner on_init,
                       ComPtr<IRadio> radio) {
  DCHECK_CALLED_ON_VALID_THREAD(thread_checker_);
  if (radio) {
    radio_ = std::move(radio);
    radio_was_powered_ = GetState(radio_.Get()) == RadioState_On;
    radio_state_changed_token_ = AddTypedEventHandler(
    radio_.Get(), &IRadio::add_StateChanged,
    base::BindRepeating(&BluetoothAdapterWinrt::OnRadioStateChanged,
                weak_ptr_factory_.GetWeakPtr()));
    if (!radio_state_changed_token_)
      BLUETOOTH_LOG(ERROR) << "Adding Radio State Changed Handler failed.";
    return;
  }
  // This happens within WoW64, due to an issue with non-native APIs.
  BLUETOOTH_LOG(ERROR)
      << "Getting Radio failed. Chrome will be unable to change the power "
     "state by itself.";

Number of Powered Radios: 1:

void BluetoothAdapterWinrt::OnPoweredRadioAdded(IDeviceWatcher* watcher,
                                                IDeviceInformation* info) {
  if (++num_powered_radios_ == 1)
    NotifyAdapterPoweredChanged(true);
  BLUETOOTH_LOG(ERROR) << "OnPoweredRadioAdded(), Number of Powered Radios: "
                       << num_powered_radios_;
}
void BluetoothAdapterWinrt::OnPoweredRadioRemoved(
    IDeviceWatcher* watcher,
    IDeviceInformationUpdate* update) {
  if (--num_powered_radios_ == 0)
    NotifyAdapterPoweredChanged(false);
  BLUETOOTH_LOG(ERROR) << "OnPoweredRadioRemoved(), Number of Powered Radios: "
                       << num_powered_radios_;
}
void BluetoothAdapterWinrt::OnPoweredRadiosEnumerated(IDeviceWatcher* watcher,
                                                      IInspectable* object) {
  BLUETOOTH_LOG(ERROR)
      << "OnPoweredRadiosEnumerated(), Number of Powered Radios: "
      << num_powered_radios_;
  // Destroy the ScopedClosureRunner, triggering the contained Closure to be
  // run. Note this may destroy |this|.
  DCHECK(on_init_);
  on_init_.reset();
}

Deep Dive

These errors are the direct impact of the changes incorporated with google-chrome as per the details within the discussion in Chrome no longer accepts certificates that fallback to common name

Solution

Ensure that:

Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v84.0 level.
Chrome is updated to current Chrome Version 84.0 level. (as per ChromeDriver v84.0 release notes)
If your base Web Client version is too old, then uninstall it and install a recent GA and released version of Web Client.

Additional considerations

However it was observed that this error can be supressed by running Chrome as root user (administrator) on Linux, but that would be a deviation from the documentation in ChromeDriver - WebDriver for Chrome where it is mentioned:

A common cause for Chrome to crash during startup is running Chrome as root user (administrator) on Linux. While it is possible to work around this issue by passing '--no-sandbox' flag when creating your WebDriver session, i.e. the ChromeDriver session as such a configuration is unsupported and highly discouraged.

Ideally, you need to configure your environment to run Chrome as a regular user instead.

Suppressing the error

Finally, as per the documentation in Selenium Chrome Driver: Resolve Error Messages Regarding Registry Keys and Experimental Options these error logs can be supressed by adding the argument:

excludeSwitches: ['enable-logging']

So your effective code block will be:

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.google.com/")

score 0 · Answer 2 · answered Apr 20 '20 at 21:32

0

Have you tried switching to the Firefox driver instead?
https://github.com/mozilla/geckodriver/releases

You'd just have to update your code:

driver = webdriver.Firefox(executable_path='/path/to/driver')

answered Apr 20 '20 at 21:32

I did. It worked for the first time and then suddenly stopped working. – Wandering Sovereign Apr 21 '20 at 07:50
@WanderingSovereign in what way did it stop working? what errors did it throw? – Apr 21 '20 at 17:27
That's the problem. It doesn't give me any errors. It just fails to get any tags from the website. – Wandering Sovereign Apr 21 '20 at 17:40
When you say it fails to retrieve any tags from the website, is your `Tags` empty, your `List` empty, or is your `Index` empty? Also, what is the purpose of the `while True:` loop? Is everything after supposed to be inside of it, or just the `Tags = driver.find_elements_by_tag_name('a')`. If the latter, then the program won't ever break out of that loop, so you'd need to remove it. – Apr 21 '20 at 21:35
Ah sorry, I set up the while true loop cause ai wanted to check whether the error was occuring because the webpage had yet to load properly. It actually shouldn't exist. I'll change it. And yes, the first list, 'Tags' is returned empty. – Wandering Sovereign Apr 22 '20 at 03:42
Can you add `print(driver.page_source)` after your `driver.get(Main + Name)` just to see that the page is actually being loaded correctly? – Apr 23 '20 at 04:49
Sure. But the resulted data is too big to post here. Check the txt file below. '''https://drive.google.com/file/d/1G78nYHlS3MMLU0De4UmKUWpZXnWv72PF/view?usp=sharing''' – Wandering Sovereign Apr 23 '20 at 05:59
The page is being loaded fine and no errors are thrown. You say it worked the first time, but it then suddenly stopped working? Did you change anything or do anything different in that period? Other than that, I can't see why `driver.find_elements_by_tag_name('a')` would suddenly fail. – Apr 24 '20 at 04:05
I don't see a reason to why it should fail either. But it does. The page loads and the driver.find command returns a blank list. – Wandering Sovereign Apr 25 '20 at 06:04