I have a PyQt5 application running in Python 3.7 that makes a lot of web request, but no more than one or two at a time. Most requests are quick, but sometimes take several seconds. The whole UI hangs until the requests complete. I have a full sample application below to demonstrate this.
Looking around, some people recommend using QThread but if I try to do that, I get complaints about accessing UI objects from the wrong thread. That method also prevents me from returning the web response to the caller. In the sample code below, the response is only used to print the status code, but in the actual application the JSON responses are parsed and used. Most answers to this question I find are more than five years old, often referencing PyQt4 or older, or referencing packages that are now obsolete such as grequests or request-threads. Some people reference httpx (still in beta) or aiohttp, which has its own event loop and doesn't always play nice inside PyQt5 applications.
The sample is as stripped down as possible to demonstrate the problem. if you click "Make Request" and immediately click on "Click Counter" you'll see that the counter doesn't increase until the web request has completed. This is not surprising given that the request is blocking the Qt event loop.
How can I just simply get the current code to work where the requests are submitted and managed asynchronously, but still being able to pass a response back to the caller?
Here is test.py:
import sys
from datetime import datetime
import requests
from PyQt5 import QtWidgets, uic
from requests import RequestException
class UI(QtWidgets.QMainWindow):
def __init__(self):
super(UI, self).__init__()
uic.loadUi('test.ui', self)
self.web = WebManager(self)
self.issue_request_button.clicked.connect(self.issue_request)
self.issue_click_button.clicked.connect(self.click_counter)
self.show()
def click_counter(self):
self.click_count_label.setText(str(int(self.click_count_label.text()) + 1))
def issue_request(self):
r = self.web.issue_get(self.request_url.text())
print(r.status_code if r is not None else 'No successful response')
class WebManager:
def __init__(self, gui):
self.gui = gui
def issue_get(self, url_request):
try:
start = datetime.now()
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
r = requests.get(url_request, headers=headers)
self.fill_request_response_info(r, datetime.now() - start)
return r
except RequestException as e:
print(f'Request failed with an Exception of type {type(e).__name__}')
return None
def fill_request_response_info(self, response, total_request_time):
request = response.request
self.gui.request_url_textbox.setText(request.method + ' ' + request.url)
self.gui.request_headers_textbox.setText('\n'.join(k + ':' + v for k, v in request.headers.items()))
self.gui.response_status_textbox.setText(f'{response.status_code} {response.reason}')
self.gui.response_elapsed_time_textbox.setText(f'{response.elapsed} / {total_request_time}')
self.gui.response_headers_textbox.setText('\n'.join(k + ':' + v for k, v in response.headers.items()))
self.gui.response_body_textbox.setText(response.content.decode('utf-8'))
app = QtWidgets.QApplication(sys.argv)
window = UI()
app.exec()
and here is the UI file that it needs test.ui:
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>MainWindow</class>
<widget class="QMainWindow" name="MainWindow">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>750</width>
<height>734</height>
</rect>
</property>
<property name="windowTitle">
<string>Test Window</string>
</property>
<widget class="QWidget" name="centralwidget">
<layout class="QVBoxLayout" name="verticalLayout_3">
<item>
<layout class="QHBoxLayout" name="horizontalLayout">
<item>
<widget class="QPushButton" name="issue_request_button">
<property name="text">
<string>Make Request</string>
</property>
</widget>
</item>
<item>
<widget class="QLabel" name="label">
<property name="text">
<string>URL:</string>
</property>
</widget>
</item>
<item>
<widget class="QLineEdit" name="request_url">
<property name="text">
<string>https://archive.org/advancedsearch.php?q=subject:google+sheets&output=json</string>
</property>
</widget>
</item>
</layout>
</item>
<item>
<layout class="QHBoxLayout" name="horizontalLayout_3">
<item>
<widget class="QPushButton" name="issue_click_button">
<property name="text">
<string>Click Counter</string>
</property>
</widget>
</item>
<item>
<widget class="QLabel" name="click_count_label">
<property name="text">
<string>0</string>
</property>
</widget>
</item>
<item>
<spacer name="horizontalSpacer">
<property name="orientation">
<enum>Qt::Horizontal</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>40</width>
<height>20</height>
</size>
</property>
</spacer>
</item>
</layout>
</item>
<item>
<widget class="QGroupBox" name="http_request_groupBox">
<property name="title">
<string>HTTP Request</string>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QLabel" name="req_url_label">
<property name="text">
<string>Request URL:</string>
</property>
</widget>
</item>
<item>
<layout class="QHBoxLayout" name="horizontalLayout_2">
<item>
<widget class="QLineEdit" name="request_url_textbox"/>
</item>
</layout>
</item>
<item>
<widget class="QLabel" name="req_headers_label">
<property name="text">
<string>Request Headers:</string>
</property>
</widget>
</item>
<item>
<widget class="QTextEdit" name="request_headers_textbox"/>
</item>
</layout>
</widget>
</item>
<item>
<widget class="QGroupBox" name="http_response_groupBox">
<property name="title">
<string>HTTP Response</string>
</property>
<layout class="QVBoxLayout" name="verticalLayout_2" stretch="0,0,1,0,1">
<item>
<layout class="QGridLayout" name="gridLayout" rowstretch="0,0">
<item row="0" column="0">
<widget class="QLabel" name="http_status_label">
<property name="text">
<string>HTTP Status:</string>
</property>
</widget>
</item>
<item row="0" column="3">
<widget class="QLabel" name="elapsed_time_label">
<property name="text">
<string>Elapsed Time:</string>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QLineEdit" name="response_status_textbox"/>
</item>
<item row="1" column="3">
<widget class="QLineEdit" name="response_elapsed_time_textbox"/>
</item>
</layout>
</item>
<item>
<widget class="QLabel" name="response_headers_label">
<property name="text">
<string>Response Headers:</string>
</property>
</widget>
</item>
<item>
<widget class="QTextEdit" name="response_headers_textbox"/>
</item>
<item>
<widget class="QLabel" name="response_body_label">
<property name="text">
<string>Response Body:</string>
</property>
</widget>
</item>
<item>
<widget class="QTextEdit" name="response_body_textbox"/>
</item>
</layout>
</widget>
</item>
</layout>
</widget>
</widget>
<tabstops>
<tabstop>issue_request_button</tabstop>
<tabstop>request_url</tabstop>
<tabstop>issue_click_button</tabstop>
<tabstop>request_url_textbox</tabstop>
<tabstop>request_headers_textbox</tabstop>
<tabstop>response_status_textbox</tabstop>
<tabstop>response_elapsed_time_textbox</tabstop>
<tabstop>response_headers_textbox</tabstop>
<tabstop>response_body_textbox</tabstop>
</tabstops>
<resources/>
<connections/>
</ui>
