Get contents of website from QWebView using Qt 5.5.1

Question

I'm using Qt 5.5.1 and I make a small browser using QWebview I want when
I open a web page and press a button, it get the content of the website like I use get method in QnetworkAccessManager without using it because I want to get the data from website that have a login page so the URL not change when I log in and have not post method to PHP to get the data.
for example when I log in to www.login.com the data of the login show on the same link
I need any idea I can solve this problem or if I can get the current data from website open in the QWebview
Note
When I login into the website and get the data from it by press on view source code in firefox the login data appear in the source code
This is what I tried

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow) {
        ui->setupUi(this);
        ui->webView->load(QUrl("https://www.login.com")); // load page that have user name and password field
        QWebPage *page = ui->webView->page(); // get the current page 
        manager = page->networkAccessManager(); // get the access manager from this page
    }

    MainWindow::~MainWindow() {
        delete ui;
    }
    // get button
    void MainWindow::on_pushButton_clicked() {
        reply = manager->get(QNetworkRequest(QUrl("https://www.login.com"))); // make get now after login
        connect(reply, SIGNAL(readyRead()),this,SLOT(readyRead()));
        connect(reply, SIGNAL(finished()),this, SLOT(finish()));
    }

    void MainWindow::readyRead() {
        QString str = QString::fromUtf8(reply->readAll()).trimmed(); // read the data
        ui->plainTextEdit->appendPlainText(str);
    }

but I get the data of the first page without the login. I want to get the page content after login, please give me any hint on what I should do.
update
I view the page source code from firefox get the text input name and use QUrlQuery to make post to it by the result was the first page without login this the part of HTML code I got the name of it

<label class="label-off" for="dwfrm_ordersignup_orderNo">Numéro de la commande</label> <input class="textinput required" id="anyname" type="text"  name="UserName"  value=""  maxlength="2147483647" placeholder="* UserName" data-missing-error="Saisis ton numéro de commande. "  data-parse-error="Ce contenu est invalide"  data-range-error="Ce contenu est trop long ou trop court"  required="required" />

and it has same code for the other field.
Code for I use in Qt to make a post

manager = new QNetworkAccessManager(this);
QUrlQuery query;
query.addQueryItem("UserName", "AAAA");
query.addQueryItem("Password", "BBB");
reply = manager->post(QNetworkRequest(QUrl(ui->lineEdit->text().trimmed())), query.toString().toUtf8());
connect(reply,&QNetworkReply::downloadProgress,this,&MainWindow::progress);
connect(reply,SIGNAL(readyRead()),this,SLOT(readyRead()));
connect(reply, SIGNAL(finished()), this,SLOT(finish()));

I try the post code with PHP page I made and it work the problem here it's only HTML page

You probably want send a post method to the server, you need to analyze the login page and see the name of the text inputs and pass it as a key to `QUrlQuery` and pass those respective values (usually the name and the password). An example of how to do post method with Qt was posted [here](http://stackoverflow.com/a/7445752/2014561). — Antonio Dias, Feb 14 '17 at 08:52
I update the question with what I have tried please take a look at this — user7179690, Feb 14 '17 at 11:20

Antonio Dias · Accepted Answer · 2017-02-15T23:06:10.200

I will use a different approach, instead to simulate a POST request, I will load the login page normally with QWebView and use Qt to fill the username, password and do a fake click on the submit button.

On how to save the after login page how what you see, not the HTML code, a good way provided by Qt is render the web view to PDF, with a downside of lose the HTML code.

If only the code is enough you can use webview->page()->mainFrame()->toHtml()

See a simple example, please note that you need to adapt the code to your environment, analyze the login page, etc.

void MainWindow::start()
{
    connect(webview, &QWebView::loadFinished, this, &MainWindow::LogIn);

    QString html = "<html>"
                   "<body>"
                   "<form action=\"https://httpbin.org/post\" method=\"POST\">"
                   "Username:<br>"
                   "<input type=\"text\" name=\"usernameinput\" value=\"abc\">"
                   "<br>"
                   "Password:<br>"
                   "<input type=\"password\" name=\"passwordinput\" value=\"123\">"
                   "<br><br>"
                   "<button name=\"button1\">Submit</button>"
                   "</form>"
                   "</body>"
                   "</html>";

    webview->setHtml(html); //Load yours https://www.login.com, using setHtml just for example
}

void MainWindow::LogIn(bool ok)
{
    disconnect(webview, &QWebView::loadFinished, this, &MainWindow::LogIn); //Disconnect the SIGNAL

    if (!ok)
        return;

    QWebElement document = webview->page()->mainFrame()->documentElement();

    QWebElement username = document.findFirst("input[name=usernameinput]"); //Find the first input with name=usernameinput

    if (username.isNull())
        return;

    username.setAttribute("value", "def"); //Change the value of the usernameinput input even
                                           //if it already has some value

    QWebElement password = document.findFirst("input[name=passwordinput]"); //Do the same for password

    if (password.isNull())
        return;

    password.setAttribute("value", "123456"); //Do the same for password

    QWebElement button = document.findFirst("button[name=button1]"); //Find the button with name "button1"

    if (button.isNull())
        return;

    connect(webview, &QWebView::loadFinished, this, &MainWindow::finished);

    button.evaluateJavaScript("this.click()"); //Do a fake click on the submit button
}

void MainWindow::finished(bool ok)
{
    disconnect(webview, &QWebView::loadFinished, this, &MainWindow::finished);

    if (!ok)
        return;

    QByteArray data;
    QBuffer buffer(&data);

    if (!buffer.open(QIODevice::WriteOnly))
        return;

    QPdfWriter pdfwriter(&buffer);

    pdfwriter.setResolution(100); //In DPI

    webview->page()->setViewportSize(QSize(pdfwriter.width(), pdfwriter.height()));

    QPainter painter;
    painter.begin(&pdfwriter);
    webview->page()->mainFrame()->render(&painter);
    painter.end();

    buffer.close();

    qDebug() << "PDF Size:" << data.size(); //Now you have a PDF in memory stored on "data"
}

Thank you very much for your reply and your effort I appreciate this I try this code but it made a pdf file that contains the login page with the fields written in it the username and password no click button pressed! I checked the source code I found the type of submit button not but — user7179690, Feb 15 '17 at 12:20
now is there any way to make the button click in the code as you did ? maybe because the type of submit not and it's — user7179690, Feb 15 '17 at 12:22
Yes, it depends of the object, if it's has type=submit, or is a button, a hyperlink, etc. You need to find a unique identifier of the object on page and pass it to `findFirst`. I've edited my answer to show a example where button has name=button1. — Antonio Dias, Feb 15 '17 at 23:12
@AntonioDias Can I use this in Qt Quick Application? the QWebView Class and QWebElement and ... ? — mohsen, Jun 21 '21 at 16:22
@mohsen: I didn't found an way to do that, maybe possible, maybe not, i'm not sure! Please note that the OP question is about C++ and not QML! — Antonio Dias, Jun 28 '21 at 06:44

Get contents of website from QWebView using Qt 5.5.1

1 Answers1