How do I save a webpage using C++? Windows or Linux System

Question

I need to know how to save a webpage using C++ on a Windows and/or Linux.

Step 1) This is my current code that opens the webpage:

ShellExecute(NULL, "open", websiteURL, NULL, NULL, SW_SHOWNORMAL);

Step 2) This is the step where I will save the webpage that is opened as a .txt

Your help here.

Step 3) This is my attempt at closing the webpage after saving it as a .txt; However, it does not work currently.

ShellExecute(NULL, "close", websiteURL, NULL, NULL, SW_SHOWNORMAL);

possible duplicate of [Saving a webpage to disk using C++](http://stackoverflow.com/questions/7937283/saving-a-webpage-to-disk-using-c) — Slizzered, Apr 09 '15 at 19:39
@Slizzered Yes, involve curl, and that job could be done easily. — πάντα ῥεῖ, Apr 09 '15 at 19:53
I would just call `SaveWebPageToDisk(char *url, char *savePath);` myself. — mah, Apr 09 '15 at 19:57
@mah How do I use SaveWebPageToDisk? Can you give a bit more details on how to set this up? What is the actual definition of SaveWebPageToDisk()? — Universum Magnus, Apr 10 '15 at 08:09

Barmak Shemirani · Accepted Answer · 2019-08-21T21:08:41.980

3

This is Windows version. Note, the Windows functions are Unicode UTF-16, but the output file could be ANSI or UTF-8.

#include <iostream>
#include <string>
#include <fstream>
#include <Windows.h>
#include <WinINet.h>

#pragma comment(lib, "WinINet.lib")

int main()
{
    std::ofstream fout(L"c:\\test\\_test.htm", std::ios::binary);
    std::wstring url = L"https://www.stackoverflow.com/questions/29547368";
    HINTERNET hopen = InternetOpen(L"MyAppName", 
                            INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0);
    if(hopen)
    {
        DWORD flags = INTERNET_FLAG_DONT_CACHE;
        if(url.find(L"https://") == 0) 
            flags |= INTERNET_FLAG_SECURE;
        HINTERNET hinternet = InternetOpenUrl(hopen, url.c_str(), NULL, 0, flags, 0);
        if(hinternet)
        {
            char buf[1024];
            DWORD received = 0;
            while(InternetReadFile(hinternet, buf, sizeof(buf), &received))
            {
                if(!received) break;
                fout.write(buf, received);
            }
            InternetCloseHandle(hinternet);
        }
        InternetCloseHandle(hopen);
    }
    return 0;
}

edited Aug 21 '19 at 21:08

answered Apr 10 '15 at 02:47

Barmak Shemirani

30,904
6
40
77

I tried compiling this code and received the following errors. {$ g++ test.cpp test.cpp: In function ‘int main()’: test.cpp:57:23: error: ‘printf’ was not declared in this scope printf(src.c_str()); } – Universum Magnus Apr 10 '15 at 08:17
I changed it to `cout` and added the header file ``. – Barmak Shemirani Apr 10 '15 at 12:26
@bamrak-shemirani Am I missing a library, a header from my local folder? When I compile, I get the following errors: $ g++ test.cpp /tmp/ccVUR0Zt.o:test.cpp:(.text+0x72): undefined reference to `WinHttpOpen' /tmp/ccVUR0Zt.o:test.cpp:(.text+0x72): relocation truncated to fit: R_X86_64_PC3 2 against undefined symbol `WinHttpOpen' – Universum Magnus Apr 10 '15 at 16:43
I compiled this code with Visual Studio, I have a feeling it won't run on gcc without a lot of modifications. Maybe gcc has a different solution. – Barmak Shemirani Apr 10 '15 at 17:47
@bamrak-shemirani I just compiled and ran this with Visual Studio successfully. It appears to have copied the HTML code to my console. Is it possible to modify it, so that it simply performs "save.as" the content of the webpage as a ".txt" to a folder requested? – Universum Magnus Apr 10 '15 at 18:24
I can't write the whole program, but added simple file save. – Barmak Shemirani Apr 10 '15 at 18:52
@bamrak-shemirani You have been a massive help. I checked your answer, and I wish I could do more to commend you. Thank you. – Universum Magnus Apr 10 '15 at 19:27
That's great Magnus, I am glad I could help out. – Barmak Shemirani Apr 10 '15 at 22:07

How do I save a webpage using C++? Windows or Linux System

1 Answers1

Linked