12

I am using Python and OpenSSL to connect to a site using TLS (in some cross-platform software, so it would be too much work to switch to CryptoAPI for everything); I don't want to distribute (and update) a custom list of certificates, though. I want to get them from the platform. On OS X and Linux this is fairly straightforward, but Windows ships with an incomplete list of trusted root certificate authorities for TLS; basically just Microsoft's own certificates, then dynamically adds trust roots to the store when high-level TLS stuff (such as loading a web page in Internet Explorer over HTTPS) has to verify a trust root it hasn't previously seen. (This process is explained here.) This means I can enumerate the Windows root certificate store with wincertstore, but it's useless because on machines with more recently-installed OSes, that store will be almost empty.

Microsoft provides detailed instructions for administrators to pre-retrieve this list so as to be able to operate machines with tightly-controlled network access; however, I cannot find any reference to an API that will do the same thing, and just download all trusted root certificates from Microsoft. (Honestly, in the age of weekly multi-megabyte system updates, I don't see why pre-downloading these is such a big deal, if it's just a cache; for bonus points please explain why this needs to happen at all.)

So: is there an API that would allow me to tell the system to just pre-cache the trusted root certificates according to whatever rules it uses? Failing that, if it's really impossible (i.e. if CryptoAPI can only download one trust root at a time and only if you feed it a certificate signed by that root), is there a way to hook up OpenSSL certificate verification to CryptoAPI's trust store so that the verification will download and cache trust roots just like a native TLS connection would?

Glyph
  • 31,152
  • 11
  • 87
  • 129
  • Interesting. I guess I'm not seeing this on my VMs because I keep them suspended most of the time? – Glyph Jan 12 '16 at 01:51
  • @HarryJohnston - Maybe shelling out to that command is in fact the correct answer to do this programmatically. But knowing what API it's calling so we don't need a subprocess would be ideal. – Glyph Jan 13 '16 at 01:29
  • So, `certutil -syncWithWU` just puts the certificates into a directory. It doesn't save them in the `"ROOT"` store, which is where they'd need to be, I believe. – Glyph Jan 13 '16 at 01:48
  • I wonder if the Windows Update API is the right place for looking for this: https://msdn.microsoft.com/en-us/library/windows/desktop/aa387287(v=vs.85).aspx – Glyph Jan 13 '16 at 02:20
  • No, I didn't. How are you querying the root store? I am using `python -c 'import wincertstore; print(len(list(wincertstore.CertSystemStore("ROOT"))))'` – Glyph Jan 13 '16 at 20:10
  • Also, Windows does not pre-retrieve certificates as far as I can tell. I left a Windows 8 Enterprise VM running for 48 hours to test, and no additional certs were added. It sounds like it might issue updates (i.e. revocations) to *existing* certificates on a daily basis, but not retrieve the new ones? – Glyph Jan 13 '16 at 20:11
  • Sorry, it looks like you're right; I can't reproduce the behaviour I was remembering either. I'll do some more testing and report back if I discover anything that might actually be useful. – Harry Johnston Jan 13 '16 at 22:15
  • I do note that simply opening the downloaded files is enough to add them to the root store. Opening the .sst file you get from `-generateSSTFromWU` isn't, but whenever you view one of the certificates inside it, that certificate is added to the root store. It must be possible to do the equivalent via the API, I'll see if I can figure it out. – Harry Johnston Jan 13 '16 at 22:31

1 Answers1

2

This is not an ideal approach, but it should do at a pinch and it may give you somewhere to start. This code will take the .sst file generated by certutil -generateSSTFromWU and add all the certificates to the root store:

#include <Windows.h>

#include <WinCrypt.h>

#pragma comment(lib, "crypt32.lib")

#include <stdio.h>

void process_cert(PCCERT_CONTEXT cert)
{
    PCCERT_CHAIN_CONTEXT ccc;
    CERT_CHAIN_PARA ccp = {sizeof(CERT_CHAIN_PARA)};
    DWORD flags;
    char certname[256];

    CertGetNameStringA(cert, CERT_NAME_SIMPLE_DISPLAY_TYPE, 0, NULL, certname, _countof(certname));

    flags = 0;

    if (!CertGetCertificateChain(HCCE_LOCAL_MACHINE, cert, NULL, NULL, &ccp, flags, NULL, &ccc))
    {
        printf("Certificate %s CertGetCertificateChain: %u\n", certname, GetLastError());
    }
    else
    {
        printf("Certificate %s : %x (%x)\n", certname, ccc->TrustStatus.dwErrorStatus, ccc->TrustStatus.dwInfoStatus);
    }
}

void mainfn(void)
{
    HCERTSTORE sst;
    PCCERT_CONTEXT cert;
    DWORD count;

    sst = CertOpenStore(CERT_STORE_PROV_FILENAME_W, 0, (HCRYPTPROV)NULL, CERT_STORE_OPEN_EXISTING_FLAG | CERT_STORE_READONLY_FLAG, L"c:\\downloads\\roots.sst");

    if (sst == NULL)
    {
        printf("CertOpenStore: %x\n", GetLastError());
        return;
    }

    for (cert = NULL, count = 0; cert = CertEnumCertificatesInStore(sst, cert); count++) process_cert(cert);

    {
        DWORD err = GetLastError();
        if (err != CRYPT_E_NOT_FOUND)
        {
            printf("CertEnumCertificate: %u\n", err);
            return;
        }
    }
}

int main(int argc, char ** argv)
{
    mainfn();
    return 0;
}

Alternatively, in your context, you might prefer to use the root certificates in the .sst file directly, without also adding them to the root store. (In that case you should probably enumerate the root store as well as the .sst file, so as to include any locally added certificates.)

Harry Johnston
  • 35,639
  • 6
  • 68
  • 158
  • Will this *remove* all the certificates from the root store, though? I want to do exactly what WU does normally; if a user has de-trusted a cert I don't want to re-trust it; if they have added custom certs I want to honor those. – Glyph Jan 14 '16 at 23:36
  • This won't remove any certificates, and it shouldn't affect certificates that are already in the store. All it does is ask Windows to confirm that each of the certificates is valid. The only difference between this and what Internet Explorer does as a matter of course is that you're asking Windows about the root certificate directly rather than asking it about a certificate chain leading to the root certificate. – Harry Johnston Jan 15 '16 at 00:11
  • 1
    (If OpenSSL gives you the option of validating the site certificate yourself, that might be an even better option. If you use CertGetCertificateChain and friends to do the validation, your program would behave *exactly* like Internet Explorer does, only loading the root certificate that the specific site actually needs into the store.) – Harry Johnston Jan 15 '16 at 00:15
  • The API I'm trying to satisfy, unfortunately, is "here's a list of certificates to verify against". It might be possible to get into OpenSSL and tell it to change how it does its verification, but in at least one of the APIs I need to talk to that's not really an option. – Glyph Jan 16 '16 at 02:18
  • 1
    Yeah, figured as much, but thought I'd better not just take it for granted. I think if you make a list from the certificates from the .sst and the certificates from the root store, there's no real need to actually import the certificates *into* the root store. It would be nice to find a way to download the .sst (or equivalent) without shelling out to `certutil` but I fear that's beyond my knowledge, and unfortunately we don't seem to have any Windows-crypto experts hanging around. (My best guess: no supported method. But I'm not sure.) – Harry Johnston Jan 16 '16 at 10:56
  • The need to import the certificates into the root store (in exactly the way that IE would do it) is that an administrator may have removed a trust root, and I want to respect that setting. If we just grab all the certificates, then I might as well be using https://pypi.python.org/pypi/certifi since that includes a cert bundle as well. The point of using the native trust store is to get both positive *and* negative trust rules. – Glyph Jan 18 '16 at 03:18
  • If an administrator deletes a certificate, Windows will just reload it automatically, won't it? Perhaps I'm wrong, but I think that if you want to block a certificate you have to disable it, not delete it, and hopefully you're already checking for that. (So you just ignore any certificates in the .sst file that are already in the root store.) But if you're not confident about that, the code as posted should behave exactly the way IE will, except that it loads all the certificates except just the one you need. – Harry Johnston Jan 18 '16 at 05:18
  • I am talking about the case where an admin "blocks" a certificate, yes. In that case I need an API to extract not just the certificates, but the rules around the certificates, from the root store, so that I can exclude some of them. – Glyph Jan 19 '16 at 07:52