Working with the Windows Driver Development Kit sample package (v 8.1), I found a sample print processor implementation that I tried to use as a starting point for my own custom print processor ("GenPrint Print Processor Sample"). I managed to successfully build it in Visual Studio 2017 Community (changing PlatformToolset to "v141"), and I'm apparently able to successfully install it (by creating the registry item, "HKLM:\SYSTEM\CurrentControlSet\Control\Print\Environments\Windows x64\Print Processors\genprint" with item property "Driver" and value "genprint.dll", and copying the freshly built dll to %WinDir%/System32/spool/prtprocs/x64/). I can then set up a test printer and select my "genprint" print processor using Printer Properties > Advanced > Print Processor..., and I can even attach to the spooler process, spoolsv, from my debugger in Visual Studio and set a breakpoint in EnumPrintProcessorDatatypesW(...), and see it get hit when I reopen the Print Processor dialog for my test printer. So far, so good!
The problem comes when I set a breakpoint anywhere else, and queue up an actual job to the printer queue. Say at the beginning of OpenPrintProcessor(...). Or PrintDocumentOnPrintProcessor(...), or, you name it. Nothing, nada, zilch. It seems these functions are just never called at all, at least not the implementations exported by my DLL. I even resorted to inserting event logs ala Event Tracing for Windows (ETW) at the beginning of every function exported by the DLL, but the only logs ever generated are the ones for EnumPrintProcessorDatatypesW(...).
What on Earth might be going on?? Is it possible that, for some odd reason, the spooler is using a different print processor than mine, even though I explicitly associated mine with the print queue in question? And if so, why?
[Update: it looks like the behavior I'm seeing is down to "print driver isolation"; when I change the isolation mode for the driver associated with the queue to "None", then I can attach to the spoolsv process. Otherwise, my print processor is running in a separate process communicating with spoolsv via DCOM, as described at: https://blogs.technet.microsoft.com/askperf/2009/10/08/windows-7-windows-server-2008-r2-print-driver-isolation/ -- I'm guessing the system-wide default has been changed to use print driver isolation since post-Windows 7, unless you explicitly configure the driver to "None" (or it configures itself that way)]
