3

I'm trying to help the SuperCollider community to try and understand how we can prevent Windows Defender from delaying the execution of one of the executable, on a up-to-date Windows 10.

The original github issue can be found on github.

Here is the test case :

  1. download the latest version of SuperCollider for Windows x64 (3.10.3)

  2. install it

  3. reboot your computer

  4. open "cmd" and start scsynth.exe

cd "\Program Files\SuperCollider-3.10.3"
scsynth.exe -u 57110

You will have to wait 50 to 60 seconds for seeing scsynth output which should start with something like 

Device options:
  - MME : Mappeur de sons Microsoft - Input   (device #0 with 2 ins 0 outs)
[...]
SuperCollider 3 server ready.

  1. Note that if you quit scsynth.exe and run the command again, scsynth.exe starts right away without delay

  2. Now put scsynth.exe process in Windows Defender exclusion list (see this article for info of how to access this exclusion list)

  3. Reboot

  4. Open "cmd" and start scsynth.exe

cd "\Program Files\SuperCollider-3.10.3"
scsynth.exe -u 57110

Now scsynth.exe starts right away.

This behaviour started when Windows Defender block at first sight feature was added.

It raises a lot of problems for SuperCollider Windows users.

Can anybody help us to circumvet the problem ?

geoffroy
  • 575
  • 6
  • 18
  • 1
    I used to investigated problems like this by using tools like Process Monitor (from Sysinternals/Microsoft). It can log all the i/o, registry, and process and thread operations being performed by the process. Comparing logs for the two situations can give you a good idea of how other programs might be interfering. Nowadays, I use ETW (Event Tracing for Windows) via UIforETW to record traces, and then I analyze them with Windows Performance Analyzer. Same deal with lots more detail to drill down into. – Adrian McCarthy Oct 14 '19 at 21:11
  • Thank you Adrian, I will try to figure out what's happening with these tools. – geoffroy Oct 16 '19 at 08:17
  • Not really. It's a well known problem with Windows Defender. They've added a warning in the recent versions of SC. It only happens on the 1st library class compilation. It takes e.g. 20secs on my machines, whereas a recompile takes 1-2 secs. The same goes for indexing the help files. Windows Defender has some special tricks for not slowing down like this commercial software (i.e. infernal whitelists.) See discussion about Rust https://youtu.be/qbKGw8MQ0i8?t=2326. Basically MS needs to be gently begged to whitelist SC. – Fizz Nov 29 '21 at 09:19
  • On a somewhat more streamlined installation (not much extra quarks/plugins), 866 SC class files take 10 seconds on a first compile. Almost exactly 12ms per file, like in that Rust guy's talk... Also the compiled help files get written to `C:\Users\\AppData\Local\SuperCollider` so that needs to be excluded as well for faster sclang boots. – Fizz Nov 29 '21 at 09:42
  • 1
    Thanks @fizz ! I added the warning in the recent versions of SC :) If anybody knows the procedure to beg MS to add SC to the whitelist, I'm in ! – geoffroy Nov 30 '21 at 10:11
  • One amusing discovery, perhaps you can confirm... If you download the (64-bit) zip distribution (instead of the exe installer), and extract it to a non-default location (i.e. not Program Files) and not the system drive... then Defender blissfully ignores SC's compilation actions, both for help files and classes, even without any explicit exclusions added. – Fizz Dec 06 '21 at 09:37
  • After a bit more experience with the above, what happens when you run SC from a non-system location is slightly more complicated. You still get it scanned by the Defender to the dog-slow pace, but *only once per **machine** boot*. Whereas if installed to Program Files, it gets scanned *on every SClang process startup* (but not e.g. classlib recompiles while the same sclang process is running), unless Defender exception paths are manually added. So, if you don't reboot your PC much but just use suspend/resume this isn't much of an issue. – Fizz Dec 16 '21 at 21:46

1 Answers1

1

I guess you're expecting a real answer from Microsoft here, but this might get lost in the comments and it's perhaps useful enough info for SC users to register as an answer, so:

After a bit more hands-on experience, what happens when you run SC from a non-system location (i.e. don't install to Program Files, and perhaps not to the system driver either) is slightly more complicated. For non-system locations you still get it scanned by the Defender to the dog-slow startup pace, but only once per machine boot. Whereas if SC is installed to Program Files, it gets scanned on every SClang process startup (but not e.g. classlib recompiles while the same sclang process is running), unless Defender exception paths are manually added.

The above is with block-at-first-sight still enabled, as far as I can tell, because MS says "To disable block at first sight, turn off Cloud-delivered protection or Automatic sample submission." And I still had both of those features enabled.

So, if you don't reboot your PC much but just use suspend/resume this isn't much of an issue if you install SC to a non-system location... for regular uses. It still provides a bad out-of-the-box experience for new SC users, no doubt.

Actually, Microsoft proved the above theory a bit wrong today. I just had a Windows update that made my machine reboot, so I totally expected the next SC startup to be slow again... but it wasn't!

So, it appears that Defender keeps a persistent cache of recently used & scanned locations, meaning it's kept on disk. So the next question is what might actually invalidate this cache. The last reboot, the one that did not cause a lengthy scan, was only for a Windows monthly update but it did not include a Defender engine or definitions update. So I think that the Defender cache probably gets invalidated on some of those more specific events, rather than any reboot. Perhaps there's some straight time-based or LRU expiry of entries, but it's hard to test that since Defender updates itself pretty often creating a confounder.

Yeah, after a quick search on the latter issue, Defender indeed keeps a persistent cache on disk of some info pertaining to its previous scans.

When Real-time protection is turned on, after about 20-30 minutes it creates hundreds/thousands of files in this location: C:\ProgramData\Microsoft\Windows Defender\Scans\History\Store

Most of these files are either 1kb or 2kb. Over a 24 hour period we ended up with roughly 950,000 files and it was taking 30 GB of space.

That issue with those Defender history files being too many was fixed back then (May 2021), by the way

This issue is current a known issue and the fix will be hit all release ring in this Thursday. The RCA is Engineer of MsMpEng.dll has some issue and causing lots of files in this folder. The affected engine version is 18100.5.

But it's certainly the case that some scan history info is still being kept on disk, which does mitigate the issue of re-scanning programs like SC to a good extent in regular use, albeit not in an out-of-the-box scenario when SC was just installed.

Aside: slightly hilarious, but it appears the Defender scan slowdown sometimes affects Microsoft's own products.

In the realm of programming solutions:

As far as I can tell, as long as Windows Defender (and presumably other A/V scanners) are running, there's no way to make the Windows I/O APIs consistently fast. You can disable A/V scanning (at your own peril). But the trick that Mercurial employs (which has later been emulated by rustup among other tools) is to use a thread pool for calling CloseHandle(). Even if you perform all file open and write I/O on a single thread and use a background thread pool only for calling CloseHandle(), you can see a >3x speedup in time to write files. This optimization should ideally be employed by any software that creates or mutates as little as a few hundred files on Windows. This includes version control tools, installers, and archive extraction tools. Fun fact: rustup can extract tar files on Windows faster than open source and commercial fast extraction/copy tools because it employs this trick and more. I believe rustup on Windows is actually faster at extracting tar archives than it is on Linux!

There are also youtube talks from the rustup devs who worked on that, such as this one.

Fizz
  • 4,782
  • 1
  • 24
  • 51