111

I am working with WSL a lot lately because I need some native UNIX tools (and emulators aren't good enough). I noticed that the speed difference when working with NPM/Yarn is incredible.

I conducted a simple test that confirmed my feelings. The test was running npx create-react-app my-test-app and the WSL result was Done in 287.56s. while GitBash finished with Done in 10.46s..

This is not the whole picture, because the perceived time was higher in both cases, but even based on that - there is a big issue somewhere. I just don't know where. The project I'm working on uses tens of libraries and changing even one of them takes minutes instead of seconds.

Is this something that I can fix? If so - where to look for clues?

Additional info:

  • my processor: Processor AMD Ryzen 7 5800H with Radeon Graphics, 3201 Mhz, 8 Core(s), 16 Logical Processors

  • I'm running Windows 11 with all the latest updates to both the system and the WSL. The chosen system is Ubuntu 20.04

  • I've seen some questions that are somewhat similar like 'npm install' extremely slow on Windows, but they don't touch WSL at all (and my pure Windows NPM works fast).

  • the issue is not limited to NPM, it's also for Yarn

  • another problem that I'm getting is that file watching is not happening (I need to restart the server with every change). In some applications I don't get any errors, sometimes I get the following:

    Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/DumpStack.log.tmp'
Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/hiberfil.sys'
Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/pagefile.sys'
Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/swapfile.sys'

  • npm start in an empty (freshly initialized) create-react-app takes ages to render something in the browser in WSL and when executed from GitBash - I can see stuff in 2-4 seconds

  • it is possible that's it's purely a WSL problem, but it just hurts the most when using NPM/Yarn

jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
WrRaThY
  • 1,533
  • 2
  • 10
  • 14
  • Yarn was slow in a Windows mounted folder (`/mnt/c`). Running yarn in a project within the Ubuntu home directory (`~`) made it much faster for me (Window 11, WSL2). – Benny Code Apr 14 '23 at 15:53
  • `git` commands was super slow in wsl 2. I converted to wsl 1 and now its fine. Moving the project files to `/home/..` was not ideal for me in the current project. I will follow the second soultion for future. – Code_Ostrich Apr 29 '23 at 10:36

5 Answers5

135

Since you mention executing the same files (with proper performance) from within Git Bash, I'm going to make an assumption here. Correct me if I'm wrong on this, and I'll delete the answer and look for another possibility.

This would be explained (and expected) if your files are stored on /mnt/c (a.k.a. C:, or /C under Git Bash) or any other Windows drive, as they would likely need to be to be accessed by Git Bash.

WSL2 uses the 9P protocol to access Windows drives, and it is currently (See Footnote) known to be very slow when compared to:

  • Native NTFS (obviously)
  • The ext4 filesystem on the virtual disk used by WSL2
  • And even the performance of WSL1 with Windows drives

I've seen a git clone of a large repo (the WSL2 Linux kernel Github) take 8 minutes on WSL2 on a Windows drive, but only seconds on the root filesystem.

Two possibilities:

  • If possible (and it is for most Node projects), convert your WSL to version 1 with wsl --set-version <distroname> 1. I always recommend making a backup with wsl --export first.

    And since you are making a backup anyway, you may as well just create a copy of the instance by wsl --importing your backup as --version 1 (as the last argument). WSL1 and WSL2 both have their uses, and you may find it helpful to keep both around.

    See this answer for more details on the exact syntax..

  • Or just move the project over to somewhere under the WSL root, such as /home/username/src/.

Footnote:

There may be some hope for improvement in this area based on recent developments. Patches for 9P have been released upstream which are reported to provide a significant performance boost. See this Github thread comment (and the parent thread) for more information.

NotTheDr01ds
  • 15,620
  • 5
  • 44
  • 70
  • 2
    hey first of all - thank you very much for a comprehensive answer. and yes - all my files are stored under `/mnt/c`. I tried your 2nd solution - it worked as fast as expected in the terminal, but the IDE froze when I first loaded the `create-react-app` project [IntelliJ Ultimate]. interestingly enough, the second run was successful - I'll test more with my actual project. When it comes to switching to WSL1 - I vaguely remember that I needed to upgrade because WSL1 was missing some features that WSL2 had (I'm running Rust with the BPF compiler for Solana dev), so I can't do that – WrRaThY Aug 30 '21 at 00:55
  • I'll move with my real project to `/home/username/...` and see how the IDE handles the whole thing. if it works - I'll be happy to accept your answer, for now I'll +1 it (as I believe it will probably help someone in the future - please don't delete it) disclaimer: yes, I use Rust as my main programming language for this project, but node is used for E2E testing and therefore it has both - hence I used npm as a more widely known example to depict the problem. also - some npm libs (for testing solana) work only under *nix systems – WrRaThY Aug 30 '21 at 00:57
  • the IDE is considerably slower and freezes with more instances open (never happened before), but I guess it's fast enough and I'll just go with that solution for now. thank you :) PS. do you happen to have any information on whether this issue might be addressed in the future or not? is there any github issue opened for it? – WrRaThY Aug 30 '21 at 02:30
  • 2
    @WrRaThY Here's the main [issue](https://github.com/microsoft/WSL/issues/4197) for tracking, and while I've read statements from the Microsoft team that they aren't happy with the performance, I'm unaware of a root cause being mentioned (but I haven't read through all 350+ comments on the issue) other than perhaps the lack of caching support in 9P. – NotTheDr01ds Aug 30 '21 at 02:53
  • 1
    @WrRaThY Also, I didn't notice the "watching" part of your question earlier, but yes, that's also due to 9P. See [this question/answer](https://stackoverflow.com/a/63455155/11810933) for more info on that. Note that watching seems to work if you run the app from within VSCode, though. – NotTheDr01ds Aug 30 '21 at 02:57
  • thanks for all the support! I'm guessing that running from vscode = running from native windows filesystem, so this is probably why. I'll follow up on those links if it makes sense. cheers! – WrRaThY Aug 30 '21 at 10:29
  • I can confirm, I just moved my whole project from `/mnt/c/...` to `/home/...` and the speed increase is astonishing. – nullromo Nov 01 '21 at 21:12
  • My project is stored under /home/user/workspace/myproject and I'm running into performance issues. Sometimes it seems yarn install is stuck for a while until it processes further. Accessing the project with WebStorm under Windows 11 has also a lot of freezes. – Marian Klühspies Nov 22 '21 at 12:21
  • For development WSL2 is a big nay. And not only it's super slow I had issues making file change detection works properly. Going back to v1.. – vdegenne Apr 14 '22 at 19:49
  • @vdegenne WSL1 is a good option, but also remember that having the files on the WSL2 ext4 filesystem *should* give you both inotify support as well as near native file access speeds. – NotTheDr01ds Apr 14 '22 at 20:13
  • I ran into the same problem and created a video based on this and other related questions. https://youtu.be/458YBx-Px-E – Sven Malvik Jan 01 '23 at 17:44
  • 1
    this explain why the NetBeans inside WSL is slow. Copy the NetBeans files to Linux dir, make it quite fast. Thanks, that is awesome! – Harun Apr 06 '23 at 06:20
  • @Harun Yup! And you're welcome! Every time there's a new WSL release, the first thing I check is whether the release notes indicate it has addressed the 9P performance ;-). Also, thanks for the upvote - It gave me my first "Great Answer" gold badge here on SO. :-) – NotTheDr01ds Apr 06 '23 at 14:57
  • The second one worked for me. Thanks a lot man!! Cheers!! – Arjun G Jul 13 '23 at 14:30
  • In addition to this. Install visual studio code (or other IDE) in wsl. This will speed up the whole process and I found git related colour changes in filename is working fine with this setup. I added a bat file in Desktop to run it in one click. put 'wsl code' in the bat file for that. – Arjun G Jul 14 '23 at 09:02
  • @ArjunG If I'm understanding you correctly, that shouldn't really help with this particular problem. Regardless of whether you are running the Linux version of WSL or the Windows version, the fact that WSL2 has to access files in the *Windows* filesystem through 9P is going to drastically slow things down. [Microsoft's recommendation](https://code.visualstudio.com/docs/remote/wsl) is to use the *Windows* version of VSC in WSL2 rather than the Linux version (see *"Install Visual Studio Code on the Windows side (not in WSL)"*), and I tend to agree. – NotTheDr01ds Jul 14 '23 at 11:02
  • @NotTheDr01ds I had some trouble opening files in an Ubuntu system when using Windows version VSC. A speed difference between the filesystem and the VSC raised my doubts about that. So I gave using VSC with WSL a try, and it worked well. I am aware that this is a bad approach. However, since certain corporate laptops come with numerous windows restrictions, we can utilise it as a workaround. – Arjun G Jul 24 '23 at 07:04
72

Building on @notthedr01ds's reponse.

If you look at Microsoft's Comparing WSL 1 and WSL 2 the 'Performance across the OS file systems' is explicitly worse in WSL2.

Comparison of WS1 and WSL2 from Comparing WSL 1 and WSL 2

My case fell into Exceptions for using WSL 1 rather than WSL 2

  • Your project files must be stored in the Windows file system. WSL 1 offers faster access to files mounted from Windows.
    • If you will be using your WSL Linux distribution to access project files on the Windows file system, and these files cannot be stored on the Linux file system, you will achieve faster performance across the OS files systems by using WSL 1.

This means I needed to swap to version 1.

wsl --set-version Ubuntu 1
Conversion in progress, this may take a few minutes...
Conversion complete.

Test before

>time git status
...
real    0m6.436s
user    0m0.055s
sys     0m36.380s

Test after

> time git status
...
real    0m0.126s
user    0m0.016s
sys     0m0.641s
tymtam
  • 31,798
  • 8
  • 86
  • 126
  • 6
    according to https://vxlabs.com/2019/12/06/wsl2-io-measurements/ best performance is with WSL-2 running on ext4 (i.e. inside WSL-2) and not outside (i.e. NTFS). They say it is faster than WSL-1 on lxfs or NTFS. – Nasser Apr 18 '22 at 23:36
4

I had exactly the same problem with WSL 2 on Windows 11. WSL uses two file systems, the WSL/Linux file system and the Windows file system.

What can be confusing is that by default the WSL prompt starts in

/mnt/c/Users/<username>

When you do a git clone in here what happens is the files are downloaded in the Windows filesystem. If you navigate in Windows to the Users/username directory you will find all your files here. The interaction between filesystems is done over a share (9P protocol). This is insanely slow. So if you are in a subdir of 'mnt' you are not using WSL filesystem.

Make sure after starting WSL you enter the WSL filesystem, e.g. your home dir

$ cd

If you clone here and start a build you will have the performance you'd expect.

https://learn.microsoft.com/en-us/windows/wsl/filesystems

edit:

I make sure wsl always starts on the local filesystem instead of wsl share. You can do this for any directory you like, I like to start in my git folder:

echo 'cd ~/git' >>~/.bashrc
FrankyHollywood
  • 1,497
  • 19
  • 18
3

For those who wants to keep their WSL version 2, I got much better performence using a NFS server on Windows mounted on WSL.

Performence

  • From the provided mounting point (2m06s)
> git status
...
git status  0.72s user 64.33s system 51% cpu 2:06.37 total
  • From NFS mounting point (~2sec)
> git status
git status  0.45s user 0.12s system 31% cpu 1.828 total

How to

I used a free software https://www.hanewin.net/nfs-e.htm (I downloaded the application not the service, as I don't have admin rights on my laptop) And closed the port on my firewall.

Add the folder you want to access on the NFS server :

enter image description here

Then it's just a line in /etc/fstab

172.18.192.1:/c/Users/.../IdeaProjects /home/.../IdeaProjects nfs defaults 0 0

Reboot wsl2 or mount the folder:

sudo mount /home/.../IdeaProjects

Done.

Erwan Daniel
  • 1,319
  • 11
  • 26
  • Thanks for adding this - I've been meaning to update my answer with similar info, but haven't had the chance yet. And it's better coming from someone who's tried it anyway, which I haven't yet. Also (for other readers), according to the Github issue thread I linked, other file servers may also work for this. Still hoping that the WSL dev team can make some direct improvements in 9P performance, though :-) – NotTheDr01ds May 24 '23 at 16:04
1

My issues got solved when I did checkout from ubuntu machine only and then running yarn/npm run finished quickly. Earlier I was doing git clone inside windows folder and running yarn/npm command from ubuntu.

Ashish Sharma
  • 574
  • 7
  • 18