You have to open a specific port to make this work.
You can verify this by temporarily disabling the firewall with this powershell command:
Set-NetFirewallProfile -Profile Domain,Public,Private -Enabled False // Disable firewall
Set-NetFirewallProfile -Profile Domain,Public,Private -Enabled True // Re-enable
I had once found this port (don't even ask how...), nagged to the people writing the documentation for MS, they pointed me to a corner on the internet that the port was mentioned, but now I still can't find it! This is outrageous...!
Update:
OK, I found it again! It is port 445!
And here is the documentation link: Port configuration on the target server
Turns out you need to know the port in advance to find it! (lol)
Here is a script to do this:
New-NetFirewallRule -DisplayName "__Allow_File_UploadDownload" -Direction Inbound -LocalPort 445 -Protocol TCP -Action Allow -RemoteAddress 192.168.1.5
(Note: Replace "192.168.1.5" with a whitelisted IP)