I have found a workaround...
I found that it was extracting several dlls to the temp folder during install and the MSI log was complaining about loading one of them. No dependency loading issues found with any of them so I tried copying them all to C:\Windows\SysWow64, but one of them (sqlunirl.dll) got access denied as it is a part of the OS. If I change the owner and permissions of that dll, I can then replace it with the extracted one, install MSDE, and afterwards replace it with the original Win10 one. However the SQL Service Manager app then refuses to start, but since exes always look for dlls in their own folder first, put that same dll in C:\Program Files (x86)\Microsoft SQL Server\80\Tools\Binn. Just in case, I've also copied it to other folders that have executables in too. I've not tested this out yet but have knocked up a quick batch file to install (put it next to Setup.exe along with the sqlunirl.dll grabbed from the temp folder from a previous install attempt):
takeown /f C:\Windows\SysWOW64\sqlunirl.dll /a
icacls C:\Windows\SysWOW64\sqlunirl.dll /grant *S-1-5-32-544:f
IF NOT EXIST C:\Windows\SysWOW64\sqlunirl.bak move C:\Windows\SysWOW64\sqlunirl.dll C:\Windows\SysWOW64\sqlunirl.bak
copy /y "%~dp0sqlunirl.dll" C:\Windows\SysWOW64
"%~dp0Setup.exe" (plus your extra parameters such as SAPWD=PASSWORD)
move /y C:\Windows\SysWOW64\sqlunirl.dll "C:\Program Files (x86)\Microsoft SQL Server\80\Tools\Binn"
copy /y C:\Windows\SysWOW64\sqlunirl.dll "C:\Program Files (x86)\Microsoft SQL Server\80\COM"
copy /y C:\Windows\SysWOW64\sqlunirl.dll "C:\Program Files (x86)\Microsoft SQL Server\Mssql$InstanceName\BinnMSSQL$InstanceName\Binn"
move /y C:\Windows\SysWOW64\sqlunirl.bak C:\Windows\SysWOW64\sqlunirl.dll
icacls C:\Windows\SysWOW64\sqlunirl.dll /remove *S-1-5-32-544
icacls C:\Windows\SysWOW64\sqlunirl.dll /grant *S-1-5-32-544:(GR,GE,WO)
icacls C:\Windows\SysWOW64\sqlunirl.dll /setowner *S-1-5-80-956008885-3418522649-1831038044-1853292631-2271478464
(NB: SIDs are used for Administrators group and TrustedInstaller so that it works on any language. Generic Read and Generic Execute rights are the default on the administrators group, but I am also applying Write Owner so that I can set it back to TrustedInstaller as I couldn't find a way to use TakeOwn.exe to set it back!)