Compiling dotnet core app for linux-musl-arm

Question

I tried compiling a simple .NET Core hello world app for OpenWRT on ARM Cortex A7 using linux-musl-arm with the following command:

dotnet publish --configuration Release --runtime linux-musl-arm --self-contained

But i get the following error when i try to execute it:

root@Routeur:~/RutxApp# ./RutxApp
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED2Ev: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_assignERKS4_: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE10_M_replaceEjjPKcj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_mutateEjjPKcj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5rfindEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5rfindEcj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12find_last_ofEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE8_M_eraseEjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE14_M_replace_auxEjjjc: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEcj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1ERKNS_12basic_stringIcS2_S3_EESt13_Ios_Openmode: symbol not found
Error relocating ./RutxApp: _ZSt7getlineIcSt11char_traitsIcESaIcEERSt13basic_istreamIT_T0_ES7_RNSt7__cxx1112basic_stringIS4_S5_T1_EES4_: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE17find_first_not_ofEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
Error relocating ./RutxApp: __time64: symbol not found
Error relocating ./RutxApp: __gmtime64: symbol not found
Error relocating ./RutxApp: __fstat_time64: symbol not found
Error relocating ./RutxApp: __dlsym_time64: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEjjRKS4_: symbol not found
Error relocating ./RutxApp: __fstatat_time64: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEjjPKc: symbol not found
Error relocating ./RutxApp: _ZTTNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEE: symbol not found
Error relocating ./RutxApp: _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE: symbol not found

I followed this tutorial : https://davidjmclaughlin.com/projects/how-to/2020/03/15/running-.net-core-on-openwrt-and-raspberry-pi-4.html and installed all required library but without success.

I'm trying to run this code on a RUTX11 from Teltonika

EDIT 1:

When i execute ldd ./RutxApp i get the following messages:

root@Routeur_VIGICAM:~/RutxApp# ldd ./RutxApp
        /lib/ld-musl-armhf.so.1 (0xb6ed6000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb6e0e000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb6df2000)
        libc.musl-armv7.so.1 => /lib/ld-musl-armhf.so.1 (0xb6ed6000)
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED2Ev: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_assignERKS4_: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE10_M_replaceEjjPKcj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_mutateEjjPKcj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5rfindEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5rfindEcj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12find_last_ofEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE8_M_eraseEjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE14_M_replace_auxEjjjc: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEcj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1ERKNS_12basic_stringIcS2_S3_EESt13_Ios_Openmode: symbol not found
Error relocating ./RutxApp: _ZSt7getlineIcSt11char_traitsIcESaIcEERSt13basic_istreamIT_T0_ES7_RNSt7__cxx1112basic_stringIS4_S5_T1_EES4_: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE17find_first_not_ofEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEj: symbol not found
Error relocating ./RutxApp: _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
Error relocating ./RutxApp: __time64: symbol not found
Error relocating ./RutxApp: __gmtime64: symbol not found
Error relocating ./RutxApp: __fstat_time64: symbol not found
Error relocating ./RutxApp: __dlsym_time64: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEPKcjj: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEjjRKS4_: symbol not found
Error relocating ./RutxApp: __fstatat_time64: symbol not found
Error relocating ./RutxApp: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEjjPKc: symbol not found
Error relocating ./RutxApp: _ZTTNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEE: symbol not found
Error relocating ./RutxApp: _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE: symbol not found
root@Routeur_VIGICAM:~/RutxApp#

I don't know what i'm missing...

EDIT 2:

Ok thanks to @albirew answer it look like the musl version mismatch... I tried to downgrade the .NET Core version to a version that use musl with x86 libc libraries (netcoreapp3.1) but now the app try to load ld-linux-armhf.so but it doesn't exist in the system, instead there is ld-musl-armf.so (this was loaded with next version version of .NET Core)

Note: it's the same error when i compile with linux-arm runtime (instead of linux-musl-arm)

More informations about OpenWrt version 21.02.0:

Maybe a version mismatch between the runtimes build against vs run against? We have premium support from Teltonika so we can send them a ticket... — Albirew, Apr 20 '22 at 16:15
@Albirew We can try that. Does they also support the creation of custom applications? — Ludovic Feltz, Apr 20 '22 at 16:44
This question was also posted here: https://forum.openwrt.org/t/dotnet-core-error-relocating/125784 — Ludovic Feltz, Apr 21 '22 at 12:12
Not sure if it gonna help, but I don’t publish any binaries for embedded Linux. I build them normally with Visual Studio in Release config, install the runtime on target device, and run software with a command like `/usr/share/dotnet/dotnet app.dll` — Soonts, Apr 21 '22 at 14:10
@Soonts Not sure it gonna help but i will give it a try. But it look like the problem comes from some C or C++ library that aren't on the linux system — Ludovic Feltz, Apr 21 '22 at 15:51
@LudovicFeltz It might work for you if there’s a bug in the way MS toolset makes these self-contained packages. On Qualcomm MSM8974 (also ARMv7 arch) with Alpine 3.15 OS, the MS-built binaries of .NET runtime work fine when launched with pre-installed `dotnet` binary. You can also try `dotnet --info` command. — Soonts, Apr 21 '22 at 17:49
Looks like it the same problem here: https://stackoverflow.com/questions/48017830/net-core-for-armv7-32-bit-processor — Ludovic Feltz, Apr 26 '22 at 09:05

score 0 · Accepted Answer · edited Apr 21 '22 at 17:55

0

Like I said, this is a version mismatch between the runtimes build against vs run against.

hnyman explained it on openwrt forums that seeing the __time64: symbol not found error made him believe you are building your app using musl 1.2.2+ with x64 time symbols, but the Teltonika router is using OpenWrt 21.02.x with the older musl 1.1.24 version and x86 time handling.

edited Apr 21 '22 at 17:55

Ludovic Feltz

11,416
4
47
63

answered Apr 21 '22 at 17:49

Albirew

300
1
5

Thank you, i will try your solution tomorrow and see if it works. can't accept this solution right now, i'm not in my works computer. – Ludovic Feltz Apr 21 '22 at 17:51
Edited my answer, now i have another issue – Ludovic Feltz Apr 22 '22 at 08:51

Compiling dotnet core app for linux-musl-arm

EDIT 1:

EDIT 2:

1 Answers1