How can I override default encoding when marshalling strings?

Question

I have a Delphi 2007 DLL with this export:

function TestSignString(s, Sign: PChar): LongBool; stdcall;

I am calling it in .NET Core v7 C# using this import:

[DllImport(@"LibraryName.dll")]
[return:MarshalAs(UnmanagedType.Bool)]
internal static extern bool TestSignString(string s, string sign);

I am passing s and sign params, when the run-time system has default encoding (EXPECTED) WIN1250 everything goes fine as expected, however when it's not, there's default system encoding involved and passed chars are encoded using default encoding (for example to WIN1252).

So my question is, how to override default encoding when marshaling strings in C#? I mean both ways - as a input params and output as well, please.

I have tried altering the import to:

[DllImport(@"LockBoxLibrary.dll", CharSet = CharSet.Ansi)]

but no luck, I am missing code page (1250) specification.

Are you missing? - https://stackoverflow.com/questions/37870084 — Rand Random, Jun 01 '23 at 08:41
@RandRandom, thank you for your reply, but this question is not the case, it's about un/availability of custom encoding in .NET Core. — J. Ogurek, Jun 01 '23 at 08:52
and you claimed `I am missing code page (1250) specification.`, `using default encoding (for example to WIN1252).` and you are using `.NET Core v7` which by default has no support for the code page you mentioned — Rand Random, Jun 01 '23 at 08:58
You cannot do this ... but you can always send byte array (`ref byte[] s, ....`) instead of string and do the encoding by yourself — Selvin, Jun 01 '23 at 09:00
@RandRandom, as I wrote, this is not an issue, I have registered custom encoding and it's working fine even in .NET Core, the problem is how to override default encoding. — J. Ogurek, Jun 01 '23 at 09:04
@Selvin, thank you, yes, this is what I whould like to do :-) Can you post a short code snipper, please? — J. Ogurek, Jun 01 '23 at 09:05
@shingo `CharSet.Unicode` will marshal the strings as UTF-16, which is compatible with `PWideChar` in Delphi. But in Delphi 2007 and earlier, `PChar` maps to `PAnsiChar` instead, so is not compatible with UTF-16. `CharSet.Ansi` or `byte[]` must be used in this situation. — Remy Lebeau, Jun 01 '23 at 17:05
@RemyLebeau, thank you for your expert insight. Can you post your solution, please? — J. Ogurek, Jun 01 '23 at 17:55
@J.Ogurek You have already been told what to do - pass in `string`s and use `CharSet.Ansi` in the `DllImport`, or else pass in (null-terminated) `byte[]` arrays instead of `string`s. You could try using `PtrInt` with `Marshal.AllocHGlobal()`, but that's pretty heavy-handed for such a simple task. — Remy Lebeau, Jun 01 '23 at 18:14

score 0 · Answer 1 · answered Jun 01 '23 at 12:35

0

I have a "dirty" solution as it's probably memory leaking. Let me know if you know better way, please. I have two extension methods for string conversion to nint and vice versa.

private static string FromAnsiPtrToUtf8Str(this nint n)
{
    var length = 0;

    while (Marshal.ReadByte(n + length) != 0)
    {
        length++;
    }

    var strbuf = new byte[length];
    Marshal.Copy(n, strbuf, 0, length);
    
    return Encoding.GetEncoding(1250).GetString(strbuf);
}

private static nint FromUtf8StrToAnsiPtr(this string s) => GCHandle.Alloc(Encoding.GetEncoding(1250).GetBytes(s), GCHandleType.Pinned).AddrOfPinnedObject();

Import changed to: private static extern bool TestSignString(nint s, nint sign);

Method call: TestSignString(s.FromUtf8StrToAnsiPtr(), sign.FromUtf8StrToAnsiPtr());

answered Jun 01 '23 at 12:35

J. Ogurek

17
3

Naming your functions `Utf8` is misleading since there is no UTF-8 involved in this code. C# strings are not UTF-8. `FromAnsiPtrToStr()` and `FromStrToAnsiPtr()` would be more accurate. But then, you are really kind of just duplicating what `Marshal.PtrToStringAnsi()` and `Marshal.StringToXXXAnsi()` already do. – Remy Lebeau Jun 01 '23 at 18:18
@RemyLebeau, `Utf8` is not involved? I thought Utf8 is default string encoding in .NET. And duplicating `Marshal.PtrToStringAnsi()`? When I decompile these methods, they do different stuff - no encoding conversion like I do. Can you post your code snippet, please? – J. Ogurek Jun 02 '23 at 08:05
"*I thought Utf8 is default string encoding in .NET.*" - Nope, the [`String`](https://learn.microsoft.com/en-us/dotnet/api/system.string) type in .NET "*Represents text as a sequence of **UTF-16** code units.*" Just like everything else Unicode-related in Windows – Remy Lebeau Jun 02 '23 at 15:04
If you look at the actual source code for `PtrToStringAnsi()`, it just calls the `sbyte*` constructor of `String`. And the doc for that constructor says the bytes "*are interpreted using the current system code page encoding (that is, the encoding specified by `Encoding.Default`).*" – Remy Lebeau Jun 02 '23 at 15:11
@RemyLebeau, your statement is clear, however take a look at the Encoding sample... https://dotnetfiddle.net/nT8Jto – J. Ogurek Jun 05 '23 at 07:35
@RemyLebeau, yes, this is it. I am interested in overriding the default encoding as the title of this thread states. – J. Ogurek Jun 05 '23 at 07:36
You can't override what codepage `Encoding.Default` or `Charset.Ansi` refer to. If you need a specific encoding, use `Encoding.GetEncoding()` instead. If .NET Core doesn't support what you need, then implement the charset manually, or use a 3rd party library. – Remy Lebeau Jun 05 '23 at 15:05
@RemyLebeau, you think WIN1250 encoding is not supported in .NET Core? It is supported, the only step is to register encoding provider like this `Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);` – J. Ogurek Jun 06 '23 at 06:28
"*you think WIN1250 encoding is not supported in .NET Core?*" - I didn't say anything about that. You did: "*I am missing code page (1250) specification*", to which RandRandom [replied to](https://stackoverflow.com/questions/76379702/76381932#comment134685060_76379702): "*you are using .NET Core v7 which by default has no support for the code page you mentioned*" – Remy Lebeau Jun 06 '23 at 15:38

score 0 · Answer 2 · answered Jun 03 '23 at 17:59

0

Try to change TestSignString to take a UTF-16 string, or introduce a widestring (W) version, it is 2023 :-)

function TestSignStringW(s, Sign: PWideChar): LongBool; stdcall;

answered Jun 03 '23 at 17:59

Dmitry Streblechenko

62,942
4
53
78

thank you for your kind help, but I am not able to change Delphi library side, that's why the topic is "How can I override default encoding when marshalling strings?" – J. Ogurek Jun 05 '23 at 07:40

How can I override default encoding when marshalling strings?

2 Answers2