3

I am trying to serialize/deserialize an object. The serialization works well, but the deserialization doesn't work. It seems that it is a unicode problem : There is a System.Text.DecoderFallbackException Exception that says that there is a Unicode problem (Translation French to English) :

Can not translate [FF] bytes to index 31 from the code page specified in Unicode.

I used the two functions for Binary serialization in deadlydog answer on the topic.

The question is, how can I specify a Unicode for deserialization and serialization in this code ?

EDIT :

Finaly I am using protobuf-net, here are my functions :

    public static void WriteToBinaryFile<T>(string filePath, T objectToWrite, bool append = false)
    {
        using (var file = File.Open(filePath, append ? FileMode.Append : FileMode.Create))
        {
            Serializer.Serialize<T>(file, objectToWrite);
        }
    }


    public static T ReadFromBinaryFile<T>(string filePath)
    {
        using (var file = File.Open(filePath, FileMode.Open))
        {
            return Serializer.Deserialize<T>(file);
        }
    }

But I have an error when I deserialize : Invalid field in source data: 0

lema
  • 470
  • 2
  • 9
  • I kinda need to raise a red flag here; that linked post uses `BinaryFormatter`, which is **almost always** a ***terrible*** idea (the only time it *should* be used is for communicating between app-domains, or for IPC/RPC between systems *that are strongly tied and will always be in sync*). Using it for file persistence is a **really bad idea** and it **will hurt you**. Can I strongly suggest you revisit your serialization choices? If you want binary, tools like protobuf-net are *very easy to use*, and will **get this right**. – Marc Gravell Jul 04 '19 at 10:20
  • FWIW: the code shown shouldn't cause encoding errors; that's very odd and unusual - are you sure you didn't corrupt the payload at some point? (that's remarkably easy to do, I see people doing this *all the time*); this **is not an endorsement**, however, and even if it is this: I really really don't think that "fix the problem that is causing the corruption" is your best approach here; again, I **very strongly urge** you to run away from `BinaryFormatter` as fast as you possibly can, and do not look back. – Marc Gravell Jul 04 '19 at 10:26
  • (if you can show your usage *surrounding* this code, I can probably point out any likely causes of corruption; the most common cause is people trying to store binary as text by running it backwards through an `Encoding`, which ... doesn't work) – Marc Gravell Jul 04 '19 at 10:36
  • JSON is HTML which will not work with binary. HTML has special characters. See Wiki : https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references. You can use System.Net.WebUtility.HtmlEncode(string) to convert special characters. Also when sending over a html connection you need to make a 64 bit string to remove special characters. The JSON serializer should handle special characters (unicode). I do not know what instruction the code is failing so I can't tell what the correct solution would be. – jdweng Jul 04 '19 at 10:42
  • @jdweng um... unless I'm missing something, everything about that statement is wrong, sorry; "JSON is HTML" - I'm guessing you actually meant "text" there? ... "which will not work with binary." - no, text works trivially with binary; that's what an encoding defines, and `BinaryFormatter` (which is doing the work here) knows perfectly well how to encode/decode text/strings; "when sending over a html connection" - do you mean http? http works fine with binary; nobody mentioned JSON/HTML, btw; "you need to make a 64 bit string" - just; no; that's "not even wrong" – Marc Gravell Jul 04 '19 at 10:46
  • The deadlydog has JSON. Marc this is absolutely wrong : " http works fine with binary". You MUST encode to 64 base string to send binary over HTTP. – jdweng Jul 04 '19 at 10:55
  • @jdweng the linked post has examples for binary, xml, and json; we're discussing the first one here; "base-64" != "64-bit" (if anything it is 128-bit i.e. ASCII), but if you mean base-64, no: that's also backwards; HTTP (not that anyone mentioned HTTP being involved here) - despite the T being "text" - works just fine with raw binary without any encoding - that's actually how most non-text content is transmitted; if you want to send binary *inside text content* (like in HTML), then yes: you need to use something like base-64, base-16, or similar; but *nobody is talking about those things* – Marc Gravell Jul 04 '19 at 11:05
  • Thank you for your help, I'll try using protobuf-net ! – lema Jul 04 '19 at 11:57
  • @MarcGravell, I think I am doing alright, I did exactly as it is said in the documentation in github. But I have an error when I deserialize : **Invalid field in source data: 0**. I have seen some topic on this, but they are not doing like me (writing to a file) – lema Jul 04 '19 at 12:25
  • @lema "Invalid field in source data: 0" - are you talking about protobuf-net ? fields must be **positive** integers (and low numbers are cheaper than high numbers): start with `[ProtoMember(1)]` etc – Marc Gravell Jul 04 '19 at 13:07
  • Yes I am talking about protobuf-net ! I believe all my fields are positive integers. – lema Jul 04 '19 at 13:12
  • I have some class that in inherit from others, maybe there is some rules that I don't know ? I have added ProtoInclude as it is said in the documentation. – lema Jul 04 '19 at 13:15
  • @lema I can definitely help you with this - I'm the library author; but I'd need to see some code – Marc Gravell Jul 04 '19 at 13:24
  • Well there is too much code/class in my project, bu I'll edit my post to give you my serialization/deserialization functions. – lema Jul 04 '19 at 13:28
  • @lema I only found this now because I remembered to check; if you don't "at" me, I don't get a notification. I see those methods, and they look fine. Can you post enough code for me to actually repro the issue you're seeing? I..e something runnable? Help me to help you. .. – Marc Gravell Jul 06 '19 at 23:29
  • @MarcGravell I finaly used JsonSerializer and it worked well immediatly, it would have been too long to add all my code (around 10 class need to be serialize). But thanks a lot for trying to help me ! – lema Jul 08 '19 at 07:39

0 Answers0