Converting Stream to String and back

Question

I want to serialize objects to strings, and back.

We use protobuf-net to turn an object into a Stream and back, successfully.

However, Stream to string and back... not so successful. After going through StreamToString and StringToStream, the new Streamisn't deserialized by protobuf-net; it raises an Arithmetic Operation resulted in an Overflow exception. If we deserialize the original stream, it works.

Our methods:

public static string StreamToString(Stream stream)
{
    stream.Position = 0;
    using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
    {
        return reader.ReadToEnd();
    }
}

public static Stream StringToStream(string src)
{
    byte[] byteArray = Encoding.UTF8.GetBytes(src);
    return new MemoryStream(byteArray);
}

Our example code using these two:

MemoryStream stream = new MemoryStream();
Serializer.Serialize<SuperExample>(stream, test);
stream.Position = 0;
string strout = StreamToString(stream);
MemoryStream result = (MemoryStream)StringToStream(strout);
var other = Serializer.Deserialize<SuperExample>(result);

shouldn't Stream be MemoryStrea? – Ehsan Jul 23 '13 at 05:01 — Ehsan, Jul 23 '13 at 05:01

score 340 · Answer 1 · edited Jun 08 '17 at 15:11

340

I have just tested this and works fine.

string test = "Testing 1-2-3";

// convert string to stream
byte[] byteArray = Encoding.ASCII.GetBytes(test);
MemoryStream stream = new MemoryStream(byteArray);

// convert stream to string
StreamReader reader = new StreamReader(stream);
string text = reader.ReadToEnd();

If stream has already been written to, you might want to seek to the beginning before first before reading out the text: stream.Seek(0, SeekOrigin.Begin);

edited Jun 08 '17 at 15:11

huysentruitw

27,376
9
90
133

answered Jul 23 '13 at 05:03

Ehsan

31,833
6
56
65

8

And don't forget a using block around StreamReader reader = new StreamReader(stream); – PRMan Feb 12 '20 at 18:20
This worked because it was tested on a string. It wouldn't work for streams containing arbitrary bytes: see Marc Gravell's answer below, regarding Convert.ToBase64 – FTWinston May 20 '22 at 11:41

score 69 · Accepted Answer · edited Oct 31 '22 at 15:13

69

This is so common but so profoundly wrong. Protobuf data is not string data. It certainly isn't ASCII. You are using the encoding backwards. A text encoding transfers:

an arbitrary string to formatted bytes
formatted bytes to the original string

You do not have "formatted bytes". You have arbitrary bytes. You need to use something like a base-n (commonly: base-64) encode. This transfers

arbitrary bytes to a formatted string
a formatted string to the original bytes

Look at Convert.ToBase64String and Convert.FromBase64String.

edited Oct 31 '22 at 15:13

MarredCheese

17,541
8
92
91

answered Jul 24 '13 at 09:32

Marc Gravell

1,026,079
266
2,566
2,900

1

could you use a [`BinaryFormatter`](http://msdn.microsoft.com/en-us/library/system.runtime.serialization.formatters.binary.binaryformatter(v=vs.110).aspx), similar to this [weird example](http://stackoverflow.com/a/1133465/1037948)? – drzaus Mar 20 '14 at 17:59
@drzaus hm...maybe not: > "Any unpaired surrogate characters are lost in binary serialization" – drzaus Mar 20 '14 at 18:00

score 16 · Answer 3 · edited Apr 28 '19 at 21:16

16

a UTF8 MemoryStream to String conversion:

var res = Encoding.UTF8.GetString(stream.GetBuffer(), 0 , (int)stream.Length)

edited Apr 28 '19 at 21:16

Levi Botelho

24,626
5
61
96

answered Jul 06 '16 at 07:37

Wolfgang Grinfeld

870
10
11

2

Use ToArray() instead. The buffer may larger than the size of used data. ToArray() returns a copy of the data with the correct size. `var array = stream.ToArray(); var str = Encoding.UTF8.GetString(array, 0, array.Length);` . See also https://msdn.microsoft.com/en-us/library/system.io.memorystream.getbuffer.aspx?ranMID=24542&ranEAID=TnL5HPStwNw&ranSiteID=TnL5HPStwNw-fWxoHnp968IcSsgF87gs3g&tduid=(af5d64506773ff2a87e3cde51de009eb)(256380)(2459594)(TnL5HPStwNw-fWxoHnp968IcSsgF87gs3g)() – Mortennobel Apr 02 '17 at 09:09
4

@Mortennobel `ToArray()` allocates a new array in memory and copies data over from the buffer, which may have serious implications if you're dealing with a lot of data. – Levi Botelho Apr 28 '19 at 21:01
1

Note the use of stream.Length, rather than stream.GetBuffer().Length. And Levi did correctly note the reason for not using ToArray(). – Wolfgang Grinfeld May 13 '20 at 14:53

score 6 · Answer 4 · answered Jul 23 '13 at 05:24

6

When you testing try with UTF8 Encode stream like below

var stream = new MemoryStream();
var streamWriter = new StreamWriter(stream, System.Text.Encoding.UTF8);
Serializer.Serialize<SuperExample>(streamWriter, test);

answered Jul 23 '13 at 05:24

Damith

62,401
13
102
153

score 6 · Answer 5 · edited Apr 30 '14 at 08:18

6

Try this.

string output1 = Encoding.ASCII.GetString(byteArray, 0, byteArray.Length)

edited Apr 30 '14 at 08:18

Ravi Dhoriya ツ

4,435
8
37
48

answered Apr 30 '14 at 07:59

user3588327

61
1
1

score 6 · Answer 6 · answered Jun 28 '22 at 05:27

6

 StreamReader reader = new StreamReader(strm, System.Text.Encoding.UTF8);
        var final1 = reader.ReadToEnd();

The main reason for most issue is encoding type... here by default System.Text.Encoding.UTF8 fixes the issue...

Enjoy...

answered Jun 28 '22 at 05:27

Madhusudhan V Indian

352
5
11

1

As `StreamReader` implements `IDisposable`, its recommended to use `using`. – user3819197 Jul 01 '22 at 03:12

score 2 · Answer 7 · answered Jan 08 '17 at 13:00

I wrote a useful method to call any action that takes a StreamWriter and write it out to a string instead. The method is like this;

static void SendStreamToString(Action<StreamWriter> action, out string destination)
{
    using (var stream = new MemoryStream())
    using (var writer = new StreamWriter(stream, Encoding.Unicode))
    {
        action(writer);
        writer.Flush();
        stream.Position = 0;
        destination = Encoding.Unicode.GetString(stream.GetBuffer(), 0, (int)stream.Length);
    }
}

And you can use it like this;

string myString;

SendStreamToString(writer =>
{
    var ints = new List<int> {1, 2, 3};
    writer.WriteLine("My ints");
    foreach (var integer in ints)
    {
        writer.WriteLine(integer);
    }
}, out myString);

I know this can be done much easier with a StringBuilder, the point is that you can call any method that takes a StreamWriter.

score 1 · Answer 8 · answered Jan 10 '19 at 20:02

I want to serialize objects to strings, and back.

Different from the other answers, but the most straightforward way to do exactly that for most object types is XmlSerializer:

        Subject subject = new Subject();
        XmlSerializer serializer = new XmlSerializer(typeof(Subject));
        using (Stream stream = new MemoryStream())
        {
            serializer.Serialize(stream, subject);
            // do something with stream
            Subject subject2 = (Subject)serializer.Deserialize(stream);
            // do something with subject2
        }

All your public properties of supported types will be serialized. Even some collection structures are supported, and will tunnel down to sub-object properties. You can control how the serialization works with attributes on your properties.

This does not work with all object types, some data types are not supported for serialization, but overall it is pretty powerful, and you don't have to worry about encoding.

score 0 · Answer 9 · answered Sep 10 '15 at 03:10

In usecase where you want to serialize/deserialize POCOs, Newtonsoft's JSON library is really good. I use it to persist POCOs within SQL Server as JSON strings in an nvarchar field. Caveat is that since its not true de/serialization, it will not preserve private/protected members and class hierarchy.

Converting Stream to String and back

9 Answers9

Linked