I know how to serialize to a Stream, but is it possible to serialize to a TextWriter?
The problem is that TextWriter does not expose a Stream object.
It seems there is no such option, but maybe I am missing something.
EDIT 1
Please, allow me to clarify. I know TextWriter does not expose a Stream object, because there are TextWriter implementations that do not have an underlying stream. That is the gist of this question - can we serialize to a TextWriter using System.Text.Json?
There is no built-in way to serialize directly to a TextWriter using System.Text.Json as of .NET 6. For confirmation:
In Try the new System.Text.Json APIs, Immo Landwerth of MSFT wrote,
We needed a new set of JSON APIs that are highly tuned for performance by using Span and can process UTF-8 directly without having to transcode to UTF-16 string instances.
I.e. not needing to serialize to some intermediate TextWriter was a design requirement.
No override of JsonSerializer.Serialize() or JsonSerializer.SerializeAsync() takes a TextWriter.
The search 'textwriter path:src/libraries/System.Text.Json/src/System/Text/Json' in dotnet/runtime currently returns nothing.
For comparison, searching for Stream in the same path returns 16 code results.
Whenever .NET needs to serialize to a Stream using some encoding other than UTF8, they do so via Encoding.CreateTranscodingStream() rather than via a StreamWriter; see this search for examples.
Thus the most straightforward way to serialize to a TextWriter using System.Text.Json would be to serialize to a string, then write that to the TextWriter.
If for some reason this causes prohibitively bad performance (e.g. because the intermediate strings would be large enough to go on the large object heap and cause memory fragmentation or garbage collection slowdowns) you could wrap your TextWriter in some synthetic Stream that takes the incoming bytes, interprets each pair as Unicode characters, and writes them to the TextWriter. Then that Stream could be in turn wrapped by a transcoding stream that accepts UTF8 and passed to JsonSerializer as shown in the following extension methods:
public static partial class JsonSerializerExtensions
{
public static void Serialize<TValue>(TextWriter textWriter, TValue value, JsonSerializerOptions? options = default)
{
if (textWriter == null)
throw new ArgumentNullException(nameof(textWriter));
using (var stream = textWriter.AsWrappedWriteOnlyStream(Encoding.UTF8, true))
{
JsonSerializer.Serialize(stream, value, options);
}
}
public static async Task SerializeAsync<TValue>(TextWriter textWriter, TValue value, JsonSerializerOptions? options = default, CancellationToken cancellationToken = default)
{
if (textWriter == null)
throw new ArgumentNullException(nameof(textWriter));
await using (var stream = textWriter.AsWrappedWriteOnlyStream(Encoding.UTF8, true))
{
await JsonSerializer.SerializeAsync(stream, value, options);
}
}
}
public static partial class TextExtensions
{
public static Encoding PlatformCompatibleUnicode { get; } = BitConverter.IsLittleEndian ? Encoding.Unicode : Encoding.BigEndianUnicode;
public static bool IsPlatformCompatibleUnicode(this Encoding encoding) => BitConverter.IsLittleEndian ? encoding.CodePage == 1200 : encoding.CodePage == 1201;
public static Stream AsWrappedWriteOnlyStream(this TextWriter textWriter, Encoding outerEncoding, bool leaveOpen = false)
{
if (textWriter == null || outerEncoding == null)
throw new ArgumentNullException();
Encoding innerEncoding;
if (textWriter is StringWriter)
innerEncoding = PlatformCompatibleUnicode;
else
innerEncoding = textWriter.Encoding ?? throw new ArgumentException(string.Format("No encoding for {0}", textWriter));
return outerEncoding.IsPlatformCompatibleUnicode()
? new TextWriterStream(textWriter, leaveOpen)
: Encoding.CreateTranscodingStream(new TextWriterStream(textWriter, leaveOpen), innerEncoding, outerEncoding, false);
}
}
sealed class TextWriterStream : Stream
{
// By sealing UnicodeTextWriterStream we avoid a lot of the complexity of MemoryStream.
TextWriter textWriter;
bool leaveOpen;
Nullable<byte> lastByte = null;
public TextWriterStream(TextWriter textWriter, bool leaveOpen) => (this.textWriter, this.leaveOpen) = (textWriter ?? throw new ArgumentNullException(), leaveOpen);
public override bool CanRead => false;
public override bool CanSeek => false;
public override bool CanWrite => true;
public override long Length => throw new NotSupportedException();
public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); }
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override int Read(Span<byte> buffer) => throw new NotSupportedException();
public override int Read(byte[] buffer, int offset, int count) => throw new NotSupportedException();
public override int ReadByte() => throw new NotSupportedException();
bool TryPopLastHalfChar(byte b, out char ch)
{
if (lastByte != null)
{
Span<byte> tempBuffer = stackalloc byte [2];
tempBuffer[0] = lastByte.Value; tempBuffer[1] = b;
ch = MemoryMarshal.Cast<byte, char>(tempBuffer)[0];
lastByte = null;
return true;
}
ch = default;
return false;
}
void PushLastHalfChar(byte b)
{
if (lastByte != null)
throw new InvalidOperationException("Last half character is already saved.");
this.lastByte = b;
}
void EnsureOpen()
{
if (textWriter == null)
throw new ObjectDisposedException(GetType().Name);
}
static void Flush(TextWriter textWriter, Nullable<byte> lastByte)
{
if (lastByte != null)
throw new InvalidOperationException(string.Format("Attempt to flush writer with pending byte {0}", (int)lastByte));
textWriter?.Flush();
}
static Task FlushAsync(TextWriter textWriter, Nullable<byte> lastByte, CancellationToken cancellationToken)
{
if (lastByte != null)
throw new InvalidOperationException("Attempt to flush writer with pending byte");
if (cancellationToken.IsCancellationRequested)
return Task.FromCanceled(cancellationToken);
return textWriter.FlushAsync(); // No overload takes a cancellation token?
}
public override void Flush()
{
EnsureOpen();
Flush(textWriter, lastByte);
}
public override Task FlushAsync(CancellationToken cancellationToken)
{
EnsureOpen();
return FlushAsync(textWriter, lastByte, cancellationToken);
}
public override void Write(byte[] buffer, int offset, int count)
{
ValidateBufferArgs(buffer, offset, count);
Write(buffer.AsSpan(offset, count));
}
public override void Write(ReadOnlySpan<byte> buffer)
{
EnsureOpen();
if (buffer.Length < 1)
return;
if (TryPopLastHalfChar(buffer[0], out var ch))
{
textWriter.Write(ch);
buffer = buffer.Slice(1);
}
if (buffer.Length % 2 != 0)
{
PushLastHalfChar(buffer[buffer.Length - 1]);
buffer = buffer.Slice(0, buffer.Length - 1);
}
if (buffer.Length > 0)
textWriter.Write(MemoryMarshal.Cast<byte, char>(buffer));
}
public override Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
{
ValidateBufferArgs(buffer, offset, count);
return WriteAsync(buffer.AsMemory(offset, count)).AsTask();
}
public override ValueTask WriteAsync(ReadOnlyMemory<byte> buffer, CancellationToken cancellationToken = default)
{
if (cancellationToken.IsCancellationRequested)
return ValueTask.FromCanceled(cancellationToken);
try
{
return WriteAsyncCore(buffer, cancellationToken);
}
catch (OperationCanceledException oce)
{
return new ValueTask(Task.FromCanceled(oce.CancellationToken));
}
catch (Exception exception)
{
return ValueTask.FromException(exception);
}
}
async ValueTask WriteAsyncCore(ReadOnlyMemory<byte> buffer, CancellationToken cancellationToken)
{
EnsureOpen();
if (buffer.Length < 1)
return;
if (TryPopLastHalfChar(buffer.Span[0], out var ch))
{
await textWriter.WriteAsync(ch);
buffer = buffer.Slice(1);
}
if (buffer.Length % 2 != 0)
{
PushLastHalfChar(buffer.Span[buffer.Length - 1]);
buffer = buffer.Slice(0, buffer.Length - 1);
}
if (buffer.Length > 0)
await textWriter.WriteAsync(Utils.Cast<byte, char>(buffer), cancellationToken);
}
protected override void Dispose(bool disposing)
{
try
{
if (disposing)
{
var textWriter = Interlocked.Exchange(ref this.textWriter!, null);
if (textWriter != null)
{
Flush(textWriter, lastByte);
if (!leaveOpen)
textWriter.Dispose();
}
}
}
finally
{
base.Dispose(disposing);
}
}
public override async ValueTask DisposeAsync()
{
var textWriter = Interlocked.Exchange(ref this.textWriter!, null);
if (textWriter != null)
{
await FlushAsync(textWriter, lastByte, CancellationToken.None);
if (!leaveOpen)
await textWriter.DisposeAsync();
}
await base.DisposeAsync();
}
static void ValidateBufferArgs(byte[] buffer, int offset, int count)
{
if (buffer == null)
throw new ArgumentNullException(nameof(buffer));
if (offset < 0 || count < 0)
throw new ArgumentOutOfRangeException();
if (count > buffer.Length - offset)
throw new ArgumentException();
}
}
public static class Utils
{
// Adapted for read only memory from this answer https://stackoverflow.com/a/54512940/3744182
// By https://stackoverflow.com/users/23354/marc-gravell
// To https://stackoverflow.com/questions/54511330/how-can-i-cast-memoryt-to-another
public static ReadOnlyMemory<TTo> Cast<TFrom, TTo>(ReadOnlyMemory<TFrom> from)
where TFrom : unmanaged
where TTo : unmanaged
{
// avoid the extra allocation/indirection, at the cost of a gen-0 box
if (typeof(TFrom) == typeof(TTo)) return (ReadOnlyMemory<TTo>)(object)from;
return new CastMemoryManager<TFrom, TTo>(MemoryMarshal.AsMemory(from)).Memory;
}
private sealed class CastMemoryManager<TFrom, TTo> : MemoryManager<TTo>
where TFrom : unmanaged
where TTo : unmanaged
{
private readonly Memory<TFrom> _from;
public CastMemoryManager(Memory<TFrom> from) => _from = from;
public override Span<TTo> GetSpan() => MemoryMarshal.Cast<TFrom, TTo>(_from.Span);
protected override void Dispose(bool disposing) { }
public override MemoryHandle Pin(int elementIndex = 0) => throw new NotSupportedException();
public override void Unpin() => throw new NotSupportedException();
}
}
And now you will be able to do something like:
JsonSerializerExtensions.Serialize(textWriter, someObject);
But I'm not sure all this effort is worth it in all honesty.
Notes:
Not tested on big-endian platforms (which are rare).
Could use more async testing.
Demo fiddle here.
