2

We have a public API call in which we would like to take the raw content of the Request body and store in a log record in our database. The basic use of our code is as follows:

public HttpResponseMessage PutObjects([FromBody] List<ObjectDTO> objectsDto)
{
    string bodyText = "";
    //HttpContext.Current.Request.GetBufferedInputStream().Close();
    using (var bodyStream = new StreamReader(HttpContext.Current.Request.InputStream))
    {
        bodyStream.BaseStream.Seek(0, SeekOrigin.Begin);
        bodyText = bodyStream.ReadToEnd();
    }
    // ... Rest of logic proceeds

Using this solution works when the Body being passed across is of valid JSON to deserialize into the objectsDto. However, when the JSON is malformed and can't be loaded into the objectsDto parameter, the following error is thrown:

System.InvalidOperationException: 'Either BinaryRead, Form, Files, or InputStream was accessed before the internal storage was filled by the caller of HttpRequest.GetBufferedInputStream.'

I had a thought that if it was because the parameter wasn't filled due to (I'm assuming) an automatic call to GetBufferedInputStream, that if I execute Close() on this, it would reset the Stream and I would be able to proceed. The line of code I've used to do this is the commented out line seen in the code example.

This appears to have worked until closer inspection. The ReadToEnd() result put in bodyText appears to have cut off in the middle rather than reading to the end. One interesting detail I noticed is that it stops at a length of 1024. In other words, only the first 1024 characters are being returned from ReadToEnd(). Of course, 1024 is a very suspicious number to me but my lack of knowledge with StreamReaders has me at a stopping point.

The main question now is: Why does the StreamReader in this case only return the first 1024 characters when calling ReadToEnd()? Again, this appears to only occur if the JSON sent over is not formatted correctly.

Also, if anyone thinks of an easier way to grab the raw Body of the Request, I'm open to trying that as well. Thanks in advance.

EDIT: I attempted to use HttpContext.Current.Request.GetBufferlessInputStream(disableMaxRequestLength:true) to see what the outcome would be. The error received mentioned not being able to use GetBufferlessInputStream after GetBufferedInputStream was already called.

My Stack Overfloweth
  • 4,729
  • 4
  • 25
  • 42
  • Is there a possibility to use: bodyStream.Seek(0, SeekOrigin.Begin); instead of bodyStream.BaseStream.Seek(0, SeekOrigin.Begin); ? – VitezslavSimon Jun 06 '19 at 19:52
  • This behavior implies your input data stream can contain some managing character (<, >, \0, etc.) which can terminate the data stack. Did you tried to convert input to base64 string? – VitezslavSimon Jun 06 '19 at 19:55
  • If you will get a value of bodyStream.EndOfStream after 1st 1024 chars read ... what you will get? True or False? – VitezslavSimon Jun 06 '19 at 20:00
  • Just checked and .Seek isn't present on the bodyStream object. bodyStream is of type StreamReader and the .BaseStream property is of type Stream. For converting to base64, we don't have the freedom to do so before it hits the API call since we would ideally expect a response that is deserialized directly into the objectsDto parameter. Is there a way to dynamically read the string as base64 and store that instead? That would be okay since we could just encode back to the original value if we wanted to inspect the log. Also, EndOfStream returns true in this case. – My Stack Overfloweth Jun 06 '19 at 20:02
  • Just for some details of where it is cutting off: it doesn't appear to be a special character. The data it cuts off at is in the middle of this JSON property: "PermitReceivedOn": "0001-01-01T00:00:00". It stops at "PermitReceivedOn": "000. I couldn't think of any reason why it would choose that character so that's when I put the result into notepad and noticed it was returning exactly 1024 characters from the stream. – My Stack Overfloweth Jun 06 '19 at 20:04
  • When you will open a stream it should be on zero position by default if it is not a "realtime changeable" stream. Do you have a chance to use setter of Position property - https://stackoverflow.com/questions/7238929/stream-seek0-seekorigin-begin-or-position-0? base64 ... it is not possible to read as base64 if you haven't encoded it before call of server side. EndOfStream returns true in this case ... it seems you really have there some managing character inside the data. What codepage has been recognized on server side? Does your client used the same codepage? – VitezslavSimon Jun 06 '19 at 20:08
  • Does your application is created from standard MS template (what version of Visual Studio)? Is it a module for IIS server? Is it self hosted web server? – VitezslavSimon Jun 06 '19 at 20:14
  • I'm able to use Position = 0 but still receive the same first 1024 characters back. As far as what codepage and what the client sees, it only fails to read the Stream past 1024 characters if the automatic deserialization to the objectsDto doesn't work. The application is a .NET MVC Web API that I have running on a local IIS server. All other calls and functionality work fine. The only problem we've had is trying to get the raw request body being sent over even if it doesn't serialize properly to the expected parameter. – My Stack Overfloweth Jun 06 '19 at 20:17
  • What is your input data class in problem case? Are all your data inside data class simple and serializable types (string, bool, enum, ...). Are you using any collections inside the class - especially dictionary? E.g. you are unable to (de)serialize dictionaries well without custom code as same as e.g. connections, Type class, non-managed code handles, etc. Have you tried e.g. standard XML or JSON (de)serialization on your input data? What JSON serializer you are using? Newtonsoft.JSON or standard Microsoft one? – VitezslavSimon Jun 06 '19 at 20:26
  • What are headers of your request when it will arrive to your server side? https://stackoverflow.com/questions/27971340/getting-header-values-in-webapi-2-controller Do you see whole data of stream on your debug watch? – VitezslavSimon Jun 06 '19 at 20:33
  • You can also be limited by max capacity of header or body on your IIS server configuration (I am not using IIS so there I can't help much): https://stackoverflow.com/questions/22923400/web-api-maximum-header-value-length#comment35026453_22925520 ; https://stackoverflow.com/questions/38563118/maximum-http-request-size-for-asp-web-api-with-json – VitezslavSimon Jun 06 '19 at 20:51
  • We're using the default .NET Web API deserializtion. If that doesn't come through, the objectsDto comes across as null which is okay. Some additional info on this: we've included some logging in our global.asax Application_BeginRequest to read the raw Body of the request. This seems to work so far but is expensive. Thankfully, the code being written only needs to be used in a test environment to help a client identify what the body of their request is coming across as. However, I'm still perplexed how the StreamReader only returns 1024 characters. Thanks for all your help! – My Stack Overfloweth Jun 06 '19 at 20:58
  • You are welcome. It will be great to know where has been the problem. :) – VitezslavSimon Jun 06 '19 at 21:12

1 Answers1

0

I had the same issue and was surprised I could not find this answer quickly. Here is what worked for me:

using (var reader = new StreamReader(streamAFLPem, System.Text.Encoding.Default, false, 2048))
    {
         //Process StreamReader
    }

I found this at MS site. You may have to tweak the settings, but it can let you increase the buffer size. enter image description here

Danimal111
  • 1,976
  • 25
  • 31