Streamreader to String not working properly

Question

I am getting HttpWebResponse encoded in Base64 following lines get the webresponse from API.

Dim myResp As HttpWebResponse = myReq.GetResponse()
Dim myreader As New System.IO.StreamReader(myResp.GetResponseStream)

the response which i get is something like following, however actual response is too long and i cannot paste here so i have manually stripped the actual response.

{"status":"1","data":"eyJiMmIiOlt7ImludiI6W3siaXRtcyI6W3sibnVtIjoxODAxLCJpdG1fZGV0Ijp7ImNzYW10IjowLCJzYW10Ijo4MDkuOTEsInJ0IjoxOCwidHh2YWwiOjg5OTksImNhbXQiOjgwOS45MX19XSwidmFsIjoxMDYxOC44MiwiaW52X3R5cCI6IlIiLCJwb3MiOiIyNCIsImlkdCI6IjExLTA3LTIwMTgiLCJyY2hyZyI6Ik4iLCJpbnVtIjoiUldHSjA3LzE4LzAwMDU4NCIsImNoa3N1bSI6IjVjMjNiY2M1ZTQ3ZDI0NjU5YWQzNTEzNTM1YjhiNTAzNmM4NGU0MzU5NWJiMTVjYzA4M2VkYzBiNTQzZTQ1MzcifSx7Iml0bXMiOlt7Im51bSI6MTgwMSwiaXRtX2RldCI6eyJjc2FtdCI6MCwic2FtdCI6NDE4LjUsInJ0IjoxOCwidHh2YWwiOjQ2NTAsImNhbXQiOjQxOC41fX1dLCJ2YWwiOjU0ODcsImludl90eXAiOiJSIiwicG9zIjoiMjQiLCJpZHQiOiIyNS0wNy0yMDE4IiwicmNocmciOiJOIiwiaW51bSI6IlJXR0owNy8xOC8wMDEyNjEiLCJjaGtzdW0iOiJjOGEyMjNmNmMzYjY5ODZiYzE2MmNjYjdmMDhlZTYxMTdjYTdkOWZhNmEzYTExMWY1MmVjNzllYmExMGM5MWQ3In1dLCJjZnMiOiJZIiwiY3RpbiI6IjI0QUFCQ1I3MTc2QzFaSiJ9LHsiaW52IjpbeyJpdG1zIjpbeyJudW0iOjEsIml0bV9kZXQiOnsiY3NhbXQiOjAsInNhbXQiOjMzNzUsInJ0IjoxOCwidHh2YWwiOjM3NTAwLCJjYW10IjozMzc1fX1dLCJ2YWwiOjQ0MjUwLCJpbnZfdHlwIjoiUiIsInBvcyI6IjI0IiwiaWR0IjoiMzEtMDctMjAxOCIsInJjaHJnIjoiTiIsImludW0iOiJULTAxNzcvMjAxOC0xOSIsImNoa3N1bSI6ImYzNzFmYjA0N2FjNTRlOTkwYzZjNzM5Zjk0NTgwMzZlMWQxNjE0N2IxYmQ0ZTkxY2FlNmEwN2IyOGVlYzE0YWUifV0sImNmcyI6IlkiLCJjdGluIjoiMjRBQURDSTIwMzJFMVo5In1dfQ=="}

I am not sure why above Base64 Encoded message starts with {"status":"1","data":" and then ends with "}.

Actual Base64 data starts after {"status":"1","data":"

Due to those unsupported characters at starting and ending of the stream , i first try to convert actual response to string as shown below.

Dim myResp As HttpWebResponse = myReq.GetResponse()
Dim myreader As New System.IO.StreamReader(myResp.GetResponseStream)

Actual stream response returns around 248000 characters (as per response received in POSTMAN with same API). Streamreader information in Debug mode also shows same 248000 number. But when i convert them into string with following code line, string gets slimmed to around only 32000 characters. I don't know why this is happening?

Dim myText As String = myreader.ReadToEnd

'''Then following code will remove all those unwanted characters from starting string, which are {"status":"1","data":" 
Dim Final_text As String = myText.Substring(myText.Substring(0, myText.LastIndexOf("""")).LastIndexOf("""") + 1)

'''Following code will remove two characters "} from end of the string.
Final_text = Final_text.Trim().Remove(Final_text.Length - 2)

''' Now Decode this proper Base64 String to JSON format
Dim data As Byte() = Convert.FromBase64String(Final_text)
Dim decodedString As String = Encoding.UTF8.GetString(data)

Dim JsonP As JObject = JObject.Parse(decodedString)       
Dim SetPointerOut As JToken = JsonP("b2b")

Two things: why converting from Stream to String cut down actual response? 248000 charters to just apprx. 32000 characters. In debug mode if i type in ?mytext.length it returns 248000 as value. But When i hover mouse and brows what is in mytext variable, it shows me around 32000 charters only.

Service provider says Response which i get from API is Base64 encoded and i have to decode it before using it as JSON. Then why do i get unsupported characters at starting of the stream (even in Postman), is it Base64 Encoded message in serialized manner?

Am I doing right process to first convert the stream to string, remove unwanted characters and then Decode it? or there is some other way around.

Answer 1

Ok, issue of 32768 character in debug mode of Visual Studio is it self. VS2015 had bug in which it does not support more than 32768 characters. Read Why strings are shown partially in the Visual Studio 2008 debugger? and Visual Studio Text Visualizer missing text

The method which i was using to remove extra unwanted characters from "mytext" string, still works and give result. But as @Steve suggested in comment to the question, I should parse the JSON string. I find that idea much better and correct method.

so final code is like below:

Dim myResp As HttpWebResponse = myReq.GetResponse()
Dim myreader As New System.IO.StreamReader(myResp.GetResponseStream)
Dim myText As String = myreader.ReadToEnd
Dim json As String = myText
Dim jsonResult = JsonConvert.DeserializeObject(Of Dictionary(Of String, Object))(json)


Dim jsonObject As Newtonsoft.Json.Linq.JObject = Newtonsoft.Json.Linq.JObject.Parse(json)
Dim jsonValue As JValue = jsonObject("data")

Dim Final_text As String = jsonValue.ToString

''' No need of following code as doing JSON parse above
''' Dim Final_text As String = myText.Substring(myText.Substring(0, myText.LastIndexOf("""")).LastIndexOf("""") + 1)

'''Final_text = Final_text.Trim().Remove(Final_text.Length - 2)

Dim data As Byte() = Convert.FromBase64String(Final_text)
Dim decodedString As String = Encoding.UTF8.GetString(data)

Dim JsonP As JObject = JObject.Parse(decodedString)       
Dim SetPointerOut As JToken = JsonP("b2b")