0

I've written a VB.Net app which connects to a remote IP/port combination that's outputting a steady stream of messages. The IP address could be localhost or it could be a remote machine.

The remote process can stop and start at any time and my process needs to connect to it and simply start receiving. There is no negotiation between the two about which ports to use.

My current code uses TcpClient and NetworkStream to read a byte at a time. This is converted to the corresponding ASCII character (all messages are ASCII text) and are appended to a string until an EOL character is received. Once it is then the line is added to a queue to be processed, the string is truncated and the process begins again. Whilst this works, listening a byte at a time is probably not the most efficient way to do this.

I now testing the app and I'm running into some serious performance issues. Although my current code works it can't handle the amount of data that is thrown at it. Whilst it can handle 500 messages a second, it really struggles at 1500 per second and peaks at 20% CPU.

I'm sure that the bottleneck is the reading of the data a byte at a time - is there a way to speed this up ? Ideally I'd love to read in a line of data at a time up until the EOL character is received but this capability doesn't seem to exist in the TcpClient.

Socket programming is not my strong point so maybe I'm missing something obvious. Help !

Visual Vincent
  • 18,045
  • 5
  • 28
  • 75
Philip Lee
  • 157
  • 3
  • 16
  • I'd say the very best way to communicate via TCP is to implement [**message framing**](http://stackoverflow.com/a/37352525/3740093), but if you want it to fit your current model you can also use a [**`StreamReader`**](https://msdn.microsoft.com/en-us/library/system.io.streamreader(v=vs.110).aspx). -- Keep in mind that by implementing message framing you'll at the same time be implementing a way to identify different kinds of data (ex. text, images, files, etc). – Visual Vincent Oct 03 '16 at 19:02
  • The messages are variable length so I can't see how message framing would work - they are just plain text, no fancy encoding. Unless I'm mistaken (and I probably am) isn't StreamReader tied to local files rather than sockets ? – Philip Lee Oct 03 '16 at 19:03
  • A `StreamReader` is a **stream _reader_**. It has the capability of reading text from other streams. To quote the documentation, a StreamReader _"reads characters from a byte stream in a particular encoding."_ See [**one of its constructors**](https://msdn.microsoft.com/en-us/library/yhfzs7at(v=vs.110).aspx). -- `The messages are variable length so I can't see how message framing would work - they are just plain text, no fancy encoding.` - I don't get it. What do you mean by "variable length"? Also, if they don't use any encoding how on earth are you sending them? – Visual Vincent Oct 03 '16 at 19:08
  • There is really no such thing as "plain text" (except for handwritten text). All (digital) texts have some sort of encoding. An encoding is a table that the system uses to convert bytes into characters, and vice versa. If you haven't specified any encoding, [**`System.Text.Encoding.Default`**](https://msdn.microsoft.com/sv-SE/library/system.text.encoding.default(v=vs.110).aspx) is used. – Visual Vincent Oct 03 '16 at 19:11
  • Sorry, I got my terminology wrong. The strings are ASCII - I thought that you were suggesting that they were some kind of propriety format. I'll have another look at StreamReader but none of the examples that I have found show it being used to receive data from a socket that simply connects to a remote machine. I've been looking at this for hours though so it could be time for a coffee to clear my head ! – Philip Lee Oct 03 '16 at 19:21
  • Receiving through a `StreamReader` is not very hard actually. It's pretty much like reading from a file. Initialize the StreamReader and set the base stream to your `NetworkStream`: `Dim Reader As New StreamReader()`, then just make it read lines: `Reader.ReadLine()`. – Visual Vincent Oct 03 '16 at 19:42

2 Answers2

0

Bytewise IO is usually shown in tutorials because it is so easy. It's not suitable for production.

When programming sockets use the most high level abstraction that you can. Ideally, you abandon raw sockets and request data over something like HTTP or websockets. Libraries for that are available.

If you insist on using raw sockets you can try a line based format with StreamReader/Writer. No need for ASCII, just use UTF8 which is the default.

The reader can be obtained similarly to this: new StreamReader(new NetworkStream(socket)).

usr
  • 168,620
  • 35
  • 240
  • 369
-2

Try using the ASCIIEncoding.GetString method from the System.Text namespace. It should convert an array of bytes to a string faster than your one byte at a time approach.

Dim ascii As New ASCIIEncoding()
'*** define bytes() byte array here ***
Dim decoded As String = ascii.GetString(bytes)

There is also a unicode variant of this method.

JohnH
  • 1,920
  • 4
  • 25
  • 32
  • How would that help him? If he gets one byte at a time, he's back where he started. If he gets more than one byte at a time, he might have bits of two messages in the buffer. – David Schwartz Oct 03 '16 at 19:48