0

I'm trying to send non-printable ASCII characters (codes 128 - 255) through telnet to a Ruby app using Socket objects to read data in.

When I try to send \x80 through telnet, I expect Ruby to receive a string of 3 bytes: 128 13 10.

I actually receive a string of 6 bytes: 92 120 56 48 13 10.

Do I need to change something about how telnet is sending the information, or how the Ruby socket is accepting it? I've read through all the telnet jargon I can comprehend. A point in the right direction would be very much appreciated.

quetzalcoatl
  • 32,194
  • 8
  • 68
  • 107
Brad Rice
  • 1,334
  • 2
  • 17
  • 36

1 Answers1

3
92 120 56 48 13 10

is in decimal ASCII:

\  x   8  0  \r \n

So you are doing something very wrong and it's not the Telnet. The escape sequence \x80 was treated literally instead of being understood as single character of code=128.

I guess you have used '\x80' instead of "\x80". Note the different quotes. If that was a single character, you can in Ruby also use ? character to denote a character: ?\x80 so for example:

"\x80\r\n" == ?\x80 + ?\r + ?\n
=> true

while of course

'\x80\r\n' == "\x80\r\n"
=> false

--

To summarize long story from the comments:

  • originally data to be sent was entered manually through telnet terminal
  • telnet terminals often don't accept any escape codes and "just send" everything they got directly, sometimes copying&pasting a text with special characters work, sometimes terminals provide some extra UI goodies to send special characters - but this time the terminal was very basic and pasting it didn't work, and there was no UI goodies
  • instead of entering the data manually, sending a file through a pipe to the telnet terminal seemed to work much better. Some data arrived, but not good
  • piping the data to nc(netcat) instead of telnet terminal almost seemed to work, binary data arrived, but it was not perfect yet
  • after examining the input file (the one that was piped to nc) with hexdump utility, it turned out that the file contained not exactly what we thought, it seems that the editor used to create the file saved the text with wrong encoding, and it has added some extra unwanted bytes
  • finally, a utility called xxd helped to produce a good binary data from a tailored hex-text; output of xxd could be directly piped to nc (netcat)
quetzalcoatl
  • 32,194
  • 8
  • 68
  • 107
  • I'm manually entering the hex code into telnet, not through Ruby. It's being _accepted_ by Ruby and interpreted character-by-character. I'm asking how to enter the special character into telnet so that Ruby will understand what I mean. – Brad Rice Jan 08 '15 at 20:59
  • @BradRice: Usually, you can't. But it greatly depends on what terminal are yo uusing. Intelligent terminals often have some side console or window when you can issue command like "now send this special character". But if yours doesn't have anything like that, then the onlu thing is to try copying a pasting it the exact character or string from some text notepad that can handle them. Also, you may try to pipe into the terminal a text file that contains the text-with-special-characters instead of typing them by hand. Often that's more handy and repeatable. Or write a script for that or (...) – quetzalcoatl Jan 08 '15 at 21:05
  • And by `terminal` I mean the `telnet terminal`, not your Bash/console/etc window. I mean the application you are using to establish the telnet connection. Telnet is a name of a protocol, and there are many different clients (terminals) that handle it. For example, `putty` can, and it supports pasting special characters (provided you manage to copy them to clipboard). – quetzalcoatl Jan 08 '15 at 21:07
  • Also, if you just want "Ruby to understand it", then send literally the `\x80` as four characters and parse/translate it in Ruby. Simple eval will do, although it's not the safest way. But if you really want to have a binary charcode 128 sent over the wire, you must first understand your terminal. – quetzalcoatl Jan 08 '15 at 21:09
  • Gotcha. Well putting the character in a file yielded the same result I've been seeing. I'm using telnet from my Mac terminal, so I have no extra help there. Unfortunately, I won't be able to have Ruby piece together the data for me. I need to be able to send the special characters themselves. – Brad Rice Jan 08 '15 at 21:20
  • @BradRice: putting the character in a file gave the same result? no way! Sorry, I really find it hard to believe.. Did you put "\x80" in that file, or did you put one bytecode=128 inside? Did you really piped it to the terminal? I really can't believe that the terminal would translate a raw byte from file that back to \x80 escaped literal.. it's just plain wrong! Did you check the received data length on Ruby side? Maybe you got correct data but displayed it directly, and maybe Ruby stringizer emitted escape sequence to show it properly? Please try again and check the received length! – quetzalcoatl Jan 08 '15 at 21:23
  • And check the size of the file. It should be one byte (or three), not four (or six, when \r\n is included) – quetzalcoatl Jan 08 '15 at 21:25
  • Well, the file size is 3 bytes. I had to cat the file and pipe it to telnet. I couldn't find how to pipe directly into telnet... – Brad Rice Jan 08 '15 at 21:30
  • I'm also doing the entire telnet process in a bash script, of which I'm not very familiar. How should I be able to pipe the contents of a file to telnet, say running on localhost at a particular port? – Brad Rice Jan 08 '15 at 21:32
  • I just got a lot closer with `cat text.txt | nc localhost 7001`. Apparently, `nc` or netcat, is the more appropriate tool for this. Thing is, now my Ruby string bytes read `194 128 10`. Have any idea where the \xC2 would be coming from? – Brad Rice Jan 08 '15 at 21:41
  • I've now gone through 3 different editors. Cleared the file, replaced my text, added more text around these characters, doesn't matter. Every time, the `\x80` is always preceeded by `\xC2`. – Brad Rice Jan 08 '15 at 21:50
  • @BradRice: IIRC both cat and netcat truly "just read/send" everything you pass to it, so I'm not suprised, but catting it to telnet should work too. I dont know why it didnt, and I dont have Mac to test. `C2 80` seems like UTF8 code for character `\u0080`. So I'd guess that your file is `C2 80 13` or `C2 80 10` instead of `80 13 10` or `80 10` or `80 13`. It could have happened if you generated it with a texteditor that saved it under not-the-encoding-you-wanted.. Have you tried hexdumping it (https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/hexdump.1.html)? – quetzalcoatl Jan 08 '15 at 23:15
  • Btw. those the three bytes you received (194/128/10), were they an received as array-of-numbers or as a string? If the file is absolutely OK with no extra C2 at start, then maybe it got transferred properly and Ruby somehow reencoded the data upon receiving and packing into a string? It'd be strange, but I can imagine it if you have something odd in global encoding configuration. But if you are receiving an array of bytes, and C2|80 bytes arrived, then surely it's at the sender's side. – quetzalcoatl Jan 08 '15 at 23:22
  • Remember that you can always check what flies over the network with some utils like Wireshark or tcpdump. They'll instantly show you whether the data transferred is OK, and you'll quickly know if there's a problem with wrong bytes being sent, or maybe the bytes being misinterpreted at receiver, or .. maybe both. If everything is OK, then the same hexdump that you see in the file should occur in the network packets and should be visible at receiver. – quetzalcoatl Jan 08 '15 at 23:23
  • But, I have a feeling that everything works and that's just the file's contents are `C2 80 10` instead of `80 13 10` or `80 10`. If that's the case, and if you have problems with removing that C2 from file, you can always chop it off with `tail` (i.e. `tail -c 2 myfile` will read last two bytes of a file) – quetzalcoatl Jan 08 '15 at 23:28
  • Thank you for clarifying why the extra character was showing up. I was very confused. I'm looking into hexdump now. Do you have any suggestions as to what other encoding for the file I could try? I see the file is using charset UTF-8. I don't know if I need to change it, or even what to change it to though – Brad Rice Jan 09 '15 at 14:11
  • I actually don't know how which encoding would fit your needs, you'd need to show an larger example of text and the desired bytes. However, if you know what bytes should be, then you can write a hexdump file yourself (`CC BB FF 33 80 ..`) and then directly translate it into a binary file. There's a commanline utility called [XXD](http://stackoverflow.com/questions/7826526/transform-a-hex-info-to-binary-using-linux-command), [it should already on your Mac machine](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/xxd.1.html). You can pipe from it, too. – quetzalcoatl Jan 09 '15 at 14:57
  • So just `xxd myHexDumpFile | nc ...` should be OK. – quetzalcoatl Jan 09 '15 at 14:58
  • Thank you very much for your help. I found the relationship between `hexdump` and `xxd` at last, and essentially confirmed your suspicion: file, OS, or some other library's interpretation of the encoding is munging the data before my Ruby code gets it. If you'd like to update your answer with brief mentions of these tools, I'd be happy to accept your answer. Again, thanks for your explanations – Brad Rice Jan 09 '15 at 16:23
  • @BradRice: I've updated the answer with a summary of our discussion and your findings, I hope I didn't mix up anything :) – quetzalcoatl Jan 09 '15 at 21:59