Endian in networking and multi platform - clarification?

Question

I have this code in JAVA :

socket = new Socket("127.0.0.1", 10);
OutputStream os = socket.getOutputStream();
os = socket.getOutputStream();


int data=50000;
os.w.write(data.toByteArray());
os.write(ByteBuffer.allocate(4).putInt(data).array());

And in the C# :

byte[] ba = readint(networkStream);
networkStream.Flush();
if (BitConverter.IsLittleEndian) Array.Reverse(ba);
int i = BitConverter.ToInt32(ba, 0); //50000

It's all working fine but :

I saw this image :

enter image description here

But the part that interested me was the "Network Order - under Big Endian"

I read in wiki that

Many IETF RFCs use the term network order, meaning the order of transmission for bits and bytes over the wire in network protocols. Among others, the historic RFC 1700 (also known as Internet standard STD 2) has defined its network order to be big endian, though not all protocols do.

Question

if Java uses big endian and the tcp also uses "Networked order" - big Endian -

So Why - in my C# - I had to check if it's big endian ?

I mean I could do :

Array.Reverse(ba);

Without checking : if (BitConverter.IsLittleEndian) Array.Reverse(ba);

Am I right ?

If so , What about the case that some unknown source sends me data and I don't know if he sent it big or small endian ? He would have to send me a first byte to indicate right ? but the first byte is also subject to endianness.....Where is my misunderstanding ?

score 3 · Accepted Answer · answered Nov 07 '14 at 08:41

3

You are assuming that this is a 32-bit in big endian, you could also assume that BitConverter has a default of little endian (or you could change it to big endian and not reverse it in the first place) BTW ByteBuffer supports little endian too.

You could also send little endian

os.write(ByteBuffer.allocate(4)
                   .order(ByteOrder.LITTLE_ENDIAN)
                   .putInt(data)
                   .array());

What about the case the some unknown source sends me data and I don't know if he sent it big or small endian ?

Then you don't know how to decode it, for sure. You can guess if you have enough data.

He would have to send me a first byte to indicate right ?

She could but you would have to know that the first byte tells you the order and how to interperate that first byte. Simpler to assume a given byte order.

but the first byte is also subject to endianness

So imagine a byte is written in little or big endian. How would it be any different? ;)

answered Nov 07 '14 at 08:41

Peter Lawrey

525,659
79
751
1,130

Can you please read my new 3 added lines at the end ? ( added them after submitting - by mistake) – Royi Namir Nov 07 '14 at 08:43
1

after your code edit - how would I know what you send me ? (big or small) ? you will probably send me a first byte which indicates that. but that byte also could be problematic as itself....(big /small) – Royi Namir Nov 07 '14 at 08:44
@RoyiNamir you can't know if data sent is big endian or small endian with its data value. You have to infer it. And the unknown source which sends that, has to set some explicit indicator for order apart from data. – nullptr Nov 07 '14 at 08:47
@RoyiNamir You are already assuming the first value is a 4-byte int. What if it's text or a long or a double? ;) At some point you have to make an assumption. – Peter Lawrey Nov 07 '14 at 08:47
Thank you - but does endianness changes across platforms ?(x86,x64,motorola,itanium etc) I mean could it be that differenxt x86 uses different endianess or is it by 64 vs 32 or vendors....? – Royi Namir Nov 07 '14 at 08:52
@RoyiNamir if you know data value, and byte array representing data value, for example you know that unknown source will always send first 4 bytes with data value of integer 1, then you can infer order from its bytes and infer the endianness used by unknown source. – nullptr Nov 07 '14 at 08:55
@Meraman , are you telling me that if he sends me 4 mb file as byte array - there could be a situation where i have to reverse all file ? – Royi Namir Nov 07 '14 at 09:08
1

@RoyiNamir it depends, if byte order doesn't match then you have reverse all bytes representing data value like integers, floats, characters with more than one byte. You dont have to reverse whole 4 MB, because I doubt if you represent some value with 4 MB :O. But if its binary file, then never have to worry about byte order. – nullptr Nov 07 '14 at 09:14
@RoyiNamir and if its text file, with characters represented in more than one byte, then you must be having byte order marker (BOM). And then still you have not to worry, because the application works with that file, will know BOM, and process it correctly. – nullptr Nov 07 '14 at 09:17
@Meraman sorry , but why wont i have to worry if you send me 4321 bytes for file when you should have sent me 1234 ? – Royi Namir Nov 07 '14 at 09:19
file of total 4321 bytes OR '4321' integer value in file? Actually what are trying to send? Not clear whether you want to send some binary files like JPEG, MPEG, TS, or some custom data packets with integer, floats or characters embedded in it..? – nullptr Nov 07 '14 at 09:20
@RoyiNamir When you read a binary file, you need to know what the bytes in that file mean. e.g. you need to know if its a 4-byte int, or float or text or an 8-byte double etc and you need to know what endian was used. There is only two types of endian in binary big and little. (note: for dates in text the US uses an order which is neither MM-DD-YYYY being the only exception I can think of) – Peter Lawrey Nov 07 '14 at 09:46
I think I'm understood - a java client want to send me jpg file and i should save it to disk and open it in future . So he starts sending me bytes in chunks . Each chunk is saved to disk . Should i be afraid of endians ? Im starting to think that it only has to do with memory representation and not disk representation .....? Thank you for all help ! Btw – Royi Namir Nov 07 '14 at 09:54
Endianess is the same on different cpus whether in memory or on disk. You always need to know what the bytes mean unless you are only copying the bytes without interpreting them. E.g. If a file is 4 MB and all you want to do is copy it you don't need to know what any of those bytes mean. – Peter Lawrey Nov 07 '14 at 10:06

Endian in networking and multi platform - clarification?

1 Answers1