What's the meaning of some sort of custom binary file?

Question

In Head First C book, there is a "there are no dumb questions" section.

Are bitfields really just used to save space?

No, They're important if you need to read low-level binary information

Such as?

If you're reading or writing some sort of custom binary file.

What is that mean?

@ModyElSayed Please edit the question rather than commenting updates. — amirali, May 28 '20 at 12:53
A binary file is a file that contains it information in binary format instead of a plain text format: https://en.wikipedia.org/wiki/Binary_file — UnholySheep, May 28 '20 at 12:54
@UnholySheep So why do I need to use bitfield if it is just a binary format? — Mody El Sayed, May 28 '20 at 12:56
You may not need to, it all depends on what data you stored there and in what form your program uses it — UnholySheep, May 28 '20 at 12:57
Instead of paraphrasing the author, can you post his exact wording? — Fiddling Bits, May 28 '20 at 12:58
There are no dumb questions, only dumb answers. Indeed. I don't think this author has ever worked with low-level C programming. You should especially _avoid_ bit-fields when writing to binary files, because files need to be 100% no matter which program that is reading it. — Lundin, May 28 '20 at 13:08
@Lundin It very likely is, given the snippet posted by OP, and the book description on amazon: *"This book helps you learn the C language with a unique method that goes beyond syntax and how-to manuals and helps you understand how to be a great programmer."* Unfortunately it still has a lot of positive reviews, but those of course seem to be primarily from students which are using the book to learn C. — Felix G, May 28 '20 at 13:32
@Lundin how should I know that I’m intermediate or expert programmer and walking in the right direction to become a better developer? — Mody El Sayed, May 28 '20 at 13:56
@ModyElSayed You won't. I started out teaching myself programming, thought I was pretty good... until I took some classes in school and realized I knew nothing. After those classes I thought I was pretty good, then I went to technical university and realized I knew nothing. After getting my degree, I thought I was pretty good, then I got my first job and realized I knew nothing. Somewhere after a university degree and 10 years of work as full-time professional programmer, you might dare to call yourself an expert. — Lundin, May 28 '20 at 14:06
@ModyElSayed: I've been programming professionally for 30 years, and I'm still reluctant to call myself an "expert". I know a lot about C, but I still get things wrong. The only way you know you're growing as a developer is that you make new mistakes instead of repeating the same ones over and over. — John Bode, May 28 '20 at 14:26

score 1 · Answer 1 · answered May 28 '20 at 13:00

You'll have to ask the author of that book what they were thinking. Generally, bit-fields do not save space in any way, but they may very likely lead to increased memory use. If you have a bit-field such as

struct foo { unsigned int a: 31; bool b: 1; }

then the compiler is free to allocate 32 bits for the int part and 32 more bits for the bool part. And that is exactly what happens on gcc. The bit-field resulted in twice the memory use.

This has to do with various details regarding internal chunks of memory inside the bit-field mentioned in the C standard as "storage units". It's an abstract and poorly-specified concept. Where such a storage unit ends, what it can contain, how it reacts to different types isn't specified. It is however specified that the compiler is free to throw in padding bytes in the middle of the bit-field.

Furthermore, bit-fields are not useful for low-level binary information, because they are incredibly poorly defined by the C standard, to the point where they are completely non-portable and nearly useless. Details here: https://stackoverflow.com/a/6044223/584518

Always use unsigned integer types, bitwise operators and bit masks instead of bit-fields.

score 0 · Answer 2 · answered May 28 '20 at 13:16

I'm not sure if this is what the author was referring to, but here's an imaginary example where one might be tempted to use a bitfield:

Let's say we define an image file format. To display the image, the software needs to know the dimensions of that image, so we decide that our file format should start with a header defining the dimensions of the image. Let's also assume that - for some reason - we don't ever want to store images larger than, say, 16x16 pixels. Therefore we then decide that we can save space by not using full bytes to specify the dimensions in x and y, but instead we just use 4 bits each.

In a case like that, someone might want to use a bitfield, because it allows easy access to parts of a byte (or integers in general). However, since bitfields are not portable, you would still be better off doing the bit-manipulation by hand.

John Bode · Answer 3 · 2020-05-28T15:31:05.300

Here's an example of how most people think bitfields should work - picture your typical IEEE 754 single-precision float:

+-+--------+-----------------------+
| |        |                       |
+-+--------+-----------------------+
 ^ ^        ^
 | |        |
 | |        +----------------------- significand
 | +-------------------------------- exponent
 +---------------------------------- sign bit

You could create a struct type with bitfields to represent this format:

struct ieee_754_s {
  unsigned    sign_bit :  1;
  unsigned    exponent :  8;
  unsigned significand : 23;
};

This could be a useful abstraction for learning and understanding how floating point formats work, and what operations need to be performed on each field in order to do floating point arithmetic, etc.

However, there also tends to be a naive expectation that this type would map directly onto a float and you could do all kinds of clever bit manipulations on float objects.

Except it doesn't. At least, it's not guaranteed to. On a little-endian system like x86 the fields don't map correctly. To make it work on my Macbook, I had to define the type as

struct ieee_754_le {
  unsigned sig_3 : 8; // lowest-order bits of the significand
  unsigned sig_2 : 8;
  unsigned sig_1 : 7; // highest-order bits of the significand
  unsigned sign  : 1;
  unsigned exp   : 8;
};

and then I could get it to map correctly. But again, it's still not guaranteed. Bit fields do not have to be laid out contiguously, and the order in which they're allocated within a storage unit is implementation-defined. The layout above just happens to map cleanly onto a sequence of storage units, and the compiler was nice enough to order the bit fields as I expected within each storage unit. It could have flipped the ordering of the sign and sig_1 fields.

Bit fields are a way to deal with objects whose sizes are not an integral number of storage units without having to do a bunch of bit masking. They are not a good way to represent and interact with low-level binary formats.

What's the meaning of some sort of custom binary file?

3 Answers3