1

I'm beginning with Qt and am stuck with a problem supposedly for quite a long time now. I'm sure it's just something I don't see in C++. Anyway, please look at the following simple code and point me what am I doing wrong:

typedef struct FILEHEADER {
    char udfSignature[8];
    char fileName[64];
    long fileVersion;
    UNIXTIME fileCreation;
    UNIXTIME lastRebuild;
    FILEPOINTER descriptor;
} fileheader;

QFile f("nanga.dat");
    if(f.open(QIODevice::ReadWrite));

f.write(fileheader);

Qt 5.2.0 trows me the following error message:

C:\sw\udb\udb\main.h:113: error: no matching function for call to
'QFile::write(FILEHEADER&)'
         file.write(header);
                          ^

Any suggestion on how I can write this struct to a QFile?

Thanks

Binoy Babu
  • 16,699
  • 17
  • 91
  • 134
Gustavo Pinsard
  • 267
  • 3
  • 10
  • 1
    You're mixing C with C++. You have to overload `operator<<` and use `QDataStream` to write to file. Read about data serialization. – prajmus Jan 18 '14 at 19:38
  • 1
    In which format do you want to store the data? (QDataStream would be one option, but not necessarily the best, depending on your requirements) – Frank Osterfeld Jan 18 '14 at 19:56
  • 1
    Following http://qt-project.org/doc/qt-5/qfile-members.html your struct should be converted to a `QByteArray` or `char *` – Tob Jan 18 '14 at 20:50
  • 1
    I want to store data as binary, with some fixed length fields composing the header structure. I want to create my own teaching material, in the hope future programmers don't get stuck with whatever version of whichever SQL database there are by the time the current programmers decide to, well, retire. For good, know what I mean? – Gustavo Pinsard Jan 18 '14 at 21:26
  • Note that storing pointers (that `FILEPOINTER` is pointer, I assume) like this is totally pointless. You have to be careful to ignore the stored value, when you read it back. – hyde Jan 19 '14 at 08:04
  • Since this is for teaching/learning purposes: you shoud write two function: `bool serializeFileHeader(QIODevice &out, const fileheader &headerOut)` and `bool deserializeFileHeader(QIODevice &in, fileheader &headerIn);` First versions can just dump and read back the raw memory contents, shown in an aswer already (and note my comment about pointer above), which can then be refined. – hyde Jan 19 '14 at 08:16
  • @hyde, forget about the FILEPOINTER. It is not definitive and was never meant to store actual memory address. I should've named it FILEOFFSET or something like that. Thanks for your sharp eye, though. It clearly demonstrates you're paying attention to the whole. – Gustavo Pinsard Jan 20 '14 at 02:04

2 Answers2

5

Given that everyone else has picked up on the obvious errors, let's note when (and only when) it's OK to do what you're trying to do.

The in-memory format of the header structure is platform- and compiler-dependent. So, it's perfectly fine to store the header the way you do it only if it's temporary data that lasts no longer than the application's runtime. If the header is in a temporary file that you delete before exiting, you're OK.

If, on the other hand, you try to "teach" this way of storing binary data permanently - to last after the application exits, you've shot your students in the foot. With a bazooka, no less. You're not guaranteed at all that the next version of your compiler will generate code that has the same in-memory arrangement for the structure fields. Or that some other compiler will do so.

Pedagogical Note

There are several pedagogical aspects worth addressing: the complexity of writing a portable and maintainable file format, and idiomatic use of the programming language C++. A good approach will exploit an inherent synergy between the two.

In most code I see on public forums, fixed-length string buffers are a gateway drug to buffer overflows and insecure code. Pedagogically, it's a disastrous habit to teach to anyone. Fixed size buffers automatically create extra problems:

  1. File bloat due to storage of the padding.

  2. Impossibility of storing arbitrarily long strings, and thus forced loss of data.

  3. Having to specify and test for "correct" behavior when strings that are too long have to be shoehorned into short buffers. This also invites off-by-one errors.

Since you're teaching in C++, it'd be a good idea to write code like other skilled people write in C++. Just because you can write it like if it were C, and braindead C at that, doesn't mean it's a good idea at all. C++, like any other language, has idioms - ways of doing things that result in both decent code and decent understanding and maintainability by others.

To that end, one should use QDataStream. It implements its own, portable-within-Qt serialization format. If you need to read this format from code that doesn't use Qt, refer to the documentation - the binary format is documented and stable. For simple data types, it's done just as decently written C code would do it, except that by default the file is always big-endian no matter what the platform's endianness is.

Homebrew file formats done by "simply" writing C structs to disk always suffer because you have no control by default over how the data is arranged in the memory. Since you merely copy memory image of the struct to the file, you lose control over how the data is represented in the file. The compiler's vendor is in control, not you.

QDataStream and QIODevice (implemented in QFile) necessarily abstract out some of the complexity, because they aim to be usable without the user having to write lots of boilerplate code to properly address the portability aspects. The following are often ignored aspects of writing binary data to files:

  1. Endianness of the numerical data.
  2. Sizes of data types.
  3. Padding between "contiguously" stored data.
  4. Future extensibility and versioning of the file format.
  5. Buffer overflows and inevitable data loss when present with fixed-size buffers.

Addressing it properly requires some forethought. This is a perfect opportunity, though, to use the debugger to trace the flow of code through QDataStream to see what really happens as the bytes are pushed to the file buffer. It is also an opportunity to examine the QDataStream API's portability aspects. Most of that code exists for a good reason, and the students could be expected to understand why it was done that way.

Ultimately, the students can reimplement some minimal subset of QDataStream (handling just a few types portably), and the files written using both Qt's and students' implementation can be compared to assess how well they succeeded in the task. Similarly, QFile can be reimplemented by deriving from QIODevice and using the C file API.

Here is how it really should be done in Qt.

// Header File

struct FileHeader { // ALL CAPS are idiomatically reserved for macros
  // The signature is an implementation detail and has no place here at all.
  QString fileName;
  // The file version is of a very dubious use here. It should only
  // be necessary in the process of (de)serialization, so ideally it should
  // be relegated to that code and hidden from here.
  quint32 fileVersion;
  QDataTime fileCreationTime;
  QDateTime lastRebiuildTime;
  // The descriptor is presumably another structure, it can be
  // serialized separately. There's no need to store a file offset for it
  // here.
};
QDataStream & operator<<(QDataStream& str, const FileHeader & hdr) {
QDataStream & operator>>(QDataStream& str, FileHeader & hdr) {

// Implementation File

static const quint32 kFileHeaderSignature = 0xC5362A99;
// Can be anything, but I set it to a product of two randomly chosen prime
// numbers that is greater or equal to 2^31. If you have multiple file
// types, that's a reasonable way of going about it.

QDataStream & operator<<(QDataStream& str, const FileHeader & hdr) {
  str << kFileHeaderSignature
      << hdr.fileName << hdr.fileVersion
      << hdr.fileCreationTime << hdr.lastRebuildTime;
  return str;
}

QDataStream & operator>>(QDataStream& str, FileHeader & hdr) {
  quint32 signature;
  str >> signature;
  if (signature != kFileHeaderSignature) {
    str.setStatus(QDataStream::ReadCorruptData);
    return;
  }
  str >> hdr.fileName >> hdr.fileVersion
      >> hdr.fileCreationTime >> hdr.lastRebuildTime;
  return str;
}

// Point of use

bool read() {
  QFile file("myfile");
  if (! file.open(QIODevice::ReadOnly) return false;
  QDataStream stream(&file);
  // !!
  // !!
  // !!
  // Stream version is a vitally important part of your file's binary format,
  // you must choose it once and keep it set that way. You can also store it
  // in the header, if you wish to go to a later version in the future, with the
  // understanding that older versions of your software won't read it anymore.
  // !!
  // !!
  // !!
  stream.setVersion(QDataStream::Qt_5_1);
  FileHeader header;
  stream >> header;
  ...
  if (stream.status != QDataStream::Ok) return false;
  // Here we can work with the data
  ...
  return true;
}
Community
  • 1
  • 1
Kuba hasn't forgotten Monica
  • 95,931
  • 16
  • 151
  • 313
  • 1
    I think it's always worth explicitly pointing out when mentioning `QDataStream`, that has it's own binary serialization format (actually several, with version as property, as your code shows). It's not for storing raw binary data, it's primarily for use with Qt applications, and if using data stored with it with non-Qt apps, extra care should be taken especially on Qt side, to store only data which the non-Qt app understands. – hyde Jan 19 '14 at 11:40
  • @hyde: QDataStream's binary format is "documented" in the form of Qt sources. It is perfectly fine to read it from code that doesn't use Qt, you just need to write that code - as would be the case when reading any other home-brew file format. – Kuba hasn't forgotten Monica Jan 19 '14 at 11:43
  • @KubaOber I think hyde meant QDataStream has its own way of arranging data for easy of use sake. Not what I'm looking for, because I want to teach students the inner mechanics of file control and processing. – Gustavo Pinsard Jan 20 '14 at 02:15
  • 1
    @GustavoPinsard: You're not teaching anyone the inner mechanics of anything when you write an opaque struct to disk. You have no way of knowing, short of examining the assembly code or the on-disk file, of how the data actually looks in the file. If you merely gave me your original code that writes the struct to the file, I'd have no way of telling you how the bytes in the file are actually arranged. On the other hand, if you gave me the C++ code that uses `QDataStream`, I could tell you exactly how the file is formatted. I'm worried that you teach this without such understanding. – Kuba hasn't forgotten Monica Jan 20 '14 at 12:40
  • @GustavoPinsard Anything which stores binary data to file needs to do what `QDataStream` does, one way or another, it's definitiely not "for ease of use sake". I just meant, that `QDataStream` has it's own format defined by Qt implementation, and for non-Qt use *another* implementation of *same* format must be created, and there's always extra compatibility concern when doing that (compared to just one implementation which must be compatible with itself). – hyde Jan 22 '14 at 07:19
  • @hyde `QDataStream` is fully backward compatible, going back to Qt 1. As long as you stick to one fixed version of stream format, like you should, you can depend on it staying compatible in the future. That's guaranteed by policy on the Qt development side. – Kuba hasn't forgotten Monica Jan 22 '14 at 13:55
  • @KubaOber Yes, of course. What I mean is for example this: if Qt app using QDataStream is changed, and starts to save some type it didn't save before, then any custom implementation used to read the data is likely in trouble. If the project does not use good practices such as testing, then this might even go unnoticed for a while, causing confounding mayhem. – hyde Jan 22 '14 at 14:21
  • @hyde This is dealt with in file format design, the implementation must merely follow the spec. There are well-established idioms in file format design that deal with this. Popular file formats such as jpeg and mp3 are fully backwards compatible in spite of presence of numerous later additions to the data. – Kuba hasn't forgotten Monica Jan 22 '14 at 14:29
  • @hyde Two ways of dealing with this are: 1. Use known-length, opaque packets that are ignored by older implementations (jpeg/mp3 approach). 2. Use a self-describing file format, such as one of the encodings of [ASN.1](http://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One). – Kuba hasn't forgotten Monica Jan 22 '14 at 14:32
  • (This is a revised older comment): For numerical types, `QDataStream`'s behavior is exactly what you would expect. That's not the case when you dump a C struct to disk. For most other types, such as `QString`, the behavior is easy to explain in one sentence: [null-terminated UTF-16 code units stored as `quint16`, or 0xFFFF if the string is null](http://code.woboq.org/qt5/qtbase/src/corelib/tools/qstring.cpp.html#_ZlsR11QDataStreamRK7QString). – Kuba hasn't forgotten Monica Apr 23 '14 at 04:50
1

QFile has write method which accepts arbitrary array of bytes. You can try something like this:

fileheader fh = { ...... };
QFile f("nanga.dat");
if(f.open(QIODevice::ReadWrite))
    f.write(reinterpret_cast<char*>(&fh), sizeof(fh));

But remember that in general, it's not a good idea to store any data this way.

tumdum
  • 1,981
  • 14
  • 19
  • 5
    "it's not a good idea to store any data this way." Indeed, it's probably the most horrible and fragile way. It starts with `long`'s size being system-dependent, and then there's struct padding. FILEPOINTER (whatever that is) sounds like a pointer, so saving it to a file it wouldn't make any sense. Using proper serialization instead of such an appalling hack will avoid many headaches, crashes and portability issues. – Frank Osterfeld Jan 18 '14 at 20:53
  • 2
    I am building the foundations to teach students how to create their own data file format, and accompanying support functions. The struct is for a file header general signature, so I can teach them how to make a disk file "discoverable" by services. – Gustavo Pinsard Jan 18 '14 at 21:23
  • @GustavoPinsard: If I were an employer and an intern argued with me that "they were taught to do it that way" I'd tell them: "Forget everything your teacher taught you for it's mostly wrong and leads to fragile, non-portable, and likely unsafe code". Gustavo, fixed-size string buffers are really a NO-NO. Don't teach anyone to write such code. You're contributing to the lamentable insecurity of most of the world's software. Buffer overflow bugs really should be the thing of the past when you use C++. Yet you're teaching precisely such anti-patterns. – Kuba hasn't forgotten Monica Jan 20 '14 at 14:58
  • @KubaOber, I appreciate your concern and mostly agree. However, there are certain occasions where one must use fixed length structures. Maybe I should've been more detailed in explaining my needs, but I thought unncessary because my question was very specific: how to dump a fixed lenght structure using QFile's write method. I understand now that QFile wasn't tailored for such a use, but that doesn't imply I'm not allowed to use fixed length structs in watever I'm creating. Thanks anyway for your observations. They helped me elaborate better my thoughts about this. – Gustavo Pinsard Jan 20 '14 at 15:12
  • @GustavoPinsard Fixed length buffers can be fine, especially when it is a file format (in any modern C++ code you should still use a container class, not raw array). What is fragile (and therefore dangerous) is using `struct` to define file contents directly. You really want a serialization step in between, which writes data to file with explicit format. – hyde Jan 22 '14 at 07:03
  • @hyde I understand. What I wonder is if serializing the data won't change its elements' sizes, which is exactly what I must avoid. – Gustavo Pinsard Jan 23 '14 at 14:29