How to compare two parsed files and append the difference to the first?

Question

Suppose I have two .dat files; one on my computer and the other one on the other side of the earth - with data constantly being serialized into them through a QDataStream.

The data is parsed the same way – first some sort of ID and then an object associated with that particular ID.

QFile file("data.dat");
QDataStream stream(&file);

file.open("QIODevice::ReadWrite");

stream << *id*;          // ID goes in.
stream << *data_object*; // Object with interesting data is serialized into the file.

file.close();

After a while – the first one might look something like this (illustratory, not syntactically correct):

//-------------------------------------DATA.DAT------------------------------------//

ID:873482025
 dataObject

ID:129845379
 dataObject

ID:836482455
 dataObject

ID:224964811
 dataObject

ID:625444876
 dataObject

ID:215548669
 dataObject

//-------------------------------------DATA.DAT------------------------------------//

But the second one hasn't caught up quite yet.

//-------------------------------------DATA.DAT------------------------------------//

ID:873482025
 dataObject

ID:129845379
 dataObject

ID:836482455
 dataObject

//-------------------------------------DATA.DAT------------------------------------//

Is it possible to take both files – detect the differences between them and then "fuse" in the ones that are missing from the second but are present in the first?

Obviously this could be achieved by writing a function extracts the innards of the files, categorizes the contents individually, compares them and so forth – but is there a way to do this by just handling the files themselves, without having to parse the contents individually?

Answer 1

1. Read both files to extract Id sets.

2. Read one of the files while appending the objects with missing Ids to the other file.

You can leverage QSet to do set arithmetic. Also, each object would need not only the streaming operators, but also a skipObject static method. I'm also ignoring how you discriminate object types.

typedef qint32_t Id;

bool isOk(const QDataStream & str) { return str.status() == QDataStream::Ok; }

class Object {
  ...
public:
  static void skipObject(QDataStream & str) {
    qint8 format;
    str >> format;
    if (format == 0)
      str.skipRawData(32); // e.g. format 0 of this object is 32 bytes long
    ...
  }
};

QPair<QSet<Id>, bool> getIds(const QString & path) {
  QSet<Id> ids;
  QFile file(path);
  if (!file.open(QIODevice::ReadOnly)) return ids;
  QDataStream stream(&file);
  while (!stream.atEnd()) {
    stream >> id;
    Object::skipObject(stream);
    if (ids.contains(id))
      qWarning() << "duplicate id" << id << "in" << path;
    ids.insert(id);
  }
  return qMakePair(ids, isOk(stream));
}

bool copyIds(const QString & src, const QString & dst, const QSet<Id> & ids) {
  QFile fSrc(src), fDst(dst);
  if (! fSrc.open(QIODevice::ReadOnly)) return false;
  if (! fDst.open(QIODevice::WriteOnly | QIODevice::Append)) return false;
  QDataStream sSrc(&fSrc), sDst(&fDst);
  while (!sSrc.atEnd()) {
    Id id;
    sSrc >> id;
    if (ids.contains(id)) {
       Object object;
       sSrc >> object;
       sDst << id << object;
    } else
       Object::skipObject(sSrc);     
  }
  return isOk(sSrc) && isOk(sDst);
}

bool copyIds(const QString & src, const QString & dst) {
  auto idsSrc = getIds(src);
  auto idsDst = getIds(dst);
  if (!idsSrc.second || !idsDst.second) return false;
  auto ids = idsSrc.first - idsDst.first; 
  return copyIds(src, dst, ids);
}