3

I have a problem with reading file, specific is I want to make a small dictionary. In file which I need to read has content like this:

a   Ph  P6
a   snsr    CA
a b c   fb  Dj
a b c - book    i+  BS
A except B gate oOPa    y
a font  kQ  BU
[....]

It has about 109.000 lines, and file just has size about 2MB. In my QT app, I coded like this to read and add items to QListWidget:

QString sWord;
QFile inFile("C:\\EV\\ev.index");

inFile.open(QIODevice::ReadOnly|QIODevice::Text);
QTextStream in(&inFile);
while(!in.atEnd())
{
     sWord = in.readLine();
     myListWidget->addItem(sWord); //myListWidget is a QListWidget
}

But it reads too long! At first I think reason is my app reads line by line, so I coded its again like this:

QString data;
QStringList listWord;
QFile inFile("C:\\EV\\ev.index");


inFile.open(QIODevice::ReadOnly|QIODevice::Text);
QTextStream in(&inFile);
data.append(in.readAll());
listWord.append(data.split('\n'));
myListWidget->addItems(listWord);
inFile.close();

It works faster!(about 5 seconds since app launched), still long, I want it to read faster. What I have to do?

Kuba hasn't forgotten Monica
  • 95,931
  • 16
  • 151
  • 313
mthe.net
  • 119
  • 2
  • 10

1 Answers1

9
  1. The list widget's layout takes too long. Set your list widget's uniformItemSizes property to true. This avoids expensive layout operations. Another way would be to set the layoutMode property to QListView::Batched. This avoids having to expensively lay out all of the items at once.

  2. Don't use a QListWidget if a lower overhead QListView would do.

  3. Additions of large numbers of elements should be batched, i.e. do not insert the elements into the model one-by-one. Insert elements from each batch in an atomic operation that emits the rowsInserted or columnsInserted signal only once.

  4. You must not do any file loading in the GUI thread. This is a source of bad user experience in a lot of applications and must be discouraged with a heapful of scorn. Don't do it.

Below is a minimal example that takes all of those into account.

// https://github.com/KubaO/stackoverflown/tree/master/questions/filemodel-18548048
#include <QtWidgets>
#include <QtConcurrent>

void makeLines(QBuffer &buf, int count = 1000000) {
   buf.open(QIODevice::WriteOnly | QIODevice::Text);
   char line[16];
   for (int i = 0; i < count; ++i) {
      int n = qsnprintf(line, sizeof(line), "Item %d\n", i);
      buf.write(line, n);
   }
   buf.close();
}

struct StringListSource : QObject {
   Q_SIGNAL void signal(const QStringList &);
   void operator()(const QStringList &data) { emit signal(data); }
   Q_OBJECT
};

int main(int argc, char *argv[]) {
   QApplication app(argc, argv);
   QListView view;
   QStringListModel model;
   StringListSource signal;
   QObject::connect(&signal, &StringListSource::signal, &model, &QStringListModel::setStringList);
   QtConcurrent::run([&signal]{
      QBuffer file;
      signal({"Generating Data..."});
      makeLines(file);
      signal({"Loading Data..."});
      QStringList lines;
      if (file.open(QIODevice::ReadOnly | QIODevice::Text))
         while (!file.atEnd())
            lines.append(QString::fromLatin1(file.readLine()));
      file.close();
      signal(lines);
   });
   view.setModel(&model);
   view.setUniformItemSizes(true);
   view.show();
   return app.exec();
}
#include "main.moc"
Kuba hasn't forgotten Monica
  • 95,931
  • 16
  • 151
  • 313
  • 1
    You saved me! In accordance with your advice in item #1, I made a call to `ui->listView->setUniformItemSizes(true)`. I was adding ~1.5 million items to a `QListView` using a custom model derived from `QAbstractItemModel` and it was taking minutes. Setting `setUniformItemSizes` to true cut the time down to under 1 second. Thanks for this tip! – Matthew Kraus Sep 08 '15 at 03:35
  • Amazing answer! Thank you. I'm designing a music library, so same concept. Printing out information on 100,000 songs read from a database file. I've been waiting about 15 minutes now for the view to load. – krose Jan 22 '21 at 21:53