1

I am new to Qt SQL, we need to insert data row by row into database.

First I need to fetch data from the application into a row, a row consists of 4 types of data: int, float, string & null. Then I insert this row into db.

To compare different methods speeds, we tried 2 solutions:

  1. Put all data of a row into std::vector<QVariant>, then use bindValue method to insert to db:
    std::vector<QVariant> getOneRow(int row)
    {
        std::vector<QVariant> outPut;

        // Get int 1, float 1.23, string "I am string", "NULL" from the application
        outPut.push_back(1);
        outPut.push_back(1.23);
        outPut.push_back("I am string");
        outPut.push_back("NULL");

        return outPut;   
    }

    void insertDataBase(std::vector<QVariant> input, QString query)
    {
         int rowLength = input.size();
    
         myQtSql.prepare(query);
         for(int i = 0; i < rowLength ; ++i)
         {
            myQtSql.bindValue(i,input[i]);
         }
         myQtSql.exec();
    }
  1. Convert every element of a row to QString, then use "INSERT INTO" statement into db:
    QString getOneRow(int row)
    {
        QString  outPut;

        // Get int 1, float 1.23, string "I am string", "NULL" from the application
        outPut += QString::number(1);
        outPut += ",";
        outPut += QString::number(1.23, 'f', 2);
        outPut += ",";
        outPut += "I am string";
        outPut += ",";
        outPut += "NULL";

        return outPut;   
    }

    void insertDataBase(QString input)
    {
        QString query="INSERT INTO MyTable VALUES("+input+");";

        myQtSql.exec(query))
    }

The whole workflow is like:

    while(row < maxRow)   //maxRow, like 1 million
    {
        getOneRow(row);
        insertDataBase();
    }

The advantage of method 1 is getOneRow execution time is 1/10 of that in method 2, since I don't need to convert every numeric value into QString (I found that QString::number is very time consuming, which takes almost 90% of the execution time.) But bindValue method is slower than "INSERT INTO"

The advantage of method 2 is "INSERT INTO" is much faster than bindValue method, overall method 2 execution time is half of method 1. I tested under Desktop Qt 6.1.3 MSVC 2019 64-bit, release mode

May I ask: 1. Are there other better ways? 2. Since QString::number is very time consuming, are there better value string conversion methods? I found Dragonbox Fastest C++ way to convert float to string

Thank you very much

  • 1
    Your second version is also very vulnerable to sql injection. use bindValue() – chehrlic Jun 20 '22 at 17:27
  • Did you try `execBatch`? https://doc.qt.io/qt-6/qsqlquery.html#execBatch. Also I suggest to pass and return `std::vector` by constant reference, to avoid data copying – Alexey Jun 20 '22 at 20:34
  • `outPut.push_back("NULL");` this would not be `NULL`, it would be a string "NULL" – Alexey Jun 20 '22 at 20:39
  • @Alexey Looks like you both prefer using bindValue than "INSERT INTO", I tried data copy, it indeed saved a little time – SuperNoooob Jun 22 '22 at 03:22
  • The main benefit you can get from using the `execBatch` method is that only one `INSERT` query would be executed to insert all the rows into the datatable and that should eliminate the overhead of multiple separate queries creation and execution – Alexey Jun 22 '22 at 07:31

0 Answers0