On a very slow query, how do I indicate percentage of progress

Question

I'm using ZEOS components to connect to an (ancient) MDB database.
I'm doing a query that reads in lots of data to bridge into a different database.

Is there a way to indicate progress as a percentage?

procedure TForm13.ActionReadInMemoryExecute(Sender: TObject);
var
  QueryLine: string;
  FullQuery: string;
  Tablename: string;
  i: integer;
begin
  i:= 0;
  TableMeter.DisableControls;
  try
    TableMeter.First;
    FullQuery:= '';
    while not TableMeter.eof do begin
      Tablename:= TableMeter.FieldByName('tabelnaam').AsString;
      QueryLine:= ReplaceStr(ImportQuerySjabloon, cTabelname, Tablename);
      FullQuery:= FullQuery + QueryLine;
      if (TableMeter.RecNo < (TableMeter.RecordCount -1)) then begin
        FullQuery:= FullQuery + ' UNION ALL ';
      end;
      TableMeter.Next;
    end; {while}
    QueryImportMeterreadings.Close;
    QueryImportMeterreadings.SQL.Text:= FullQuery;
    QueryImportMeterreadings.Open;  <<-- takes a long time
  finally
    TableMeter.EnableControls;
  end;
end;

Is there a way to indicate progress of the query, or can I only do this if I split up the individual queries and eliminate the UNION's.
It takes about 1 minute to run, involving 8 unions.

I don't see any event that I can use for this purpose:

Or should I fake an OnCalcField on a field in the Query to do this (not sure if that will even work in principle).
Or attach a sequence? nope, gives unsupported operation on a Access DB

Answer 1

I say split up the individual queries and eliminate the union, make a timer around each query, depending on the avg time taken * number of queries remaining you should give an estimate / update a text field to say x out of y queries completed (time remaining: -time-)

Answer 2

I would split the huge query into individual queries; in code, you iterate over each query's result set and manually insert the values into a clientdataset (cds). The cds can be connected to a dbgrid. Then you can show when each query completes - you could also show progress after each tuple is handled, but you won't know how many tuples in total there are, unless you perform a separate query which returns a count of tuples. The problem with using such an unconnected cds is that you have to define the fields in code. Here is an example of something similar which I wrote last night - the queries all update one field in the cds.

const
 field1 = 'id';
 field2 = 'customer name';
 field3 = 'total debt';

procedure TTotalCustDebt.FormCreate(Sender: TObject);
var
 strings: tstrings;

begin
 with qTotalDebt do   // this is the clientdataset
  begin
   fielddefs.add (field1, ftInteger, 0, false);
   fielddefs.add (field2, ftString, 32, false);
   fielddefs.add (field3, ftInteger, 0, false);
   createdataset;
   fieldbyname (field1).visible:= false;
   open;
   addindex ('idx0', field2, [], '', '', 0);
   addindex ('idx1', field2, [ixDescending], '', '', 0);
   addindex ('idx2', field3, [], '', '', 0);
   addindex ('idx3', field3, [ixDescending], '', '', 0);
   strings:= tstringlist.create;
   getindexnames (strings);
   strings.free;
  end;
end;

procedure TTotalCustDebt.PopulateCDS;
begin
 dsTotalDebt.dataset:= nil;
 with qTotalDebt do
  begin
   emptydataset;
   indexfieldnames:= field1;  // initially sort by customer.id
  end;

 with qDBills do
  begin
   params[0].asdate:= dt;
   open;
   while not eof do
    begin
     qTotalDebt.append;
     qTotalDebt.fieldbyname (field1).asinteger:= qDBillsID.asinteger;
     qTotalDebt.fieldbyname (field2).asstring:= qDBillsName.asstring;
     qTotalDebt.fieldbyname (field3).asinteger:= qDBillsTot.asinteger;
     qTotalDebt.post;
     next
    end;
   close
  end;

  // show progress indicator

  with qDReceipts do
   begin
    params[0].asdate:= dt;
    open;
    while not eof do
     begin
      if qTotalDebt.findkey ([qDReceiptsID.asinteger]) then
       begin  // customer already exists
        qTotalDebt.edit;
        qTotalDebt.fieldbyname (field3).asinteger:= - qDReceiptsTot.asinteger
                                  + qTotalDebt.fieldbyname (field3).asinteger;
      end
     else
      begin  // add new record
       qTotalDebt.append;
       qTotalDebt.fieldbyname (field1).asinteger:= qDReceiptsID.asinteger;
       qTotalDebt.fieldbyname (field2).asstring:= qDReceiptsName.asstring;
       qTotalDebt.fieldbyname (field3).asinteger:= - qDReceiptsTot.asinteger;
      end;
     qTotalDebt.post;
     next
    end;
   close
  end;

 // show progress indicator
 // more queries
 // at end, attach the clientdataset to the TDataSource
 dsTotalDebt.dataset:= qTotalDebt;
end;