0

I am developing a Delphi 10 Seattle based application that performs multiple tasks using FireDAC's toolset to performe backup, restore and maintenance routines on Firebird databases.

The components included in this toolset (TFDIBBackup, TFDIBRestore and TFDIBValidade) all have the Verbose option, that lets you intercept the output of it and be able to output it to somewhere in order to read it.

I'm trying to output that to a memo, but I'm having some problems with it.

The main problem is: the process gets significantly slowed down the bigger the database, the more lines you output is directly related to the amount of time it takes to backup / restore the database. If you disable verbose completely the amount of time that the process takes drops down to like, 10% of the overall time. It's insane.

During my experiments I found out that if I use another thread to output the verbose to the memo, the process doesn't take that big of a hit in performance, it's like 90% of the non-verbose option, which is great. But I've ran into an issue, using threads makes the application create some sort of stack of threads that will eventually be outputted to the memo, but the problem is that the process itself is already done and the lines will keep being outputted to the memo for a long time.

If I don't use threads to do this, the process takes the big hit on it's performance and the application gets essentially locked down until the process is done.

Does anyone know of any good ways to output the verbose of this toolset without locking down the UI or cause the problem that I explained above?

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
Nickolas de Luca Alberton
  • 320
  • 1
  • 3
  • 17
  • 3
    You mustn't touch VCL controls from a thread different from the main thread (without some kind of synchronisation). BTW, do you use `BeginUpdate` and `EndUpdate` when you output lines to the memo? You could also add the lines to an in-memory non-GUI object and then only once in a while populate the memo. – Andreas Rejbrand Aug 11 '20 at 14:58
  • 1
    Surely you are creating the problem yourself by funneling your "verbose output" through something which is fundamentally ill-suited to dealing with it, namely a gui control like a memo. Store your output to text files and provide a decent filter for loading a relevant part of of their content when someone actually needs to look at it. Like when they have a post portem to do. – MartynA Aug 11 '20 at 14:59
  • @AndreasRejbrand I tried your method, while it solved my issue, I didn't like the way it turned out to be. but it is a good way to tackle the issue. – Nickolas de Luca Alberton Aug 11 '20 at 16:24
  • @MartynA that's the way the software is working at the moment, I store everything and if the user chooses to, they can save the output to a file. It's efficient, solves the main problem but I can't see the progress "live". – Nickolas de Luca Alberton Aug 11 '20 at 16:25
  • Seeing the process "live" clearly has a substantial impact on performance. You can't really have it both ways; you either have it fast but not seen or visible but much slower because of the overhead of displaying it and updating that display. Take your pick of one of them. – Ken White Aug 11 '20 at 18:07
  • 1
    If there's **volatile** output needed I tend to use a console window (once `AllocConsole()`) and then simply output thru `Writeln()`. Advantages: the buffer of the console is configurable, output auto-rotates, different colors. – AmigoJack Aug 11 '20 at 20:02
  • 2
    In my case, I write output to a log file, and then have a separate app displaying that log file in real-time, updating the UI at intervals whenever the file is written to. This way, I still get to see a display of events in near-real-time, but without hindering the app that is doing the logging. – Remy Lebeau Aug 11 '20 at 20:18
  • Does this answer your question? [Thread update GUI](https://stackoverflow.com/questions/34803149/thread-update-gui) – Arioch 'The Aug 13 '20 at 13:08
  • It was discussed many times already, consider: https://stackoverflow.com/questions/34803149 and https://stackoverflow.com/a/44162039/976391 and https://stackoverflow.com/a/17706142/976391 and many others. In general, you should 1) Throttle screen output to 2-4 updates per second, no more frequent. Plus one out-of-schedule last update after job aborted or completed; and 2) track if the user is seeing last line or if user scrolled back to prior lines to re-read slowly; and 3) if user did not scrolled back - to scroll memo down to last line using SetCaret text cursor positioning – Arioch 'The Aug 13 '20 at 13:10
  • @RemyLebeau frankly, i only see there a benefit of having a file left, if an app crashes badly and dies instantly without being able even to squeak. But the added boilerplate would be non-buffered (or precisely flushed) file output, including decision where to create the file and how to name it and how to inform user where to find last log if the program disappears in a puff of smoke, then all the IPC... Unless you really need that all for "the file would remain even in worst case" warranty it seems much easier and leaner to do it just inside one multithreaded application. – Arioch 'The Aug 13 '20 at 14:19
  • I'll try to implement what @AmigoJack suggested. I'll post back if its good or not. – Nickolas de Luca Alberton Aug 13 '20 at 19:26
  • 1
    i would still suggest you to throttle output, console or memo - it is all the same for human eye. If wall of text is scrolled too fast - it becomes a blurred grey goo. My personal opinion is that 1/3 seconds gap between updates is the best compromise. – Arioch 'The Aug 13 '20 at 21:09

1 Answers1

0

I decided to implement AmigoJack's suggestion of using AllocConsole() and Writeln() to display Firebird's verbose output. It works quite well and doesn't lock my application's GUI.

To prevent the user from closing the application when closing the console you are able to get the handle to that console and then remove the close button.

You will need to declare this function first:

function GetConsoleWindow: HWND; stdcall; external kernel32;

And this is the code to allocate a new console and then hide the button:

var
  handle: HWND;
  consoleMenuItem: HMENU;
begin
  AllocConsole();

  handle := GetConsoleWindow;

  if IsWindow(handle) then
  begin
    consoleMenuItem := GetSystemMenu(Handle, False);

    if IsMenu(consoleMenuItem) then
    begin
      DeleteMenu(consoleMenuItem, SC_CLOSE, MF_BYCOMMAND);
    end;
  end;
end;

Just remember to add a way for the user to call FreeConsole() in order to get rid of the console when you don't need it anymore, or call it yourself.

Nickolas de Luca Alberton
  • 320
  • 1
  • 3
  • 17
  • 1
    https://stackoverflow.com/questions/11959643/why-does-closing-a-console-that-was-started-with-allocconsole-cause-my-whole-app – Andreas Rejbrand Aug 25 '20 at 18:56
  • Ok I did manage to get it to respond to the close event, but I could not stop it from closing. The console calls whatever I set it to do but it still closes the application at the end, even Abort; can't stop it. Do you know how to stop it from closing the application? – Nickolas de Luca Alberton Aug 25 '20 at 20:54
  • 1
    No, the link above kind of confirms that it isn't really possible. But it does offer a few workarounds. – Andreas Rejbrand Aug 25 '20 at 20:55
  • 1
    Disable the console's (window) close button, so it can't be closed at all anymore. Give your program/user a way to hide/free the console (window) again. – AmigoJack Oct 16 '20 at 22:21