Delphi Adding Items to ComboBox Speed

Question

I have a fairly complex and large application that hands loads and loads of data. Is there a speedy way to add items to ComboBox that doesn't take so long? On my P3 3.2ghz, the following snippet takes just under a second to add around 32,000 items. (MasterCIList is a StringList with strings typically 20 - 30 bytes long).

with LookupComboBox do
 begin
  Items.BeginUpdate;
  Items.Clear;
  for i := 0 to MasterCIList.Count - 1 do
   Items.Add(MasterCIList[i]);
  Items.EndUpdate;
 end;

Drilling down into the VCL, it appears that in TComboBoxStrings.Add, there is a call to

Result := SendMessage(ComboBox.Handle, CB_ADDSTRING, 0, Longint(PChar(S)));

I'm guessing this is really taking up time (okay, I know it is). Is there another way to populate the Items that is speedier? Any high-speed combo boxes available? I have the TMS components but they seem to be extensions of TComboBox.

For instance, I have the PlusMemo which seems to be a total rewrite of a TMemo. I can easily add a million line in a second to a PlusMemo. A TMemo, I don't think so.

Thank you so much for your time!

Not an answer, but why on earth would you want 32,000 items in a combo box? That is a terrible way to store that much data. — Ed S., Mar 20 '09 at 21:33
*"How do you expect a user to sort through that many items to make a decision?"* autocomplete. But his question is still valid with ~270 items (i.e. one for each country). Takes 80 ms to add ~270 items to a `TComboBox`; which is just rediculous. So, everyone, feel free to answer the question that was asked. It's a valid question, and applicable to a wider audience than just the exact specifics of this particular scenario. There is no reason a `TComboBox` shouldn't be able to add ~100,000 items in < 10 ms. Don't worry about the UX - that's not part of this question. Answer the **question**. — Ian Boyd, Aug 18 '23 at 19:08
[Related question](https://stackoverflow.com/questions/10617551/combobox-items-addrange-performance) - people having the same problem from within C#. — Ian Boyd, Aug 18 '23 at 19:34

score 16 · Answer 1 · edited Feb 08 '18 at 12:42

Sorry if I'm a nuisance, but I doubt a TComboBox with 32,000 items is even remotely ''usable'' --- I'd say there's a reason why it's slow: it was never meant to do this :)

Would there be a possibility to filter the data, and only load a subset? To be more concrete, in one particular database application I've been working on, the user can search for a person. We let the user type at least 3 or 4 characters of the name, and only then begin to return results in a listbox. This has greatly increased usability of the search form, also greatly speeding up the whole process.

Would you be able to follow a similar approach?

Or, on a completely different take, perhaps you could take a look at the VirtualTreeView component --- either for direct use, or for inspiration.

It's a good tip. Maybe add the strings in the OnDropDown handler, filtered to begin with the chars that are already entered. That way the user has influence on the delay. In any case, a lookup list with 32000 items is unusable, even if instantly populated. — mghie, Mar 20 '09 at 21:42

score 3 · Answer 2 · answered Mar 21 '09 at 01:29

I agree that 32K items is a ridiculous amount to have in a combobox... That being said, you might try adding the items to a TStringList first and then use a combination of Begin/EndUpdate and AddStrings:

SL := TStringList.Create;
try
  // Add your items to the stringlist
  ComboBox.Items.BeginUpdate;
  try
    ComboBox.Items.AddStrings(YourStringList);
  finally
    ComboBox.Items.EndUpdate;
  end;
finally
  SL.Free;
end;

The code compiles, but I didn't test it further than that; I've never felt the need to add more than a few dozen items to a combobox or listbox. If any more items might be needed than that, I find a way to filter before populating the list so there are fewer items.

Just out of curiosity, how do you expect a user to sort through that many items to make a decision?

*"How do you expect a user to sort through that many items to make a decision?"* autocomplete. The question is just as valid with 200 items. — Ian Boyd, Aug 18 '23 at 19:12
Calling **AddStrings** internally already calls **BeginUpdate** / **EndUpdate** for you. Calling **BeginUpdate** on a **TComboBox.Items** sets `WM_SETREDRAW` on the control. The issue is that **AddStrings** still calls **AddObject** in a loop, which sends `CB_SETITEMDATA` message each time through the loop. A regular **TStringList** takes 0.2 ms to fill. A "combo-box TStrings" is ~280x slower. — Ian Boyd, Aug 18 '23 at 19:22

score 2 · Answer 3 · answered Dec 07 '13 at 00:01

var
  Buffer: TStringList;
begin
  Buffer := TStringList.Create;

  try
    // --> Add items to Buffer here <--

    ComboBox.Items := Buffer;
  finally
    FreeAndNil(Buffer);
  end;
end;

This is the fastest way we've found to update a visual control.

The VCL does BeginUpdate, Clear, and EndUpdate internally.

If you don't believe me, profile it.

score 1 · Answer 4 · answered Apr 13 '12 at 08:11

use backgroundworker for adding MasterCIList items.after complete adding items use only AddStrings.

procedure TForm2.BackgroundWorker1Work(Worker: TBackgroundWorker);
var
  I: Integer;
begin
  MasterCIList.BeginUpdate;
  try
    MasterCIList.Capacity := 32 * 1024; // if derminate  count of items
    for I := 1 to 32 * 1024 do
    begin
      MasterCIList.Add(IntToStr(I));
      { if need progess }
      if I mod 300 = 0 then
        Worker.ReportProgress((I * 100) div MasterCIList.Capacity);
      { if need cancelable }
      if (I mod 100 = 0) and Worker.CancellationPending then
        Break;

    end;

  finally
    MasterCIList.EndUpdate;
  end;

end;

procedure TForm2.BackgroundWorker1WorkComplete(Worker: TBackgroundWorker;
  Cancelled: Boolean);
begin

  LookupComboBox.Items.AddStrings(MasterCIList );


// AddStrings use beginupdate..endupdate in itself

 end;

score 1 · Answer 5 · answered Apr 13 '12 at 08:57

1

Maybe you can try this?

"Speeding up adding items to a combobox or listbox" http://blogs.msdn.com/b/oldnewthing/archive/2004/06/10/152612.aspx

answered Apr 13 '12 at 08:57

André

8,920
1
24
24

score 1 · Answer 6 · answered Mar 20 '09 at 21:35

perhaps cmb.Items.Assign(myStringList) will help.

here's a wild idea: i haven't tried it but you might check to see if there's a way to virtually load the combobox by setting the number of items and then owner drawing. please pardon this crazy idea but i think i've heard of this being available somehow. irrelevant: this is how it's done in Palm OS...where the faster way to load the combobox is to not load it all... ;-)

Not an answer, but why on earth would you want 32,000 items in a combo box? That is a terrible way to store that much data.

i agree; it's a bad practice...

score 1 · Answer 7 · answered Mar 21 '09 at 03:05

1

It's me again. I'm adding 32,000 items cause I need to. That's one of many controls in my application that has "lots" of items. I need to have that many items. It works just fine looking things up. Perfectly in fact. I'm just trying to optimize things. The users find things just fine since they are in a certain logical order.

Everything I've seem so far with Assign and AddStrings is that they eventually end up in Add with the SendMessage call. So I'll keep looking.

Thanks for the feedback.

answered Mar 21 '09 at 03:05

1

First, posting an "answer" like this is wrong. Edit your original post. Second, again, I've been programming for 20+ years, Win apps for 15+, have done some extraordinarily complex apps, and have never found the need for even 1K items in a combobox. But good luck to you (and the users of your app). – Ken White Mar 21 '09 at 03:23
Agree with Ken. To be clear - it's fine to post an answer if it's really an answer to your question. If you have a comment then just edit your question. – Argalatyr Mar 22 '09 at 02:20

score 0 · Answer 8 · edited Dec 15 '12 at 01:54

I implement this in a different manner. First i removed the combobox control and take textbox control and assign it autocomplete to custom source where the custom source string collection is 32k items.I get the selected value from a new query on controls validation.

So it can replace combobox functionality. Mostly about 32k items people dont scroll but they keep entering key strokes and is catched by our custom auto complete source..

score 0 · Answer 9 · answered Mar 21 '09 at 09:01

Perhaps you can use a database engine in the back-end and use a data aware component. Then the things will be much more quicker and usable. Perhaps if you'll try to describe what do you try to accomplish we'll help you further. In any case, your UI design is, let's say, odd. And for this perhaps the Embarcadero forums will help you better.

Ian Boyd · Answer 10 · 2023-08-21T15:51:03.093

Nobody has the solution that solves the speed problem. But lets take a tour of the various answers that speed it up somewhat (although nowhere near fast enough to consider the problem solved).

Our test will be of adding 250 items.

Test 1 - Add to raw TStrings - 120,000 items/sec

First we'll add 250 items to a TStringList. Without any other overhead or cost we will be how have it can happen.

var
   i: Integer;
   sl: TStrings;
begin
   sl := TStringList.Create;
   sl.BeginUpdate;
   for i := 1 to 250 do
      sl.Items.Add('Test');
   sl.EndUpdate;
end;

That adds 250 items in 0.0081 ms (120,000 items/sec) on my 3.5 GHz i7-2700.

This is the fastest we could ever reasonably expect.

Test 2 - Adding to TComboBox.Items TStrings - 8 items/sec (99.993% slower)

Now our test function will actually add things to a TComboBox.Items.

var
   i: Integer;
begin
   for i := 1 to 250 do
      AComboBox.Items.Add('Test');
end;

That adds 250 items in 124.9 ms (8.00 items/sec).

Test 3 - Turn off Redraw - 24 items/sec (99.98% slower)

Next we turn off drawing of the control by calling WM_SETREDRAW

var
   i: Integer;
begin
   AComboBox.Perform(WM_SETREDRAW, 0, 0); // turn OFF redraw
   for i := 1 to 250 do
      AComboBox.Items.Add('Test');
   AComboBox.Perform(WM_SETREDRAW, 1, 0); // turn ON redraw
end;

That adds 250 items in 41.3397 ms (24.2 items/sec).

Test 4 - Items.BeginUpdate - 24 items/sec (99.98% slower)

You should note that the TCustomComboBoxStrings class (which is the TStrings descendant used internally by the TComboBox already calls WM_SETREDRAW for you when you call BeginUpdate / EndUpdate:

procedure TCustomComboBoxStrings.SetUpdateState(Updating: Boolean);
begin
  SendMessage(ComboBox.Handle, WM_SETREDRAW, Ord(not Updating), 0);
  if not Updating then ComboBox.Refresh;
end;

So that means that you can dispense with the call to WM_SETREDRAW and just use the canonical Delphi pattern:

var
   i: Integer;
begin
   AComboBox.Items.BeginUpdate; // turn OFF redraw
   for i := 1 to 250 do
      AComboBox.Items.Add('Test');
   AComboBox.Items.EndUpdate; // turn ON redraw
end;

That adds 250 items in 42.1127 ms (23.74 items/sec). The extra overhead of calling the TStrings.BeginUpdate wrapper is negligible.

Test 5 - CB_INITSTORAGE - 24 items/sec (99.98% slower)

Then we come to the mytical CB_INITSTORAGE method. The intention is that you tell the ComboBox how many items you are about to add, and how much memory (in bytes) it is expected to take. This lets the Combobox preallocate memory once.

var
   i: Integer;
begin
   AComboBox.Perform(CB_INITSTORAGE, 250, 250*6*2); // 12 bytes for "Test\r\n"
   AComboBox.Items.BeginUpdate; // turn OFF redraw
   for i := 1 to 250 do
      AComboBox.Items.Add('Test');
   AComboBox.Items.EndUpdate; // turn ON redraw
end;

That adds 250 items in 41.8486 ms (23.9 items/sec).

Which means it adds no improvement whatsoever. In fact i could not get CB_INITSTORAGE to give any improvmenet anywhere along the line. Perhaps the COMBOBOX control of Windows XP in 2004 benefitted from pre-allocating memory. But it seems that the control has internally solved the problem of repeated incremental memory allocations, and now allocates memory in chunks, and the usefulness of CB_INITSTORAGE no longer exists.

Summary

Test	Time to add 250 items (ms)	Speed (items/sec)
Add to ComboBox	124.9 ms	8
Set Redraw	41.3397 ms	24.2
BeginUpdate	42.1127 ms	23.8
Init Storage	41.8486 ms	23.9
Raw TStringList	0.0081 ms	120000

Using the methods described above, you boost your performance:

Before: 99.9933% slower than it should be
After: 99.9809% slower than it should be

For a dramatic 0.01% speedup!

Bonus Chatter

Some people might suggest calling .AddStrings (or .AddObjects as your case may be):

begin
   AComboBox.Items.AddStrings(SourceList);
end;

What they dont' relize is that AddStrings is identical to Case 4 above:

procedure TStrings.AddStrings(Strings: TStrings);
var
  I: Integer;
begin
  BeginUpdate;
  try
    for I := 0 to Strings.Count - 1 do
      AddObject(Strings[I], Strings.Objects[I]);
  finally
    EndUpdate;
  end;
end;

So we not speak of AddStrings (or AddObjects) anymore, as they are not relevant to our discussion.

Bonus Reading

The Old New Thing: Speeding up adding items to a combobox or listbox
ComboBox.Items.AddRange Performance
What is the fastest way to add items in combo box dropdown using C#?
NirSoft: Faster items adding in ComboBox (with item data)
How do I add combobox items more quickly?
Reddit: Slow comboboxes: performance tricks?
[ComboBoxEdit AddRange method very slow for large quantity of items](https://supportcenter.devexpress.com/ticket/details/b204649/comboboxedit-addrange-method-very-slow-for-large-quantity-of-items}
ComboBoxEdit is really slow when ItemsSource has a lot of items