3

I am working on an application built in Delphi 2010 that uses UIB to connect to a Firebird 2.5 database. The application has been running using the default character set for a long time, i.e. nobody gave character sets any special thought and it has simply been working. Currently I am trying to make it work correctly with UTF-8 data.

In doing so I have hit upon a problem with TUIBQuery and parameterized queries. When using Database.Charset=csUTF8 and setting a parameter value for a CHAR(n)-field and retrieving it before executing the query the value is truncated.

Unfortunately some of my code writes and reads parameter like this in a number of places and therefore dies an ugly death.

To isolate and demonstrate the problem I created a simple fresh database with DEFAULT CHARACTER SET UTF-8 and a table like this:

  CREATE TABLE TEST (
    CHARFIELD CHAR(20),
    VARCHARFIELD VARCHAR(20)
  );

I set up the application to connect to the database using TUIBDatabase and TUIBTransaction. Then I created a TUIBQuery-instance, set SQL to a parameterized INSERT-statement into this table, and set the parameters:

  Query := TUIBQuery.Create(NIL);
  Query.Transaction := Transaction;
  Query.SQL.Text := 'INSERT INTO TEST (CHARFIELD, VARCHARFIELD) VALUES (:CHARFIELD, :VARCHARFIELD)';
  Query.Prepare(True);

  s:= 'ABC';

  Query.Params.ByNameAsString['CHARFIELD'] := s;
  Query.Params.ByNameAsString['VARCHARFIELD'] := s;

When I now read the parameter-values back like this:

  s := Query.Params.ByNameAsString['CHARFIELD'];
  s := Query.Params.ByNameAsString['VARCHARFIELD'];

The results are correct for Database.Charset=csNone. But when I instead specify DataBase.Charset=csUTF8 the value for CHARFIELD is truncated to 'A' instead of 'ABC'. The value for VARCHARFIELD is fine. The behaviour is independent of the actual data, I do not have to actually use non-ASCII-characters to provoke it, as the sample shows.

Calling ExecSQL() on the query works correctly and INSERTs the data as expected in both cases.

I have uploaded sourcecode to my simple test program as UIB_UTF8_Test.zip.

Does someone here have any idea what I may be doing wrong and how to do it right?

Marian Aldenhövel
  • 677
  • 1
  • 6
  • 24
  • did you try more recent Delphi ? for example up to September 9 you can grab free Delphi 10.1 Starter and can test there. I just have some prejudice towards D2010 - it is just next release after uber-buggy 2009 – Arioch 'The Aug 29 '16 at 08:44
  • you say u use firebird 2.5 and your tags say u use 2.1 - so which is the exact version? post the database as well. Also post the uib inc-file because the library is target-compiled for selected db sever and UI language – Arioch 'The Aug 29 '16 at 08:45
  • The DB is indeed Firebird 2.5, but when I posted the question SO did not offer me a tag of that version. So I picked a close one. Corrected now. I have uploaded the DB: [UTF8.FDB](http://www.qvwx.de/tmp/UTF8.FDB). I did not know about uib.inc and it targetting specific Interbase/Firebird-versions so I have learned something already. The version of the file we have used so far is vanilla from the distribution and does not $DEFINE any specific release. I tried with `{.$DEFINE FB25}` but that did not change the behaviour. – Marian Aldenhövel Aug 29 '16 at 09:31
  • And no, I did not try a different version of Delphi yet. Keeping down the number of variables at this point. Thank you very much for giving my problem some thought. – Marian Aldenhövel Aug 29 '16 at 09:32
  • u see, "Keeping down the number of variables" is good, but "Keeping down the number of constants" is not. You still have to try a bit different things to find what makes the difference. Or analyse the sources. PS. you are using last github snapshot of uib or what ? – Arioch 'The Aug 29 '16 at 09:47
  • "I tried with {.$DEFINE FB25}" - u mean without the dot ? I hope u use the latest fb 2.5.6 then – Arioch 'The Aug 29 '16 at 09:47
  • I am using a github snapshot from last week. And yes, I did remove the dot, sorry. – Marian Aldenhövel Aug 29 '16 at 10:28
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/122086/discussion-between-marian-aldenhovel-and-arioch-the). – Marian Aldenhövel Aug 29 '16 at 10:35
  • i'll see if i can find something later // but actually I think what you are trying to do is not what params were intended to. First, UIB is about speed, and reading/writing TDataSet params back and forth is not the fastest thing on earth. Second, I think the main idea for reading from parameters is returning results for INSERT-RETURNING and EXECUTE_PROCEDURE kinds of statement. Then, I do not say the bug should not be marked as the bug, just that maybe the lib is really was not designed wit hsuch usecases in mind – Arioch 'The Aug 29 '16 at 20:49
  • I agree that may not be the standard use of parameters. My main usage pattern is the creation of master- and detail-records within the same request. In those cases the system passes the query holding the INSERT-statement for the master to the subroutines that set up the details. All the master-data required to build the details is available and key information for the details can be copied from the master-parameters directly. I could of course create special models for that and pass those around, but they would look remarkably similar to a typed and named parameter-list. Thank you! – Marian Aldenhövel Aug 30 '16 at 09:17

0 Answers0