0

We are (still) using ZF1, to write/read from a Microsoft SQL database.

As long as a table field is of type NVARCHAR, everything is fine, special characters (like, chinese, polish, etc.) and emojis are saved correctly, 1:1 equal to the entered text in an html form.

But when it comes to column type TEXT, there are Question marks instead of the special characters/emojis in these columns.

Funny enough that everything is fine on servers where we got PHP5.6 running, with FreeTDS and php-mssql installed on them. This problem occurs on a system where PHP7.4 is installed, using only SQLSRV as the db driver.

According to this thread, a simple N should be enough...as long as the fields are set as NVARCHAR fields. We are using the update and insert methods of Zend Framework, so we got no clue how to save all these characters into TEXT columns.

When connecting, our bootstrap file is using UTF-8 as the character set, used as an driver option.

Is there any workaround we could try to accomplish this? We cannot change the column type because there are reasons why some colums are TEXT.

  • Is the db configuration where PHP5.6 runs identical as the one with PHP7.4 ? If the DB is the same, then this is likely a PHP input charset issue – Simon Jan 21 '21 at 18:01
  • @Simon Indeed, same database. As I mentioned, it's working fine if we write into NVARCHAR columns with PHP7.4, whilst having trouble with TEXT columns. –  Jan 21 '21 at 23:39

2 Answers2

0

Why are you using text? It (text) has been deprecated for 16 years. text, however, does not support unicode characters, that would be ntext; but that too has been deprecated for 16 years.

To store up to 2GB of characters, that need to be unicode, use nvarchar(MAX). For a column that already exists, ALTER it.

ALTER TABLE dbo.YourTable ALTER COLUMN YourColumn nvarchar(MAX);
Thom A
  • 88,727
  • 11
  • 45
  • 75
0

As per OP response in the comments the database setup can be put aside given that the TEXT column type behaves as expected when using PHP 5.6 and the operation fails on the same database when using PHP 7.4.

This allows to narrow the issue to runtime PHP charset handling. Not knowing all the specifics between the upgrade of PHP 5.6 to 7.4, the issue may lie in:

  • Different default charset in php.ini
  • Different behavior in the mssql driver with charset handling
  • Different operating system default locale which may affect the script
  • Different php file encoding (the encoding of the .php file itself may alter the charset used at runtime)

Since the issue at hand cannot be unambiguously identified based on the provided information, I may only suggest to check and eliminate the different possibilities one by one.

  1. try detecting the actual encoding of the input text using any of the mb_* functions (maybe refer to this question)
  2. try adding a comment in your script file containing special characters and be sure to save the file as UTF8 (or desired encoding)
  3. try setting explicitly the driver charset (refer to this question) (or maybe in the connection string, if supported)
  4. try setting the default php locale using setlocale and other functions of the sort

Those steps may guide you through a resolution but without guarantees.

Simon
  • 2,353
  • 1
  • 13
  • 28