I am using MySQL 5.6 and I want to modify the default encoding of one table (from latin1 to utf8) WITHOUT modifying the existing columns and rows. Based on documentation I have tried the following command:
ALTER TABLE mytable DEFAULT CHARACTER SET utf8;
It modified the default character set encoding of my table and did NOT modify the collation of the columns, as expected, BUT I was really surprised to see:
Query OK, 32141 rows affected (6.31 sec)
Records: 32141 Duplicates: 0 Warnings: 0
Except "32141 rows affected", the results are as expected as you can see below:
MySQL> select count(*) from mytable;
+----------+
| count(*) |
+----------+
| 32141 |
+----------+
1 row in set (0.01 sec)
MySQL> show table status like 'mytable';
+-----------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-----------------+----------+----------------+---------+
| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |
+-----------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-----------------+----------+----------------+---------+
| mytable | InnoDB | 10 | Compact | 16723 | 20798 | 347815936 | 0 | 21561344 | 15728640 | NULL | NULL | NULL | NULL | utf8_general_ci | NULL | partitioned | |
+-----------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-----------------+----------+----------------+---------+
MySQL> show create table mytable;
CREATE TABLE `mytable` (
`ID` varchar(255) NOT NULL,
`COL1` double DEFAULT NULL,
`COL2` longtext CHARACTER SET latin1,
`COL3` datetime DEFAULT NULL,
`COL4` varchar(255) CHARACTER SET latin1 DEFAULT NULL,
`COL5` int(11) DEFAULT NULL,
`COL6` datetime DEFAULT NULL,
`COL7` varchar(255) CHARACTER SET latin1 DEFAULT NULL,
`COL8` datetime(3) NOT NULL,
`COL9` int(11) NOT NULL DEFAULT '-1',
`COL10` int(11) DEFAULT '0',
`COL11` double DEFAULT '0',
PRIMARY KEY (`ID`,`COL9`),
KEY `idx1` (`COL7`,`COL3`,`COL6`),
KEY `idx2` (`COL1`,`COL4`,`COL3`,`COL6`),
KEY `idx3` (`ID`,`COL3`,`COL6`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
/*!50100 PARTITION BY RANGE (`COL9`)
(PARTITION p0 VALUES LESS THAN (1) ENGINE = InnoDB,
PARTITION p1 VALUES LESS THAN (2) ENGINE = InnoDB,
PARTITION p2 VALUES LESS THAN (3) ENGINE = InnoDB,
PARTITION p3 VALUES LESS THAN (4) ENGINE = InnoDB,
PARTITION p4 VALUES LESS THAN (5) ENGINE = InnoDB,
PARTITION p5 VALUES LESS THAN (6) ENGINE = InnoDB,
PARTITION p6 VALUES LESS THAN (7) ENGINE = InnoDB,
PARTITION p7 VALUES LESS THAN (8) ENGINE = InnoDB,
PARTITION p8 VALUES LESS THAN (9) ENGINE = InnoDB,
PARTITION p9 VALUES LESS THAN (10) ENGINE = InnoDB,
PARTITION p10 VALUES LESS THAN (11) ENGINE = InnoDB,
PARTITION p11 VALUES LESS THAN (100) ENGINE = InnoDB,
PARTITION p12 VALUES LESS THAN (101) ENGINE = InnoDB,
PARTITION p13 VALUES LESS THAN (102) ENGINE = InnoDB,
PARTITION p14 VALUES LESS THAN (103) ENGINE = InnoDB,
PARTITION p15 VALUES LESS THAN (104) ENGINE = InnoDB,
PARTITION p16 VALUES LESS THAN (105) ENGINE = InnoDB,
PARTITION p17 VALUES LESS THAN (106) ENGINE = InnoDB,
PARTITION p18 VALUES LESS THAN (107) ENGINE = InnoDB,
PARTITION p19 VALUES LESS THAN (108) ENGINE = InnoDB,
PARTITION p20 VALUES LESS THAN (109) ENGINE = InnoDB,
PARTITION p21 VALUES LESS THAN (110) ENGINE = InnoDB,
PARTITION p22 VALUES LESS THAN (111) ENGINE = InnoDB,
PARTITION p23 VALUES LESS THAN (200) ENGINE = InnoDB,
PARTITION p24 VALUES LESS THAN (201) ENGINE = InnoDB,
PARTITION p25 VALUES LESS THAN (202) ENGINE = InnoDB,
PARTITION p26 VALUES LESS THAN (203) ENGINE = InnoDB,
PARTITION p27 VALUES LESS THAN (204) ENGINE = InnoDB,
PARTITION p28 VALUES LESS THAN (205) ENGINE = InnoDB,
PARTITION p29 VALUES LESS THAN (206) ENGINE = InnoDB,
PARTITION p30 VALUES LESS THAN (207) ENGINE = InnoDB,
PARTITION p31 VALUES LESS THAN (208) ENGINE = InnoDB,
PARTITION p32 VALUES LESS THAN (209) ENGINE = InnoDB,
PARTITION p33 VALUES LESS THAN (210) ENGINE = InnoDB,
PARTITION p34 VALUES LESS THAN (211) ENGINE = InnoDB,
PARTITION p35 VALUES LESS THAN MAXVALUE ENGINE = InnoDB) */
MySQL> show full columns from mytable;
+--------------------------+--------------+-------------------+------+-----+---------+-------+---------------------------------+---------+
| Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment |
+--------------------------+--------------+-------------------+------+-----+---------+-------+---------------------------------+---------+
| ID | varchar(255) | latin1_swedish_ci | NO | PRI | NULL | | select,insert,update,references | |
| COL1 | double | NULL | YES | MUL | NULL | | select,insert,update,references | |
| COL2 | longtext | latin1_swedish_ci | YES | | NULL | | select,insert,update,references | |
| COL3 | datetime | NULL | YES | | NULL | | select,insert,update,references | |
| COL4 | varchar(255) | latin1_swedish_ci | YES | | NULL | | select,insert,update,references | |
| COL5 | int(11) | NULL | YES | | NULL | | select,insert,update,references | |
| COL6 | datetime | NULL | YES | | NULL | | select,insert,update,references | |
| COL7 | varchar(255) | latin1_swedish_ci | YES | MUL | NULL | | select,insert,update,references | |
| COL8 | datetime(3) | NULL | NO | | NULL | | select,insert,update,references | |
| COL9 | int(11) | NULL | NO | PRI | -1 | | select,insert,update,references | |
| COL10 | int(11) | NULL | YES | | 0 | | select,insert,update,references | |
| COL11 | double | NULL | YES | | 0 | | select,insert,update,references | |
+--------------------------+--------------+-------------------+------+-----+---------+-------+---------------------------------+---------+
My connection parameters are as follows:
MySQL> show variables where variable_name like '%char%' or variable_name like '%collation%';
+--------------------------+--------------------------------------------------+
| Variable_name | Value |
+--------------------------+--------------------------------------------------+
| character_set_client | utf8mb4 |
| character_set_connection | utf8mb4 |
| character_set_database | utf8mb4 |
| character_set_filesystem | binary |
| character_set_results | utf8mb4 |
| character_set_server | utf8mb4 |
| character_set_system | utf8 |
| collation_connection | utf8mb4_general_ci |
| collation_database | utf8mb4_general_ci |
| collation_server | utf8mb4_general_ci |
+--------------------------+--------------------------------------------------+
Note that:
- data was created from a java application
- at the time of data creation, the connection parameters were set to utf8
- there are no FK linked with this table
When I try to reproduce with some newly created tables, it seems that the rows are not modified. See below "0 rows affected":
MySQL> select count(*) from mytesttable;
+----------+
| count(*) |
+----------+
| 3 |
+----------+
3 row in set (0.10 sec)
MySQL> alter table mytesttable character set utf8;
Query OK, 0 rows affected (0.03 sec)
Records: 0 Duplicates: 0 Warnings: 0
I tried to changed my connection parameters back to latin1 during the data creation but it didn't change the result: still "0 rows affected".
So my questions:
- Is my understanding of the command correct? (that it shouldn't modify the rows)
- What could explain that the rows are affected in the 1st case?
EDIT
I have just found out that the problem doesn't happen if I remove the partition.
- With partition I get "XXX affected rows"
- Without partition I get "0 affected rows"
Is it expected?
EDIT 2 with SUMMARY
Initially:
- The table was using
latin1
as default encoding (same for the columns) - The connection was declared as
utf8
What works:
- Before any
ALTER TABLE
command, characters like "é" seem to belatin1
encoded (E9
) - Running command
ALTER TABLE mytable CHARACTER SET utf8mb4;
does not modify the data (hex command still showsE9
) The column is still declaredlatin1
. - Running command
ALTER TABLE mytable MODIFY COL2 LONGTEXT CHARACTER SET utf8mb4
changes the column toutf8mb4
(C3A9
)
So far so good.
Remaining questions:
- How to make sure that all data present in the table is
latin1
? I have triedSELECT COL2 FROM mytable WHERE LENGTH(COL2) != CHAR_LENGTH(COL2) LIMIT 1
and I got 0 results. Is it enough? - Why the command
ALTER TABLE mytable CHARACTER SET utf8mb4;
shows "32141 rows affected" when it seems that the data is not modified? (it happens when the table has partitions and index on the same column) - Following the previous point, is it safe (needed?) to also change the default encoding of the table? Or shall I just stick to the modification of the columns?
Thanks a lot for your help