MySQL - How do I search for a GUID

Question

I am using a 3rd party .NET library (Rhino Security) that stores it's identifiers as guids in binary(16) fields in my mysql db. Everything works perfectly from the application but when I attempt to manually run a query via a query editor (TOAD for mysql) no rows are returned for identifiers I know to exist. For instance, if i run the following query, i get no results:

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where EntitySecurityKey  =  '02a36462-49b7-406a-a3b6-d5accd6695e5'

Running the same query with no filter returns many results, including one with the above GUID in the EntitySecurityKey field. Is there another way to write a query to search on a guid/binary field?

Thanks!

EDIT

I found it interesting that TOAD returned a string and not an ugly blob. Using a different editor to return the results (for the unfiltered query) I get the raw binary data. I would have assume that my query would work using the binary keyword but neither of the following worked:

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where EntitySecurityKey  =  BINARY '02a36462-49b7-406a-a3b6-d5accd6695e5'

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where BINARY  EntitySecurityKey  =  '02a36462-49b7-406a-a3b6-d5accd6695e5'

score 3 · Answer 1 · edited May 23 '17 at 10:28

I am not familiar with Rhino Security, but it uses NHibernate to store to the database as far as I know.

The .net Guid uses 1 Int32, 2 Int16 and 8 Bytes internally to store the 128-bit Guid value. When NHibernate (3.2) stores/retrieves a Guid value to/from a BINARY(16) column, it uses the .Net ToByteArray() method and Guid(Byte[] ...) constructor respectively. These methods basically swap the order of the bytes for the Int32 and Int16 values. You can reflect on the methods to see the exact code, but here is a simple example of the first 4 bytes:

guidBytes[0] = (byte)this._int32member;
guidBytes[1] = (byte)(this._int32member >> 8), 
guidBytes[2] = (byte)(this._int32member >> 16), 
guidBytes[3] = (byte)(this._int32member >> 24);

This may be the cause of your issue.

For example, your Guid is stored in MySql differently.
02a36462-49b7-406a-a3b6-d5accd6695e5 - Your Guid from Guid.ToString()
6264a302-b749-6a40-a3b6-d5accd6695e5 - Your Guid in MySql using MySql hex(Guid)
Note: hex(x) does not add the dashes; I added dashes manually for comparison purposes. Notice that the order of the first 8 bytes is different.

To test if this is your issue, run the following query:

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where EntitySecurityKey = UNHEX(REPLACE(6264a302-b749-6a40-a3b6-d5accd6695e5,'-',''));

If so, some other items I have noticed in my own travels:

Toad for MySql complicates this too, since it will display the Guid as the .net equivalent in a select statements output. It does not apply the same conversion if you include that Guid in a select's where clause (at least from my experience). You would need to use the Guid as stored by MySql.

You will either need to manually convert the Guids when performing DB queries or you may need to introduce a custom NHibernate type to convert the Guid differently.

You can refer to:

How to read a .NET Guid into a Java UUID
http://pastebin.com/M07wnK5K (I haven't used this code, but found it useful for reference)

Update: I just tried the custom NH type on the pastebin link above. The byte order is incorrect, at least for my environment. I would need:

private static readonly int[] ByteOrder = new[] { 3, 2, 1, 0, 5, 4, 7, 6, 8, 9, 10, 11, 12, 13, 14, 15 };

to make by .Net output match the select hex(Guid column) from table output.

score 2 · Answer 2 · edited May 23 '17 at 12:26

What strikes me as interesting is that you're storing your GUIDs in fields that are binary(16), emphasis on the 16. Per the manual, a binary field's length is in bytes and will truncate anything that goes over it (only in strict mode though). Is it possible that your GUID's are being truncated? With your sample GUID, 02a36462-49b7-406a-a3b6-d5accd6695e5, try querying the database with the first 16 characters:

WHERE EntitySecurityKey = '02a36462-49b7-40'

Per the accepted answer to this question, a field should be char(16) binary to store a GUID, not just binary(16). However, I couldn't get this to work in my sample table.

What did work for me were using char(36) and also binary(36). Try updating your field-lengths to 36 instead of 16 (I'd do this on a test-table first, just to be safe).

Here was my test table:

CREATE TABLE test_security_keys (
    test_key1 char(16) not null,
    test_key2 char(16) binary not null,
    test_key3 char(36) not null,
    test_key4 binary(16) not null,
    test_key5 binary(36) not null
);

Then, I ran a script to insert numerous GUIDs (the same one for each column in a row). You can test it with your sample GUID:

INSERT INTO test_security_keys
    VALUES ('02a36462-49b7-406a-a3b6-d5accd6695e5', '02a36462-49b7-406a-a3b6-d5accd6695e5', '02a36462-49b7-406a-a3b6-d5accd6695e5', '02a36462-49b7-406a-a3b6-d5accd6695e5', '02a36462-49b7-406a-a3b6-d5accd6695e5');

Using a simple SELECT * FROM test_security_keys will show all of the columns except the ones with size 36 to be truncated. Also, binary or not, I was able to successfully query the columns with a regular string-comparison:

SELECT * FROM test_security_keys WHERE test_key3 = '02a36462-49b7-406a-a3b6-d5accd6695e5';
SELECT * FROM test_security_keys WHERE test_key5 = '02a36462-49b7-406a-a3b6-d5accd6695e5';

If you've confirmed that your current columns with binary(16) aren't truncating, I would then-suggest to use a CAST() in your WHERE clause. The following should work (with your sample query):

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
WHERE
    EntitySecurityKey = CAST('02a36462-49b7-406a-a3b6-d5accd6695e5' AS binary(16));

If you CAST(.. AS binary(16)) and the input-data is longer than 16 bytes, MySQL should issue a warning (should be unseen and not affect anything) stating that it had to truncated the data (try SHOW WARNINGS; if you get them). This is to be expected, but also means that you can't use binary(16) to store the GUIDs and you'll need to use binary(36) or char(36).

* I have not tried any of this using TOAD, but I've used both command-line MySQL and Navicat and have the same results for both.

This isn't the issue, a GUID is just a 128-bit number which is 16 bytes and can be stored in a binary field holding 16 bytes - which is what binary(16) is. The problem is that .NET guid.ToByteArray() handles endianness differently compared to other systems. Read @user1748279 answer below for the proper info or just go to http://stackoverflow.com/questions/5745512/how-to-read-a-net-guid-into-a-java-uuid — Mani Gandham, Aug 11 '15 at 23:51

score 2 · Answer 3 · edited May 23 '17 at 10:28

I came across this question when I was evaluating how others "searched" for records where the primary key is a binary type (mainly the binary(16) indicative of GUID's) to compare it to my own. I must admit I was a little taken a back that the answers quickly diverged from the users desire to do manual searches based on the binary field but instead focused on changing the column types themselves to characters which, in this case, is murder on InnoDB storage.

An attempted solution for the lack of results would be to search against the Id column instead of the EntitySecurityKey column. Given JP's original query, the modified, working version would be:

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where Id  =  UNHEX(REPLACE('02a36462-49b7-406a-a3b6-d5accd6695e5', '-', ''));

Again, this assumes the Id field is the binary(16) type. (I wasn't able to determine which column: Id, EntitySecurityKey was the binary(16)). If the EntitySecurityKey is the binary(16) column, then:

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where EntitySecurityKey  =  UNHEX(REPLACE('02a36462-49b7-406a-a3b6-d5accd6695e5', '-', ''));

If that failed to yield the desired results, per this question (How to read a .NET Guid into a Java UUID), it would be of interest to try the same query but with the endianness of the first three parts of the GUID reversed:

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where Id  =  UNHEX(REPLACE('6264a302-b749-6a40-a3b6-d5accd6695e5', '-', ''));

The only other thing I can think of is that EntitySecurityKey is a virtual column based on Id, but I don't have enough information on the table set-up to validate this statement.

Cheers.

score 1 · Answer 4 · answered Oct 12 '12 at 18:13

From your EDIT, which I've a bit modifed:

Please, try

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences
WHERE EntitySecurityKey  = X'02a3646249b7406aa3b6d5accd6695e5'

Since the strings have 16 hexadecimal byes each, it may works...

score 0 · Answer 5 · edited May 23 '17 at 12:02

Have you checked what the GUID data actually looks like in the table? Since it's the primary key, it quite possibly isn't stored as a GUID string. If that's the case querying the string value clearly won't work.

Possibly it's in binary format? Or possibly it's a string but without the hyphens? Or maybe it's been converted some other way? I don't know without seeing your data. But whatever the case, if a GUID that you know exists isn't being found when you query it, then whatever format it is in, it clearly isn't stored in the format you're querying.

The way I would solve this would be by searching for a different value in a record that you know exists. Or even just query the top few records without a where clause at all.

This will show you what the data actually looks like. With any luck, that will be sufficient for you to work out how it's been formatted.

If that doesn't help, one possible clue might come from this question: Why are binary Guids different from usual representation

Hope that helps.

score -1 · Answer 6 · answered Oct 12 '12 at 13:07

Not sure how your database is set up, but I would try these two variants:

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where EntitySecurityKey  =  x'02a3646249b7406aa3b6d5accd6695e5'

SELECT Id, EntitySecurityKey, Type
FROM mydb.security_entityreferences 
where EntitySecurityKey  =  x'6264a302b7496a40b6a3d5accd6695e5'

MySQL - How do I search for a GUID

6 Answers6