1

Hey I've a few issues with my solution and I can't figure it out

Below is my creation table

    CREATE TABLE FreeTextSearch
(
    [ID] BIGINT NOT NULL IDENTITY(1,1),
    [Content] VARBINARY(MAX) NOT NULL,
    [SubjectClass] VARCHAR(30) NOT NULL,
    [SubjectID] VARCHAR(30) NOT NULL,
    [ColumnName] VARCHAR(128) NOT NULL,
    CONSTRAINT PK_FreeTextSearch PRIMARY KEY (ID)
);


ALTER TABLE FreeTextSearch
Add FileExtension As '.html';

CREATE INDEX FreeTextSearch_SubjectClass
ON FreeTextSearch ([SubjectClass]); 


CREATE FULLTEXT CATALOG [freetext_catalog]


CREATE FULLTEXT INDEX ON FreeTextSearch
(
    [Content] TYPE COLUMN FileExtension
)
    KEY INDEX PK_FreeTextSearch ON freetext_catalog;

This will create a FreeTextSearch table, with FileExtension of HTML etc.

I've the data converted to VARBINARY automatically inserted into the table based on a trigger but there are two issues right now

Content like:

<p>Test note for free text bla!</p>

Won't be detected by query like

    SELECT
    Id,
    CONVERT(VARCHAR(MAX), [Content]) AS [Content]
FROM dbo.FreeTextSearch
WHERE FREETEXT ([Content], 'Test')

Additionally

    SELECT
    Id,
    CONVERT(VARCHAR(MAX), [Content]) AS [Content]
FROM dbo.FreeTextSearch

Will return

ID: 1 Content <
ID: 2 Content <

While

    SELECT
    Id,
    CONVERT(XML, [Content]) AS [Content]
FROM dbo.FreeTextSearch

Will Return

ID: 1 Content: <p>Test note for free text bla!</p>
ID: 2 Content: <p>Something is very wrong with this one i think</p><p>Not really sure what.</p>

Which is the correct data but it's unclear why

  1. FREETEXT doesn't find anything
  2. Why VARCHAR(MAX) returns only <
VoidEssy
  • 23
  • 7
  • 2
    Recheck yourself. Works perfectly for me. – David דודו Markovitz Mar 09 '20 at 13:47
  • Hey @DavidדודוMarkovitz Thanks to answer from Lptr I see that the cast issue comes because it needs to be NVARCHAR rather than VARCHAR. Still trying to see what causes the issue with FREETEXT finding nothing, maybe I messed up my DB in the process of editing it when I forgot to set IDENTITY, will try with clean instance. – VoidEssy Mar 09 '20 at 14:17
  • Perhaps the better question is why you believe you need to store HTML as varbinary. And then why you think you can safely cast this to XML. Presumably this is related to your previous question. – SMor Mar 09 '20 at 18:49
  • @SMor XML was just a test to underline the point of confusion. HTML is in varbinary because i want to apply html filter on my FREETEXT queries which can't be applied to VARCHAR or NVARCHAR – VoidEssy Mar 09 '20 at 23:19

1 Answers1

1
select test, cast(test as varchar(max)), cast(test as nvarchar(max))
from
(
select cast('a' as varbinary(max)) + 0x0 /* nul in ascii, anything after that not printed*/ + cast('123' as varbinary(max)) as test
) as src;


select test, cast(test as varchar(max)), cast(test as nvarchar(max))
from
(
select cast(N'<p>Test note for free text bla!</p>' as varbinary(max)) as test
) as src;
lptr
  • 1
  • 2
  • 6
  • 16
  • Hey ran your code and yeah NVARCHAR WORKS that resolves one of the issues not sure about the FREETEXT issue yet. https://i.ibb.co/ZcDpPb4/Output.png Below image with named columns for clarity if anyone looks here https://i.ibb.co/VN26SJm/Output.png – VoidEssy Mar 09 '20 at 14:21
  • 1
    BOM the varbinary to indicate unicode: `insert into FreeTextSearch([Content], [SubjectClass], [SubjectID], [ColumnName]) values(0xFFFE + cast(N'

    Test note for free text bla!

    ' as varbinary(max)), 'abc', 'abc', 'abc')`
    – lptr Mar 09 '20 at 16:08
  • Hey, I'm not very sure on what BOM means google tells me byte order mark, need to read up on that :) Alright so this one does come up in the free text search. Properly, is something wrong with my trigger maybe that gets the data into it? Even thought I don't think it should be an issue. I'll attach the Trigger script https://pastebin.com/DQsrxja2 (trigger script) https://i.ibb.co/s9W3Wjs/Output.png (Output from the test) PS. I'm sorry if I'm being dense or stupid here really. EDIT: I assume you're suggesting i spike my entries with BOM? – VoidEssy Mar 10 '20 at 09:42
  • Update: So after adding 0xFFFE to my Trigger code passing it does work with freetext on NVARCHAR conversion which is great, but If you could explain it a bit further I'd highly appreciate it. – VoidEssy Mar 10 '20 at 11:01