sql server full text search on json

Question

i'm in need for effective representation of json in sql server so i can perform a very fast search operation.

What i have:

json to be stored :

{"person": {
  "name": "1234",
  "age": "99",
  "parameters": {
    "param1": "1",
    "param2": "2"
  }
}}

or

{"person": {
  "name": "12345",
  "age": "996",
  "parameters": {
    "param1": "1",
    "param5": "5",
    "param7": "7"
  }
}}

Parameters section can contain up to 20 of 60 different parameters. I need to look up for person using only some parameters. If some person has 12 i can use 0-12 parameters in search query. Name and age are always provided in search query and each person has them both. I have around 30m jsons in table.

Is it possible to do it with sql server without nosql/solr/elastic?

we already have a sql server and migration is not an option these days — omali, Sep 25 '17 at 19:15

score 3 · Answer 1 · answered Aug 31 '18 at 11:42

Let's take your case and create a table, that would store json in a column and all parameters that you want to search for in virtual computed columns:

CREATE TABLE [dbo].[JsonTest](
    [_id] [bigint] IDENTITY(1,1) NOT NULL,
    [Json] [nvarchar](max) NOT NULL,
    [Parameter1]  AS (CONVERT([varchar](20),json_value([Json],'$.person.parameters.param1'))),
    [Parameter2]  AS (CONVERT([varchar](20),json_value([Json],'$.person.parameters.param2'))),
CONSTRAINT [PK_JsonTest] PRIMARY KEY CLUSTERED 
(
    [_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]


ALTER TABLE [dbo].[JsonTest]  WITH CHECK ADD  CONSTRAINT [CK_JsonTest_Json] CHECK  ((isjson([Json])=(1)))

then let's insert two examples you provided:

INSERT INTO [JsonTest] (Json) VALUES ('{"person": {
  "name": "1234",
  "age": "99",
  "parameters": {
    "param1": "1",
    "param2": "2"
  }
}}')

INSERT INTO [JsonTest] (Json) VALUES ('{"person": {
  "name": "12345",
  "age": "996",
  "parameters": {
    "param1": "1",
    "param5": "5",
    "param7": "7"
  }
}}')

Now when we query table:

SELECT TOP 100 * FROM [dbo].[JsonTest]

Then we get a result:

Notice, that computed columns work also when there is no such parameter.

Next step is to create indexes on computed columns:

CREATE NONCLUSTERED INDEX [IX_JsonTest_Parameter1] ON [dbo].[JsonTest]
(
    [Parameter1] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)

CREATE NONCLUSTERED INDEX [IX_JsonTest_Parameter2] ON [dbo].[JsonTest]
(
    [Parameter2] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)

And now finally you can query your table very fast:

SELECT [Json], Parameter1, Parameter2
FROM [dbo].[JsonTest]
WITH (INDEX(IX_JsonTest_Parameter1),INDEX(IX_JsonTest_Parameter2))
WHERE Parameter1 = 1 and Parameter2 = 2

This approach queries a table with 2m records in less than 1 second. And have in mind that we only keep json values.

If you want to play with full-text search, you would have to enable it first. Everything is described here: Cannot use a CONTAINS or FREETEXT predicate on table or indexed view because it is not full-text indexed

And now query can look similar to this:

SELECT [Json]
FROM JsonTest
Where Contains(Json,'Near((param1,1), MAX, True)')

score 0 · Answer 2 · answered Oct 01 '17 at 11:20

High search performance on sql server can be obtained by adding computed columns by using ADD col1 AS JSON_VALUE(data,'$.person.parameters.param1') and then indexing these computed columns.

E.g. Index JSON data

(Please provide an example on how you json is stored in a sql table, if we need to provide specific code examples.)

sql server full text search on json

2 Answers2