As of now, this is still a problem in Entity Framework Core when using the SQL Server Database Provider.
Still on Entity Framework 6 (non-core)? skip to the next section.
I wrote QueryableValues to solve this problem in a flexible and performant way; with it you can compose the values from an IEnumerable<T> in your query, like if it were another entity in your DbContext.
In contrast to other solutions out there, QueryableValues achieves this level of performance by:
- Resolving with a single round-trip to the database.
- Preserving the query's execution plan regardless of the provided values.
Usage example:
// Sample values.
IEnumerable<int> values = Enumerable.Range(1, 10);
// Using a Join.
var myQuery1 =
from e in dbContext.MyEntities
join v in dbContext.AsQueryableValues(values) on e.Id equals v
select new
{
e.Id,
e.Name
};
// Using Contains.
var myQuery2 =
from e in dbContext.MyEntities
where dbContext.AsQueryableValues(values).Contains(e.Id)
select new
{
e.Id,
e.Name
};
You can also compose complex types!
It's available as a nuget package and the project can be found here. It's distributed under the MIT license.
The benchmarks speak for themselves.
An Alternative for Entity Framework 6 (non-core)
NEW! QueryableValues EF6 Edition has arrived!
I'll explain how to manually provide some of the functionality of QueryableValues on this legacy version of Entity Framework, specifically, the ability to compose an IEnumerable<int> with any of your entities in the same way that QueryableValues does on EF Core. You can use this same technique to support collections of other simple types like long, string, etc.
Requirements
- Must use the SQL Server provider
- Must use the database-first strategy OR you already have a way to map a TVF using the code-first strategy
Instructions Summary
- Create a method that takes an
IEnumerable<int> and returns XML.
- Create a TVF in your database that takes XML and returns a rowset.
- Add the TVF to the EDMX using the designer.
- Encapsulate the code that glues the functions created on step 1 and 2 and return an
IQueryable<int>.
- Use the
IQueryable<int> in your queries as desired.
Instructions
1. Create a method that takes a IEnumerable<int> and returns XML
This method will serialize the provided values as XML, so later on it can be transmitted as a parameter in your query.
static string GetXml<T>(IEnumerable<T> values)
{
var sb = new StringBuilder();
using (var stringWriter = new System.IO.StringWriter(sb))
{
var settings = new System.Xml.XmlWriterSettings
{
ConformanceLevel = System.Xml.ConformanceLevel.Fragment
};
using (var xmlWriter = System.Xml.XmlWriter.Create(stringWriter, settings))
{
xmlWriter.WriteStartElement("R");
foreach (var value in values)
{
xmlWriter.WriteStartElement("V");
xmlWriter.WriteValue(value);
xmlWriter.WriteEndElement();
}
xmlWriter.WriteEndElement();
}
}
return sb.ToString();
}
If the above method is provided with new[] { 1, 2, 3 }, it will return a XML string with the following structure:
<R><V>1</V><V>2</V><V>3</V></R>
2. Create a TVF in your database that takes XML and returns a rowset
The following table-valued function (TVF) will take the XML created by the previous function and project it as a rowset with a single column (V), that can then be used from SQL Server's side in your query. Must be created in the database associated with your EDMX file, so it can be added to your EDMX model in the next step.
CREATE FUNCTION dbo.udf_GetIntValuesFromXml
(
@Values XML
)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN
(
SELECT I.value('. cast as xs:integer?', 'int') AS V
FROM @Values.nodes('/R/V') N(I)
)
The above function when provided with the <R><V>1</V><V>2</V><V>3</V></R> XML, will return the following rowset:
3. Add the TVF to the EDMX using the designer
Table-Valued Functions (TVFs) - EF Docs
After adding this function to your EDMX model, ensure to save the changes to the EDMX file so that your DbContext generated code is up to date.
4. Encapsulate the code that glues the functions created on step 1 and 2 and return an IQueryable<int>
The following code encapsulates the XML serializer function explained above and everything else you need on the .NET side to make this work:
using System.Collections.Generic;
using System.Linq;
public static class QueryableValuesClassicDbContextExtensions
{
private static string GetXml<T>(IEnumerable<T> values)
{
var sb = new StringBuilder();
using (var stringWriter = new System.IO.StringWriter(sb))
{
var settings = new System.Xml.XmlWriterSettings
{
ConformanceLevel = System.Xml.ConformanceLevel.Fragment
};
using (var xmlWriter = System.Xml.XmlWriter.Create(stringWriter, settings))
{
xmlWriter.WriteStartElement("R");
foreach (var value in values)
{
xmlWriter.WriteStartElement("V");
xmlWriter.WriteValue(value);
xmlWriter.WriteEndElement();
}
xmlWriter.WriteEndElement();
}
}
return sb.ToString();
}
public static IQueryable<int> AsQueryableValues(this IQueryableValuesClassicDbContext dbContext, IEnumerable<int> values)
{
return dbContext.GetIntValuesFromXml(GetXml(values));
}
}
public interface IQueryableValuesClassicDbContext
{
IQueryable<int> GetIntValuesFromXml(string xml);
}
The IQueryableValuesClassicDbContext interface is intended to be explicitly implemented on your DbContext class to provide access to the TVF that was added to the EDMX model.
You can do this by creating a partial class for your DbContext. For example, if your DbContext name is TestDbContext:
using System.Linq;
partial class TestDbContext : IQueryableValuesClassicDbContext
{
IQueryable<int> IQueryableValuesClassicDbContext.GetIntValuesFromXml(string xml)
{
return udf_GetIntValuesFromXml(xml).Select(i => i.Value);
}
}
5. Use the IQueryable<int> in your queries as desired (via AsQueryableValues)
using (var db = new TestDbContext())
{
var valuesQuery = db.AsQueryableValues(new[] { 1, 2, 3, 4, 5 });
var resultsUsingContains = db.MyEntity
.Where(i => valuesQuery.Contains(i.MyEntityID))
.Select(i => new { i.MyEntityID, i.PropA })
.ToList();
var resultsUsingJoin = (
from i in db.MyEntity
join v in valuesQuery on i.MyEntityID equals v
select new { i.MyEntityID, i.PropA }
)
.ToList();
}
Below is the T-SQL generated behind the scenes for the above EF queries. As you can see, it's completely parameterized.
exec sp_executesql N'SELECT
[Extent1].[MyEntityID] AS [MyEntityID],
[Extent1].[PropA] AS [PropA]
FROM [dbo].[MyEntity] AS [Extent1]
WHERE EXISTS (SELECT
1 AS [C1]
FROM [dbo].[udf_GetIntValuesFromXml](@Values) AS [Extent2]
WHERE ([Extent2].[V] = [Extent1].[MyEntityID]) AND ([Extent2].[V] IS NOT NULL)
)',N'@Values nvarchar(4000)',@Values=N'<R><V>1</V><V>2</V><V>3</V><V>4</V><V>5</V></R>'
exec sp_executesql N'SELECT
[Extent1].[MyEntityID] AS [MyEntityID],
[Extent1].[PropA] AS [PropA]
FROM [dbo].[MyEntity] AS [Extent1]
INNER JOIN [dbo].[udf_GetIntValuesFromXml](@Values) AS [Extent2] ON [Extent1].[MyEntityID] = [Extent2].[V]',N'@Values nvarchar(4000)',@Values=N'<R><V>1</V><V>2</V><V>3</V><V>4</V><V>5</V></R>'
Limitations
- The provided
IEnumerable<int> is enumerated at query build time, not at execution time.
- The final query cannot reference more than one
IQueryable<T> returned by the AsQueryableValues extension method. This is another limitation around composing the same TVF more than once. EF will create two parameters with the same name, which is illegal and you will get the following error:
A parameter named 'Values' already exists in the parameter collection. Parameter names must be unique in the parameter collection.
- Incorrect type used for the XML type parameter of the TVF (notice the use of
nvarchar instead of xml in the T-SQL above). This is a deficiency in the EF infrastructure (ObjectParameter) that's used to compose the TVF. Not using the correct parameter type has a detrimental effect in performance due to the implicit casting that must be done by SQL Server.
Conclusion
Despite the limitations, this is still a robust solution when compared to not using parameterized T-SQL queries. To understand the underlying issue that this mitigates you can continue reading here.
Legal Stuff
Feel free to use the code and examples above as you wish. I'm releasing it under the MIT license:
MIT License
Copyright (c) Carlos Villegas (yv989c)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
