1

I want to retrieve users form User table where

  1. Users are not deleted
  2. Users who have user id in provided list of user ids
  3. Users who have team id in provided list of team ids
  4. Users who have role id and team id in provided tuple list of role and team ids

For that, I'm writing stored procedure with three table-valued parameters like below:

create type RoleTeamTableType as table
    ( 
    RoleId bigint,
    TeamId bigint 
    );  
go 

create type UserTableType as table
    ( 
    UserId bigint
    );  
go  

create type TeamTableType as table
    ( 
    TeamId bigint
    );  
go

create proc spCurrentUsersInRolesAndTeams
@roleTeamTableType RoleTeamTableType readonly,
@userTableType UserTableType readonly,
@teamTableType TeamTableType readonly
as
set nocount on

select * from Users as U where U.Deleted<>1 and
                        (
                            exists 
                            (select * from @roleTeamTableType as V
                            where (
                                    (V.TeamId <> 0 and V.TeamId = U.TeamId and V.RoleId = U.RoleId) or
                                    (V.TeamId = 0 and V.RoleId = U.RoleId)
                                  )) /*if team id is 0 then only check for role id match else match both*/

                            or exists 
                            (select * from @userTableType as W
                            where ( W.UserId = U.Id))

                            or exists 
                            (select * from @teamTableType as X
                            where ( X.TeamId = U.TeamId))
                        )
return                             
go



 /*Usage*/

declare @roleTeamTableType as RoleTeamTableType;  
insert into @roleTeamTableType (RoleId, TeamId)  
    values (1,0), (1,1), (2,1)

declare @userTableType as UserTableType 
insert into @userTableType (UserId) values (1),(2),(3)


declare @teamTableType as TeamTableType 
insert into @teamTableType (TeamId) values (1),(2),(3)


exec spCurrentUsersInRolesAndTeams @roleTeamTableType, @userTableType, @teamTableType

While the above code works, I have following questions:

  1. Considering, I'm not an expert in writing efficient SQL queries, is this the right way of achieving what I want, or is there any better way?
  2. I've written LINQ to Entities version of this query using EF but I want to optimize that by calling this stored procedure instead. So, how do I create table valued parameters in .Net code?
  3. Consider the scenario where the provided table valued parameter lists contain thousands of values lets say 100000, it will surely degrade the performance. What is the preferred and efficient way to get table rows by providing list of ids?

Update

Comments below suggest to add problematic query, so here it is

 public IEnumerable<long> GetCurrentUserIds(IEnumerable<long> userIds, IEnumerable<Tuple<long, long>> roleTeamIds, IEnumerable<long> teamIds)
        {
            //setting timeout to avoid timeout exceptions
            Context.Database.CommandTimeout = 180;

            var users = Context.Users.AsNoTracking().Where(user => !user.Deleted);

            //following loop is only for testing with userIds1
            //working at i = 30,000

            var userIds1 = new List<long>();
            for (long i = 1; i < 30000; i++)
                userIds1.Add(i);

            //at i = 35,000
            //Internal error: An expression services limit has been reached. Please look for potentially complex expressions in your query, and try to simplify them.

            //at i =  40,000
            //"The query processor ran out of internal resources and could not produce a query plan. 
            //This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. 
            //Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information."

            var result = users.Where(user => (userIds1.Contains(user.Id) ||
                                            (user.TeamId.HasValue && teamIds.Contains(user.TeamId.Value)))).Select(u=>u.Id);

            foreach (var roleTeamId in roleTeamIds)
            {
                var filtered = users.Where(user => (roleTeamId.Item2 != 0 && roleTeamId.Item2 == user.TeamId && roleTeamId.Item1 == user.RoleId) ||
                                            (roleTeamId.Item2 == 0 && roleTeamId.Item1 == user.RoleId)).Select(u => u.Id);
                result = result.Concat(filtered);
            }

            return result.Distinct().ToList();
        }

SQL Profiler query resulted with parameter values of

userIds: 6,7

roleTeamIds: (4,1), (6,1)

teamIds: 0

exec sp_executesql N'SELECT 
    [Distinct1].[C1] AS [C1]
    FROM ( SELECT DISTINCT 
        [UnionAll2].[Id] AS [C1]
        FROM  (SELECT 
            [Extent1].[Id] AS [Id]
            FROM [dbo].[Users] AS [Extent1]
            WHERE ([Extent1].[Deleted] <> 1) AND ([Extent1].[Id] IN (cast(6 as bigint), cast(11 as bigint)))
        UNION ALL
            SELECT 
            [Extent2].[Id] AS [Id]
            FROM [dbo].[Users] AS [Extent2]
            WHERE ([Extent2].[Deleted] <> 1) AND (((0 <> @p__linq__0) AND (@p__linq__1 = [Extent2].[TeamId]) AND (@p__linq__2 = [Extent2].[RoleId])) OR ((0 = @p__linq__3) AND (@p__linq__4 = [Extent2].[RoleId])))
        UNION ALL
            SELECT 
            [Extent3].[Id] AS [Id]
            FROM [dbo].[Users] AS [Extent3]
            WHERE ([Extent3].[Deleted] <> 1) AND (((0 <> @p__linq__5) AND (@p__linq__6 = [Extent3].[TeamId]) AND (@p__linq__7 = [Extent3].[RoleId])) OR ((0 = @p__linq__8) AND (@p__linq__9 = [Extent3].[RoleId])))) AS [UnionAll2]
    )  AS [Distinct1]',N'@p__linq__0 bigint,@p__linq__1 bigint,@p__linq__2 bigint,@p__linq__3 bigint,@p__linq__4 bigint,@p__linq__5 bigint,@p__linq__6 bigint,@p__linq__7 bigint,@p__linq__8 bigint,@p__linq__9 bigint',@p__linq__0=1,@p__linq__1=1,@p__linq__2=4,@p__linq__3=1,@p__linq__4=4,@p__linq__5=1,@p__linq__6=1,@p__linq__7=6,@p__linq__8=1,@p__linq__9=6
Saad
  • 198
  • 2
  • 12
  • 1
    Your question is kind of all over the place here. I would probably use a more generic table type for the first two since they are essentially the same thing. They have a single column of type bigint. That isn't a performance issue, just more stuff to maintain. I would probably make a few changes to the procedure to make it easier to understand but nothing major. For creating tvps in dotnet I suggest using a search engine. There are thousands of examples. And #3 will be answered as you do some research for #2. – Sean Lange May 11 '18 at 13:38
  • If you have a performance issue in fetching 3 users by id you have a problem a stored procedure is not going to solve. To me this sounds like premature optimization. Anyway, you question far too broad. – Gert Arnold May 12 '18 at 21:31
  • @GertArnold It was just an example. The 3rd point clears that out, what if there are 100,000 items in the list? It is not premature optimization as Linq to entities does not event send command to SQL server if my user ids are lets say 600,000 or even less. I've checked it in Profiler. The Profiler even crashes while displaying such a query with only 4000 user ids. I need to design this to work with said user ids. I've figured out my question number 2 so you can omit that part if that is too broad. Thanks! – Saad May 13 '18 at 07:35
  • Is that a realistic scenario? What do you need that kind of amounts for? As for being too broad, you better narrow down the question yourself. Focus on the EF part. Show the queries you tried and indicate where the shoe pinches. The outcome may be: use a stored procedure, but *how* to do this effectively using a stored procedure is a different question. – Gert Arnold May 13 '18 at 08:12
  • @GertArnold updated my question with EF query. Irrespective of scenario, I'm just interested in finding out whats wrong with my query. – Saad May 13 '18 at 13:16
  • OK, first thing to do is: build [one query with predicates](https://stackoverflow.com/a/14622200/861716) instead of the `Union`s. Also, the scenario does matter. If you have to fetch large numbers of users you probably shouldn't use EF at all. For small numbers (like in regular applications' authorization checks) it's OK. – Gert Arnold May 13 '18 at 14:20
  • Thanks, the library surely is a nice addition. The query at Sql end is now more appropriate and optimized. However, it did not solve my problem. With more than 30000 ids, EF is still not able to send the command to Sql, may be EF has some limit while emitting expressions. And you are right, that's why I've written the stored procedure in the first place. I'll probably use the stored procedure but I really hope to find out whats actually going wrong with EF way. – Saad May 13 '18 at 17:46

0 Answers0