EF Linq- Dynamic Lambda expression trees

Question

I have a generic repository that uses generic expressions for returning data from Entity Framework Core.

public async Task<T2> GetFieldsAsync<T2>(Expression<Func<T, T2>> expression)
  {
            return await context.Set<T>()
                                .Select(expression)
                                .FirstOrDefaultAsync();
  }

Now if I want to select specific fields, at compile time I can write the following statement:

var test = await _repositoryBase.GetFieldsAsync(x => new { x.Id, x.Name });

I want to be able to do above at run time instead. I can create an expression at run time that returns a single parameter as follows:

var expression = Expression.Parameter(typeof(Ingredient), "ingr");
var param1 = Expression.Property(expression, "Id");
var lambda = Expression.Lambda<Func<Ingredient, Guid>>(param1, expression);
var test = await _repositoryBase.GetFieldsAsync(lambda);

The above lambda only returns a single property from the Ingredient class. Is it possible to create a run time lambda that returns an anonymous object using expression trees? I.e.

x => new { x.Id, x.Name }

Note that users might request different fields (e.g. Name, Description, DateCreated, etc.) so need to dynamically create the lambda expression.

I know I can use https://github.com/StefH/System.Linq.Dynamic.Core to pass in strings to select statements via its built-in IQueryable extension methods. I am wondering if there is a way to dynamically select specific fields at run time via a list of fields passed in by the user.

---

Currently, my approach is to get all properties of the class from the database, and use an ExpandoObject to only select the fields that user requests. The issue here is that although it works, I am returning more data from the database than is required. I'd like to avoid this by only selecting the required data.

//user did not provide any fields so include all fields
    if (string.IsNullOrEmpty(field))
     {
         myPropertyInfoList.AddRange(typeProperties);
     }
    else
      {
        foreach (var item in fields)
        {
            myPropertyInfoList.Add(type.GetProperty(item, BindingFlags.IgnoreCase | BindingFlags.Public | BindingFlags.Instance));
        }

    var expandoObj = (IDictionary<string, object>)new ExpandoObject();

        foreach (var item in myPropertyInfoList)
        {
            expandoObj.Add(item.Name, item.GetValue(ingrView));
        }

Answer 1

This is a simplified anonymous type creator. It uses public fields instead of building properties, it has no methods implemented (it gets a default constructor).

First, a simple extension method I use later:

public static class StringExt {
    public static string Join(this IEnumerable<string> strings, string sep) => String.Join(sep, strings);    
}

Here, the code to create a new anonymous type from a Dictionary<string,Type> that describes the anonymous type. A generic type with type parameters for the type of every field (property) is generated and then fixed to the actual types being used - which is how the C# compiler generates anonymous types. This code creates (once) a dynamic Assembly and Module, then adds new types as needed. It uses a cache to (attempt) to prevent creating the same type more than once.

public static class AnonymousExt {
    private static readonly CustomAttributeBuilder CompilerGeneratedAttributeBuilder = new CustomAttributeBuilder(typeof(CompilerGeneratedAttribute).GetConstructor(Type.EmptyTypes), new object[0]);
    private static ModuleBuilder AnonTypeMB;
    private static int AssemCount = 25;

    // create a pseudo anonymous type (POCO) from an IDictionary of property names and values
    // using public fields instead of properties
    // no methods are defined on the type
    public static Type MakeAnonymousType(IDictionary<string, Type> objDict) {
        // find or create AssemblyBuilder for dynamic assembly
        if (AnonTypeMB == null) {
            var assemblyName = new AssemblyName($"MyDynamicAssembly{AssemCount}");
            var ab = AppDomain.CurrentDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run);
            AnonTypeMB = ab.DefineDynamicModule("MyDynamicModule");
        }
        // get a dynamic TypeBuilder
        var typeBuilder = AnonTypeMB.DefineType($"<>f__AnonymousType{AssemCount++}`{objDict.Keys.Count}", TypeAttributes.AnsiClass | TypeAttributes.Class | TypeAttributes.AutoLayout | TypeAttributes.NotPublic | TypeAttributes.Sealed | TypeAttributes.BeforeFieldInit);
        typeBuilder.SetCustomAttribute(CompilerGeneratedAttributeBuilder);

        // create generic parameters for every field
        string gtpName(string fieldName) => $"<{fieldName}>j__TPar";
        var gtpnames = objDict.Keys.Select(k => gtpName(k)).ToArray();
        var gtpbs = typeBuilder.DefineGenericParameters(gtpnames);
        var gtpN2Bs = gtpnames.Zip(gtpbs, (n, pb) => new { n, pb }).ToDictionary(g => g.n, g => g.pb);

        // add public fields to match the source object
        var fbs = new List<FieldBuilder>();
        foreach (var srcFieldName in objDict.Keys)
            fbs.Add(typeBuilder.DefineField(srcFieldName, gtpN2Bs[gtpName(srcFieldName)], FieldAttributes.Public));

        // Fix the generic class
        var fieldTypes = objDict.Values.ToArray();        
        return typeBuilder.CreateType().MakeGenericType(fieldTypes);
    }

    static string MakeAnonymousTypeKey(IDictionary<string, Type> objDict) => objDict.Select(d => $"{d.Key}~{d.Value}").Join("|");

    public static Dictionary<string, Type> PrevAnonTypes = new Dictionary<string, Type>();
    public static Type FindOrMakeAnonymousType(IDictionary<string, Type> objDict) {
        var wantedKey = MakeAnonymousTypeKey(objDict);
        if (!PrevAnonTypes.TryGetValue(wantedKey, out var newType)) {
            newType = MakeAnonymousType(objDict);
            PrevAnonTypes[wantedKey] = newType;
        }

        return newType;
    }    
}

And here is some sample code using it with a SQL Table named Accounts which has a class type named Accounts. Because my anonymous type doesn't have a constructor that takes the field values, I use a MemberInitExpression instead of the normal NewExpression (which seems unnecessarily complicated anyway?).

var objDescr = new Dictionary<string, Type> { { "Actid", typeof(Int32) }, { "Application", typeof(string) }, { "Username", typeof(string) }};
var aType = AnonymousExt.FindOrMakeAnonymousType(objDescr);

var parma = Expression.Parameter(typeof(Accounts), "a");
var fxbe = aType.GetFields().Select(fi => Expression.Bind(fi, Expression.Field(parma, fi.Name))).ToArray();
var mie = Expression.MemberInit(Expression.New(aType), fxbe);
var myf = Expression.Lambda<Func<Accounts, object>>(mie, parma);

var ans = Accounts.Select(myf).Take(2);

Answer 2

Anonymous types are just types that C# builds for you at compile time. Instead of trying to build an anonymous type, how about returning an object[] for each row of your generated query. As a bonus, that reduces the complexity of dealing with the returned data.

List<string> properties = ... ;
var parameter = Expression.Parameter(typeof(T), "e");

var selectExpression = Expression.Lambda<Func<T, object[]>>(
    Expression.NewArrayInit(
        typeof(object),
        properties.Select(p =>
        {
            var ret = Expression.Property(parameter, p);
            if (ret.Type != typeof(object))
                ret = Expression.Convert(ret, typeof(object));
            return ret;
        })
    ),
    parameter);