Preamble:
- All data connection strings, connections, etc are created using DbProviderFactories.
- Code is mixed C# and VB.Net from mulitple libraries.
I am mapping a DbDataReader to entities and have some benchmarks:
[0] retrieved 159180 records in 45135 ms
[1] retrieved 159180 records in 45008 ms
[2] retrieved 159180 records in 44814 ms
[3] retrieved 159180 records in 44987 ms
[4] retrieved 159180 records in 44914 ms
[5] retrieved 159180 records in 45224 ms
[6] retrieved 159180 records in 45829 ms
[7] retrieved 159180 records in 60762 ms
[8] retrieved 159180 records in 52128 ms
[9] retrieved 159180 records in 47982 ms
This is a significant amount of time and extremely poor considering it only takes 17 seconds to query from Sql Server Management Studio. My select statement:
"SELECT * FROM tbl_MyTable"
Table contains 43 fields and probably isn't indexed as best as it should; however, performing a select all, I wouldn't expect indexing to be problematic. So ... here is what I am doing:
Define an entity:
public class Concept
{
#region Columns
[DataParameter("ConceptID", DbType.Int32)]
public Int32 ConceptID
{ get; set; }
[DataParameter("ConceptName", DbType.String)]
public string ConceptName
{ get; set; }
[DataParameter("ConceptTypeID", DbType.Int32)]
public Int32 ConceptTypeID
{ get; set; }
[DataParameter("ActiveYN", DbType.Boolean)]
public bool ActiveYN
{ get; set; }
#endregion
}
Query DataReader:
for (int i = 0; i <= 99; i++)
{
sw.Start();
var results = session.QueryReader<Concept>(
new SqlCommand(command), dr => new Concept());
sw.Stop();
Console.WriteLine("[{0}] retrieved {1} records in {2} ms", i, results.Count(), sw.ElapsedMilliseconds);
sw.Reset();
}
... calling:
Public Function QueryReader(Of TEntity As {Class, New})(ByVal Command As DbCommand, _
ByVal Projection As Func(Of DbDataReader, TEntity)) _
As IEnumerable(Of TEntity)
Dim list As IEnumerable(Of TEntity)
Command.Connection = dataReader.NewConnection
Command.Connection.Open()
Using _reader As DbDataReader = Command.ExecuteReader()
list = _reader.Query(Of TEntity)(Projection).ToList()
End Using
Command.Connection.Close()
Return list
End Function
... and extension method QueryReader<T>
: edit placement of new TEntity() - thanks @Henk
public static IEnumerable<TEntity> Query<TEntity>(this DbDataReader Reader,
Func<DbDataReader, TEntity> Projection)
where TEntity : class, new()
{
// moving this reflection to another class
Dictionary<string, PropertyInfo> props;
while (Reader.Read())
{
TEntity entity = new TEntity();
if (!entities.TryGetValue(typeof(TEntity).ToString(), out props))
{
// reflection over TEntity
props = (from p in entity.GetType().GetProperties()
from a in p.GetCustomAttributes(typeof(DataParameterAttribute), false)
select p)
.ToDictionary(p => p.Name);
entities.Add(typeof(TEntity).ToString(), props);
}
foreach (KeyValuePair<string, PropertyInfo> field in props)
{
if (null != Reader[field.Key] && Reader[field.Key] != DBNull.Value)
{ field.Value.SetValue(entity, Reader[field.Key], null); }
}
yield return entity;
}
}
Any suggestions on increasing performance would be greatly appreciated ...
Update
I implemented dapper-dot-net as @EtienneT suggested - here are the retrieval times:
[0] retrieved 159180 records in 6874 ms
[1] retrieved 159180 records in 6866 ms
[2] retrieved 159180 records in 6570 ms
[3] retrieved 159180 records in 6785 ms
[4] retrieved 159180 records in 6693 ms
[5] retrieved 159180 records in 6735 ms
[6] retrieved 159180 records in 6627 ms
[7] retrieved 159180 records in 6739 ms
[8] retrieved 159180 records in 6569 ms
[9] retrieved 159180 records in 6666 ms