VB optimize loop avoid with an Evaluate()

Question

I am running a loop that evaluates input given by the user over more than 150k objects. The user sets the info to read like "obj.Name", "obj.Path", "obj.Date"... which may contain also logic evaluations such as "IIf(obj.Params>5,1,0)". It is then provided into the programm as a string.

I used the Evaluate() functions and it does work well, just that it is slow. It takes almost 6h to go over all elements. I was thinking if there is a way in which I can take the requested info and turn it into a straight-forward executable somehow, and avoid using the Evaluate expression in the whole loop (it runs for the number of requested data by the user * 150k).

This is a schematic of the loop I am running:

For Each Object in ObjectsList
    For Each UserRequest in Requests
        ResultsMatrix(i,j) = Evaluate(Requests(i))
        j += 1
    Next
    i += 1
Next

I store then the results in a Matrix which is pasted in an Excel file at the end. Is there a way in which I can do sort of working the string to be evaluated into a function's return? I'd like to avoid usig the Eval function and parse directly the string in an executable and dont evaluate it for each object. Any tips on speeding up the loop?

If the same request happen multiple times, you could store the value instead of evaluating it again. — the_lotus, Jul 19 '18 at 13:51
The thing that is the most important as far as the code goes is what is computed in the `Evaluate(...)` call, but that's missing. We need a [mcve]. — Enigmativity, Jul 19 '18 at 13:52
https://stackoverflow.com/q/1707854/11683 sends you in the right direction, but that fact you are using VB for the code will probably limit your choices. — GSerg, Jul 19 '18 at 13:55
I'm voting to close this question as off-topic because it is asking for a review. Please send this over to [Code Review](https://codereview.stackexchange.com/) and then close this question. — Trevor, Jul 19 '18 at 16:06

score 1 · Answer 1 · answered Jul 20 '18 at 14:02

It might be worth considering writing the requests into a set of functions and using the .NET CodeDom compilers to build it into a DLL. You can then load the assembly, find the right functions using reflection and put them into an array, then call them using reflection - that way you'll be calling .NET code and it should be far faster. Some (incomplete) code to get you started from a project where I have done this...

        Private Function CombineCode() As String

            Dim ret As New System.Text.StringBuilder
            ret.AppendLine("Imports System")
            ret.AppendLine("Imports Microsoft.VisualBasic")
            ret.AppendLine()
            ret.AppendLine("Namespace " & MainNamespace)
            ret.AppendLine("Public Class " & MainClassName)

            For Each e In _Entries
                ret.AppendLine(e.Value.Code)
            Next

            ret.AppendLine("End Class")
            ret.AppendLine("End Namespace")

            Return ret.ToString
        End Function

        Private Function Compile(Code As String) As Assembly
            'Dim d As New Dictionary(Of String, String)
            'd.Add("langversion", "14")
            Dim VBP As New Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider()

            Dim PM As New System.CodeDom.Compiler.CompilerParameters
            'PM.GenerateInMemory = True
            PM.GenerateExecutable = False
            PM.OutputAssembly = IO.Path.Combine(_Path, GenerateFileName() & ".dll") ' "Generated.dll"
            PM.MainClass = MainClassName
            PM.IncludeDebugInformation = True


            Dim ASM As System.Reflection.Assembly
            For Each ASM In AppDomain.CurrentDomain.GetAssemblies()
                Try
                    If ASM.Location <> "" Then PM.ReferencedAssemblies.Add(ASM.Location)
                Catch
                End Try
            Next

            PM.ReferencedAssemblies.Add("System.Web.dll")

            'Get compilation results
            Dim Results As System.CodeDom.Compiler.CompilerResults
            Results = VBP.CompileAssemblyFromSource(PM, Code)

            'Show possible compilation errors
            Dim Err As System.CodeDom.Compiler.CompilerError
            For Each Err In Results.Errors
                Throw New SyntaxErrorException("Error N. " & Err.ErrorNumber &
            " Message: " & Err.ErrorText & " Line " & Err.Line & " in code " & vbCrLf & Code)
            Next

            Return Results.CompiledAssembly

        End Function
        Private Sub FindMethods()
            Dim dt = (From t In _LatestAssembly.GetTypes() Where t.Name = MainClassName).Single
            For Each e In _Entries.Values
                e.Method = dt.GetMethod(e.MethodName)
            Next
        End Sub


Assembly = Assembly.LoadFrom(System.IO.Path.Combine(Path, sd.LatestAssemblyFile))

score 0 · Answer 2 · answered Jul 19 '18 at 17:23

The Evaluate function is just resources on the computer itself. It's a great candidate for using Parallel.For.

In this case, j is the implied index.

For Each Object in ObjectsList

    Parallel.For(0, Requests.Length, New ParallelOptions(), Sub(j, loopState)
           ResultsMatrix(i,j) = Evaluate(Requests(j))
        End Sub
    )
    i += 1
Next

Note, that Requests(i) is getting called repeatedly and produces the same result, so I assume you mean Requests(j).

VB optimize loop avoid with an Evaluate()

2 Answers2