Four Loops using two ranges to find highest PNL

Question

I am trying to get this code to run faster as it has billions of combinations. I need to look through four loops and based on those parameters find the highest profit. The dictionary could have 500 records and I usually use excel to find patterns of the top performing settings and after a few minutes I end up with about 100 entries. What approach do you guys think its best for me or what recommendations do you have?

using System;
using System.Collections.Concurrent;
using System.Data;
using System.Diagnostics;
using System.Linq;
using System.Windows.Forms;

namespace WindowsFormsApp1 {
    public partial class Form1 : Form {
        public Form1() {
            InitializeComponent();
        }

        public class Stock {
            public double PNL { get; set; }
            public double OD { get; set; }
            public double DR { get; set; }
            public double Volume { get; set; }
            public double Liquidity { get; set; }

            public Stock(double iPNL, double iOD, double iDR, double iVolume, double iLiquidity) {
                PNL = iPNL;
                OD = iOD;
                DR = iDR;
                Volume = iVolume;
                Liquidity = iLiquidity;
            }
        }

        private ConcurrentDictionary<string, Stock> StockData = new();

        private void button1_Click(object sender, EventArgs e) {

            StockData["A"] = new Stock(-109, 60, 0.92, 32, 5.99);
            StockData["B"] = new Stock(41, 25, 0.96, 37, 6.75);
            StockData["C"] = new Stock(41, 62, 1.12, 79, 8.4);


            long iLoops = 0;
            Stopwatch stopwatch = new Stopwatch();
            stopwatch.Start();
            double iODMax = 999;
            double iDRMax = 1500;
            double iVolMax = 999;
            double iLiqMax = 2000;
            double PNL = 0;

            for (int iODMin = 0; iODMin <= iODMax; iODMin += 6) {

                for (double iDRMin = 1; iDRMin <= iDRMax; iDRMin += 6) {

                    for (int iVolMin = 0; iVolMin <= iVolMax; iVolMin += 6) {

                        for (double iLiqMin = 1; iLiqMin <= iLiqMax; iLiqMin += 6) {

                            iLoops += 1;

                            var aaaa = StockData.Where(n => n.Value.OD >= iODMin && n.Value.OD <= iODMax &&
                                                    n.Value.DR >= iDRMin && n.Value.DR <= (iDRMax / 100) &&
                                                    n.Value.Liquidity >= iLiqMin && n.Value.Liquidity <= (iLiqMax / 100) &&
                                                    n.Value.Volume >= iVolMin && n.Value.Volume <= iVolMax).FirstOrDefault();
                            if (aaaa.Value != null) {
                                PNL += aaaa.Value.PNL;
                            }

                        }
                    }
                }
            }

            stopwatch.Stop();

            Debug.Print(stopwatch.Elapsed.ToString() + " | " + iLoops.ToString("#,#") + " | " + PNL.ToString("#,#"));
        }
    }
}

@XK8ER - your data seems not sharing resources, hence you can implement [Parallel.For](https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/how-to-write-a-simple-parallel-for-loop) loop rather than normal for loop. Parallelism makes your performance better. — Pashyant Srivastava, Jun 08 '21 at 04:12
Try not to use LINQ inside of nested loops. Every time you do so a new object is allocated by the framework to capture your local variables. More heap allocations mean more time spent with the Garbage Collector cleaning up. If you're really going for performance use another for or foreach loop to replace the LINQ statement. — Frederik Hoeft, Jun 08 '21 at 07:14

Pashyant Srivastava · Answer 1 · 2021-06-08T06:53:42.313

Here is one of way you can implement Parallelism in your logic which can give you better performance.

public partial class Form1 : Form
{
    public Form1()
    {
        InitializeComponent();
    }
    public class Stock
    {
        public double PNL { get; set; }
        public double OD { get; set; }
        public double DR { get; set; }
        public double Volume { get; set; }
        public double Liquidity { get; set; }

        public Stock(double iPNL, double iOD, double iDR, double iVolume, double iLiquidity)
        {
            PNL = iPNL;
            OD = iOD;
            DR = iDR;
            Volume = iVolume;
            Liquidity = iLiquidity;
        }
    }
    private ConcurrentDictionary<string, Stock> StockData = new ConcurrentDictionary<string, Stock>();
    public IEnumerable<int> SteppedIntegerList(int startIndex,
        double endEndex, int stepSize)
    {
        for (int i = startIndex; i < endEndex; i += stepSize)
        {
            yield return i;
        }
    }
    object lockObject = new object();
    private void button1_Click(object sender, EventArgs e)
    {
        StockData["A"] = new Stock(-109, 60, 0.92, 32, 5.99);
        StockData["B"] = new Stock(41, 25, 0.96, 37, 6.75);
        StockData["C"] = new Stock(41, 62, 1.12, 79, 8.4);


        long iLoops = 0;
        Stopwatch stopwatch = new Stopwatch();
        stopwatch.Start();
        double iODMax = 499;
        double iDRMax = 1000;
        double iVolMax = 499;
        double iLiqMax = 1500;
        double PNL = 0;


        Parallel.ForEach(SteppedIntegerList(0, iODMax, 6), iODMin =>
        {
            Parallel.ForEach(SteppedIntegerList(1, iDRMax, 6), iDRMin =>
            {
                Parallel.ForEach(SteppedIntegerList(0, iVolMax, 6), iVolMin =>
                {
                    Parallel.ForEach(SteppedIntegerList(1, iLiqMax, 6), iLiqMin =>
                    {
                        Interlocked.Increment(ref iLoops);
                        var aaaa = StockData.Where(n => n.Value.OD >= iODMin && n.Value.OD <= iODMax &&
                                                n.Value.DR >= iDRMin && n.Value.DR <= (iDRMax / 100) &&
                                                n.Value.Liquidity >= iLiqMin && n.Value.Liquidity <= (iLiqMax / 100) &&
                                                n.Value.Volume >= iVolMin && n.Value.Volume <= iVolMax).FirstOrDefault();
                        if (aaaa.Value != null)
                        {
                            lock (lockObject)
                            {
                                PNL += aaaa.Value.PNL;
                            }
                        }
                    });
                });
            });
        });
        stopwatch.Stop();
        Debug.Print(stopwatch.Elapsed.ToString() + " | " + iLoops.ToString("#,#") + " | " + PNL.ToString("#,#"));
    }
}

//Output analysis

I have reduced sample data mentioned below to test for my own purpose

double iODMax = 499;
double iDRMax = 1000;
double iVolMax = 499;
double iLiqMax = 1500;

With the above data out is like this :

// the logic you have given : 00:03:44.7069770 | 294,588,000 | 12,628

// new logic with Parallelism : 00:01:13.9397930 | 294,588,000 | 12,628

I need to introduced locking of object as these are shared resource among all the threads, hence you can extend your logic accordingly.

More info on Parallel.For

Could probably use `Interlocked.Add` instead of a lock. – Kit Jun 08 '21 at 22:36 — Kit, Jun 08 '21 at 22:36

Rufus L · Accepted Answer · 2021-06-09T00:02:39.507

Removing linq from the nested for loops should improve performance, because linq adds some additional overhead (see the answers to this question for more info).

This could be combined with the parallelism answer from Pashyant.

for (int iODMin = 0; iODMin <= iODMax; iODMin += 6)
{
    for (double iDRMin = 1; iDRMin <= iDRMax; iDRMin += 6)
    {
        for (int iVolMin = 0; iVolMin <= iVolMax; iVolMin += 6)
        {
            for (double iLiqMin = 1; iLiqMin <= iLiqMax; iLiqMin += 6) 
            {
                iLoops++;

                foreach(var stock in StockData)
                {
                    if (stock.Value.OD >= iODMin && 
                        stock.Value.OD <= iODMax &&
                        stock.Value.DR >= iDRMin && 
                        stock.Value.DR <= iDRMax / 100 &&
                        stock.Value.Liquidity >= iLiqMin && 
                        stock.Value.Liquidity <= iLiqMax / 100 &&
                        stock.Value.Volume >= iVolMin && 
                        stock.Value.Volume <= iVolMax)
                    {
                        PNL += stock.Value.PNL;
                        break;
                    }
                }
            }
        }
    }
}

Also, if there is any pattern in your data that suggests some of the conditions in the if statement are more likely to be false than others, put those first (since the conditional evaluation happens from first to last).

Four Loops using two ranges to find highest PNL

2 Answers2

//Output analysis