Faster KV vs dictionary vs concurrent dictionary which one to use when only updating value of dictionary

Question

Streaming live prices for stocks, each stock is key (int) and its value type is struct (not class) Dictionary<int,Tick>. This dictionary is initialized once with all keys, and updating (no new additions) values every millisecond randomly from websocket callback. From another thread this dictionary is continuously iterating every 20 milliseconds for latest values. Size of the dictionary is 5000 keys. So dictionary are not thread safe, ConcurrentDictionary has little performance overhead. So is it better to go for Microsoft FASTER KV?

Will normal dictionary is thread safe if we don't add new values, just updating existing key value from one thread and reading from another thread is safe? in the below code block "Point A" and "Point B" will have same value?

 /// <summary>
/// this struct from 3rdpary library
/// </summary>
public struct Tick
{
    public DateTime DateTime { get; set; }
    public uint InstrumentToken { get; set; }
    public decimal LastPrice { get; set; }
}

class TickDataProcessor
{
    private Dictionary<UInt32, Tick> AllStocksTickData { get; set; }
    private Dictionary<UInt32, Tick> AllPreviousStocksTickData { get; set; }


    private void Initialize()
    {
        AllPreviousStocksTickData = new Dictionary<uint, Tick>(capacity: 5000);
        AllPreviousStocksTickData = new Dictionary<uint, Tick>(capacity: 5000);

        // initialize dict with all empty values, after this no more additions
        UInt32 key = 1000001;
        for (int i = 0; i < 5000; i++)
        {
            AllStocksTickData[key] = new Tick();
            AllPreviousStocksTickData[key] = new Tick();
            key++;
        }
    }

    /// <summary>
    ///real time streaming of stock data from stock broker
    /// this method is call back of websocket, websocket callback comes every millisecond for random key
    /// it will update the value of that key.
    /// </summary>
    /// <param name="tickData"></param>
    public void OnNewTickData(Tick tickData)
    {
        AllStocksTickData[tickData.InstrumentToken] = tickData;
    }

    public void Start()
    {
        var ts = new ThreadStart(OneBackgroundMethodStockPriceAnalyzer);
        var backgroundThread = new Thread(ts);
        backgroundThread.Start();

        var ts2 = new ThreadStart(TwoBackgroundMethodStockPriceAnalyzer);
        var backgroundThread2 = new Thread(ts2);
        backgroundThread2.Start();
    }

    private void TwoBackgroundMethodStockPriceAnalyzer()
    {
        while (true)
        {
            foreach (var entry in AllStocksTickData)
            {
                // second thread
            }
        }
    }

  private  void OneBackgroundMethodStockPriceAnalyzer()
    {
        while (true)
        {
            foreach (var entry in AllStocksTickData)
            {
                var key = entry.Key;
                var tickData = entry.Value; // Point A
                /*
                 * process tickdata
                 *
                 */
                if (AllPreviousStocksTickData[key].LastPrice >= tickData.LastPrice)
                {
                    // calculate % of change
                    // other calculations...
                }

                AllPreviousStocksTickData[key] = entry.Value; // Point B
            }
            Thread.Sleep(20);
        }
    }
}

class Program

{
    public static void Main(string[] args)
    {
        var tickDataProcessor = new TickDataProcessor();
        tickDataProcessor.Start();
        Console.ReadLine();
    }
}

Which one to use when only updating value of dictionary considering high performance.

FASTER KV Quick-Start Guide

The [`FasterKV`](https://microsoft.github.io/FASTER/docs/fasterkv-basics/) looks like a highly sophisticated component with numerous configuration options, with which not many people are familiar. My feeling is that the info provided in the question might not be sufficient for a meaningful comparison. Choosing between this and a `ConcurrentDictionary` might require detailed info about the usage pattern, like frequency of updates, frequency of enumerations, and number of threads. The more metrics that you can provide, the better. — Theodor Zoulias, Jul 03 '22 at 06:45

Theodor Zoulias · Answer 1 · 2022-07-02T00:21:11.027

Will normal dictionary is thread safe if we don't add new values, just updating existing key value from one thread and reading from another thread is safe?

I'll address this part of the question only. No, using a normal Dictionary<int, decimal> in a multithreaded environment, where the number of keys is fixed and the values can be updated, is definitely not thread-safe. That's because updating a decimal is not an atomic operation, so a decimal value can be torn. You can find an experimental demonstration of this phenomenon here: Reproduce torn reads of decimal in C#.

Faster KV vs dictionary vs concurrent dictionary which one to use when only updating value of dictionary

1 Answers1