Managing timeseries in c#

Question

I wanted to have your opinion on what is the best way to manage time series in c# according to you. I need to have a 2 dimensions matrix-like with Datetime object as an index of rows (ordered and without duplicate) and each columns would represent the stock value for the relevant Datetime. I would like to know if any of those objects would be able to handle missing data for a date: adding a column or a time serie would add the missing date in the row index and would add "null" or "N/a" for missing values for existing dates.

A lot of stuff are already available in c# compared to c++ and I don't want to miss something obvious.

Answer 1

TeaFiles.Net is a library for time series storage in flat files. As I understand you only want to have the data in memory, in which case you would use a MemoryStream and pass it to the ctor.

// the time series item type
struct Tick
{
    public DateTime Time;
    public double Price;
    public int Volume;
}

// create file and write some values
var ms = new MemoryStream();
using (var tf = TeaFile<Tick>.Create(ms))
{
    tf.Write(new Tick { Price = 5, Time = DateTime.Now, Volume = 700 });
    tf.Write(new Tick { Price = 15, Time = DateTime.Now.AddHours(1), Volume = 1700 });
    // ...
}

ms.Position = 0; // reset the stream

// read typed
using (var tf = TeaFile<Tick>.OpenRead(ms))
{
    Tick value = tf.Read();
    Console.WriteLine(value);
}

https://github.com/discretelogics/TeaFiles.Net

You can install the library via NuGet packages Manager "TeaFiles.Net"
A vsix sample Project is also available in the VS Gallery.

Answer 2

You could use a mapping between the date and the stock value, such as Dictionary<DateTime, decimal>. This way the dates can be sparse.

If you need the prices of multiple stocks at each date, and not every stock appears for every date, then you could choose between Dictionary<DateTime, Dictionary<Stock, decimal>> and Dictionary<Stock, Dictionary<DateTime, decimal>>, depending on how you want to access the values afterwards (or even both if you don't mind storing the values twice).

Answer 3

The DateTime object in C# is a value Type which means it initializes with its default value and that is Day=1 Month=1 Year=1 Hour=1 Minute=1 Second=1. (or was it hour=12, i am not quite sure).

If I understood you right you need a datastructure that holds DateTime objects that are ordered in some way and when you insert a new object the adjacent dateTime objects will change to retain your order.

In this case I would focus mor on the datastructure than on the dateTime object.

Write a simple class that inherits from Lits<> for example and include the functionality you want on an insert oder delete operation.

Something like:

public class DateTimeList : List<DateTime> {

public void InsertDateTime (int position, DateTime dateTime) {

    // insert the new object
    this.InsertAt(position, dateTime)

    // then take the adjacent objects (take care of integrity checks i.e.
    // exists the index/object? in not null ? etc.        

    DateTime previous = this.ElementAt<DateTime>(position - 1);

    // modify the previous DateTime obejct according to your needs.

    DateTime next = this.ElementAt<DateTime>(position + 1);

    // modify the next DateTime obejct according to your needs.    

}
}

Answer 4

The is a time series library called TimeFlow, which allows smart creation and handling of time series.

The central TimeSeries class knows its timezone and is internally based on a sorted list of DatimeTimeOffset/Decimal pairs with specific frequency (Minute, Hour, Day, Month or even custom periods). The frequency can be changed during resample operations (e.g. hours -> days). It is also possible to combine time series unsing the standard operators (+,-,*,/) or advanced join operations using cusom methods.

Further more, the TimeFrame class combines multiple time series of same timezone and frequency (similar to pythons DataFrame but restricted to time series) for easier access.

Additional there is the great TimeFlow.Reporting library that provides advanced reporting / visualization (currently Excel and WPF) of time frames.

Disclaimer: I am the creator of these libraries.

Answer 5

As you mentioned in your comment to Marc's answer, I believe the SortedList is a more appropriate structure to hold your time series data.

UPDATE

As zmbq mentioned in his comment to Marc's question, the SortedList is implemented as an array, so if faster insertion/removal times are needed then the SortedDictionary would be a better choice.

See Jon Skeet's answer to this question for an overview of the performance differences.