Using CsvHelper can I translate white space to a nullable?

Question

I have some really lame Csv files I need to parse. I am using CsvHelper and it is working awesome. Except I have some lines that have whitespace where normaly I have a double.

File:

Text,SomeDouble,MoreText

"Good",1.23,"Good"

"Bad", ,"Bad"

if I try and map this into

public class Test
{
  [CsvField(Name = "Text")]
  public string Text { get; set; }

  [CsvField(Name = "SomeDouble")]
  public double? SomeDouble{ get; set; }

  [CsvField(Name = "MoreText")]
  public string MoreText{ get; set; }
}

then I get an error like this:

CsvHelper.CsvReaderException: An error occurred trying to read a record of type

Row: '2' (1-based)

Field Index: '1' (0-based)

Field Name: 'SomeDouble'

Field Value: ' '

System.Exception: is not a valid value for Double. ---> System.FormatException: Input string was not in a correct format.
at System.Number.ParseDouble(String value, NumberStyles options, NumberFormatInfo numfmt) at System.ComponentModel.DoubleConverter.FromString(String value, NumberFormatInfo formatInfo) at System.ComponentModel.BaseNumberConverter.ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, Object value) --- End of inner exception stack trace --- at System.ComponentModel.BaseNumberConverter.ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, Object value) at System.ComponentModel.NullableConverter.ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, Object value) at lambda_method(Closure , ICsvReader ) at CsvHelper.CsvReader.d__0`1.MoveNext()

As I see it, my options are to create a custom parser, or map my value into a string property and do the parsing there.

Are there any other options?

It would be nice if I could configure that I want to treat white space as null.

As requested, here is a code sample that reproduces the problem

 static class Program
    {
        public class Test
        {
            [CsvField(Name = "Text")]
            public string Text { get; set; }

            [CsvField(Name = "SomeDouble")]
            public double? SomeDouble { get; set; }

            [CsvField(Name = "MoreText")]
            public string MoreText { get; set; }
        }

        static void Main(string[] args)
        {
            // create fake in memory file
            var memoryStream = new MemoryStream();
            var streamWriter = new StreamWriter(memoryStream);
            streamWriter.WriteLine("Text,SomeDouble,MoreText");
            streamWriter.WriteLine("Good, 1.23, Good");
            streamWriter.WriteLine("Bad, ,Bad");

            streamWriter.Flush();

            //reset the file to the begining
            memoryStream.Position = 0;

            using (
                var csv =
                    new CsvReader(
                        new StreamReader(memoryStream)))
            {
                // this call will blow up with the exception.
                var records = csv.GetRecords<Test>().ToList();

                //carry on and do stuff with 'records'...
            }
    }

Thanks.

Can you post the code for the method that's actually failing, I'm not familiar with CSVHelper. — Tejas Sharma, Oct 24 '12 at 23:52
I don't think out-of-the-package it does that, but Does this help: [custom typeconverter](https://github.com/JoshClose/CsvHelper/wiki/Custom-TypeConverter) I'm going to see if I get anywhere with it, I could use that feature too. Please post if you find a solution. — Levin Magruder, Oct 25 '12 at 18:39

RMK · Accepted Answer · 2012-10-29T14:17:20.470

In the end I went with creating my own type converter that will treat whitespace the same as a null.

public class WhiteSpaceToNullableTypeConverter<T> : TypeConverter where T : struct
{
    public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
    {
        return sourceType == typeof (string);
    }

    public override bool CanConvertTo(ITypeDescriptorContext context, Type destinationType)
    {
        return destinationType == typeof (T?);
    }

    public override object ConvertFrom(ITypeDescriptorContext context, System.Globalization.CultureInfo culture,
                                       object value)
    {
        T? result = null;

        var stringValue = (string) value;
        if (!string.IsNullOrWhiteSpace(stringValue))
        {
            var converter = TypeDescriptor.GetConverter(typeof(T));
            result = (T)converter.ConvertFrom(stringValue.Trim());
        }

        return result;
    }

    public override object ConvertTo(ITypeDescriptorContext context, System.Globalization.CultureInfo culture,
                                     object value, Type destinationType)
    {
        var result = (T?) value;
        return result.ToString();
    }
}

Apply it to your model like this

public class Test
{
    [CsvField(Name = "Text")]
    public string Text { get; set; }

    [CsvField(Name = "SomeDouble")]
    [TypeConverter( typeof( WhiteSpaceToNullableTypeConverter<Double> ) )]
    public double? SomeDouble{ get; set; }

    [CsvField(Name = "MoreText")]
    public string MoreText{ get; set; }
}

score 5 · Answer 2 · edited Aug 12 '20 at 09:06

A simple way is to use ConvertUsing() in your ClassMap:

Map(x => x.SomeDouble)
    .ConvertUsing(row => 
        string.IsNullOrWhiteSpace(row.GetField("SomeDouble")) ?
            (double?) null : 
            Convert.ToDouble(row.GetField("SomeDouble")));

I like to make little helper functions and call them

Map(x => x.SomeDouble)
    .ConvertUsing(row => GetOddballDouble(row.GetField("SomeDouble")));

Using CsvHelper can I translate white space to a nullable?

2 Answers2

Linked