C# Regex Split - commas outside quotes

Question

I got quite a lot of strings (segments of SQL code, actually) with the following format:

('ABCDEFG', 123542, 'XYZ 99,9')

and i need to split this string, using C#, in order to get:

• 'ABCDEFG'
• 123542
• 'XYZ 99,9'

I was originally using a simple Split(','), but since that comma inside the last parameter is causing havoc in the output i need to use Regex to get it. The problem is that i'm still quite noobish in regular expressions and i can't seem to crack the pattern mainly because inside that string both numerical and alpha-numerical parameters may exist at any time...

What could i use to split that string according to every comma outside the quotes? Cheers

Answer 1

You could split on all commas, that do have an even number of quotes following them , using the following Regex to find them:

",(?=(?:[^']*'[^']*')*[^']*$)"

You'd use it like

var result = Regex.Split(samplestring, ",(?=(?:[^']*'[^']*')*[^']*$)");

Answer 2

//this regular expression splits string on the separator character NOT inside double quotes. 
//separatorChar can be any character like comma or semicolon etc. 
//it also allows single quotes inside the string value: e.g. "Mike's Kitchen","Jane's Room"
Regex regx = new Regex(separatorChar + "(?=(?:[^\"]*\"[^\"]*\")*(?![^\"]*\"))"); 
string[] line = regx.Split(string to split);

Answer 3

I had a problem where it wasn't capturing empty columns. I modified it as such to get empty string results

var results = Regex.Split(source, "[,]{1}(?=(?:[^\"]*\"[^\"]*\")*(?![^\"]*\"))");

Answer 4

although I too like a challenge some of the time, but this actually isn't one. please read this article http://secretgeek.net/csv_trouble.asp and then go on and use http://www.filehelpers.com/

[Edit1, 3]: or maybe this article can help too (the link only shows some VB.Net sample code but still, you can use it with C# too!): http://msdn.microsoft.com/en-us/library/cakac7e6.aspx

I've tried to do the sample for C# (add reference to Microsoft.VisualBasic to your project)

using System;
using System.IO;
using Microsoft.VisualBasic.FileIO;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            TextReader reader = new StringReader("('ABCDEFG', 123542, 'XYZ 99,9')");
            TextFieldParser fieldParser = new TextFieldParser(reader);

            fieldParser.TextFieldType = Microsoft.VisualBasic.FileIO.FieldType.Delimited;
            fieldParser.SetDelimiters(",");

            String[] currentRow; 

            while (!fieldParser.EndOfData)
            {
                try
                {
                     currentRow = fieldParser.ReadFields();

                     foreach(String currentField in currentRow)
                     {
                        Console.WriteLine(currentField);                        
                     }
                }
                catch (MalformedLineException e)
                {
                    Console.WriteLine("Line {0} is not valid and will be skipped.", e);
               }

            } 

        }
    }
}

[Edit2]: found another one which could be of help here: http://www.codeproject.com/KB/database/CsvReader.aspx

-- reinhard

Answer 5

Try (hacked from Jens') in the split method:

",(?:.*?'[^']*?')"

or just add question marks after Jens' *'s, that makes it lazy rather than greedy.

Answer 6

... or you could have installed NuGet package LumenWorks CsvReader and done something like below where I read a csv file which has content like for example

"hello","how","hello, how are you"
"hi","hello","greetings"
...

and process it like this

public static void ProcessCsv()
        {
            var filename = @"your_file_path\filename.csv";
            DataTable dt = new DataTable("MyTable");

            List<string> product_codes = new List<string>();
            using (CsvReader csv = new CsvReader(new StreamReader(filename), true))
            {
                int fieldCount = csv.FieldCount;

                string[] headers = csv.GetFieldHeaders();
                for (int i = 0; i < headers.Length; i++)
                {
                     dt.Columns.Add(headers[i], typeof(string));
                }

                while (csv.ReadNextRecord())
                {
                    DataRow dr = dt.NewRow();
                    for (int i = 0; i < fieldCount; i++)
                    {
                        product_codes.Add(csv[i]);
                        dr[i] = csv[i];
                    }
                    dt.Rows.Add(dr);
                }
            }
        }

Answer 7

The accepted answer does not work for me (can put in and test at Regexr-dot-com and see that does not work). So I had to read the lines into an array of lines. Use (C#) Regex.Matches to get an array of any strings found between escaped quotes (your in-field commas should be in fields wrapped in quotes), and replace commas with || before splitting each line into columns/fields. After splitting each line, I looped each line and column to replace || with commas.

        private static IEnumerable<string[]> ReadCsv(string fileName, char delimiter = ';')
    {
        string[] lines = File.ReadAllLines(fileName, Encoding.ASCII);
        // Before splitting on comma for a field array, we have to replace commas witin the fields
        for(int l = 1; l < lines.Length; l++)
        {
            //(\\")(.*?)(\\")
            MatchCollection regexGroup2 = Regex.Matches(lines[l], "(\\\")(.*?)(\\\")");
            if (regexGroup2.Count > 0)
            {
                for (int g = 0; g < regexGroup2.Count; g++)
                {
                    lines[l] = lines[l].Replace(regexGroup2[g].Value, regexGroup2[g].Value.Replace(",", "||"));
                }
            }
        }

        // Split
        IEnumerable<string[]> lines_split = lines.Select(a => a.Split(delimiter));

        // Reapply commas
        foreach(string[] row in lines_split)
        {
            for(int c = 0; c < row.Length; c++)
            {
                row[c] = row[c].Replace("||", ",");
            }
        }

        return (lines_split);
    }