0

I have this script task which must convert a number of Tab Delimited Files from utf8 -> unicode.

It runs rather slow. How can i improve the script code?

public void Main()
{
    Dts.TaskResult = (int)ScriptResults.Success;
    string path= (string)Dts.Variables["dataPath"].Value;
    string name = (string)Dts.Variables["fileName"].Value;
    string from = Path.Combine(path, name) + ".tsv";
    string to = Path.ChangeExtension(from, "txt");
    using (StreamReader reader = new StreamReader(from, Encoding.UTF8, false, 1000000))
    using (StreamWriter writer = new StreamWriter(to,false, Encoding.Unicode, 1000000))
    {
        while (!reader.EndOfStream)
        {
            var line = reader.ReadLine();
            if (line.Length>0)
                writer.WriteLine(line);
        }
    }       

}

I have done whatever i can in order to improve the disk activity , everything is running on the same raid array disk which is the fastest i can get.

Any suggestions ofr the upper code to be faster in the SSIS pipeline?

Additional Info:

i tried to replace the while with this:

        while (reader.Peek() >= 0)
        {
            writer.WriteLine(reader.ReadLine());
        }

i dont know if it is beneficial i am testing it now.

e4rthdog
  • 5,103
  • 4
  • 40
  • 89
  • My method didn't work in the end so I'm deleting the answer, sorry about that – James Rosser Jan 08 '15 at 10:50
  • utf-8 is unicode, why do you need convert it? See this link: http://stackoverflow.com/questions/11293994/how-to-convert-a-utf-8-string-into-unicode – Blindsniper Jan 12 '15 at 02:37
  • I am getting data from 4 non-unicode AS400s via text files. If i dont do this i cant get correct characters... – e4rthdog Jan 12 '15 at 07:36

0 Answers0