Distinct() doesn't see uppercase letter changed by ToLower() method

Question

This is my code:

string textToEncode = File.ReadAllText(@"C:\Users\ASUS\Desktop\szyfrowanie2\TextSample.txt");
textToEncode = textToEncode.ToLower();
char[] distinctLetters = textToEncode.Distinct().ToArray();
var count = textToEncode.Distinct().Count();
Console.WriteLine("Letters used in text: \n\n");
for (int i = 0; i < count; i++)
{
    if (Equals(distinctLetters[i]," "))
    {
        Console.Write("<space>"); 
    }
    else
    {
        Console.Write(" " + distinctLetters[i] + " ");
    }
}

I want to read the .txt file, turn it all to lowercases by ToLower(); method, but then when I want to read all the distinct characters from .txt file and then write them on screen, they don't show up. Yet later when I use

for (int i = 0; i < distinctLetters.Length; i++)
{
    Console.Write("Swap " + distinctLetters[i] + " with ");

it shows the letter that indeed was changed into a lowercase, but wasn't visible on screen by first for loop. First word in my TextSample.txt file is "With". The first loop only shows

i t h

But as the second loop starts, it asks

Swap w with

and I have no idea why. Also the if statement in first loop doesn't work, it doesn't detect the space.

`Distinct` [does not keep the original order](https://stackoverflow.com/a/4734876/11683), so your "w" must be later in the array. The `Equals` apparently does not work because it's `Something.Equals`, not `string.Equals`. That does not explain why `distinctLetters[i]` shows "Swap w with" at `i = 0`, but it will probably be explained if you show more code. — GSerg, Nov 10 '19 at 23:17
The if() doesn't work because " " is a string and distinctLetters[i] is a char. As they're both different types, it always evaluates to false. Replace " " with ' '. (Single quotes, not double!) — Wim ten Brink, Nov 10 '19 at 23:22
@GSerg, Distinct generally doesn't change the order so the 'w' should go first. But as he's also printing a carriage return and a newline character, that 'w' actually gets overwritten by a space! That's why it's not visible... :-) — Wim ten Brink, Nov 10 '19 at 23:51
Since whole code was too long for a comment, there you go: https://pastebin.com/ZtxZwZQG still have no idea how to solve the fact that Distinct().ToArray(); doesn't show uppercases that were changed by ToLower() method. — virouz98, Nov 11 '19 at 07:33

score 2 · Answer 1 · edited Jun 20 '20 at 09:12

I modified your code a bit and apart from "fixing" if statement (as it was never true because of the fact your array contains chars, but was compared to a string (" " is a string, ' ' is a char)) included a new loop which highlights and fixes what turned out to be the main problem (carriage return and new line symbols, which was kindly pointed out by Wim ten Brink in the comments). The string value I used is there only to demonstrate the issue.

string textToEncode = "abc\rdefabdg";

textToEncode = textToEncode.ToLower();

char[] distinctLetters = textToEncode.Distinct().ToArray();
var count = distinctLetters.Length;

Console.WriteLine("Letters used in text (old):");

for (int i = 0; i < count; i++)
{
    var letter = distinctLetters[i];
    if (Equals(letter, " "))
    {
         Console.Write("<space>");
    }
    else
    {
         Console.Write(distinctLetters[i]);
    }
}

Console.WriteLine();
Console.WriteLine("Letters used in text (new):");

for (int i = 0; i < count; i++)
{
   var letter = distinctLetters[i];
       
   if (!char.IsLetter(letter))
       continue;

   Console.Write(distinctLetters[i]);
}

And the output is:

Letters used in text (old):

defg

Letters used in text (new):

abcdefg

Interesting, but you have the same flaw! Thing is, your string starts with a \r\n so you won't notice the problem, but you are also printing a carriage return and line feed character, which will overwrite the first character on the screen with a space! See my answer! ;-) — Wim ten Brink, Nov 10 '19 at 23:45
@WimtenBrink nice observation! Those are the first resulting chars in an output array which are simply ignored as they compensate each other. My example simply hid it as those were first. — Stas Ivanov, Nov 10 '19 at 23:49
Dang! I just upvoted your answer as it is now correct, but now you have more votes than me. :) — Wim ten Brink, Nov 13 '19 at 11:04

Wim ten Brink · Accepted Answer · 2019-11-10T23:58:16.740

I've also modified your code a bit:

string textToEncode = File.ReadAllText(@"C:\Users\ASUS\Desktop\szyfrowanie2\TextSample.txt").ToLower();
char[] distinctLetters = textToEncode.Distinct().ToArray();
var count = distinctLetters.Count();
Console.WriteLine("Letters used in text: \n\n");
for (int i = 0; i < count; i++)
{
    if (Equals(distinctLetters[i], ' ')) { Console.Write("<space>"); }
    else if (Equals(distinctLetters[i], '\r')) { Console.Write("<cr>"); }
    else if (Equals(distinctLetters[i], '\n')) { Console.Write("<lf>"); }
    else { Console.Write(" " + distinctLetters[i] + " "); }
}

Just a few minor things. I merged the two first lines, changed " " into ' ' so it now compares characters, changed the counting of characters to use distinctLetters instead of executing the same Distinct() command again and I added two conditions to handle the carriage return and line feed. (I always mix them up, btw.)
This now shows the right result but should also explain why characters went missing! A simple reason, actually. Your text file has a carriage return character, which will send the cursor back to the left. This will cause the first character to be overwritten by a space...

So your code actually prints " w i ..." but then gets the '\r'. It will then print a space, go back to the beginning of the line and writes another space over the ' '! Then the newline will come next, which prints a second space over the 'w', moves to the next line and prints a space again. Then the rest gets printed...

Simple, isn't it? But by capturing these two special characters with the two extra if statements, it is fixed... :-) The '\r' and '\n' characters are often overlooked in console applications, giving unexpected results when they get printed.

It's great, thank you. Now I'm wondering how to make a for loop that skips the '\r' and '\n' marks — virouz98, Nov 11 '19 at 08:04
@virouz98 if you want to skip some chars in your for loop, you could simply use continue adding this in the beginning of the loop: if (Equals(distinctLetters[i], '\r') || Equals(distinctLetters[i], '\n')) { continue; } — Stas Ivanov, Nov 11 '19 at 14:57

Distinct() doesn't see uppercase letter changed by ToLower() method

2 Answers2