What is the Best Way to Clean a URL with a Title in it

Question

What is the best way to clean a URL? I am looking for a URL like this

what_is_the_best_headache_medication

My current code

public string CleanURL(string str)
{
    str = str.Replace("!", "");
    str = str.Replace("@", "");
    str = str.Replace("#", "");
    str = str.Replace("$", "");
    str = str.Replace("%", "");
    str = str.Replace("^", "");
    str = str.Replace("&", "");
    str = str.Replace("*", "");
    str = str.Replace("(", "");
    str = str.Replace(")", "");
    str = str.Replace("-", "");
    str = str.Replace("_", "");
    str = str.Replace("+", "");
    str = str.Replace("=", "");
    str = str.Replace("{", "");
    str = str.Replace("[", "");
    str = str.Replace("]", "");
    str = str.Replace("}", "");
    str = str.Replace("|", "");
    str = str.Replace(@"\", "");
    str = str.Replace(":", "");
    str = str.Replace(";", "");
    str = str.Replace(@"\", "");
    str = str.Replace("'", "");
    str = str.Replace("<", "");
    str = str.Replace(">", "");
    str = str.Replace(",", "");
    str = str.Replace(".", "");
    str = str.Replace("`", "");
    str = str.Replace("~", "");
    str = str.Replace("/", "");
    str = str.Replace("?", "");
    str = str.Replace("  ", " ");
    str = str.Replace("   ", " ");
    str = str.Replace("    ", " ");
    str = str.Replace("     ", " ");
    str = str.Replace("      ", " ");
    str = str.Replace("       ", " ");
    str = str.Replace("        ", " ");
    str = str.Replace("         ", " ");
    str = str.Replace("          ", " ");
    str = str.Replace("           ", " ");
    str = str.Replace("            ", " ");
    str = str.Replace("             ", " ");
    str = str.Replace("              ", " ");
    str = str.Replace(" ", "_");
    return str;
}

Answer 1

Regular expressions for sure:

public string CleanURL(string str)
{
    str = Regex.Replace(str, "[^a-zA-Z0-9 ]", "");
    str = Regex.Replace(str, " +", "_");
    return str;
}

(Not actually tested, off the top of my head.)

Let me explain:

The first line removes everything that's not an alphanumeric character (upper or lowercase) or a space . The second line replaces any sequence of spaces (1 or more, sequentially) with a single underscore.

Answer 2

Generally your best bet is to go with a white list regular expression approach instead of removing all the unwanted characters because you definitely are going to miss some.

The answers here are fine so far but I personally did not want to remove umlauts and characters with accent marks entirely. So the final solution I came up with looks like this:

public static string CleanUrl(string value)
{
    if (value.IsNullOrEmpty())
        return value;

    // replace hyphens to spaces, remove all leading and trailing whitespace
    value = value.Replace("-", " ").Trim().ToLower();

    // replace multiple whitespace to one hyphen
    value = Regex.Replace(value, @"[\s]+", "-");

    // replace umlauts and eszett with their equivalent
    value = value.Replace("ß", "ss");
    value = value.Replace("ä", "ae");
    value = value.Replace("ö", "oe");
    value = value.Replace("ü", "ue");

    // removes diacritic marks (often called accent marks) from characters
    value = RemoveDiacritics(value);

    // remove all left unwanted chars (white list)
    value = Regex.Replace(value, @"[^a-z0-9\s-]", String.Empty);

    return value;
}

The used RemoveDiacritics method is based on the SO answer by Blair Conrad:

public static string RemoveDiacritics(string value)
{
    if (value.IsNullOrEmpty())
        return value;

    string normalized = value.Normalize(NormalizationForm.FormD);
    StringBuilder sb = new StringBuilder();

    foreach (char c in normalized)
    {
        if (CharUnicodeInfo.GetUnicodeCategory(c) != UnicodeCategory.NonSpacingMark)
            sb.Append(c);
    }

    Encoding nonunicode = Encoding.GetEncoding(850);
    Encoding unicode = Encoding.Unicode;

    byte[] nonunicodeBytes = Encoding.Convert(unicode, nonunicode, unicode.GetBytes(sb.ToString()));
    char[] nonunicodeChars = new char[nonunicode.GetCharCount(nonunicodeBytes, 0, nonunicodeBytes.Length)];
    nonunicode.GetChars(nonunicodeBytes, 0, nonunicodeBytes.Length, nonunicodeChars, 0);

    return new string(nonunicodeChars);
}

Hope that helps somebody challenged by slugifying URLs and keeping umlauts and friends with their URL friendly equivalent at the same time.

Answer 3

You should consider using a regular expression instead. It's much more efficient than what you're trying to do above.

More on Regular Expressions here.

Answer 4

1. How do you define "friendly" URL - I'm assuming you mean to remove _'s etc.
2. I'd look into a regular expression here.

If you want to persist with the method above, I would suggest moving to StringBuilder over a string. This is because each of your replace operations is creating a new string.

Answer 5

I can tighten up one piece of that:

while (str.IndexOf("  ") > 0)
    str = str.Replace("  ", " ");

...instead of your infinite number of " " replacements. But you almost certainly want a regular expression instead.

Answer 6

Or, a bit more verbose, but this only allows alphanumeric and spaces (which are replaced by '-')

string Cleaned = String.Empty;
foreach (char c in Dirty)
    if (((c >= 'a') && (c <= 'z')) ||
         (c >= 'A') && (c <= 'Z') ||
         (c >= '0') && (c <= '9') ||
         (c == ' '))
           Cleaned += c;
Cleaned = Cleaned.Replace(" ", "-");

Answer 7

The way stackoverflow is doing it can be found here:

https://stackoverflow.com/a/25486/142014

optimized for speed ("This is the second version, unrolled for 5x more performance") and taking care of a lot of special characters.