Uri.EscapeDataString() - Invalid URI: The Uri string is too long

Question

I'm using compact framework/C# on windows mobile.

In my application I am uploading data to the server by serializing objects and using a HttpWebRequest/POST request to send the information up. On the server the post data is de-serialised and saved to the db.

The other day I realised that I had a problem with special characters in the post data (ampersands etc..). So I introduced Uri.EscapeDataString() into the method and all was well.

However, today I have discovered that there is a problem when the application attempts to upload a large amount of data (I'm unsure of what exactly denotes "large" at the moment!)

Existing code (Kind of)

var uploadData = new List<Things>();

uploadData.Add(new Thing() { Name = "Test 01" });
uploadData.Add(new Thing() { Name = "Test 02" });
uploadData.Add(new Thing() { Name = "Test with an & Ampersand " }); // Do this a lot!!

var postData = "uploadData=" + Uri.EscapeDataString(JsonConvert.SerializeObject(uploadData, new IsoDateTimeConverter()));

Problem

The call to Uri.EscapeDataString() is causing the following exception:

System.UriFormatException: Invalid URI: The Uri string is too long.

Question

Are there any other ways to prepare the data for upload?

As far as I can see HttpUtility (which has its own Encode/Decode methods) is not available for the compact framework.

Answer 1

Or you could simply split your string and call Uri.EscapeDataString(string) for each block, in order to avoid reimplementing the function.

Sample Code:

        String value = "large string to encode";
        int limit = 2000;

        StringBuilder sb = new StringBuilder();
        int loops = value.Length / limit;

        for (int i = 0; i <= loops; i++)
        {
            if (i < loops)
            {
                sb.Append(Uri.EscapeDataString(value.Substring(limit * i, limit)));
            }
            else
            {
                sb.Append(Uri.EscapeDataString(value.Substring(limit * i)));
            }
        }

Answer 2

The answer of "Alberto de Paola" is good.

Nonetheless, to unescape the escaped data is little bit trickier, because you have to avoid cutting the encoded string at the middle of an encoded char (or you will break the integrity of the original string).

Here's my way of fixing this issue :

public static string EncodeString(string str)
{
    //maxLengthAllowed .NET < 4.5 = 32765;
    //maxLengthAllowed .NET >= 4.5 = 65519;
    int maxLengthAllowed = 65519;
    StringBuilder sb = new StringBuilder();
    int loops = str.Length / maxLengthAllowed;

    for (int i = 0; i <= loops; i++)
    {
        sb.Append(Uri.EscapeDataString(i < loops
            ? str.Substring(maxLengthAllowed * i, maxLengthAllowed)
            : str.Substring(maxLengthAllowed * i)));
    }

    return sb.ToString();
}

public static string DecodeString(string encodedString)
{
    //maxLengthAllowed .NET < 4.5 = 32765;
    //maxLengthAllowed .NET >= 4.5 = 65519;
    int maxLengthAllowed = 65519;

    int charsProcessed = 0;
    StringBuilder sb = new StringBuilder();

    while (encodedString.Length > charsProcessed)
    {
        var stringToUnescape = encodedString.Substring(charsProcessed).Length > maxLengthAllowed
            ? encodedString.Substring(charsProcessed, maxLengthAllowed)
            : encodedString.Substring(charsProcessed);

        // If the loop cut an encoded tag (%xx), we cut before the encoded char to not loose the entire char for decoding
        var incorrectStrPos = stringToUnescape.Length == maxLengthAllowed ? stringToUnescape.IndexOf("%", stringToUnescape.Length - 4, StringComparison.InvariantCulture) : -1;
        if (incorrectStrPos > -1)
        {
            stringToUnescape = encodedString.Substring(charsProcessed).Length > incorrectStrPos
                ? encodedString.Substring(charsProcessed, incorrectStrPos)
                : encodedString.Substring(charsProcessed);
        }

        sb.Append(Uri.UnescapeDataString(stringToUnescape));
        charsProcessed += stringToUnescape.Length;
    }

    var decodedString = sb.ToString();

    // ensure the string is sanitized here or throw exception if XSS / SQL Injection is found
    SQLHelper.SecureString(decodedString);
    return decodedString;
}

To test these functions :

var testString = "long string to encode";
var encodedString = EncodeString(testString);
var decodedString = DecodeString(encodedString);

Console.WriteLine(decodedString == testString ? "integrity respected" : "integrity broken");

Hope this can help avoiding some headaches ;)

Answer 3

StringBuilder stringBuilder = new StringBuilder();
for (int i = 0; i < originalString.Length; i++)
{
    if ((originalString[i] >= 'a' && originalString[i] <= 'z') || 
        (originalString[i] >= 'A' && originalString[i] <= 'Z') || 
        (originalString[i] >= '0' && originalString[i] <= '9'))
    {
        stringBuilder.Append(originalString[i]);
    }
    else
    {
        stringBuilder.AppendFormat("%{0:X2}", (int)originalString[i]);
    }
}

string result = stringBuilder.ToString();

Answer 4

I have been using System.Web.HttpUtility.UrlEncode and seems to handle the longer strings much better.

Answer 5

Use System.Web.HttpUtility.UrlEncode (based on this answer):

        value = HttpUtility.UrlEncode(value)
            .Replace("!", "%21")
            .Replace("(", "%28")
            .Replace(")", "%29")
            .Replace("*", "%2A")
            .Replace("%7E", "~"); // undo escape

Answer 6

I needed another solution because the solution from Pouki does not work when Cyrillic is processed and symbol is cut.

The alternative solution is as follow:

    protected const int MaxLengthAllowed = 32765;
    private static string UnescapeString(string encodedString)
    {
        var charsProccessed = 0;

        var sb = new StringBuilder();

        while (encodedString.Length > charsProccessed)
        {
            var isLastIteration = encodedString.Substring(charsProccessed).Length < MaxLengthAllowed;

            var stringToUnescape = isLastIteration
                ? encodedString.Substring(charsProccessed)
                : encodedString.Substring(charsProccessed, MaxLengthAllowed);

            while (!Uri.IsWellFormedUriString(stringToUnescape, UriKind.RelativeOrAbsolute) || stringToUnescape.Length == 0)
            {
                stringToUnescape = stringToUnescape.Substring(0, stringToUnescape.Length - 1);
            }

            sb.Append(Uri.UnescapeDataString(stringToUnescape));
            charsProccessed += stringToUnescape.Length;
        }

        return sb.ToString();
    }