Parsing GoogleTranslate especial characters

Question

First of all, is not a solution for me to pay for a GoogleTranslate API.

I'm trying to translate with a Get method a simple phrase that contains a especial character "&".

"Me & You"

This is the method I wrote:

Public Function Google_Translate(ByVal Input As String, _
                                 ByVal From_Language As Languages, _
                                 ByVal To_Language As Languages) As String

    Dim webClient As New System.Net.WebClient

    Dim str = webClient.DownloadString( _
    "http://translate.google.com/translate_a/t?client=t&text=" & Input & _
    "&sl=" & Formatted_From_Language & _
    "&tl=" & Formatted_To_Language & "")

    ' Debug: MsgBox(str)

    Return (str.Substring(4, str.Length - 4).Split(ControlChars.Quote).First)

End Function

This is a usage example:

Google_Translate("Me and you", Languages.en, Languages.en)

The result is the same string 'cause I've translated from english to english:

"Me and you"

The problem is when I try to use any especial HTML character, for example the "&":

Google_Translate("Me & you", Languages.en, Languages.en)

Result:

"Me"

This is the string without the split:

[[["Me","Me","",""]],,"en",,,,,,,0]

This is all I've tried:

Unicode identifiers:

Google_Translate("Me \u0026 you")

HTML Entities:

Google_Translate("Me & you")

HTML Escaped entities:

Google_Translate("Me &H38; you")

...And HTML percents:

Google_Translate("Me %26 you")

...Calling the method using the percents I get a string with an Unicode identifier:

[[["Me \u0026 you","Me \u0026 you","",""]],,"en",,,,,,[["en"]],0]

That maybe will mean the only thing that I need to do is to get the string from Google and translate the unicode identifier and that's all? ...NOT! 'cause if I call Google using other special character, I don't get any unicode identifier:

Google_Translate("Hello·"" World¿?", GoogleTranslate_Languages.en, GoogleTranslate_Languages.en)

Result:

"Hello·\" World¿?"

Result without the split:

[[["Hello·\" World¿?","Hello·\" World¿?","",""]],,"en",,,,,,[["en"]],0]

What I'm missing?

How I can send/get the data in the correct way using especial characters as &%$"¿? ?

Answer 1

Firstly by using their web interface for automated queries you are almost certainly violating Google's Terms of use and also will make your application extremely brittle as there is nothing stopping Google from changing their front end code as they do regularly. Chances are if you are only doing two or three translations a month you will only get twenty translation before it breaks. Depending on how you value your time you may spend more time on fixing it to respond to Google's changes than using the API would cost. The API is cheap compared to most translation services.

If cost is a big issue it may be worth looking at the bing translation API which is free.

Finally the correct way to send the data is to use URI encoding (which you call html percents) for anything other than alphanumeric characters. The response is a JSON encoded string. Use a framework like JSON.NET to deserialize it.

Answer 2

Here's how (in C#):

First, you create your name value collection using the inputs you need like:

var nvc = new NameValueCollection
                {
                    {"q", input},
                    {"source", "en"},
                    {"target", "en"},
                    {"key","Your translate API key here"}
                };

Then you can call a function like this one:

internal string Post(string url, ref CookieContainer cookieJar, NameValueCollection nvc, string referer = null)
{
    var postdata = string.Join("&", Array.ConvertAll(nvc.AllKeys, key => string.Format("{0}={1}", HttpUtility.UrlEncode(key), HttpUtility.UrlEncode(nvc[key]))));
    var request = (HttpWebRequest)WebRequest.Create(url);
    request.CookieContainer = cookieJar;
    request.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0";
    request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
    request.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
    request.Headers.Add("Accept-Encoding", "gzip, deflate");
    request.Headers.Add("Accept-Language", "en-us");
    request.Method = "POST";
    request.KeepAlive = true;
    request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = postdata.Length;
    if (!string.IsNullOrEmpty(referer))
        request.Referer = referer;

    var writer = new StreamWriter(request.GetRequestStream());
    writer.Write(postdata);
    writer.Close();

    var response = (HttpWebResponse)request.GetResponse();
    var resp = (new StreamReader(response.GetResponseStream())).ReadToEnd();
    return resp;
}

Here's the call:

var result=Post("https://www.googleapis.com/language/translate/v2/detect",ref new CookieContainer(),nvc);

for small values of input, you can use GET instead of POST, just append postdata onto the url after a ?.

For google translate API, the response is in JSON format, there are many posts on how to parse JSON responses, so I won't go into that here, but these should help you get started: How to decode a JSON string using C#? Convert JSON File to C# Object

Answer 3

input: Web.HttpUtility.UrlEncode("You & Me")

output: Web.HttpUtility.HtmlDecode(result)