1

Hi I am just beginner on Web Scraping and I am trying to scrape website that has Post Request Method. When I scrape the website I saw "�" instead of Turkish characters and I have no idea what to do to fix it :/

var client = new HttpClient();
var values = new Dictionary<string, string>
{
    { "city_id", "01"    },
    { "district_id", "000" },
    {"post","1"    }
};

var content = new FormUrlEncodedContent(values);
var response = await client.PostAsync("url-address", content);
string responseString = await response.Content.ReadAsStringAsync();
atiyar
  • 7,762
  • 6
  • 34
  • 75
Himelu5
  • 15
  • 3
  • Check out the accepted answer here: https://stackoverflow.com/questions/22649473/getting-an-utf-8-response-with-httpclient-in-windows-store-apps Does that help? – LocEngineer Oct 18 '20 at 12:20
  • Just to ease your frustration: Characters, encoding and regionalization is just horrible to work with. Because there are so many elements that impact characters/text. Host OS, source OS, Encoding defaults in settings on either host or source, setup of the .net project... the list goes on. I have ended up running applications on docker linux containers forcing "invariant" regionalization in all aspects and still have this issue from time to time. Try to cover most things with UTF8 and then only the most important scenarios on a per-case basis :-) – Frank R. Haugen Oct 18 '20 at 13:02
  • @LocEngineer yes I checked that answer aswell but It didn't solve my problem either :/ – Himelu5 Oct 18 '20 at 13:10
  • Please specify `I saw "�" instead of Turkish` - saw *how* ? Where do you check / display the content? Need a bit more info. – LocEngineer Oct 18 '20 at 14:44

1 Answers1

0

By help of @LocEngineer's comment I just changed words of that answer's page and finally fixed my issue

var buffer = await response.Content.ReadAsByteArrayAsync(); 
            var byteArray = buffer.ToArray(); 
            var responseString = Encoding.Default.GetString(byteArray, 0, byteArray.Length);
Himelu5
  • 15
  • 3