0

I've read multiple threads about this but I simply can't get it to work.

I must admit also admit that I don't really know much about how encoding works which might be clear in my question below.

Here is the issue:

I've written a script that gets a response from an API (Atlassian - Confluence Cloud) and it worked perfectly on my laptop (VSCODE). I then moved the script to the intended machine it should run on by copy-pasting it into PowerShell ISE and the response I get from the API suddenly have a new malformed character - "Â".

The general idea of the script is that it parses a table (response is XHTML / XML and I strip all the headers) and fetches the value for every row and then depending if a date is soon to be expired or not sends a notification to the specific entry's "team".

My laptop (VSCODE) running $OutputEncoding: UTF-8
^ PSVersion : 7.1.2 [PSCORE]
Other machine (ISE) running $OutputEncoding: Windows-1252
^ PSVersion : 5.1.14393.3866 [PowerShell]

I use "Invoke-Restmethod" although if I use "Invoke-WebRequest" to view which encoding the response uses then I get:

($test = "Invoke-WebRequest" ...)
My laptop (VSCODE): $test.encoding = System.Text.UTF8Encoding+UTF8EncodingSealed
My laptop (VSCODE): $test.ParsedHtml.defaultCharset = no such property ("parsedhtml")
My laptop (VSCODE): $test.ParsedHtml.charset = no such property ("parsedhtml")
Other machine: $test.encoding = no such property ("encoding")
Other machine: $test.ParsedHtml.defaultCharset = windows-1252
Other machine: $test.ParsedHtml.charset = unicode

So just to I understand what happens here:

When I receive the response it is automatically encoded using the machines output encoding, when the machine has UTF-8 my response displays correctly but when it has windows-1252 it shows a malformed character?

I've tried enforcing which encoding the response should send using headers;

$headers = @{
    'Authorization' = "Basic $EncodedText"
    'Content-Type' = 'application/json ; charset=utf-8'
    'Charset' = 'utf-8'
}

which makes no difference.

What is weird is that the malformed character which appears when the charset on the machine is set to windows-1252 is just...gone when I run it on the machine that is using utf-8.

I would understand if a character like the swedish "Ä" is displayed as "Â" when not using utf-8 but there simply isn't any character at all when using utf-8. The correctly formated response should still have a character in place of the malformed character but there simply isn't any.

Really weird IMHO.

Any ideas on how to solve it on the machine that is using windows-1252?

I've tried saving the output to a file and to specify the encoding as "utf-8" and then reading back the content of the file but I still have the character since the actual response I get includes the malformed character.

So AFAIK it is the response that is returned malformed and any transformation after the response won't work since it will still contain the malformed character.

Thanks for your time and sorry for the long post!

TheSwede86
  • 85
  • 2
  • 8
  • Have a look at https://stackoverflow.com/questions/58539438/powershell-invoke-restmethod-umlauts-issues-with-utf-8-and-windows-1252 - it's basically the same issue. The Confluence API encodes the HTTP response using a non-default encoding (i.e. UTF8) but it doesn't add a header to tell the client it's done that, so in a lot of cases the client ends up using the default decoder (ISO-8859-1) and gets mangled text as a result. – mclayton Mar 05 '21 at 13:27
  • Sadly that isn't working: $get_certificatescontent = Invoke-RestMethod ... $certificates_body_content_unenc = $get_certificatescontent.body.view.value $bytes = [System.Text.Encoding]::GetEncoding(1252).GetBytes($certificates_body_content_unenc) $certificates_body_content = [System.Text.Encoding]::UTF8.GetString($bytes) Echoing $certificates_body_content still shows the malformed character – TheSwede86 Mar 05 '21 at 14:25
  • Have a look at [this answer](https://stackoverflow.com/a/66411315/9898643) too. – Theo Mar 11 '21 at 15:18

0 Answers0