0

I'm using a fairly simple and standard jQuery post to post data to another page. Both pages have

<meta charset="utf-8">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<% Response.CharSet = "UTF-8" %>

The function part of the jQuery post just has alert(data) and all that I return is the value of one form field enclosed in brackets. So when I add a British pound sign (£) it alerts [£] accordingly

However, I also write to a text file on the page I post to and there it actually writes £ rather than just £

So when I do my database insert, it actually inserts £ as well rather than just £

Any reason why this is happening?

Page 1

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Intelligence Point </title>
    <meta name="description" content="">

<meta charset="utf-8">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="">

<!-- Bootstrap core CSS -->
<link href="/css/bootstrap.min.css" rel="stylesheet">

<!-- Custom styles for this template -->
<link href="/css/custom.css" rel="stylesheet">
<link href='https://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="//ajax.googleapis.com/ajax/libs/jqueryui/1.10.4/themes/smoothness/jquery-ui.css" />

<!-- HTML5 shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
      <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
      <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
    <![endif]--> 
    <style type="text/css">        p,textarea {
            padding: 5px;
        }
        textarea {
        border:1px solid #ccc;}
    </style>
</head>
<body>


    <div class="container" style="margin-top:-30px;">
        <p>&nbsp;</p>

    <p class="bg-primary">Comments last updated by JJ004 on 2014-05-07 at 16:28</p>

    <p><textarea class="bg-success" maxlength="4000" style="width:100%;" rows="4" wrap="soft" id="comment1">£</textarea></p>
    <p><textarea class="bg-success" maxlength="4000" style="width:100%;" rows="4" wrap="soft" id="comment2">£</textarea></p>
    <p><textarea class="bg-success" maxlength="4000" style="width:100%;" rows="4" wrap="soft" id="comment3">£</textarea></p>
    <p><textarea class="bg-success" maxlength="4000" style="width:100%;" rows="4" wrap="soft" id="comment4">£</textarea></p>

    <button type="button" class="btn btn-success" id="postcomment">Save comments</button>
    </div>


    <script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
    <script src="../js/bootstrap.min.js"></script>



    <script type="text/javascript">
        $(document).ready(function () {
            window.resizeTo(800, 750);
            $("#postcomment").click(function () {
                $.post(
                   "putComment.asp",
                   {
                       yR: "123",
                       yP: "456",
                       yN: "",
                       yF1: "XXX",
                       yF2: "YYY",
                       yF3: "ZZZ",
                       yF4: "",
                       yC1: $('#comment1').val(),
                       yC2: $('#comment2').val(),
                       yC3: $('#comment3').val(),
                       yC4: $('#comment4').val()
                   },

                   function (data) {

                       //alert(data);
                       var url = "SOMEWHERE.COM";
                       alert("Narrative saved");
                       window.opener.location.href = url;
                       window.close(); // or self.close();

                   }

                );
            });
        });
    </script>

</body>
</html>

putComment.asp

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <% Response.CharSet = "UTF-8" %>
</head>
<body>

<!--#include virtual="/comments/connProd.inc"-->
<%
Function cleanSQL(thisWord)
       Dim newWord
       If thisWord <> "" Then
                       newWord = Replace(thisWord,"/*","")
                       newWord = Replace(newWord,"*/","")
                       newWord = Replace(newWord,"UNION","")
                       newWord = Replace(newWord,"'","''")
                       newWord = Replace(newWord,"""","&amp;quot;")
                       newWord = Replace(newWord,"<script>","[script]")
                       newWord = Trim(newWord)
       End If
       cleanSQL = newWord
       End Function

       Function in_array(element, arr)
    For i=0 To Ubound(arr) 
        If Trim(arr(i)) = Trim(element) Then 
            in_array = True
            Exit Function
        Else 
            in_array = False
        End If  
    Next 
End Function

zR=cleanSQL(request.form("yR"))
zP=cleanSQL(request.form("yP"))
zN=cleanSQL(request.form("yN"))

zC1=cleanSQL(request.form("yC1"))
zC2=cleanSQL(request.form("yC2"))
zC3=cleanSQL(request.form("yC3"))
zC4=cleanSQL(request.form("yC4"))

zF1=cleanSQL(request.form("yF1"))
zF2=cleanSQL(request.form("yF2"))
zF3=cleanSQL(request.form("yF3"))
zF4=cleanSQL(request.form("yF4"))

    'response.Write "["&zC1 & "]<br/>"

sql="insert into IPPR.Comments values("
sql=sql+"'"+zR+"',"
sql=sql+"'"+zP+"',"
sql=sql+"'"+zF1+"',"
sql=sql+"'"+zF2+"',"
sql=sql+"'"+zF3+"',"
sql=sql+"'"+zF4+"',"
sql=sql+"'"+zC1+"',"
sql=sql+"'"+zC2+"',"
sql=sql+"'"+zC3+"',"
sql=sql+"'"+zC4+"',"
sql=sql+"getdate(),"
sql=sql+"'"+zN+"')"


dim fs,f
set fs=Server.CreateObject("Scripting.FileSystemObject") 
set f=fs.OpenTextFile("c:\inetpub\wwwroot\text\log.txt",8,true)
f.WriteLine("")
f.WriteLine("["&zC1 & "]" & sql)
f.close
set f=nothing
set fs=nothing


    'response.write sql
    'response.flush
set rs=conn.execute(sql)

%>
    </body>
</html>
pee2pee
  • 3,619
  • 7
  • 52
  • 133
  • This has been dealt with multiple time on this site, you might find [this answer](http://stackoverflow.com/a/17680939/692942) I gave useful. – user692942 May 07 '14 at 15:16
  • The character is being inserted before SQL it seems. nvarchar and 2012 @Lankymart - no go for me, still doesn't work – pee2pee May 07 '14 at 15:27
  • 1
    @marc_s The OP is saying it's already mismatched encoding before it gets to the database. The issue is most likely their pages are not saved as `UTF-8`. – user692942 May 07 '14 at 15:27
  • 1
    @pee2pee I guarantee it will work, I've heard *"It doesn't work"* regarding this exact issue time and time again, in the end it always does. Can you update your question and show what code you have tried and I'll help you to resolve it. [Another link](http://stackoverflow.com/a/21914278/692942) where the OP concedes they where missing something. – user692942 May 07 '14 at 15:29
  • Edit - I might well have missed something. Can't see the wood through the trees... – pee2pee May 07 '14 at 15:34
  • Yes, your DB encoding as well as your content, needs to be saved as UTF-8. – Diodeus - James MacFarlane May 07 '14 at 15:37
  • You should go back and read my answer in that previous question, it explains how you need certain things in place before encoding will work correctly without ending up with mismatches (i.e. `£`). Your pages *should* be saved as `UTF-8` (65001) encoding not `Windows-1252` or some other regional default. You also need to tell IIS how it expects to process the pages this is done with a processing directive at the top of your page code `<%@ CodePage = 65001 %>`. Just telling the server to returns strings as `UTF-8` is not enough, so this `Response.Charset = "UTF-8"` is inadequate. – user692942 May 07 '14 at 15:39
  • Also any `#includes` should also be saved as `UTF-8` encoding, basically any pages in the chain have to be encoded correctly, one missing link and the whole thing falls apart. – user692942 May 07 '14 at 15:42
  • I've added `<% Response.CharSet = "UTF-8" Response.CodePage = 65001 %>` to every single page and still no luck – pee2pee May 07 '14 at 15:51
  • Have you made sure those pages are saved as `UTF-8`? Method for this varies depending on what you are using, if you open the files with `notepad.exe` for example there is an option in `Save` Dialog for `Encoding`. Remember this includes any `#include` files as well. – user692942 May 07 '14 at 15:56
  • Also your still not telling IIS to handle the processing of the page in `UTF-8` so far you are only setting `Response` properties. That is the job of the `<%@ CodePage = 65001 %>` asp processing directive. – user692942 May 07 '14 at 15:59
  • Done everything noted but the response headers still say `Content-Type: text/html; Charset=ISO-8859-1` – pee2pee May 07 '14 at 16:03
  • Which pages response headers and what are you using to check? Can you show your updated code? Your `Response.Charset`, `Response.CodePage` etc. should be inserted in your code **before** any HTML is returned because once HTML is written it is too late to effect the encoding. – user692942 May 07 '14 at 16:04
  • Both! I've set mime type in IIS for ASP pages to UTF-8 as well – pee2pee May 07 '14 at 16:08
  • It's hard to convey a process in comments so I've added an answer. If still having problems after trying this I will be surprised and either you haven't followed it or there is an unknown you haven't mentioned. Not sure what setting mime type is going to do `UTF-8` isn't a mime type?? – user692942 May 07 '14 at 16:24

2 Answers2

1

Note:

It's really difficult to convey code in comments so I'm writing this.

As I have explained your getting a mismatch in encoding because you are missing key steps, try this;

  1. Add the following lines to the beginning of Page 1 and putComments.asp

    <%@Language="VBScript" CodePage = 65001 %>
    <%
    Response.CodePage = 65001
    Response.CharSet = "UTF-8"
    %> 
    
    • First line tells IIS to process the content of the page as UTF-8.

    • Second line instructs all Response strings to be encoded as UTF-8.

    • Third line is the equivalent of setting ;Charset=UTF-8 in Content-Type header.

  2. Make sure all ASP files in the chain (including any #include files) are saved with UTF-8 encoding otherwise IIS processing the files as UTF-8 when they are not will cause encoding mismatches.

  3. Any data posted should be sent as UTF-8 this differs depending on the mechanism in your case you're using a JQuery Ajax Post (uses UTF-8 by default) so no change should be needed here.


Useful links

Community
  • 1
  • 1
user692942
  • 16,398
  • 7
  • 76
  • 175
  • Will give it a go tomorrow with fresh eyes. It's the first time in over 10 years of coding Classic ASP and however many since jQuery has been out that I've come across such an issue! How about any CSS and JS files that are referenced? – pee2pee May 07 '14 at 17:32
  • Response.ContentType = "text/html" Response.AddHeader "Content-Type", "text/html;charset=UTF-8" Response.CodePage = 65001 Response.CharSet = "UTF-8" – pee2pee May 07 '14 at 18:13
  • @pee2pee CSS and JS files should not matter it's how the HTML returned by the ASP is encoded. Sometimes just leaving it a while and coming back to it is the best thing you can do. – user692942 May 07 '14 at 19:28
  • How do I know which code page my files are saved as? – Ian Warburton Dec 11 '15 at 16:36
  • @IanWarburton Depends what you are using, if you use Visual Studio to edit your files you can check using the `Advanced Save Options` command *(which isn't shown by default)*. Customise the menu bar to add it, you can find it under the `File` menu commands. – user692942 Dec 11 '15 at 16:46
  • @IanWarburton NotePad++ is a good way as opening the file then shows the encoding in the bottom right of the status bar shown at the bottom of the GUI. Beware that these tools are just making best guesses based on the files header which can get confusing when you start talking about the BOM and can cause guesses to be wrong. Ref http://stackoverflow.com/a/14247070/692942 – user692942 Dec 11 '15 at 16:53
  • @IanWarburton Failing that you could always open the file in a HEX Editor and check the BOM - Ref [How to Determine Text File Encoding](http://codesnipers.com/?q=node/68) – user692942 Dec 11 '15 at 16:57
  • @Lankymart Yes, looks like UTF-8. EF BB BF. – Ian Warburton Dec 11 '15 at 17:06
  • @IanWarburton Excellent, so all you need to do now is tell the ASP page to send the response in `UTF-8`. Including the lines in my post above should work. – user692942 Dec 11 '15 at 17:17
  • @Isn't it possible to set this globally? – Ian Warburton Dec 11 '15 at 17:21
  • @IanWarburton You can use `Session.CodePage = 65001` or set that it the ASP section of IIS Manager. Personally though I'd just try it first and get it working before affecting the whole site. – user692942 Dec 11 '15 at 17:22
  • @Lankymart "Session.CodePage" seems to be the output codepage, where as the code page setting in IIS seems to be the file's codepage. Does this sounds correct? I've fixed my problem by leaving the ASP defaults but setting "Session.CodePage = 28591" in "Session_OnStart" in order to override files that are saved in UTF-8. Not doing this appears to result in "Response.Codepage" being set to 65001. I've done it this way around because I can't see how to set the default char set. – Ian Warburton Dec 12 '15 at 17:44
0
Response.ContentType = "text/html" 
Response.AddHeader "Content-Type", "text/html;charset=UTF-8" 
Response.CodePage = 65001 
Response.CharSet = "UTF-8"

Added this and was fine!

pee2pee
  • 3,619
  • 7
  • 52
  • 133
  • 1
    As I have already explained `Response.CharSet` is the same as setting `;charset` in `Response.AddHeader("Content-Type",` also `Response.ContentType = "text/html"` is the same as setting `Response.AddHeader("Content-Type", "text/html`. That line is completely irrelevant, so how is that any different to my answer? I told you it would work. – user692942 May 08 '14 at 12:09
  • I just added the first two lines to all the files in question and it worked. I took them out and it stopped working. I left one of each in and it didn't work. I put them both back and it worked. Whilst I appreciate the help and guidance, these two lines were the final piece to the jigsaw - http://stackoverflow.com/a/7195451/544542 – pee2pee May 08 '14 at 12:53
  • I find that very strange as the two statements are the same as `Response.AddHeader("Content-Type", "text/html;charset=UTF-8")`, but you are now the second person to [suggest this works](http://stackoverflow.com/questions/1453864/classic-asp-and-utf-8/7195451#7195451). Will have to look into it, but in all the years I've dealt with encoding in Classic ASP I have never come across this issue. – user692942 May 08 '14 at 13:05
  • It's literally the first time I've ever come across the encoding issue at all - from ASP, ColdFusion, PHP etc in over 10 years – pee2pee May 08 '14 at 13:10
  • If you had ever used the `¬` character as a string delimiter you might have encountered it before now, that one is a real pain. – user692942 May 08 '14 at 14:02