5

I'm having a problem getting the PayPal Payflow Gateway accept my HTTPS POST with non US-ASCII characters in it. Not matter what I post with these special characters, it seems to want to accept only US-ASCII encoded bytes. If I send UTF-8 encoded bytes, it still works, but fails to parse some of the request NVP values. I know it is possible because I've posted with another developer's test page to my account (http://ccaples.com/ NVP Quick Test) and that post seems to preserve the special characters.

Here's an example. I've tried with MANY different Content-Type headers and Accept/Accept-Charset values including specifying "; charset=UTF-8" or "; charset=utf-8" in the Content-Type after the text/namevalue.

POST https://pilot-payflowpro.paypal.com/ HTTP/1.1
Content-Type: text/namevalue
User-Agent: KLMS Payflow API for Java
X-VPS-Request-ID: 36A4ED051A8B492ABF70E6BE51CB13D5
X-VPS-CLIENT-TIMEOUT: 20
Connection: close
X-VPS-VIT-INTEGRATION-PRODUCT: KLMS Payflow API for Java
X-VPS-VIT-INTEGRATION-VERSION: 2.0.008
X-VPS-VIT-PROXY: Y
X-VPS-VIT-RUNTIME-VERSION: 20.45-b01
X-VPS-VIT-OS-ARCHITECTURE: amd64
X-VPS-VIT-OS-VERSION: 6.1
X-VPS-VIT-OS-NAME: Windows 7
Cache-Control: no-cache
Pragma: no-cache
Host: pilot-payflowpro.paypal.com
Content-Length: 694

USER[8]=XXXXXXXX&VENDOR[13]=XXXXXXXXXXXXX&PARTNER[6]=PayPal&PWD[8]=XXXXXXXX&VERBOSITY[4]=HIGH&BILLTOEMAIL[28]=XXXXXXXX@kineticlearning.com&TRXTYPE[1]=A&TENDER[1]=C&ACCT[16]=4111111111111111&EXPDATE[4]=1117&CVV2[3]=123&BILLTOFIRSTNAME[4]=Josè&BILLTOLASTNAME[7]=Elkjærd&BILLTOSTREET[14]=123 Elm Street&BILLTOCITY[9]=Elm Creek&BILLTOSTATE[2]=VA&BILLTOZIP[5]=22203&BILLTOCOUNTRY[2]=US&BILLTOPHONENUM[12]=763-221-5593&CUSTBROWSER[108]=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 Safari/537.36&CUSTHOSTNAME[12]=192.168.1.10&CUSTIP[12]=192.168.1.10&AMT[6]=600.00&CURRENCY[3]=USD&COMMENT1[26]=Law Enforcement Curriculum&COMMENT2[16]=Standard Version

HTTP/1.1 200 OK
Connection: close
Server: VPS-3.033.00
X-VPS-Request-ID: 36A4ED051A8B492ABF70E6BE51CB13D5
Date: Sun, 25 Jan 2015 17:45:05 GMT
Content-type: text/namevalue
Content-length: 199

RESULT=104&PNREF=B70P7B6EBAE7&RESPMSG=Timeout waiting for Processor response&TRANSTIME=2015-01-25 09:44:47&BILLTOFIRSTNAME=Jos &BILLTOLASTNAME=NotProvided&AMT=600.00&ACCT=1111&EXPDATE=1117&CARDTYPE=0

Ignore the 104 timeout for now. PayPal is working on that. What you can see is that the BILLTOFIRSTNAME is Jos(weird character that's not è) and the æ character in BILLTOLASTNAME must have caused the server's NVP parsing algorithm to have completely failed to parse because it says NotProvided.

Here's the code that converts the Java string in NVP formatted form to bytes:

        final byte[] requestBytes = requestString.getBytes(Charsets.UTF_8);

Any idea what character encoding the server-side NVP parser is looking for by default, or how to tell it what character encoding I'm sending the body of the POST in?

J.C. Hamlin
  • 111
  • 7
  • 1
    [PayFlow Gateway Docs:](https://developer.paypal.com/webapps/developer/docs/classic/payflow/integration-guide/#supported-languages) `The Payflow Gateway only supports customer input and API parameter values that are in regular ASCII (English language) characters. Payflow does not support extended ASCII characters or any other character sets other than regular ASCII at this time` – EdSF Jan 27 '15 at 01:48

2 Answers2

4

I have found the solution. It appears that it is not supported by the Payflow Gateway Docs. The docs seem to be out of date and need updating. I worked with PayPal support and it does appear that the Payflow Gateway supports UTF-8. Here is the trick: either you need to NOT specify the length parameter in the name/value pairs you submit (i.e. BILLTOFIRSTNAME=Josè) or you have to specify in the length parameter as not the length in CHARACTERS (i.e. not value.length()), but the actual length in BYTES of the UTF-8 encoded byte stream for the value (i.e. value.getBytes("UTF-8").length). This transaction worked, stored the UTF-8 characters and displayed properly in the PayPal Manager, and the proper value was returned in the name/value pairs of the Payflow Gateway response. As you can see, the difference from above is that the BILLTOFIRSTNAME and BILLTOLASTNAME length tags are now larger and specify the length in bytes instead of the length in characters. Josè [4 characters but 5 bytes: 4A 6F 73 C3 A8] and Elkjærd & Grün [14 characters but 16 bytes: 45 6C 6B 6A C3 A6 72 64 20 26 20 47 72 C3 BC 6E].

POST https://pilot-payflowpro.paypal.com/ HTTP/1.1
Content-Type: text/namevalue
Connection: close
User-Agent: KLMS Payflow for Java/2.0.008
X-VPS-Request-ID: B8DABD9BDFE246EC909B4CF741030133
X-VPS-CLIENT-TIMEOUT: 20
X-VPS-VIT-INTEGRATION-PRODUCT: KLMS Payflow for Java
X-VPS-VIT-INTEGRATION-VERSION: 2.0.008
X-VPS-VIT-PROXY: Y
X-VPS-VIT-RUNTIME-VERSION: 20.45-b01
X-VPS-VIT-OS-ARCHITECTURE: amd64
X-VPS-VIT-OS-VERSION: 6.1
X-VPS-VIT-OS-NAME: Windows 7
Cache-Control: no-cache
Pragma: no-cache
Host: pilot-payflowpro.paypal.com
Content-Length: 771

USER[8]=XXXXXXXX&VENDOR[13]=XXXXXXXXXXXXX&PARTNER[6]=PayPal&PWD[8]=XXXXXXXX&VERBOSITY[4]=HIGH&BILLTOEMAIL[28]=XXXXXX@kineticlearning.com&TRXTYPE[1]=A&TENDER[1]=C&ACCT[16]=4111111111111111&EXPDATE[4]=0216&CVV2[3]=123&BILLTOFIRSTNAME[5]=Josè&BILLTOLASTNAME[16]=Elkjærd & Grün&BILLTOSTREET[14]=123 Elm Street&BILLTOSTREET2[10]=Elm & Vine&BILLTOCITY[9]=Elm Creek&BILLTOSTATE[2]=VA&BILLTOZIP[5]=12345&BILLTOCOUNTRY[2]=US&BILLTOPHONENUM[12]=763-221-5593&CUSTBROWSER[108]=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 Safari/537.36&CUSTHOSTNAME[12]=192.168.1.10&CUSTIP[12]=192.168.1.10&AMT[6]=600.00&CURRENCY[3]=USD&COMMENT1[45]=Victim Advocate Curriculum (Standard Version)&COMMENT2[36]=Purchased for J.C. Hamlin (jchamlin)

HTTP/1.1 200 OK
Connection: close
Server: VPS-3.033.00
X-VPS-Request-ID: B8DABD9BDFE246EC909B4CF741030133
Date: Wed, 28 Jan 2015 01:45:29 GMT
Content-type: text/namevalue
Content-length: 209

RESULT=104&PNREF=B10P7D2F1643&RESPMSG=Timeout waiting for Processor response&TRANSTIME=2015-01-27 17:45:12&BILLTOFIRSTNAME=Josè&BILLTOLASTNAME[16]=Elkjærd & Grün&AMT=600.00&ACCT=1111&EXPDATE=0216&CARDTYPE=0
J.C. Hamlin
  • 111
  • 7
  • In Java, `value.length` does not tell the number of characters in a string. It merely reports the number of 16-bit code-units, which is a very unuseful and low-level detail. To count characters, you must count entire code points, not their individual components under this or that encoding. – tchrist Jan 28 '15 at 04:24
  • String.length() works, but not entirely accurate. What is misleading is that String.length() is what the Payflow's Java SDK payflow.jar passes to the server. Also the Payflow Java SDK uses the system default encoding by calling String.getBytes() and new String(bytes). The payflow.jar is wrong in both cases (i.e. both things are real bugs in the Payflow Java SDK's implementation and are incompatible with the Payflow Gateway server). – J.C. Hamlin Jan 30 '15 at 20:43
  • This doesn't appear to work on my end after testing extensively. I see see Josè Elkjærd & Grün displaying as "Josè Elkjærd & Grün" – Devin Walker Nov 19 '18 at 23:07
0

For anyone else stumbling across this I ran into the same issue as Devin where the è character was being replaced with è when I looked up the transaction in Payflow. I had to convert the encoding to ISO-8859-1. I'm using c# not java so this may not be the same but it was as simple as specifying the encoding in my Stream Writer. 

using (StreamWriter sw = new StreamWriter(requestStream, Encoding.GetEncoding("ISO-8859-1")))

An explanation of why this may need to be done can be found here: https://www.i18nqa.com/debug/bug-utf-8-latin1.html

Jake Tigchelaar
  • 1