How to use "quoted-printable" content-transfer-encoding with BizTalk AS2 receiving?

Question

I'm currently using BizTalk Server 2013 R2 to exchange EDI as well as non-EDI documents using AS2 with a number of different trading partners. I recently added a new trading partner and after receiving a number of documents successfully I started seeing this error occur every now and then:

An output message of the component "Microsoft.BizTalk.EdiInt.PipelineComponents" in receive pipeline "Microsoft.BizTalk.EdiInt.DefaultPipelines.AS2Receive, Microsoft.BizTalk.Edi.EdiIntPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" is suspended due to the following error: The content transfer encoding quoted-printable is not supported.. The sequence number of the suspended message is 2.

After some investigation I found that the AS2 platform of the trading partner in question will sometimes set the Content-Transfer-Encoding of the MIME body part to quoted-printable when the enclosed XML payload contains non-ASCII characters. When this happens the message is suspended (non-resumable) with the error above.

Messages received from this trading partner are encrypted and signed, but not compressed - and received using a HTTP request-response (two-way) port configured with the out-of-the-box AS2Receive pipeline. I've tried using a custom pipeline with the AS Decoder, S/MIME decoder and AS2 disassembler components, but this does not seem to have any effect - the error stays the same.

I've also tried receiving unencrypted messages from the trading partner (by mutual agreement) but seem to be doing something wrong here as well as the message passed to the Message Box then ends up not being disassembled properly (the MIME part boundaries and AS2 signature is still visible in the actual message payload). Since the trading partner won't allow sending of unencrypted messages in a production environment anyway, I need to get this working with encryption. They also cannot change their platform's behavior as this will reportedly affect all of their other trading partners.

Here are the unfolded HTTP headers (ellipses denotes redacted values) of the encrypted and signed AS2 message received at the point of being suspended:

Date: Mon, 20 Jan 2020 17:30:53 GMT
Content-Length: 8014
Content-Type: application/pkcs7-mime; name="smime.p7m"; smime-type=enveloped-data
From: ...
Host: ...
User-Agent: Jakarta Commons-HttpClient/3.1
AS2-To: ...
Subject: AS2 Message from ... to ...
Message-Id: <1C20200120-173053-740219@xxx.xxx.130.163>
Disposition-Notification-To: <mailto:...> ...
Disposition-Notification-Options: signed-receipt-protocol=optional, pkcs7-signature; signed-receipt-micalg=optional, sha1
AS2-From: ...
AS2-Version: 1.1
content-disposition: attachment; filename="smime.p7m"
X-Original-URL: /as2

Here is the unencrypted (ellipses denotes redacted content) payload when exact same message is sent from source party without encryption:

------=_Part_16155_1587439544.1579506174880
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

...
------=_Part_16155_1587439544.1579506174880
Content-Type: application/pkcs7-signature; name=smime.p7s; smime-type=signed-data
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

...
------=_Part_16155_1587439544.1579506174880--

Question: does BizTalk Server support the quoted-printable encoding method? If it does, what am I doing wrong? If it does not, what are my options in terms of a workaround?

Answer 1

For anyone else that may encounter this same issue, I thought I'd share the solution I ended up with.

Since the error was encountered during AS2 receive pipeline processing, naturally my solution was focussed around creating a custom receive pipeline component that does more or less the same than the out-of-the-box AS2 decoder component, but with support for the quoted-printable encoding method:

1. Decode and decrypt the CMS/PKCS#7 data envelope

This is actually the easiest step with only 5 lines of code:

EnvelopedCms envelopedCms = new EnvelopedCms();
envelopedCms.Decode(encryptedData);
envelopedCms.Decrypt();
byte[] decryptedData = envelopedCms.Encode();
string decryptedMessageString = Encoding.ASCII.GetString(decryptedData);

-encryptedData is a byte-array instantiated from the body-part data stream of the AS2 message received bythe HTTP adapter.

-The Decrypt method automatically searches the user and computer certificate stores for the appropriate certificate private key and uses this to decrypt the AS2 payload. For more information on the `EnvelopedCms' class follow this link.

2. Convert any quoted-printable content in the payload to normal UTF-8 text

First we have to get the MIME boundary name from the content type string at the beginning of the decrypted payload:

int firstBlankLineInMessage = decryptedMessageString.IndexOf(Environment.NewLine + Environment.NewLine);
string contentType = decryptedMessageString.Substring(0, firstBlankLineInMessage);

Regex boundaryRegex = new Regex("boundary=\"(?<boundary>.*)\"");
Match boundaryMatch = boundaryRegex.Match(contentType);
if (!boundaryMatch.Success)
   throw new Exception("Failed to get boundary name from content type");

string boundary = "--" + boundaryMatch.Groups["boundary"].Value;

Then we split the envelope and re-merge without the content-type header part:

string[] messageParts = decryptedMessageString.Split(new string[] {boundary}, StringSplitOptions.RemoveEmptyEntries);
string signedMessageString = boundary + messageParts[1] + boundary + messageParts[2] + boundary + "--\r\n";

Next we get the `Content-Transfer-Encoding' value in the MIME body-part header:

int firstBlankLineInBodyPart = messageParts[1].IndexOf(Environment.NewLine + Environment.NewLine);
string partHeaders = messageParts[1].Substring(0, firstBlankLineInBodyPart);

Regex cteRegex = new Regex("Content-Transfer-Encoding: (?<cte>.*)");
Match cteMatch = cteRegex.Match(partHeaders);
if (!cteMatch.Success)
   throw new Exception("Failed to get CTE from body part headers");

string cte = cteMatch.Groups["cte"].Value;
string payload = messageParts[1].Substring(firstBlankLineInBodyPart).Trim(); 

And finally we check the CTE and decode if neccessary:

string payload = messageParts[1].Substring(firstBlankLineInBodyPart).Trim();
if (cte == "quoted-printable")
{
   // Get charset
   Regex charsetRegex = new Regex("Content-Type: .*charset=(?<charset>.*)");
   Match charsetMatch = charsetRegex.Match(partHeaders);
   if (!charsetMatch.Success)
      throw new Exception("Failed to get charset from body part headers");

   string charset = charsetMatch.Groups["charset"].Value;

   QuotedPrintableDecode(payload, charset);
}

Note: There are many different implementations out there for decoding QP, including a .NET implementation that has (reportedly) been found buggy by some users. I decided to use this implementation shared by Gonzalo.

3. Update the Content-Type HTTP header and BizTalk message body-part stream

string httpHeaders = objHttpHeaders.ToString().Replace("Content-Type: application/pkcs7-mime; name=\"smime.p7m\"; smime-type=enveloped-data", "Content-Type: application/xml");
inMessage.Context.Write("InboundHttpHeaders", "http://schemas.microsoft.com/BizTalk/2003/http-properties", httpHeaders);

MemoryStream payloadStream = new MemoryStream(Encoding.UTF8.GetBytes(payload));
payloadStream.Seek(0, SeekOrigin.Begin);
pipelineContext.ResourceTracker.AddResource(payloadStream);
inMessage.BodyPart.Data = payloadStream;

-pipelineContext is the IPipelineContext variable passed to the Execute method of the custom pipeline component
-inMessage is the IBaseMessage variable passed to the Execute method

Last Thoughts

The code above can still be improved in a number of ways:

• Checking HTTP headers for encryption before attempting to decrypt
• Re-encrypting payload before passing message to AS2 disassembler component (if required by BizTalk party configuration)
• Adding support for compression

If you'd like a copy of the source code drop me a message and I'll see about upping it to an online repo.

Answer 2

I had ticket opened with Microsoft BizTalk tech support on the issue. Their response is that

The quoted-printable encoding is not supported by MS BizTalk Server 2013R2" and most likely is not supported by MS BizTalk Server 2020