1

I am working on an integration with an xml based API. I am able to sign xml requests successfully using xmlsec1 like this...

xmlsec1 --sign --lax-key-search --privkey-pem test_privkey.pem,test_cert.pem --output xml/signed.xml xml/template.xml

template.xml: <root><Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><SignedInfo><CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"></CanonicalizationMethod><SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"></SignatureMethod><Reference URI=""><Transforms><Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"></Transform><Transform Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"></Transform></Transforms><DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></DigestMethod><DigestValue></DigestValue></Reference></SignedInfo><SignatureValue/></Signature></root>

(I've removed the actual content in an attempt to simplify/debug the issue explained below)

This works fine, but now I'd like to do this in Python...

# Load the private key
with open(KEY_PATH, "rb") as key_file:
    private_key = load_pem_private_key(key_file.read(), None)

# Load the XML file
tree = etree.parse(f"xml/{XML_FILENAME}")
root = tree.getroot()

# Create a Signature element
signature_element = etree.Element("Signature")
signature_element.attrib["xmlns"] = "http://www.w3.org/2000/09/xmldsig#"

# Create a SignedInfo element
signed_info_element = etree.SubElement(signature_element, "SignedInfo")

# Create a CanonicalizationMethod element
canonicalization_method_element = etree.SubElement(signed_info_element, "CanonicalizationMethod")
canonicalization_method_element.attrib["Algorithm"] = "http://www.w3.org/TR/2001/REC-xml-c14n-20010315"

# Create a SignatureMethod element
signature_method_element = etree.SubElement(signed_info_element, "SignatureMethod")
signature_method_element.attrib["Algorithm"] = "http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"

# Create a Reference element
reference_element = etree.SubElement(signed_info_element, "Reference")
reference_element.attrib["URI"] = ""

# Create Transforms
transforms_element = etree.SubElement(reference_element, "Transforms")
transform_element1 = etree.SubElement(transforms_element, "Transform")
transform_element1.attrib["Algorithm"] = "http://www.w3.org/2000/09/xmldsig#enveloped-signature"
transform_element2 = etree.SubElement(transforms_element, "Transform")
transform_element2.attrib["Algorithm"] = "http://www.w3.org/TR/2001/REC-xml-c14n-20010315"

# Create DigestMethod
digest_method_element = etree.SubElement(reference_element, "DigestMethod")
digest_method_element.attrib["Algorithm"] = "http://www.w3.org/2001/04/xmlenc#sha256"

# Compute the digest value for the entire XML document and add it to the DigestValue element
digest = hashlib.sha256(etree.tostring(root, method="c14n")).digest()
digest_value_element = etree.SubElement(reference_element, "DigestValue")
digest_value_element.text = b64encode(digest).decode()

# Canonicalize the SignedInfo XML
c14n_signed_info = etree.tostring(signed_info_element, method="c14n")

# Create a SHA256 digest of the SignedInfo
digest = hashlib.sha256(c14n_signed_info).digest()

# Sign the digest
signature = private_key.sign(
    digest,
    padding.PKCS1v15(),
    hashes.SHA256()
)

# Embed the SignatureValue in the Signature
signature_value_element = etree.SubElement(signature_element, "SignatureValue")
signature_value_element.text = b64encode(signature).decode()

# Add the Signature element to the root of the document
root.append(signature_element)

# Save the signed XML
tree = etree.ElementTree(root)
with open('xml/signed.xml', 'wb') as f:
    tree.write(f)

XML_FILENAME (simplified for debugging): <root></root>

When I submit the contents of signed.xml to the API it fails and this API provider is not able to help. In troubleshooting, I noticed that the SignatureValue is different between the Python and xmlsec1 versions (the digest value is the same).

Any ideas on why the SignatureValue would differ, or any other differences you see that would explain why it works with xmlsec1 and not with the Python code. I also tried the signxml Python library with the same results.

Thanks.

Bafsky
  • 761
  • 1
  • 7
  • 16
  • Using signxml looks like the way to go instead of a custom implementation https://github.com/XML-Security/signxml – LMC May 31 '23 at 22:49
  • @LMC as I mentioned, I tried signxml with the same results. I actually tried that first, and then tried the custom approach when it didn't work. – Bafsky Jun 01 '23 at 02:18
  • Custom security might be a [bad idea](https://security.stackexchange.com/a/52688). I would suggest to get signxml to work. – LMC Jun 01 '23 at 17:42
  • @LMC this code just signs an xml payload for a 3rd party API. It's not really custom security, just manually populating the xml signature elements rather than using a library. There's no real risk... it will either work or get rejected by the API. In any case I've found there's a Python version of xmlsec which works so I'm good to go. – Bafsky Jun 01 '23 at 20:58
  • good. You can post your own answer so others can benefit from it :-) – LMC Jun 01 '23 at 21:10
  • @LMC I don't have an answer for the question of why the signatures differ. The Python version of xmlsec can be found here... https://pypi.org/project/xmlsec/. That provides the same signature as the xmlsec1 command line. – Bafsky Jun 02 '23 at 01:17
  • I meant to post an answer with the working code with xmlsec. – LMC Jun 02 '23 at 06:02

1 Answers1

0

I have not looked closely but you appear to be adding elements to the XML before you do the digest. The digest should be across the canonical version of the original XML. The Signed info and signature is added after the digest is calculated.

Update:

Take a look at https://github.com/perl-net-saml2/perl-XML-Sig/blob/8cd5c375e1f29469c13b9925b28ab02cf1024468/lib/XML/Sig.pm#L319 if you can read perl :-). I have not run into any issues with signatures and I test against xmlsec1 regularly

Timothy Legge
  • 459
  • 1
  • 4
  • 5
  • Thanks, but nothing is being added to the root until the end, after the digest. Also, the digest matches the one from xmlsec1. It's the signature that doesn't match. – Bafsky Jun 01 '23 at 02:16
  • Take a look at my update. perls XML::Sig has a debug option that might help and generates a xmlsec1 valid signature – Timothy Legge Jun 01 '23 at 15:03
  • I've found there's a Python version of xmlsec which works so I'm good to go. – Bafsky Jun 01 '23 at 20:59