Sign XML Digital Signatures Without the ds Prefix and Pass Verification: SignXML Limits, xmlsec Fix

Learn how to sign XML without ds prefix, why SignXML verification fails with default namespaces, and how xmlsec fixes it for reliable signing and validation.

Signing XML without the ds: prefix: why SignXML verification breaks and how to make it work with xmlsec

Some legacy systems expect XML Digital Signature elements to live in the correct namespace but without the ds: prefix. In other words, Signature and its children must be in http://www.w3.org/2000/09/xmldsig# with a default namespace, yet no explicit namespace prefix on the tags. Generating such a signature with SignXML is straightforward by overriding namespaces, but verification may fail. Below is a practical walkthrough that reproduces the issue and a working alternative using xmlsec.

Reproducible example with SignXML

The following code signs the XML twice: once with the default SignXML behavior (ds: present) and once with the default namespace forced (ds: removed). The second verification is where the failure occurs.

"""
Test Certificate Setup

Run the following commands to generate a self-signed cert and private key:

.. code-block:: bash

    openssl genrsa -out private.key 4096
    openssl req -new -key private.key -out request.csr
    openssl x509 -req -days 365 -in request.csr -signkey private.key -out certificate.crt
"""

from xml.etree.ElementTree import Element, SubElement, tostring

import lxml
import signxml
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization

KEY_FILE = "/app/private.key"
CERT_FILE = "/app/certificate.crt"


def load_cert_and_pubkey(path: str):
    with open(path, "rb") as fh:
        raw = fh.read()
    cert = x509.load_pem_x509_certificate(raw, default_backend())
    return cert, cert.public_key()


def load_privkey(path: str, password=None):
    with open(path, "rb") as fh:
        raw = fh.read()
    return serialization.load_pem_private_key(raw, password=password, backend=default_backend())


def build_source_xml():
    root = Element("ROOT")

    data_node = SubElement(root, "DATA")

    id_node = SubElement(data_node, "ID")
    id_node.text = "encrypted_some_data_data"

    key_node = SubElement(data_node, "SESSION_KEY")
    key_node.text = "encrypted_session_key"

    return tostring(root, encoding="unicode")


private_key_obj = load_privkey(KEY_FILE)
cert_obj, pubkey_obj = load_cert_and_pubkey(CERT_FILE)


def sign_with_ds_prefix_and_verify():
    xml_payload = build_source_xml()
    doc = lxml.etree.fromstring(xml_payload.encode("utf-8"))

    signer = signxml.XMLSigner(
        method=signxml.methods.enveloped,
        signature_algorithm="rsa-sha256",
        digest_algorithm="sha256",
        c14n_algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315",
    )

    signed_doc = signer.sign(
        doc,
        key=private_key_obj,
        exclude_c14n_transform_element=True,
    )

    serialized = lxml.etree.tostring(signed_doc, encoding="utf-8")

    parsed = lxml.etree.fromstring(serialized)
    signxml.XMLVerifier().verify(parsed, x509_cert=cert_obj)
    # Verification succeeds here.


def sign_without_ds_prefix_and_verify():
    xml_payload = build_source_xml()
    doc = lxml.etree.fromstring(xml_payload.encode("utf-8"))

    signer = signxml.XMLSigner(
        method=signxml.methods.enveloped,
        signature_algorithm="rsa-sha256",
        digest_algorithm="sha256",
        c14n_algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315",
    )

    # Force default namespace to xmldsig, removing the ds: prefix
    signer.namespaces = {None: signxml.namespaces.ds}

    signed_doc = signer.sign(
        doc,
        key=private_key_obj,
        exclude_c14n_transform_element=True,
    )

    serialized = lxml.etree.tostring(signed_doc, encoding="utf-8")

    parsed = lxml.etree.fromstring(serialized)
    # This call raises an exception in this scenario:
    # signxml.exceptions.InvalidInput: Expected to find XML element DigestMethod in {http://www.w3.org/2000/09/xmldsig#}Reference
    signxml.XMLVerifier().verify(parsed, x509_cert=cert_obj)

What actually breaks during verification

Verification fails only when the ds: prefix is removed by assigning the Digital Signature namespace as default. The error shows that the verifier expects to find a DigestMethod under the ds namespace. The stack trace indicates the lookup is made using ds:DigestMethod, and despite the elements being in the correct namespace, the lookup fails in this configuration.

signxml.exceptions.InvalidInput: Expected to find XML element DigestMethod in {http://www.w3.org/2000/09/xmldsig#}Reference

Signing with a ds: prefix works and verifies in the same environment. The same document structure without the prefix does not verify.

A working alternative: xmlsec without the ds: prefix

Creating a signed XML without the ds prefix using SignXML didn’t verify successfully here. Since the package hasn’t received recent updates, switching to xmlsec resolved the issue. With xmlsec, both signing and verification worked as required without the ds prefix, and it plays well with current lxml versions.

The sample below builds the same XML, signs it with an enveloped signature using a default namespace for xmldsig (no ds:), and then verifies it.

"""
Test Certificate Setup

Run the following commands to generate a self-signed cert and private key:

.. code-block:: bash

    openssl genrsa -out private.key 4096
    openssl req -new -key private.key -out request.csr
    openssl x509 -req -days 365 -in request.csr -signkey private.key -out certificate.crt
"""

import lxml.etree as LX
import xmlsec
from xml.etree.ElementTree import Element, SubElement, tostring

from cryptography import x509
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.backends import default_backend


PEM_KEY = "/app/tmp/private.key"
PEM_CERT = "/app/tmp/certificate.crt"


def read_pem_cert(p: str) -> bytes:
    with open(p, "rb") as f:
        return f.read()


def read_pem_key(p: str) -> bytes:
    with open(p, "rb") as f:
        return f.read()


def make_xml_payload() -> str:
    root = Element("ROOT")

    sec = SubElement(root, "DATA")

    rid = SubElement(sec, "ID")
    rid.text = "encrypted_some_data_data"

    skey = SubElement(sec, "SESSION_KEY")
    skey.text = "encrypted_session_key"

    return tostring(root, encoding="unicode")


priv_pem = read_pem_key(PEM_KEY)
cert_pem = read_pem_cert(PEM_CERT)


def sign_and_verify_with_xmlsec():
    xml_str = make_xml_payload()
    doc = LX.fromstring(xml_str.encode("utf-8"))

    # Create signature template with default namespace (ns=None)
    sig_node = xmlsec.template.create(
        doc,
        c14n_method=xmlsec.Transform.C14N,
        sign_method=xmlsec.Transform.RSA_SHA256,
        ns=None,
    )

    # Append the signature node under the document root
    doc.append(sig_node)

    # Reference the whole document and add enveloped transform
    ref = xmlsec.template.add_reference(sig_node, xmlsec.Transform.SHA256, uri="")
    xmlsec.template.add_transform(ref, xmlsec.Transform.ENVELOPED)

    # Signing
    ctx = xmlsec.SignatureContext()
    ctx.key = xmlsec.Key.from_memory(priv_pem, xmlsec.KeyFormat.PEM, None)
    ctx.sign(sig_node)

    # Re-parse and verify
    reparsed = LX.fromstring(LX.tostring(doc))
    sig_for_verify = xmlsec.tree.find_node(reparsed, xmlsec.Node.SIGNATURE)

    vctx = xmlsec.SignatureContext()
    vctx.key = xmlsec.Key.from_memory(cert_pem, xmlsec.KeyFormat.CERT_PEM, None)
    vctx.verify(sig_for_verify)

Why this behavior matters

Integration with legacy systems often comes with strict XML shape requirements that go beyond schema correctness. Even when elements are in the correct namespace, the presence or absence of a ds: prefix can make or break interoperability. Having a toolchain that cleanly supports default namespaces for xmldsig, and can verify those signatures, avoids brittle workarounds and reduces risk during handoffs.

Practical takeaways

If your target system mandates XML Digital Signature elements without the ds: prefix, be aware that SignXML may fail verification in this configuration. In contrast, xmlsec signs and verifies such documents reliably, and it aligns well with current dependencies. Choosing a library that reflects these constraints is the shortest path to a stable implementation.