XPath returns empty results on SOAP Fault details? Fix lxml queries with proper XML namespace mapping

Learn why XPath returns nothing in SOAP Faults when default XML namespaces are used, and how to fix it in Python with lxml by mapping namespaces correctly.

When pulling data out of SOAP faults, everything may look fine until you hit elements living in a different XML namespace. A common symptom is that some XPath queries work while others silently return nothing, even though the element clearly exists in the response. Here is a minimal, real-world pattern of that situation and how to fix it with lxml and XPath.

Reproducing the issue

The response contains a SOAP Fault. Reading soap:Subcode/soap:Value and soap:Reason/soap:Text succeeds, but fetching the text of message under WebServiceFault fails with an empty XPath result.

from lxml import etree as ET

payload = '''<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
  <soap:Body>
    <soap:Fault>
      <soap:Code>
        <soap:Value>soap:Sender</soap:Value>
        <soap:Subcode>
          <soap:Value xmlns:ns1="http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd">
            ns1:unauthorized
          </soap:Value>
        </soap:Subcode>
      </soap:Code>
      <soap:Reason>
        <soap:Text xml:lang="en">AccessResult: result: Access Denied | AuthenticationAsked: true |
          ErrorCode: IDP_ERROR:
          137 | ErrorReason: null</soap:Text>
      </soap:Reason>
      <soap:Detail>
        <WebServiceFault xmlns="http://www.taleo.com/ws/integration/toolkit/2005/07">
          <code>SystemError</code>
          <message>AccessResult: result: Access Denied | AuthenticationAsked: true | ErrorCode:
            IDP_ERROR: 137 |
            ErrorReason: null</message>
        </WebServiceFault>
      </soap:Detail>
    </soap:Fault>
  </soap:Body>
</soap:Envelope>'''

doc = ET.fromstring(payload)
nsmap = {
    'soap': 'http://www.w3.org/2003/05/soap-envelope',
    'ns1': 'http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd"'
}

print(doc.xpath('//soap:Subcode/soap:Value', namespaces=nsmap)[0].text)
print(doc.xpath('//soap:Reason/soap:Text', namespaces=nsmap)[0].text)
print(doc.xpath('//soap:Detail/WebServiceFault/message', namespaces=nsmap)[0].text)

The last line tries to reach an element that visually sits right where we expect it, yet the query returns no nodes and indexing triggers IndexError: list index out of range.

What’s really happening

The node WebServiceFault declares a default namespace: xmlns="http://www.taleo.com/ws/integration/toolkit/2005/07". Inside this scope, elements like WebServiceFault, code and message are not in the empty namespace; they belong to that Taleo namespace.

XPath treats unprefixed names as belonging to no namespace. That’s why //soap:Detail/WebServiceFault/message does not match anything. The earlier queries worked because they addressed elements in the SOAP namespace using the soap prefix, which was mapped in namespaces=....

Fixing the query

The solution is to add the Taleo namespace to the namespace mapping and use that prefix explicitly for every element in that namespace.

from lxml import etree as ET

xml_text = payload  # reuse the same XML from above
root_node = ET.fromstring(xml_text)
ns_alias = {
    'soap': 'http://www.w3.org/2003/05/soap-envelope',
    'ns1': 'http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd"',
    'ns2': 'http://www.taleo.com/ws/integration/toolkit/2005/07'
}

result_text = root_node.xpath(
    '//soap:Detail/ns2:WebServiceFault/ns2:message',
    namespaces=ns_alias
)[0].text

print(result_text)

AccessResult: result: Access Denied | AuthenticationAsked: true | ErrorCode:
         IDP_ERROR: 137 |
         ErrorReason: null

Why this matters

SOAP payloads routinely mix namespaces: the envelope and fault live under the SOAP namespace, while domain-specific details often sit under a service-defined default namespace. If those defaults are not accounted for in XPath, queries appear correct but silently return empty results. Being explicit with namespace prefixes keeps your selectors accurate and resilient.

Takeaways

Inspect XML for default namespaces and map each one in your XPath execution context. Use those prefixes consistently in queries, including for elements that look unprefixed in the payload. This alone prevents elusive empty node sets and the downstream errors they trigger.