Recently, we had a security audit on our code, and one of the problem is that our application is subject to the Xml eXternal Entity (XXE) attack.
Basically, the application is a calculator that receives inputs as XML, through a Web-Service.
Here is an example of such an XXE attack on our application:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<foo:calculateStuff>
<!--Optional:-->
<xmlInput><![CDATA[<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE currency [
<!ENTITY include SYSTEM "file:///d:/" >]>
<calcinput>...</calcinput>
]]></xmlInput>
</foo:calculateStuff>
</soapenv:Body>
</soapenv:Envelope>
As you can see, we can refer to an entity that points to an external file ("file:///d:/").
Regarding the XML input itself (the <calcinput>...</calcinput> part) is unmarshalled with JAXB (v2.1). The web-service part is based on jaxws-rt (2.1).
What do I need to do to secure my web-service?
JAXB
You can prevent the Xml eXternal Entity (XXE) attack by unmarshalling from an XMLStreamReader that has the IS_SUPPORTING_EXTERNAL_ENTITIES and/or XMLInputFactory.SUPPORT_DTD properties set to false.
JAX-WS
A JAX-WS implementation should take care of this for you. If it doesn't I would recommend opening a bug against the specific implmententation.
EXAMPLE
Demo
package xxe;
import javax.xml.bind.*;
import javax.xml.stream.*;
import javax.xml.transform.stream.StreamSource;
public class Demo {
public static void main(String[] args) throws Exception {
JAXBContext jc = JAXBContext.newInstance(Customer.class);
XMLInputFactory xif = XMLInputFactory.newFactory();
xif.setProperty(XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES, false);
xif.setProperty(XMLInputFactory.SUPPORT_DTD, false);
XMLStreamReader xsr = xif.createXMLStreamReader(new StreamSource("src/xxe/input.xml"));
Unmarshaller unmarshaller = jc.createUnmarshaller();
Customer customer = (Customer) unmarshaller.unmarshal(xsr);
Marshaller marshaller = jc.createMarshaller();
marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
marshaller.marshal(customer, System.out);
}
}
input.xml
This XML document contains an entity that has been setup to get the listing of files I used to create this example.
<?xml version="1.0"?>
<!DOCTYPE customer
[
<!ENTITY name SYSTEM "/Users/bdoughan/Examples/src/xxe/">
]
>
<customer>
<name>&name;</name>
</customer>
Customer
package xxe;
import javax.xml.bind.annotation.XmlRootElement;
#XmlRootElement
public class Customer {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
Output - Default Configuration
By default the entity will be resolved.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<customer>
<name>Customer.java
Demo.java
input.xml
</name>
</customer>
Output when XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES property is set to false
When this property is set the entity is not resolved.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<customer>
<name></name>
</customer>
Output when XMLInputFactory.SUPPORT_DTD property is set to false
When this property is set an exception is thrown trying to resolve the entity.
Exception in thread "main" javax.xml.bind.UnmarshalException
- with linked exception:
[javax.xml.stream.XMLStreamException: ParseError at [row,col]:[8,15]
Message: The entity "name" was referenced, but not declared.]
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.handleStreamException(UnmarshallerImpl.java:436)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:372)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:342)
at xxe.Demo.main(Demo.java:18)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[8,15]
Message: The entity "name" was referenced, but not declared.
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:598)
at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:196)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:370)
... 2 more
Related
Got this error
java.lang.IllegalArgumentException: Unsupported element: net
from this example xml file
<?xml version="1.0" encoding="UTF-8"?>
<net>
<node label="A">
...
</node>
<node label="B">
...
</node>
<node label="C">
...
</node>
</net>
with these java code lines
...
FileInputStream file = new FileInputStream("example.xml");
XMLDecoder decoder = new XMLDecoder(file);
Object decodedResistors = (Object) decoder.readObject();
file.close();
...
Do not use java.beans.XMLDecoder for deserialisation custom XML payloads. It was not designed for that. Read article Long Term Persistence of JavaBeans Components: XML Schema. It contains some example XML payloads which can be deserialised back by XMLDecoder:
<?xml version="1.0" encoding="UTF-8" ?>
<java version="1.4.0" class="java.beans.XMLDecoder">
<void id="myController" property="owner"/>
<object class="javax.swing.JButton">
<void method="addActionListener">
<object class="java.beans.EventHandler" method="create">
<class>java.awt.event.ActionListener</class>
<object idref="myController"/>
<string>doIt</string>
</object>
</void>
</object>
</java>
If you need to deserialise custom XML use JAXB or Jackson XML. You need to create a POJO model with JAXB annotations:
#XmlRootElement(name = "net")
#XmlAccessorType(XmlAccessType.FIELD)
class Net {
#XmlElement(name = "node")
private List<Node> nodes;
// getters, setters, toString
}
#XmlAccessorType(XmlAccessType.FIELD)
class Node {
#XmlAttribute
private String label;
// getters, setters, toString
}
Example usage:
import javax.xml.bind.JAXBContext;
import javax.xml.bind.Unmarshaller;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import java.io.File;
import java.util.List;
public class JaxbApp {
public static void main(String[] args) throws Exception {
File xmlFile = new File("./resource/test.xml").getAbsoluteFile();
JAXBContext jaxbContext = JAXBContext.newInstance(Net.class);
Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
Object net = unmarshaller.unmarshal(xmlFile);
System.out.println(net);
}
}
prints:
Net{nodes=[Node{label='A'}, Node{label='B'}, Node{label='C'}]}
See also:
java.lang.IllegalArgumentException: Unsupported element: rss
I have the following result:
<Result xmlns="urn:buscape" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" totalResultsAvailable="1" totalResultsReturned="1" totalPages="1" page="1" totalLooseOffers="0" xsi:schemaLocation="http://developer.buscape.com/admin/lomadee.xsd">
<details>
<applicationID>999999999999999</applicationID>
<applicationVersion>1.0</applicationVersion>
<applicationPath/>
<date>2016-09-12T23:50:19.722-03:00</date>
<elapsedTime>19</elapsedTime>
<status>success</status>
<code>0</code>
<message>success</message>
</details>
<lomadeeLinks>
<lomadeeLink>
<id>1</id>
<originalLink>link</originalLink>
<redirectLink>link2</redirectLink>
<code>0</code>
</lomadeeLink>
</lomadeeLinks>
</Result>
Looking at "2.7.4 Retrieving XML data via HTTP GET", I have to map this XML to POJO-like object in Java, here is the question, I can't find the #Root and #Element annotation and I'm not sure how to correctly map the XML into a Java object.
If you are using spring/springboot then simply you can use
(YourPOJO)getWebServiceTemplate().marshalSendAndReceive(yourSOAPService);
POJO :-
#XmlAccessorType(XmlAccessType.FIELD)
#XmlType(name = "", propOrder = {
"getOrdersResponse"
})
#XmlRootElement(name = "nameOfTag", namespace = "http://example.org/yourResource")
public class GetOrdersByDateResult {
#XmlElement(name = "GetOrdersResponse")
protected GetOrdersResponseType getOrdersResponse;
getter();
setter();
}
I am trying to create XML using jaxb like below format, where child element has separate name space.
<soap:Envelope xmlns:soap="http://demo.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Header>
<element1 xmlns="http://childnamespacehere">
<att1>test</att1>
<att2>test</att2>
</element1>
</soap:Header>
<soap:Body>
<element2 xmlns="http://childnamespacehere">
<att1>test</att1>
<att2>test</att2>
</element2 >
</soap:Body>
</soap:Envelope>
my class
#XmlRootElement(name = "soap:Envelope", namespace = "http://schemas.xmlsoap.org/soap/envelope/")
public class Envelope
private Element1 element1;
private Element2 element2;
#XmlElementWrapper(name = "soap:Header")
#XmlElement(name = "Element1", namespace = "http://childelementnamespace/")
public void setElement1(Element1 element){ }
#XmlElementWrapper(name = "soap:Body")
#XmlElement(name = "Element2" , namespace = "http://childelementnamespace/")
public void setElement2(Element2 element){ }
but i am getting xml generated like below, where child schema is at root level.
<soap:Envelope xsi:schemaLocation="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://childelementnamespace/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soap:Header>
<ns2:Element1>
<att1>value</att1>
<att2>value</att2>
</ns2:Element1>
</soap:Header>
<soap:Body>
<ns2:Element2>
<att1>value</att1>
<att2>value</att2>
</ns2:Element2>
</soap:Body>
</soap:Envelope>
i have #xmlschema defined in package-info.java
#XmlSchema(namespace = "http://schemas.xmlsoap.org/soap/envelope/",
xmlns = { #javax.xml.bind.annotation.XmlNs(prefix = "Element1", namespaceURI = "http://childelementnamespace"),
#javax.xml.bind.annotation.XmlNs(prefix = "Element2", namespaceURI = "http://childelementnamespace") },
elementFormDefault = XmlNsForm.QUALIFIED)
package com.model;
import javax.xml.bind.annotation.XmlNsForm;
import javax.xml.bind.annotation.XmlSchema;
when i generate xml , name space for child elements are not getting generated , i only get namespace for root element.
i have solved by adding "xmlns" attribute to objects (childnode) Element1 and Element2.
class Elemenet1
#XmlAttribute(name="xmlns")
String xmlns = "http://childnamespacehere";
public void setXmlns(String namespace){};
public String getXmlns(){};
Output
<soap:Envelope xmlns:soap="http://demo.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Header>
<element1 xmlns="http://childnamespacehere">
<att1>test</att1>
<att2>test</att2>
</element1>
</soap:Header>
<soap:Body>
<element2 xmlns="http://childnamespacehere">
<att1>test</att1>
<att2>test</att2>
</element2 >
</soap:Body>
</soap:Envelope>
In where you say what is being generated is has xmlns:ns2="http://childelementnamespace/" up the top, this is declaring the namespace and used in this fashion <ns2:Element2> using ns2 here uses tha namspace declared previously.
So what you are expecting and what you are getting are the exact same just declared in different places, the jaxB method is more correct as it is not declaring the same namespace more than once.
This is a temporary solution. This triggers major problems when you want to unmarshall a xml document.
But you can use different packages for marshall and unmarshall process too.
Edit: here's how I'm loading the XML document, as I used it in Blaise's answer. I'm loading it like this because I want to work with a node, not the whole doc. Even using the whole document I'm still having trouble when loading in this manner.
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setValidating(false);
factory.setNamespaceAware(false);
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse("[path to doc]/input.xml");
TestClass testClass = (TestClass) unmarshaller.unmarshal(doc);
I've got XML that looks like this:
<test>
<items>
<item type="cookie">cookie</item>
<item type="crackers">crackers</item>
</items>
</test>
And a class:
#XmlAccessorType(XmlAccessType.FIELD)
#XmlRootElement(name = "test")
public class TestClass
{
#XmlPath("items/item/text()")
#XmlElement
private ArrayList<String> itemList = new ArrayList<String>();
// getters, setters omitted
}
The above code will work whether or not I have #XmlElement, and I get an ArrayList containing [cookie, crackers].
If I change the declaration above to
#XmlPath("items/item/#type")
#XmlElement
private ArrayList<String> itemList = new ArrayList<String>();
my ArrayList is empty.
My ultimate goal is to just have attributes so my XML would look like this:
<test>
<items>
<item type="cookie"/>
<item type="crackers"/>
</items>
</test>
Is what I'm trying to do, pull out a list of attributes using XPath, possible, and if so, how?
Thank you.
UPDATE
I have been able to confirm the issue you are seeing (https://bugs.eclipse.org/353763). A fix has been added into our EclipseLink 2.3.1 and 2.4.0 streams and can be obtained from the nightly download page starting August 4th, 2011:
http://www.eclipse.org/eclipselink/downloads/nightly.php
Workaround:
You can workaround this issue by setting your DocumentBuilderFactory to be namespace aware:
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setNamespaceAware(true);
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse("src/forum6907225/input.xml");
testClass = (TestClass) unmarshaller.unmarshal(doc);
marshaller.marshal(testClass, System.out);
You are doing the mapping correctly (see below). Have you included a jaxb.properties file to specify EclipseLink MOXy as your JAXB provider?:
http://blog.bdoughan.com/2011/05/specifying-eclipselink-moxy-as-your.html
Test Class
package forum6907225;
import java.util.ArrayList;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import org.eclipse.persistence.oxm.annotations.XmlPath;
#XmlAccessorType(XmlAccessType.FIELD)
#XmlRootElement(name = "test")
public class TestClass
{
#XmlPath("items/item/#type")
#XmlElement
private ArrayList<String> itemList = new ArrayList<String>();
// getters, setters omitted
}
Demo
package forum6907225;
import java.io.File;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.Marshaller;
import javax.xml.bind.Unmarshaller;
import org.eclipse.persistence.Version;
public class Demo {
public static void main(String[] args) throws Exception {
JAXBContext jc = JAXBContext.newInstance(TestClass.class);
System.out.println(Version.getVersionString());
Unmarshaller unmarshaller = jc.createUnmarshaller();
File xml = new File("src/forum6907225/input.xml");
TestClass testClass = (TestClass) unmarshaller.unmarshal(xml);
Marshaller marshaller = jc.createMarshaller();
marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
marshaller.marshal(testClass, System.out);
}
}
input.xml
<?xml version="1.0" encoding="UTF-8"?>
<test>
<items>
<item type="cookie">cookie</item>
<item type="crackers">crackers</item>
</items>
</test>
Output
2.3.1.qualifier
<?xml version="1.0" encoding="UTF-8"?>
<test>
<items>
<item type="cookie"/>
<item type="crackers"/>
</items>
</test>
My xml file is,
<?xml version="1.0"?>
<type xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
xs:noNamespaceSchemaLocation="datatype.xsd">
<int>integer</int>
<varchar>varcharacter</varchar>
<double>doubles</double>
</type>
In this xml, I want to set <float></float> as mandatory. But i didn't use this tag. So how to validate the <float> is present or not in my xml file, using xsd with java.? Thanks in advance.
The following can be used to validate your XML against a schema:
import java.io.File;
import javax.xml.XMLConstants;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import javax.xml.validation.Validator;
public class Demo {
public static void main(String[] args) throws Exception {
SchemaFactory sf = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Schema schema = sf.newSchema(new File("datatype.xsd"));
Validator validator = schema.newValidator();
validator.setErrorHandler(new MyErrorHandler());
validator.validate(source);
}
}
For a more detailed example see:
http://bdoughan.blogspot.com/2010/11/validate-jaxb-object-model-with-xml.html