In an open source project I maintain, we have at least three different ways of reading, processing and writing XML files and I would like to standardise on a single method for ease of maintenance and stability.
Currently all of the project files use XML from the configuration to the stored data, we're hoping to migrate to a simple database at some point in the future but will still need to read/write some form of XML files.
The data is stored in an XML format that we then use a XSLT engine (Saxon) to transform into the final HTML files.
We currently utilise these methods:
- XMLEventReader/XMLOutputFactory (javax.xml.stream)
- DocumentBuilderFactory (javax.xml.parsers)
- JAXBContext (javax.xml.bind)
Are there any obvious pros and cons to each of these?
Personally, I like the simplicity of DOM (Document Builder), but I'm willing to convert to one of the others if it makes sense in terms of performance or other factors.
Edited to add:
There can be a significant number of files read/written when the project runs, between 100 & 10,000 individual files of around 5Kb each
It depends on what you are doing with the data.
If you are simply performing XSLT transforms on XML files to produce HTML files then you may not need to touch a parser directly:
import java.io.File;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
public class Demo {
public static void main(String[] args) throws Exception {
TransformerFactory tf = TransformerFactory.newInstance();
StreamSource xsltTransform = new StreamSource(new File("xslt.xml"));
Transformer transformer = tf.newTransformer(xsltTransform);
StreamSource source = new StreamSource(new File("source.xml"));
StreamResult result = new StreamResult(new File("result.html"));
transformer.transform(source, result);
}
}
If you need to make changes to the input document before you transform it, DOM is a convenient mechanism for doing this:
import java.io.File;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import org.w3c.dom.Document;
public class Demo {
public static void main(String[] args) throws Exception {
TransformerFactory tf = TransformerFactory.newInstance();
StreamSource xsltTransform = new StreamSource(new File("xslt.xml"));
Transformer transformer = tf.newTransformer(xsltTransform);
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
Document document = db.parse(new File("source.xml"));
// modify the document
DOMSource source = new DOMSource(document);
StreamResult result = new StreamResult(new File("result.html"));
transformer.transform(source, result);
}
}
If you prefer a typed model to make changes to the data then JAXB is a perfect fit:
import java.io.File;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.Unmarshaller;
import javax.xml.bind.util.JAXBSource;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
public class Demo {
public static void main(String[] args) throws Exception {
TransformerFactory tf = TransformerFactory.newInstance();
StreamSource xsltTransform = new StreamSource(new File("xslt.xml"));
Transformer transformer = tf.newTransformer(xsltTransform);
JAXBContext jc = JAXBContext.newInstance("com.example.model");
Unmarshaller unmarshaller = jc.createUnmarshaller();
Model model = (Model) unmarshaller.unmarshal(new File("source.xml"));
// modify the domain model
JAXBSource source = new JAXBSource(jc, model);
StreamResult result = new StreamResult(new File("result.html"));
transformer.transform(source, result);
}
}
This is a very subjective topic. It primarily depends on how you are going to use the xml and size of XML. If XML is (always) small enough to be loaded in to memory, then you don't have to worry about memory foot print. You can use DOM parser. If you need to a parse through 150 MB xml you may want to think of using SAX. etc.
Related
When I use colon in the tag name like in the example below, it ends up in error (there is no problem with tags without the colon).
package test;
import java.io.StringReader;
import java.io.StringWriter;
import javax.xml.stream.XMLOutputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamWriter;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
public class SomeClass{
public StringWriter test() throws XMLStreamException, TransformerConfigurationException, TransformerException {
StringWriter stringOut = new StringWriter();
XMLStreamWriter xmlWriter = XMLOutputFactory.newInstance().createXMLStreamWriter(stringOut);
xmlWriter.writeStartDocument("UTF-8", "1.0");
xmlWriter.writeStartElement("SomeWordHere");
{
xmlWriter.writeStartElement("SomeName:enable");//<--- notice the colon
xmlWriter.writeCharacters("true");
xmlWriter.writeEndElement();
}
xmlWriter.writeEndElement();
xmlWriter.writeEndDocument();
xmlWriter.flush();
xmlWriter.close();
Transformer transformer = TransformerFactory.newInstance().newTransformer();
transformer.setOutputProperty(OutputKeys.STANDALONE, "no");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
transformer.setOutputProperty(OutputKeys.METHOD, "xml");
transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "4");
StringWriter formattedStringWriter = new StringWriter();
transformer.transform(new StreamSource(new StringReader(stringOut.toString())), new StreamResult(formattedStringWriter));
return formattedStringWriter;
}
}
How to write the tag that would still conain the colon and would not end up in error?
I am trying to emulate the XML output (Collada DAE) produced by LEGO Stud.io software, there are sections like the one below containing tag names with colons.
<library_materials>
<material id="material_id_7" name="SOLID-BLUE">
<instance_effect url="#effect_id_7-fx" />
<extra>
<technique profile="eyesight">
<ScratchBump:enable> true </ScratchBump:enable>
<MinScratchStrength:value> 0 </MinScratchStrength:value>
<MaxScratchStrength:value> 0.2 </MaxScratchStrength:value>
<BigScratch:enable> true </BigScratch:enable>
<SmallScratch:enable> true </SmallScratch:enable>
</technique>
</extra>
</material>
</library_materials>
Colon is used for namespaces and per "Namespaces in XML" specification, it cannot be used in entity names.
The specification states:
[Definition: A document is namespace-well-formed if it conforms to
this specification. ]
It follows that in a namespace-well-formed document:
All element and attribute names contain either zero or one colon;
No entity names, processing instruction targets, or notation names contain any colons.
You can use a trick that is to declare "SomeName" as a namespace as it is suggested in this question: xml schema validation error "prefix is not bound".
On the other hand, "Extensible Markup Language" Specification state that:
Note:
The Namespaces in XML Recommendation [XML Names] assigns a meaning to
names containing colon characters. Therefore, authors should not use
the colon in XML names except for namespace purposes, but XML
processors must accept the colon as a name character.
If you change the parser you can get what you want:
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
public class CreateXmlFileDemo {
public static void main(String[] args) {
try {
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.newDocument();
Element rootElement = doc.createElement("SomeName:enable");
doc.appendChild(rootElement);
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
DOMSource source = new DOMSource(doc);
StreamResult consoleResult = new StreamResult(System.out);
transformer.transform(source, consoleResult);
} catch (Exception e) {
e.printStackTrace();
}
}
}
Reference: https://www.w3.org/TR/REC-xml-names/
I am using XMLStreamReader to achieve my goal(splitting xml file). It looks good, but still does not give the desired result. My aim is to split every node "nextTag" from an input file:
<?xml version="1.0" encoding="UTF-8"?>
<firstTag>
<nextTag>1</nextTag>
<nextTag>2</nextTag>
</firstTag>
The outcome should look like this:
<?xml version="1.0" encoding="UTF-8"?><nextTag>1</nextTag>
<?xml version="1.0" encoding="UTF-8"?><nextTag>2</nextTag>
Referring to Split 1GB Xml file using Java I achieved my goal with this code:
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.StringWriter;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamReader;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stax.StAXSource;
import javax.xml.transform.stream.StreamResult;
public class Demo4 {
public static void main(String[] args) throws Exception {
InputStream inputStream = new FileInputStream("input.xml");
BufferedReader in = new BufferedReader(new InputStreamReader(inputStream));
XMLInputFactory factory = XMLInputFactory.newInstance();
TransformerFactory tf = TransformerFactory.newInstance();
Transformer t = tf.newTransformer();
XMLStreamReader streamReader = factory.createXMLStreamReader(in);
while (streamReader.hasNext()) {
streamReader.next();
if (streamReader.getEventType() == XMLStreamReader.START_ELEMENT
&& "nextTag".equals(streamReader.getLocalName())) {
StringWriter writer = new StringWriter();
t.transform(new StAXSource(streamReader), new StreamResult(
writer));
String output = writer.toString();
System.out.println(output);
}
}
}
}
Actually very simple. But, my input file is in form from a single line:
<?xml version="1.0" encoding="UTF-8"?><firstTag><nextTag>1</nextTag><nextTag>2</nextTag></firstTag>
My Java code does not produce the desired output anymore, instead just this result:
<?xml version="1.0" encoding="UTF-8"?><nextTag>1</nextTag>
After spending hours, I am pretty sure to already find out the reason:
t.transform(new StAXSource(streamReader), new StreamResult(writer));
It is because, after the transform method being executed, the cursor will automatically moved forward to the next event. And in the code, I have this fraction:
while (streamReader.hasNext()) {
streamReader.next();
...
t.transform(new StAXSource(streamReader), new StreamResult(writer));
...
}
After the first transform, the streamReader gets directly 2 times next():
1. from the transform method
2. from the next method in the while loop
So, in case of this specific line XML, the cursor can never achive the second open tag .
In opposite, if the input XML has a pretty print form, the second can be reached from the cursor because there is a space-event after the first closing tag
Unfortunately, I could not find anything how to do settings, so that the transformator does not automatically spring to next event after performing the transform method. This is so frustating.
Does anybody have any idea how I can deal with it? Also semantically is very welcome. Thank you so much.
Regards,
Ratna
PS. I can surely write a workaround for this problem(pretty print the xml document before transforming it, but this would mean that the input xml was being modified before, this is not allowed)
As you elaborated did the transformation step proceed to the next create element if the element-nodes follow directly each other.
In order to deal with this, you can rewrite you code using nested while loops, like this:
while(reader.next() != XMLStreamConstants.END_DOCUMENT) {
while(reader.getEventType() == XMLStreamConstants.START_ELEMENT && reader.getLocalName().equals("nextTag")) {
StringWriter writer = new StringWriter();
// will transform the current node to a String, moves the cursor to the next START_ELEMENT
t.transform(new StAXSource(reader), new StreamResult(writer));
System.out.println(writer.toString());
}
}
In case your xml file fits in memory, you can try with the help of the JOOX library, imported in gradle like:
compile 'org.jooq:joox:1.3.0'
And the main class, like:
import java.io.File;
import java.io.IOException;
import org.joox.JOOX;
import org.joox.Match;
import org.w3c.dom.Document;
import org.xml.sax.SAXException;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import static org.joox.JOOX.$;
public class Main {
public static void main(String[] args)
throws IOException, SAXException, TransformerException {
DocumentBuilder builder = JOOX.builder();
Document document = builder.parse(new File(args[0]));
Transformer transformer =
TransformerFactory.newInstance().newTransformer();
transformer.setOutputProperty("omit-xml-declaration", "no");
final Match $m = $(document);
$m.find("nextTag").forEach(tag -> {
try {
transformer.transform(
new DOMSource(tag),
new StreamResult(System.out));
System.out.println();
}
catch (TransformerException e) {
System.exit(1);
}
});
}
}
It yields:
<?xml version="1.0" encoding="UTF-8"?><nextTag>1</nextTag>
<?xml version="1.0" encoding="UTF-8"?><nextTag>2</nextTag>
I have a SAX ContentHandler and want to bridge this to an StAX XMLStreamReader. Is this possible?
The following (untested) code should provide the “bridge” you require, using an XML copy transformation.
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stax.StAXSource;
import javax.xml.transform.sax.SAXResult;
void bridge(ContentHandler ch, XMLStreamReader sr) {
StAXSource src = new StAXSource(sr);
SAXResult res = new SAXResult(ch);
TransformerFactory tf = TransformerFactory.newInstance();
Transformer tr = tf.newTransformer();
tr.transform(src, res);
}
I am using this link to generate XML file using DOM. It says that "Xerces parser is bundled with the JDK 1.5 distribution.So you need not download the parser separately."
However, when I write the following line in my Eclipse Helios it gives compile-time error even though I have Java 1.6 in my system.
import org.apache.xml.serialize.XMLSerializer;
Why is it so?
Xerces is indeed bundled with the JDK but you should use it with the JAXP API under javax.xml.parsers. Check the output of the program below.
Also, to serialize an XML Document, you should use DOM Level 3 Load and Save (present in the JDK) or an XSLT transformation with no stylesheet (identity transformation). The rest is dependent on a specific implementation. The Xerces XMLSerializer is deprecated:
Deprecated. This class was deprecated in Xerces 2.9.0. It is recommended that new applications use the DOM Level 3 LSSerializer or JAXP's Transformation API for XML (TrAX) for serializing XML. See the Xerces documentation for more information.
Here is an example of serialization with DOM level 3:
import org.w3c.dom.*;
import org.w3c.dom.bootstrap.DOMImplementationRegistry;
import org.w3c.dom.ls.*;
public class DOMExample3 {
public static void main(String[] args) throws Exception {
DOMImplementationRegistry registry = DOMImplementationRegistry.newInstance();
DOMImplementationLS impl = (DOMImplementationLS) registry.getDOMImplementation("XML 3.0 LS 3.0");
if (impl == null) {
System.out.println("No DOMImplementation found !");
System.exit(0);
}
System.out.printf("DOMImplementationLS: %s\n", impl.getClass().getName());
LSParser parser = impl.createLSParser(
DOMImplementationLS.MODE_SYNCHRONOUS,
"http://www.w3.org/TR/REC-xml");
// http://www.w3.org/2001/XMLSchema
System.out.printf("LSParser: %s\n", parser.getClass().getName());
if (args.length == 0) {
System.exit(0);
}
Document doc = parser.parseURI(args[0]);
LSSerializer serializer = impl.createLSSerializer();
LSOutput output = impl.createLSOutput();
output.setEncoding("UTF-8");
output.setByteStream(System.out);
serializer.write(doc, output);
System.out.println();
}
}
Here is an example with an identity transformation:
import org.w3c.dom.Document;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Result;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
public class DOMExample2 {
public static void main(String[] args) throws Exception {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder parser = factory.newDocumentBuilder();
System.out.println("Parsing XML document...");
Document doc;
doc = parser.parse(args[0]);
// Xerces Java 2
/* Deprecated. This class was deprecated in Xerces 2.9.0.
* It is recommended that new applications use the DOM Level 3
* LSSerializer or JAXP's Transformation API for XML (TrAX)
* for serializing XML and HTML.
* See the Xerces documentation for more information.
*/
/*
System.out.println("XERCES: Displaying XML document...");
OutputFormat of = new OutputFormat(doc, "ISO-8859-1", true);
PrintWriter pw = new PrintWriter(System.out);
BaseMarkupSerializer bms = new XMLSerializer(pw, of);
bms.serialize(doc);
*/
// JAXP
System.out.println("JAXP: Displaying XML document...");
TransformerFactory transFactory = TransformerFactory.newInstance();
System.out.println(transFactory.getClass().getName());
//transFactory.setAttribute("indent-number", 2);
Transformer idTransform = transFactory.newTransformer();
idTransform.setOutputProperty(OutputKeys.METHOD, "xml");
idTransform.setOutputProperty(OutputKeys.INDENT,"yes");
// Apache default indentation is 0
idTransform.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "2");
Source input = new DOMSource(doc);
Result output = new StreamResult(System.out);
idTransform.transform(input, output);
}
}
It will be in, IIRC, com.sun.org.apache.xml.serialize.XMLSerializer. However, those are private classes and likely to change at any time. I suggest using the standard public APIs (javax.* and friends) instead. (Use the transform API without any XSLT.)
I found and followed an example from Stackoverflow (http://stackoverflow.com/questions/2310139/how-to-read-xml-response-from-a-url-in-java) of how to read an XML file from a URL (as you can see in my code pasted below). My only trouble is that now that I got the program to read the XML, how do I get it to store it? For example, could I make it save the information to a XML file built into the project (this would be the best solution for me, if it's possible)? Such as, take for example, I have a blank XML file built into the project. The program runs, reads the XML code off of the URL, and stores it all into the pre-built blank XML file. Could I do this?
If I sound confusing or un-clear about anything, just ask me to clarify what I'm looking for.
And here is my code, if you'd like to look at what I have so far:
package xml.parsing.example;
import java.io.IOException;
import java.net.URL;
import java.net.URLConnection;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.w3c.dom.Document;
import org.xml.sax.SAXException;
public class XmlParser {
public static void main (String[] args) throws IOException, ParserConfigurationException, SAXException, TransformerException {
URL url = new URL("http://totheriver.com/learn/xml/code/employees.xml");
URLConnection conn = url.openConnection();
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse(conn.getInputStream());
TransformerFactory tfactory = TransformerFactory.newInstance();
Transformer xform = tfactory.newTransformer();
// that’s the default xform; use a stylesheet to get a real one
xform.transform(new DOMSource(doc), new StreamResult(System.out));
}
}
Very simply:
File myOutput = new File("c:\\myDirectory\\myOutput.xml");
xform.transform(new DOMSource(doc), new StreamResult(myOutput));
This page has some great examples of how to serialize the DOM object to a neatly formatted XML file.