We Keep Coding

Java 8 - Split huge XML file using Stax gives unexpected results

When splitting a huge XML file I saw a very nice solution using Stax and Transformer.transform(). Nice BUT I see that some tags got lost. Why is that?
An XML file with Name... gives the following result. In the EVENT occasions the element tag is ommited.
Element: <?xml version="1.0" encoding="UTF-8"?><car><name>car1</name></car>
Element: <?xml version="1.0" encoding="UTF-8"?><name>car2</name>
Element: <?xml version="1.0" encoding="UTF-8"?><car><name>car3</name></car>
Element: <?xml version="1.0" encoding="UTF-8"?><name>car4</name>
How can I get the right elements? Has this to do with that transform( s, r) interferes with the input stream reading?
This is my code (which I saw in many places like this one). There is no change when using a StringReader or a FileReader.
I expected this: loop { advance to start-tag; get access to that element }
What I see is: 1st: the element + 2nd: parts of the element + repeated.
String testCars = "<root><car><name>car1</name></car><car><name>car2</name></car><car><name>car3</name></car><car><name>car4</name></car></root>";
String element = "car";
try {
XMLInputFactory factory = XMLInputFactory.newInstance();
XMLStreamReader streamReader = factory.createXMLStreamReader(new StringReader(testCars));
streamReader.nextTag();
TransformerFactory tf = TransformerFactory.newInstance();
Transformer t = tf.newTransformer();
while(streamReader.nextTag() == XMLStreamConstants.START_ELEMENT) {
StringWriter writer = new StringWriter();
StreamResult result = new StreamResult(writer);
t.transform(new StAXSource(streamReader), result);
System.out.println("Element: " + writer.toString());
}
} catch (Exception e) { ... }

Thanks to Andreas, this is the solution:
String testCars = "<root><car><name>car1</name></car><other><something>Unknown</something></other><car><name>car2</name></car></root>";
XMLInputFactory factory = XMLInputFactory.newInstance();
try {
XMLStreamReader streamReader = factory.createXMLStreamReader(new StringReader(testCars));
streamReader.nextTag();
TransformerFactory tf = TransformerFactory.newInstance();
Transformer t = tf.newTransformer();
streamReader.nextTag();
while ( streamReader.isStartElement() ||
( ! streamReader.hasNext() && streamReader.nextTag() == XMLStreamConstants.START_ELEMENT)) {
StringWriter writer = new StringWriter();
StreamResult result = new StreamResult(writer);
t.transform(new StAXSource(streamReader), result);
System.out.println( "XmlElement: " + writer.toString());
}
} catch (Exception e) { ... }
Input is:
<root>
<car>
<name>car1</name>
</car>
<other>
<something>Unknown</something>
</other>
<car>
<name>car2</name>
</car>
</root>
Output is:
XmlElement: <?xml version="1.0" encoding="UTF-8"?><car><name>car1</name></car>
XmlElement: <?xml version="1.0" encoding="UTF-8"?><other><something>Unknown</something></other>
XmlElement: <?xml version="1.0" encoding="UTF-8"?><car><name>car2</name></car>

Remove Namespace from tag

I searched in SO but I did not found nothing that solves my problem. I hope some one can help me.
I am building a XML file and I need to remove the Namespace xmlns.
That is my code
Document xmlDocument = new Document();
Namespace ns1 = Namespace.getNamespace("urn:iso:std:iso:20022:tech:xsd:pain.001.001.03");
Namespace ns2 = Namespace.getNamespace("xsi", "http://www.w3.org/2001/XMLSchema-instance");
Element root = new Element("Document", ns1);
root.addNamespaceDeclaration(ns2);
xmlDocument.setRootElement(root);
Element CstmrCdtTrfInitn = new Element("CstmrCdtTrfInitn");
root.addContent(CstmrCdtTrfInitn);
PrintDocumentHandler pdh = new PrintDocumentHandler();
pdh.setXmlDocument(xmlDocument);
request.getSession(false).setAttribute("pdh", pdh);
ByteArrayOutputStream sos = new ByteArrayOutputStream();
XMLOutputter outputter = new XMLOutputter(Format.getPrettyFormat());
Format format = outputter.getFormat();
format.setEncoding(SOPConstants.ENCODING_SCHEMA);
outputter.setFormat(format);
outputter.output(root, sos);
sos.flush();
return sos;
And this is the created XML-File
<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.001.03">
<CstmrCdtTrfInitn xmlns=""/>
</Document>
I have to remove the namespace xmlns from the tag CstmrCdtTrfInitn.
Many thanks in advance.

Namespace declaration without prefix (xmlns="...") is known as default namespace. Notice that, unlike prefixed namespace, descendant elements without prefix inherit ancestor's default namespace implicitly. So in the XML below, <CstmrCdtTrfInitn> is considered in the namespace urn:iso:std:iso:20022:tech:xsd:pain.001.001.03 :
<Document xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.001.03">
<CstmrCdtTrfInitn/>
</Document>
If this is the wanted result, instead of trying to remove xmlns="" later, you should try to create CstmrCdtTrfInitn using the same namespace as Document in the first place :
Element CstmrCdtTrfInitn = new Element("CstmrCdtTrfInitn", ns1);

I did some code for you, I dont know what package You are using... If You only want to remove the xmlns attribute from the CstmrCdtTrfInitn tag try with this code as other method to generate XML (I just modified this):
DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder docBuilder = docFactory.newDocumentBuilder();
// root elements
Document doc = docBuilder.newDocument();
Element rootElement = doc.createElement("Document");
rootElement.setAttribute("xmlns", "urn:iso:std:iso:20022:tech:xsd:pain.001.001.03");
rootElement.setAttribute("xmlns:xsi", "http://www.w3.org/2001/XMLSchema-instance");
doc.appendChild(rootElement);
// staff elements
Element CstmrCdtTrfInitn = doc.createElement("CstmrCdtTrfInitn");
rootElement.appendChild(CstmrCdtTrfInitn);
// write the content into xml file
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
DOMSource source = new DOMSource(doc);
// Output to console for testing
StreamResult result = new StreamResult(System.out);
transformer.transform(source, result);
} catch (ParserConfigurationException pce) {
pce.printStackTrace();
} catch (TransformerException tfe) {
tfe.printStackTrace();
}

How to add namespace prefix in Java DOM XML builder

I need to set prefix of the element in my xml. I need to print the xml in the format below:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<cars xmlns:dcterms="http://purl.org/dc/terms/" xmlns:msxsl="urn:schemas-microsoft-com:xslt">
<supercars company="Ferrari">
<dcterms:carname type="formula one">Ferrari 101</dcterms:carname>
<msxsl:carname type="sports">Ferrari 202</msxsl:carname>
</supercars>
</cars>
However, I am not able to add the prefix in my XML. What I am getting is this.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<cars xmlns:dcterms="http://purl.org/dc/terms/" xmlns:msxsl="urn:schemas-microsoft-com:xslt">
<supercars company="Ferrari">
<carname type="formula one">Ferrari 101</carname>
<carname type="sports">Ferrari 202</carname>
</supercars>
</cars>
Following is my Java Code:
public static void main(String args[]) {
try {
DocumentBuilderFactory dbFactory =
DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder =
dbFactory.newDocumentBuilder();
Document doc = dBuilder.newDocument();
// root element
Element rootElement = doc.createElement("cars");
rootElement.setAttributeNS("http://www.w3.org/2000/xmlns/", "xmlns:dcterms", "http://purl.org/dc/terms/");
rootElement.setAttributeNS("http://www.w3.org/2000/xmlns/", "xmlns:msxsl", "urn:schemas-microsoft-com:xslt");
doc.appendChild(rootElement);
// supercars element
Element supercar = doc.createElement("supercars");
rootElement.appendChild(supercar);
// setting attribute to element
Attr attr = doc.createAttribute("company");
attr.setValue("Ferrari");
// supercar.setAttributeNodeNS(attr);
supercar.setAttributeNode(attr);
// carname element
Element carname = doc.createElement("carname");
Attr attrType = doc.createAttribute("type");
attrType.setValue("formula one");
carname.setAttributeNode(attrType);
carname.appendChild(
doc.createTextNode("Ferrari 101"));
supercar.appendChild(carname);
Element carname1 = doc.createElement("carname");
Attr attrType1 = doc.createAttribute("type");
attrType1.setValue("sports");
carname1.setAttributeNode(attrType1);
carname1.appendChild(
doc.createTextNode("Ferrari 202"));
supercar.appendChild(carname1);
// write the content into xml file
TransformerFactory transformerFactory =
TransformerFactory.newInstance();
Transformer transformer =
transformerFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "2");
DOMSource source = new DOMSource(doc);
StreamResult result =
new StreamResult(new File("cars.xml"));
transformer.transform(source, result);
// Output to console for testing
StreamResult consoleResult =
new StreamResult(System.out);
transformer.transform(source, consoleResult);
} catch (Exception e) {
e.printStackTrace();
}
}
}
And I checked this link Adding namespace prefix XML String using XML DOM. However, this only talks about the SAX way of doing thing. I need to do this in DOM.
I tried setPrefix method and it is throwing me NAMESPACE_ERR

First of all, make sure you set dbFactory.setNamespaceAware(true);, if you want to work with a DOM supporting namespaces.
Then use createElementNS, e.g. Element carname = doc.createElementNS("http://purl.org/dc/terms/", "dcterms:carname");, to create elements in a namespace. Use the same approach for the element in the other namespace.

How to reposition a node in XML with DOM?

So I have this wich should put into an xml file a movie.
DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder docBuilder = docFactory.newDocumentBuilder();
Document doc = docBuilder.parse("D:\\College\\Java Eclipse\\tema5\\Movies\\Movies.xml");
try {
Element rootElement = doc.createElement("Movie");
doc.appendChild(rootElement);
// firstname elements
Element id = doc.createElement("Id");
id.appendChild(doc.createTextNode("3"));
rootElement.appendChild(id);
// lastname elements
Element name = doc.createElement("Name");
name.appendChild(doc.createTextNode("Movie 3"));
rootElement.appendChild(name);
// nickname elements
Element category = doc.createElement("Category");
category.appendChild(doc.createTextNode("Animation"));
rootElement.appendChild(category);
// salary elements
Element releasedate = doc.createElement("ReleaseDate");
releasedate.appendChild(doc.createTextNode("10-Jun-2012"));
rootElement.appendChild(releasedate);
Element rating = doc.createElement("Rating");
rating.appendChild(doc.createTextNode("10"));
rootElement.appendChild(rating);
// write the content into xml file
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
DOMSource source = new DOMSource(doc);
StreamResult result = new StreamResult(new File("D:\\College\\Java Eclipse\\tema5\\Movies\\Movies.xml"));
// Output to console for testing
// StreamResult result = new StreamResult(System.out);
transformer.transform(source, result);
} catch (TransformerException tfe) {
tfe.printStackTrace();
}
The problem is that in my xml file I already have two movies, when it tries to put the third one it succeeds but at the forth one it dies. I think it's because of the nodes and I want to know how to reposition the last to the end of the file so I can place more movies. This is the xml doc after the first insertion.
<?xml version="1.0" encoding="UTF-8" standalone="no"?><Movie>
<Movie>
<Id>1</Id>
<Name>Movie 1</Name>
<Category>Action</Category>
<ReleaseDate>22-JUN-2010</ReleaseDate>
<Rating>9</Rating>
</Movie>
<Movie>
<Id>2</Id>
<Name>Movie 2</Name>
<Category>Comedy</Category>
<ReleaseDate>2-JUN-2011</ReleaseDate>
<Rating>8</Rating>
</Movie>
</Movie>
<Movie>
<Id>3</Id>
<Name>Movie 3</Name>
<Category>Animation</Category>
<ReleaseDate>10-Jun-2012</ReleaseDate>
<Rating>10</Rating>
</Movie>

Element docRoot = doc.getDocumentElement();
Element rootElement = doc.createElement("Movie");
docRoot.appendChild(rootElement);
I would suggest renaming your rootElement variable to newMovie as it is missleading. You can only have one root element in xml, and you get it by doc.getDocumentElement()

Merging xml file using java NodeList

I'm trying to merge two xml files as shown below but i can't able to get the desired output please help me thank you
Java code:
DocumentBuilderFactory domFactory = DocumentBuilderFactory.newInstance();
domFactory.setIgnoringComments(true);
DocumentBuilder builder = domFactory.newDocumentBuilder();
Document doc = builder.parse(new File("file1.xml"));
Document doc1 = builder.parse(new File("file2.xml"));
NodeList nodes = doc.getElementsByTagName("staff");
NodeList nodes1 = doc1.getElementsByTagName("staff");
for(int i=0;i<nodes1.getLength();i=i+1){
Node n= (Node) doc.importNode(nodes1.item(i), true);
nodes.item(i).getParentNode().appendChild(n);
}
Transformer transformer = TransformerFactory.newInstance().newTransformer();
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
StreamResult result = new StreamResult(new StringWriter());
DOMSource source = new DOMSource(doc);
transformer.transform(source, result);
Writer output = null;
output = new BufferedWriter(new FileWriter("mergedxml.xml"));
String xmlOutput = result.getWriter().toString();
output.write(xmlOutput);
output.close();
System.out.println("merge complete");
File1.xml
<company>
<staff>
<name>john</name>
<phone>465456433</phone>
<email>gmail1</email>
</staff>
</company>
File2.xml
<company>
<staff>
<area>area1</area>
<city>city1</city>
</staff>
</company>
Current output:
<company>
<staff>
<name>john</name>
<phone>465456433</phone>
<email>gmail1</email>
</staff>
<staff>
<area>area1</area>
<city>city1</city>
</staff>
</company>
Expected Output:
<company>
<staff>
<name>john</name>
<phone>465456433</phone>
<email>gmail1</email>
<area>area1</area>
<city>city1</city>
</staff>
</company>

In order to do that on your own. You should do this following :
public static void mergeXML(){
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = null;
Document doc = null;
Document doc2 = null;
try {
db = dbf.newDocumentBuilder();
doc = db.parse(new File("D:\\Loic_Workspace\\Test2\\res\\test.xml"));
doc2 = db.parse(new File("D:\\Loic_Workspace\\Test2\\res\\test2.xml"));
NodeList ndListFirstFile = doc.getElementsByTagName("staff");
Node nodeArea = doc.importNode(doc2.getElementsByTagName("area").item(0), true);
Node nodeCity = doc.importNode(doc2.getElementsByTagName("city").item(0), true);
ndListFirstFile.item(0).appendChild(nodeArea);
ndListFirstFile.item(0).appendChild(nodeCity);
TransformerFactory tFactory = TransformerFactory.newInstance();
Transformer transformer = tFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
DOMSource source = new DOMSource(doc);
StreamResult result = new StreamResult(new StringWriter());
transformer.transform(source, result);
Writer output = new BufferedWriter(new FileWriter("D:\\Loic_Workspace\\Test2\\res\\testFinal.xml"));
String xmlOutput = result.getWriter().toString();
output.write(xmlOutput);
output.close();
} catch (ParserConfigurationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SAXException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (TransformerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Final output of testFinal.xml :
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<company>
<staff>
<name>john</name>
<phone>465456433</phone>
<email>gmail1</email>
<area>area1</area>
<city>city1</city>
</staff>
</company>
As you want it ;-)
Hope it helps,

This solution is for files where you need to iterate and verify something before to merge.
file1.xml:
<?xml version="1.0" encoding="UTF-8"?>
<reactions>
<reaction>
<ID>07402</ID>
<type>irreversible</type>
<substrate>15666</substrate>
<product>07756</product>
</reaction>
<reaction>
<ID>03063</ID>
<type>irreversible</type>
<substrate>00916</substrate>
<product>04712</product>
</reaction>
file2.xml:
<?xml version="1.0" encoding="UTF-8"?><reactions>
<reaction>
<ID>00001</ID>
<reactionName>polyphosphate polyphosphohydrolase</reactionName>
<reactionDescription> Polyphosphate + n H2O <=> (n+1) Oligophosphate</reactionDescription>
</reaction>
<reaction>
<ID>00002</ID>
<reactionName>Reduced ferredoxin:dinitrogen oxidoreductase (ATP-hydrolysing)</reactionName>
<reactionDescription> 16 ATP + 16 H2O + 8 Reduced ferredoxin <=> 8 e- + 16 Orthophosphate + 16 ADP + 8 Oxidized ferredoxin</reactionDescription>
</reaction>
<reaction>
<ID>03063</ID>
<reactionName>cephalosporin-C:2-oxoglutarate aminotransferase</reactionName>
<reactionDescription> Cephalosporin C + 2-Oxoglutarate <=> (7R)-7-(5-Carboxy-5-oxopentanoyl)aminocephalosporinate + D-Glutamate</reactionDescription>
</reaction>
<reaction>
<ID>07402</ID>
<reactionName>(7R)-7-(4-carboxybutanamido)cephalosporanate amidohydrolase</reactionName>
<reactionDescription> (7R)-7-(4-Carboxybutanamido)cephalosporanate + H2O <=> 7-Aminocephalosporanic acid + Glutarate</reactionDescription>
</reaction>
</reactions>
Result.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<reactions>
<reaction>
<ID>07402</ID>
<type>irreversible</type>
<substrate>15666</substrate>
<product>07756</product>
<reactionName>(7R)-7-(4-carboxybutanamido)cephalosporanate amidohydrolase</reactionName>
<reactionDescription> (7R)-7-(4-Carboxybutanamido)cephalosporanate + H2O <=> 7-Aminocephalosporanic acid + Glutarate</reactionDescription>
</reaction>
<reaction>
<ID>03063</ID>
<type>irreversible</type>
<substrate>00916</substrate>
<product>04712</product>
<reactionName>cephalosporin-C:2-oxoglutarate aminotransferase</reactionName>
<reactionDescription> Cephalosporin C + 2-Oxoglutarate <=> (7R)-7-(5-Carboxy-5-oxopentanoyl)aminocephalosporinate + D-Glutamate</reactionDescription>
</reaction>
</reactions>
Java program to do this:
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.StringWriter;
import java.io.Writer;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
public class MergeXML {
public static void main(String[] args) {
MergeXML m = new MergeXML();
try {
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db;
db = dbf.newDocumentBuilder();
Document secondaryMetabolismXML = db
.parse(new File("/home/bioinfo/workspace/teste/src/file1.xml"));
Document generalMetabolismXML = db
.parse(new File("/home/bioinfo/workspace/teste/src/file2.xml"));
NodeList secondaryReactions = secondaryMetabolismXML.getElementsByTagName("reaction");
NodeList generalReactions = generalMetabolismXML.getElementsByTagName("reaction");
for (int s = 0; s < secondaryReactions.getLength(); s++) {
Node secondaryReaction = secondaryReactions.item(s);
for (int g = 0; g < generalReactions.getLength(); g++) {
Node generalReaction = generalReactions.item(g);
if (getChildrenByNodeName(secondaryReaction, "ID").getTextContent()
.equals(getChildrenByNodeName(generalReaction, "ID").getTextContent())) {
if (getChildrenByNodeName(generalReaction, "reactionName") != null) {
secondaryReaction.appendChild(secondaryMetabolismXML
.importNode(getChildrenByNodeName(generalReaction, "reactionName"), true));
}
if (getChildrenByNodeName(generalReaction, "reactionAlternativeName") != null) {
secondaryReaction.appendChild(secondaryMetabolismXML.importNode(
getChildrenByNodeName(generalReaction, "reactionAlternativeName"), true));
}
if (getChildrenByNodeName(generalReaction, "reactionDescription") != null) {
secondaryReaction.appendChild(secondaryMetabolismXML
.importNode(getChildrenByNodeName(generalReaction, "reactionDescription"), true));
}
}
}
}
TransformerFactory tFactory = TransformerFactory.newInstance();
Transformer transformer = tFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
DOMSource source = new DOMSource(secondaryMetabolismXML);
StreamResult result = new StreamResult(new StringWriter());
transformer.transform(source, result);
Writer output = new BufferedWriter(
new FileWriter("/home/bioinfo/workspace/teste/src/Result.xml"));
String xmlOutput = result.getWriter().toString();
output.write(xmlOutput);
output.close();
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* Returns a node child when you have a match with a given node name
*
* #param node
* #param nodeName
* #return
*/
public static Node getChildrenByNodeName(Node node, String nodeName) {
for (Node childNode = node.getFirstChild(); childNode != null;) {
Node nextChild = childNode.getNextSibling();
if (childNode.getNodeName().equalsIgnoreCase(nodeName)) {
return childNode;
}
childNode = nextChild;
}
return null;
}
}

Problem is, you want to append child elements to "staff" element but what you actually do is :
nodes.item(i).getParentNode().appendChild(n);
meaning you're looking for the parent node of one of the "staff" nodes of the list, and that node is "company". So you're appending a new "staff" node (the one imported from doc1) to the "company" node of doc
Now, what you want to do is to iterate over "staff" child nodes of doc1 and to append them one by one to the "staff" node of doc.
So you would want to change nodes1 definition as following :
// Retrieving child nodes of first "staff" element of doc1
NodeList nodes1 = doc1.getElementsByTagName("staff").item(0).getChildNodes();
Then change the node you append that to by replacing
nodes.item(i).getParentNode().appendChild(n);
by
nodes.item(0).appendChild(n);
So now you'll be appending all "staff" child nodes (/!\ only for the first "staff" element) of doc1 to the first "staff" element of doc
Note 1 : Dont use the iteration variable (i) you use to run through list A to select an item of another list unless you're aware of what you're doing (for example both lists are of the same length)
Note 2 : That solution will append the nodes of first "staff" element of doc1 to the first "staff" element of doc. You will maybe want to add some iterations here and there.