We Keep Coding

JDOM Transformer - don't contract empty elements

I'm using JDOM 2.0.6 to transform an XSLT into an HTML, but I'm coming across the following problem - sometimes the data should be empty, that is, I'll have in my XSLT the following:
<div class="someclass"><xsl:value-of select="somevalue"/></div>
and when somevalue is empty, the output I get is:
<div class="someclass"/>
which may be perfectly valid XML, but is not valid HTML, and causes problems when displaying the resulting page.
Similar problems occur for <span> or <script> tags.
So my question is - how can I tell JDOM not to contract empty elements, and leave them as <div></div>?
Edit
I suspect the problem is not in the actual XSLTTransformer, but later when using JDOM to write to html. Here is the code I use:
XMLOutputter htmlout = new XMLOutputter(Format.getPrettyFormat());
htmlout.getFormat().setEncoding("UTF-8");
Document htmlDoc = transformer.transform(model);
htmlDoc.setDocType(new DocType("html"));
try (OutputStreamWriter osw = new OutputStreamWriter(new FileOutputStream(outHtml), "UTF-8")) {
htmlout.output(htmlDoc, osw);
}
Currently the proposed solution of adding a zero-width space works for me, but I'm interested to know if there is a way to tell JDOM to treat the document as an HTML (be it in the transform stage or the output stage, but I'm guessing the problem lies in the output stage).

You can use a zero-width-space between the elements. This doesn't affect the HTML output, but keeps the open-close-tags separated because they have a non-empty content.
<div class="someclass">​<xsl:value-of select="somevalue"/></div>
Downside is: the tag is not really empty anymore. That would matter if your output would be XML. But for HTML - which is probably the last stage of processing - it should not matter.

In your case, the XML transform is happening directly to a file/stream, and it is no longer in the control of JDOM.
In JDOM, you can select whether the output from the JDOM document has expanded, or not-expanded output for empty elements. Typically, people have output from JDOM like:
XMLOutputter xout = new XMLOutputter(Format.getPrettyFormat());
xout.output(document, System.out);
You can modify the output format, though, and expand the empty elements
Format expanded = Format.getPrettyFormat().setExpandEmptyElements(true);
XMLOutputter xout = new XMLOutputter(expanded);
xout.output(document, System.out);
If you 'recover' (assuming it is valid XHTML?) the XSLT transformed xml as a new JDOM document you can output the result with expanded empty elements.

If you want to transform to a HTML file then consider to use Jaxp Transformer with a JDOMSource and a StreamResult, then the Transformer will serialize the transformation result as HTML if the output method is html (either as set in your code or as done with a no-namespace root element named html.

In addition to the "expandEmptyElements" option, you could create your own writer and pass it to the XMLOutputter:
XMLOutputter outputter = new XMLOutputter(Format.getPrettyFormat().setExpandEmptyElements(true));
StringWriter writer = new HTML5Writer();
outputter.output(document, writer);
System.out.println(writer.toString());
This writer can then modify all HTML5 void elements. Elements like "script" for example won't be touched:
private static class HTML5Writer extends StringWriter {
private static String[] VOIDELEMENTS = new String[] { "area", "base", "br", "col", "command", "embed", "hr",
"img", "input", "keygen", "link", "meta", "param", "source", "track", "wbr" };
private boolean inVoidTag;
private StringBuffer voidTagBuffer;
public void write(String str) {
if (voidTagBuffer != null) {
if (str.equals("></")) {
voidTagBuffer.append(" />");
super.write(voidTagBuffer.toString());
voidTagBuffer = null;
} else {
voidTagBuffer.append(str);
}
} else if (inVoidTag) {
if (str.equals(">")) {
inVoidTag = false;
}
} else {
for (int i = 0; i < VOIDELEMENTS.length; i++) {
if (str.equals(VOIDELEMENTS[i])) {
inVoidTag = true;
voidTagBuffer = new StringBuffer(str);
return;
}
}
super.write(str);
}
}
}
I know, this is dirty, but I had the same problem and didn't find any other way.

Java Modify XML

I want to read an XML file in Java and then update certain elements in that file with new values. My file is > 200mb and performance is important, so the DOM model cannot be used.
I feel that a StaX Parser is the solution, but there is no decent literature on using Java StaX to read and then write XML back to the same file.
(For reference I have been using the java tutorial and this helpful tutorial to get what I have so far)
I am using Java 7, but there doesn't seem to be any updates to the XML parsing API since...a long time ago. So this probably isn't relevant.
Currently I have this:
public static String readValueFromXML(final File xmlFile, final String value) throws FileNotFoundException, XMLStreamException
{
XMLEventReader reader = new XMLInputFactory.newFactory().createXMLEventReader(new FileReader(xmlFile));
String found = "";
boolean read = false;
while (reader.hasNext())
{
XMLEvent event = reader.nextEvent();
if (event.isStartElement() &&
event.asStartElement().getName().getLocalPart().equals(value))
{
read = true;
}
if (event.isCharacters() && read)
{
found = event.asCharacters().getData();
break;
}
}
return found;
}
which will read the XMLFile and return the value of the selected element. However, I have another method updateXMLFile(final File xmlFile, final String value) which I want to use in conjunction with this.
So my question is threefold:
Is there a StaX implementation for editing XML
Will XPath be any help? Can that be used without converting my file to a Document?
(More Generally) Why doesn't Java have a better XML API?

There are two things you may want to look at. The first is to use JAXB to bind the XML to POJOs which you can then have your way with and serialize the structure back to XML when needed.
The second is a JDBC driver for XML, there are several available for a fee, not sure if there are any open source ones or not. In my experience JAXB is the better choice. If the XML file is too large to handle efficiently with JAXB I think you need to look at using a database as a replacement for the XML file.

This is my approach, which reads events from the file using StaX and writes them to another file. The values are updated as the loop passes over the correctly named elements.
public void read(String key, String value)
{
try (FileReader fReader = new FileReader(inputFile); FileWriter fWriter = new FileWriter(outputFile))
{
XMLEventFactory factory = XMLEventFactory.newInstance();
XMLEventReader reader = XMLInputFactory.newFactory().createXMLEventReader(fReader);
XMLEventWriter writer = XMLOutputFactory.newFactory().createXMLEventWriter(fWriter);
while (reader.hasNext())
{
XMLEvent event = reader.nextEvent();
boolean update = false;
if (event.isStartElement() && event.asStartElement().getName().getLocalPart().equals(key))
{
update = true;
}
else if (event.isCharacters() && update)
{
Characters characters = factory.createCharacters(value);
event = characters;
update = false;
}
writer.add(event);
}
}
catch (XMLStreamException | FactoryConfigurationError | IOException e)
{
e.printStackTrace();
}
}

Creating an XML based on another XML in Java

I'd like to take an XML file, heavily structured and about half gig in size, and create from it another XML file, containing only selected elements of the original one.
1) How can I do that?
2) can it be done with DOM Parser? What is the size limit of the DOM parser?
Thanks!

If you have a very large source XML (like your 0.5 GB file), and wish to extract information from it, possibly creating a new XML, you might consider using an event-based parser which does not require loading the entire XML in memory. The simplest of these implementations is the SAX parser, which requires that you write an event listener which will capture events like document-start, element-start, element-end, etc, where you can inspect the data you are reading (the name of the element, the attributes, etc.) and decide if you are going to ignore it or do something with the data.
Search for a SAX tutorial using JAXP and you should find several examples. Another strategy which you might want to consider, depending on what you want to do is StAX.
Here is a simple example using SAX to read data from a XML file and extract some information based on search criteria. It's a very simple example I use to teach SAX processing. I think it might help your understanding of how it works. The search criteria is hardwired and consists of names of movie directors to search in a giant XML with a movie selection generated from IMDB data.
XML Source example ("source.xml" ~300MB file)
<Movies>
...
<Movie>
<Imdb>tt1527186</Imdb>
<Title>Melancholia</Title>
<Director>Lars von Trier</Director>
<Year>2011</Year>
<Duration>136</Duration>
</Movie>
<Movie>
<Imdb>tt0060390</Imdb>
<Title>Fahrenheit 451</Title>
<Director>François Truffaut</Director>
<Year>1966</Year>
<Duration>112</Duration>
</Movie>
<Movie>
<Imdb>tt0062622</Imdb>
<Title>2001: A Space Odyssey</Title>
<Director>Stanley Kubrick</Director>
<Year>1968</Year>
<Duration>160</Duration>
</Movie>
...
</Movies>
Here is an example of an event handler. It selects the Movie elements by matching strings. I extended DefaultHandler and implemented startElement() (called when an opening tag is found), characters() (called when a block of characters are read), endElement() (called when an end tag is found) and endDocument() (called once, when the document finished). Since the data that is read is not retained in memory, you have to save the data you are interested in yourself. I used some boolean flags and instance variables to save the current tag, current data, etc.
class ExtractMovieSaxHandler extends DefaultHandler {
// These are some parameters for the search which will select
// the subtrees (they will receive data when we set up the parser)
private String tagToMatch;
private String tagContents; // OR match
private boolean strict = false; // if strict matches will be exact
/**
* Sets criteria to select and copy Movie elements from source XML.
*
* #param tagToMatch Must contain text only
* #param tagContents Text contents of the tag
* #param strict If true, match must be exact
*/
public void setSearchCriteria(String tagToMatch, String tagContents, boolean strict) {
this.tagToMatch = tagToMatch;
this.tagContents = tagContents;
this.strict = strict;
}
// These are the temporary values we store as we parse the file
private String currentElement;
private StringBuilder contents = null; // if not null we are in Movie tag
private String currentData;
List<String> result = new ArrayList<String>(); // store resulting nodes here
private boolean skip = false;
...
These methods are the implementation of the ContentHandler. The first one detects an element was found (start tag). We save the name of the tag (child of Movie) in a variable, because it might be one we use in the search:
...
#Override
public void startElement(String uri, String localName, String qName, Attributes atts) throws SAXException {
// Store the current element that started now
currentElement = qName;
// If this is a Movie tag, save the contents because we might need it
if (qName.equals("Movie")) {
contents = new StringBuilder();
}
}
...
This one is called every time a block of characters is called. We check if those characters are occurring inside an element which interests us. If it is, we match the contents and save it if it matches.
...
#Override
public void characters(char[] ch, int start, int length) throws SAXException {
// if we discovered that we don't need this data, we skip it
if (skip || currentElement == null) {
return;
}
// If we are inside the tag we want to search, save the contents
currentData = new String(ch, start, length);
if (currentElement.equals(tagToMatch)) {
boolean discard = true;
if (strict) {
if (currentData.equals(tagContents)) { // exact match
discard = false;
}
} else {
if (currentData.toLowerCase().indexOf(tagContents.toLowerCase()) >= 0) { // matches occurrence of substring
discard = false;
}
}
if (discard) {
skip = true;
}
}
}
...
This is called when an end tag is found. We can now append it to the document we are building in memory if we wish.
...
#Override
public void endElement(String uri, String localName, String qName) throws SAXException {
// Rebuild the XML if it's a node we didn't skip
if (qName.equals("Movie")) {
if (!skip) {
result.add(contents.insert(0, "<Movie>").append("</Movie>").toString());
}
// reset the variables so we can check the next node
contents = null;
skip = false;
} else if (contents != null && !skip) {
contents.append("<").append(qName).append(">")
.append(currentData)
.append("</").append(qName).append(">");
}
currentElement = null;
}
...
Finally, this one is called when the document ends. I also used it to print the result at the end.
...
#Override
public void endDocument() throws SAXException {
StringBuilder resultFile = new StringBuilder();
resultFile.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
resultFile.append("<Movies>");
for (String childNode : result) {
resultFile.append(childNode.toString());
}
resultFile.append("</Movies>");
System.out.println("=== Resulting XML containing Movies where " + tagToMatch + " is one of " + tagContents + " ===");
System.out.println(resultFile.toString());
}
}
Here is a small Java application which loads that file, and uses an event handler to extract the data.
public class SAXReaderExample {
public static final String PATH = "src/main/resources"; // this is where I put the XML file
public static void main(String[] args) throws ParserConfigurationException, SAXException, IOException {
// Obtain XML Reader
SAXParserFactory spf = SAXParserFactory.newInstance();
SAXParser sp = spf.newSAXParser();
XMLReader reader = sp.getXMLReader();
// Instantiate SAX handler
ExtractMovieSaxHandler handler = new ExtractMovieSaxHandler();
// set search criteria
handler.setSearchCriteria("Director", "Kubrick", false);
// Register handler with XML reader
reader.setContentHandler(handler);
// Parse the XML
reader.parse(new InputSource(new FileInputStream(new File(PATH, "source.xml"))));
}
}
Here is the resulting file, after processing:
<?xml version="1.0" encoding="UTF-8"?>
<Movies>
<Movie>
<Imdb>tt0062622</Imdb>
<Title>2001: A Space Odyssey</Title>
<Director>Stanley Kubrick</Director>
<Year>1968</Year>
<Duration>160</Duration>
</Movie>
<Movie>
<Imdb>tt0066921</Imdb>
<Title>A Clockwork Orange</Title>
<Director>Stanley Kubrick</Director>
<Year>1972</Year>
<Duration>136</Duration>
</Movie>
<Movie>
<Imdb>tt0081505</Imdb>
<Title>The Shining</Title>
<Director>Stanley Kubrick</Director>
<Year>1980</Year>
<Duration>144</Duration>
</Movie>
...
</Movies>
Your scenario might be different, but this example shows a general solution which you can probably adapt to your problem. You can find more information in tutorials about SAX and JAXP.

500Mb is well within the limits of what can be achieved using XSLT. It depends a little bit on how much effort you want to expend to develop an optimum solution: i.e., which is more expensive, your time or the machine's time?

Handling XML escape characters (e.g. quotes) using JAXB Marshaller

I need to serialize an XML java object to a XML file using the JAXB Marshaller (JAXB version 2.2). Now in the xml object, I have a tag which contains String value such that:
"<"tagA>
**"<"YYYYY>done"<"/YYYYY>**
"<"/tagA>
Now as you can see that this string value again contains tags.
I want this to be written in the same way in the xml file.
But JAXB Marshaller converts these values such as:
"&"lt;YYYYY"&"gt;"&"#xD;done ...& so on
I am not able to treat these escape characters separately using JAXB 2.2
Is it possible anyways?
Any help in this regard will be great..
Thanks in advance,
Abhinav Mishra

Done it by setting the following property for the JAXB Marshaller:
marshaller.setProperty("jaxb.encoding", "Unicode");

There is one simpler way. First use custom escape sequence:
m.setProperty(CharacterEscapeHandler.class.getName(), new CharacterEscapeHandler() {
#Override
public void escape(char[] ch, int start, int length, boolean isAttVal, Writer out) throws IOException {
out.write( ch, start, length );
}
});
Then marshal it to a String like mentioned below
StringWriter writer = new StringWriter();
m.marshal(marshalObject, writer);
and then create a document object from the writer mentioned below
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
InputSource is = new InputSource( new StringReader( writer.toString() ) );
Document doc = builder.parse( is );
escape characters issue will be resolved

With JAXB marshaller if you want full control over which characters to escape(e.g. "\'") you will have to add property :
Marshaller marshaller = jc.createMarshaller();
marshaller.setProperty(CharacterEscapeHandler.class.getName(), new CustomCharacterEscapeHandler());
and create a new CustomCharacterEscapeHandler class
import com.sun.xml.bind.marshaller.CharacterEscapeHandler;
import java.io.IOException;
import java.io.Writer;
public class CustomCharacterEscapeHandler implements CharacterEscapeHandler {
public CustomCharacterEscapeHandler() {
super();
}
public void escape(char[] ch, int start, int length, boolean isAttVal, Writer out) throws IOException {
// avoid calling the Writerwrite method too much by assuming
// that the escaping occurs rarely.
// profiling revealed that this is faster than the naive code.
int limit = start+length;
for (int i = start; i < limit; i++) {
char c = ch[i];
if(c == '&' || c == '<' || c == '>' || c == '\'' || (c == '\"' && isAttVal) ) {
if(i!=start)
out.write(ch,start,i-start);
start = i+1;
switch (ch[i]) {
case '&':
out.write("&");
break;
case '<':
out.write("<");
break;
case '>':
out.write(">");
break;
case '\"':
out.write(""");
break;
case '\'':
out.write("&apos;");
break;
}
}
}
if( start!=limit )
out.write(ch,start,limit-start);
}
}
Hope that helps.

You can leverage the CDATA structure. Standard JAXB does not cover this structure. There is an extension in EclipseLink JAXB (MOXy) for this (I'm the tech lead). Check out my answer to a related question:
How to generate CDATA block using JAXB?
It describes the #XmlCDATA annotation in MOXy:
import javax.xml.bind.annotation.XmlRootElement;
import org.eclipse.persistence.oxm.annotations.XmlCDATA;
#XmlRootElement(name="c")
public class Customer {
private String bio;
#XmlCDATA
public void setBio(String bio) {
this.bio = bio;
}
public String getBio() {
return bio;
}
}
For more information see:
http://bdoughan.blogspot.com/2010/07/cdata-cdata-run-run-data-run.html

Depending on what you are exactly looking for you can either :
disable character escaping
or use CDATA string which support can be added into JAXB with just a bit of configuration

Java Plist XML Parsing

I'm parsing a (not well formed) Apple Plist File with java.
My Code looks like this:
InputStream in = new FileInputStream( "foo" );
XMLInputFactory factory = XMLInputFactory.newInstance();
XMLEventReader parser = factory.createXMLEventReader( in );
while (parser.hasNext()){
XMLEvent event = parser.nextEvent();
//code to navigate the nodes
}
The parts I"m parsing are looking like this:
<dict>
<key>foo</key><integer>123</integer>
<key>bar</key><string>Boom & Shroom</string>
</dict>
My problem is now, that nodes containing a ampersand are not parsed like they should because the ampersand is representing a entity.
What can i do to get the value of the node as a complete String, instead of broken parts?
Thank you in advance.

You should be able to solve your problem by setting the IS_COALESCING property on the XMLInputFactory (I also prefer XMLStreamReader over XMLEventReader, but ymmv):
XMLInputFactory factory = XMLInputFactory.newInstance();
factory.setProperty(XMLInputFactory.IS_COALESCING, Boolean.TRUE);
InputStream in = // ...
xmlReader = factory.createXMLStreamReader(in, "UTF-8");
Incidentally, to the best of my knowledge none of the JDK parsers will handle "not well formed" XML without choking. Your XML is, in fact, well-formed: it uses an entity rather than a raw ampersand.

There is a predefined method getElementText(), which is buggy in jdk1.6.0_15, but works ok with jdk1.6.0_19. A complete program to easily parse the plist file is this:
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import javax.xml.stream.XMLEventReader;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.events.XMLEvent;
public class Parser {
public static void main(String[] args) throws XMLStreamException, IOException {
InputStream in = new FileInputStream("foo.xml");
XMLInputFactory factory = XMLInputFactory.newInstance();
XMLEventReader parser = factory.createXMLEventReader(in);
assert parser.nextEvent().isStartDocument();
XMLEvent event = parser.nextTag();
assert event.isStartElement();
final String name1 = event.asStartElement().getName().getLocalPart();
if (name1.equals("dict")) {
while ((event = parser.nextTag()).isStartElement()) {
final String name2 = event.asStartElement().getName().getLocalPart();
if (name2.equals("key")) {
String key = parser.getElementText();
System.out.println("key: " + key);
} else if (name2.equals("integer")) {
String number = parser.getElementText();
System.out.println("integer: " + number);
} else if (name2.equals("string")) {
String str = parser.getElementText();
System.out.println("string: " + str);
}
}
}
assert parser.nextEvent().isEndDocument();
}
}

This library enables your Java application to handle property lists of various formats.
Read / write property lists from / to files, streams or byte arrays
Convert between property list formats
Property list contents are provided as objects from the NeXTSTEP environment (NSDictionary, NSArray, NSString, etc.)
Serialize native java data structures to property list objects
Deserialize from property list objects to native java data structures
<dependency>
<groupId>com.googlecode.plist</groupId>
<artifactId>dd-plist</artifactId>
<version>1.26</version>
</dependency>
dd-plist