We Keep Coding

Parse XML with BIRT

I am pretty new to BIRT and XML but trying to fix a performance issue. Right now we have a report that is using a big XML file as the data source and it is taking to long to load. So I've been exploring different options. I found that you can use a Scripted Data Source and use java to parse the XML.
My problem is that it is leaving some of rows blank even if there is information in the XML file.
I have a test XML file that looks like this:
<?xml version="1.0"?>
<library>
<book category="COOKING">
<title lang="en">Everyday Italian</title>
<author name="Giada De Laurentiis" country="it"/>
<year>2005</year>
<info>asdfghjkl</info>
</book>
<book category="CHILDREN">
<title lang="en">Harry Potter</title>
<author name="J K. Rowling" country="uk" />
<year>2005</year>
<info>asdfghjkl</info>
</book>
<book category="WEB">
<title lang="en">XQuery Kick Start</title>
<author name="James McGovern" country="us" />
<author name="Per Bothner" country="us" />
<info>asdfghjkl</info>
</book>
<book category="WEB">
<title lang="en">Learning XML</title>
<author name="Erik T. Ray" country="us" />
<year>2003</year>
<info>asdfghjkl</info>
</book>
</library>
In my Data Set under open I have
importPackage(javax.xml.parsers);
importPackage(javax.xml.xpath);
importPackage(org.w3c.dom);
importPackage(org.xml.sax);
var factory = DocumentBuilderFactory.newInstance();
var builder = null;
doc = null;
var expr = null;
factory.setNamespaceAware(true);
builder = factory.newDocumentBuilder();
doc = builder.parse(params["FileName"].value);
doc.getDocumentElement().normalize();
// create an XPathFactory
var xFactory = XPathFactory.newInstance();
// create an XPath object
xpath = xFactory.newXPath();
// compile the XPath expression
expr = xpath.compile("/library/*");
// run the query and get a nodeset
var result = expr.evaluate(doc, XPathConstants.NODESET);
//initialize object variables to be used in the fetch method
nodes = result;
x = 0;
y = 0;
cnt = nodes.getLength();
Under fetch I have
var lboolAllowed = false;
if (x < cnt)
{
var nodeList = nodes.item(x).getChildNodes();
var nodeListLength = nodes.item(x).getChildNodes().getLength();
for(var i=0; i<nodeListLength; i++)
{
var nodeName = nodeList.item(i).getNodeName();
var nodeText = nodeList.item(i).getTextContent();
if(nodeName == "category")
{
row["category"] = nodeText;
}
if(nodeName == "year")
{
row["year"] = nodeText;
}
else if(nodeName == "info")
{
row["info"] = nodeText;
}
else if(nodeName == "title")
{
row["title"] = nodeText;
}
else if(nodeName == "author")
{
row["author"] = nodeText;
}
else if(nodeName == "country")
{
row["country"] = nodeText;
}
y++
}
x++;
y=0;
return (true);
}
return (false);
Then I have a Report Parameter called FileName that points to where the XML file is.
The rows that are coming up blank are category, author and country. I am using version 3.2.23.
Here is the XML Source from the report if that helps.
<?xml version="1.0" encoding="UTF-8"?>
<report xmlns="http://www.eclipse.org/birt/2005/design" version="3.2.23" id="1">
<property name="createdBy">Eclipse BIRT Designer Version 4.6.0.v201606072122</property>
<property name="units">in</property>
<property name="iconFile">/templates/blank_report.gif</property>
<property name="bidiLayoutOrientation">ltr</property>
<property name="imageDPI">120</property>
<parameters>
<scalar-parameter name="FileName" id="11">
<property name="hidden">true</property>
<property name="valueType">static</property>
<property name="dataType">string</property>
<property name="distinct">true</property>
<simple-property-list name="defaultValue">
<value type="constant">C:\Users\1122335\Desktop\Workspace\test\new207.xml</value>
</simple-property-list>
<list-property name="selectionList"/>
<property name="paramType">simple</property>
<property name="controlType">text-box</property>
<structure name="format">
<property name="category">Unformatted</property>
</structure>
</scalar-parameter>
</parameters>
<data-sources>
<script-data-source name="Data Source" id="9"/>
</data-sources>
<data-sets>
<script-data-set name="Data Set" id="12">
<list-property name="resultSetHints">
<structure>
<property name="position">1</property>
<property name="name">year</property>
<property name="dataType">integer</property>
</structure>
<structure>
<property name="position">2</property>
<property name="name">info</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">3</property>
<property name="name">category</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">4</property>
<property name="name">title</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">5</property>
<property name="name">author</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">6</property>
<property name="name">country</property>
<property name="dataType">string</property>
</structure>
</list-property>
<list-property name="columnHints">
<structure>
<property name="columnName">year</property>
<text-property name="displayName">year</text-property>
</structure>
<structure>
<property name="columnName">info</property>
<text-property name="displayName">info</text-property>
</structure>
<structure>
<property name="columnName">category</property>
<text-property name="displayName">category</text-property>
</structure>
<structure>
<property name="columnName">title</property>
<text-property name="displayName">title</text-property>
</structure>
<structure>
<property name="columnName">author</property>
<text-property name="displayName">author</text-property>
</structure>
<structure>
<property name="columnName">country</property>
<text-property name="displayName">country</text-property>
</structure>
</list-property>
<structure name="cachedMetaData">
<list-property name="resultSet">
<structure>
<property name="position">1</property>
<property name="name">year</property>
<property name="dataType">integer</property>
</structure>
<structure>
<property name="position">2</property>
<property name="name">info</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">3</property>
<property name="name">category</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">4</property>
<property name="name">title</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">5</property>
<property name="name">author</property>
<property name="dataType">string</property>
</structure>
<structure>
<property name="position">6</property>
<property name="name">country</property>
<property name="dataType">string</property>
</structure>
</list-property>
</structure>
<property name="dataSource">Data Source</property>
<method name="open"><![CDATA[importPackage(javax.xml.parsers);
importPackage(javax.xml.xpath);
importPackage(org.w3c.dom);
importPackage(org.xml.sax);
var factory = DocumentBuilderFactory.newInstance();
var builder = null;
doc = null;
var expr = null;
factory.setNamespaceAware(true);
builder = factory.newDocumentBuilder();
//doc = builder.parse("C:\Users\1122335\Desktop\Workspace\test\new207.xml");
doc = builder.parse(params["FileName"].value);
doc.getDocumentElement().normalize();
// create an XPathFactory
var xFactory = XPathFactory.newInstance();
// create an XPath object
xpath = xFactory.newXPath();
// compile the XPath expression
expr = xpath.compile("/library/*");
// run the query and get a nodeset
var result = expr.evaluate(doc, XPathConstants.NODESET);
//initialize object variables to be used in the fetch method
nodes = result;
x = 0;
y = 0;
cnt = nodes.getLength();
]]></method>
<method name="fetch"><![CDATA[var lboolAllowed = false;
if (x < cnt)
{
var nodeList = nodes.item(x).getChildNodes();
var nodeListLength = nodes.item(x).getChildNodes().getLength();
for(var i=0; i<nodeListLength; i++)
{
var nodeName = nodeList.item(i).getNodeName();
var nodeText = nodeList.item(i).getTextContent();
if(nodeName == "#category")
{
row["category"] = nodeText;
}
if(nodeName == "year")
{
row["year"] = nodeText;
}
else if(nodeName == "info")
{
row["info"] = nodeText;
}
else if(nodeName == "//title")
{
row["title"] = nodeText;
}
else if(nodeName == "//author")
{
row["author"] = nodeText;
}
else if(nodeName == "//country")
{
row["country"] = nodeText;
}
y++
}
x++;
y=0;
return (true);
}
return (false);]]></method>
<method name="close"><![CDATA[factory = null;
builder = null;
doc = null;
expr = null;
builder = null;
xFactory = null;
// create an XPath object
xpath = null;
result = null;
//initialize object variables to be used in the fetch method
nodes = null;
x = 0;
y = 0;
cnt = null;]]></method>
</script-data-set>
</data-sets>
<page-setup>
<simple-master-page name="Simple MasterPage" id="2">
<page-footer>
<text id="3">
<property name="contentType">html</property>
<text-property name="content"><![CDATA[<value-of>new Date()</value-of>]]></text-property>
</text>
</page-footer>
</simple-master-page>
</page-setup>
<body>
<table id="356">
<property name="dataSet">Data Set</property>
<list-property name="boundDataColumns">
<structure>
<property name="name">year</property>
<text-property name="displayName">year</text-property>
<expression name="expression" type="javascript">dataSetRow["year"]</expression>
<property name="dataType">integer</property>
</structure>
<structure>
<property name="name">info</property>
<text-property name="displayName">info</text-property>
<expression name="expression" type="javascript">dataSetRow["info"]</expression>
<property name="dataType">string</property>
</structure>
<structure>
<property name="name">category</property>
<text-property name="displayName">category</text-property>
<expression name="expression" type="javascript">dataSetRow["category"]</expression>
<property name="dataType">string</property>
</structure>
<structure>
<property name="name">title</property>
<text-property name="displayName">title</text-property>
<expression name="expression" type="javascript">dataSetRow["title"]</expression>
<property name="dataType">string</property>
</structure>
<structure>
<property name="name">author</property>
<text-property name="displayName">author</text-property>
<expression name="expression" type="javascript">dataSetRow["author"]</expression>
<property name="dataType">string</property>
</structure>
<structure>
<property name="name">country</property>
<text-property name="displayName">country</text-property>
<expression name="expression" type="javascript">dataSetRow["country"]</expression>
<property name="dataType">string</property>
</structure>
</list-property>
<column id="390"/>
<column id="391"/>
<column id="392"/>
<column id="393"/>
<column id="394"/>
<column id="395"/>
<header>
<row id="357">
<cell id="358">
<label id="359">
<text-property name="text">year</text-property>
</label>
</cell>
<cell id="360">
<label id="361">
<text-property name="text">info</text-property>
</label>
</cell>
<cell id="362">
<label id="363">
<text-property name="text">category</text-property>
</label>
</cell>
<cell id="364">
<label id="365">
<text-property name="text">title</text-property>
</label>
</cell>
<cell id="366">
<label id="367">
<text-property name="text">author</text-property>
</label>
</cell>
<cell id="368">
<label id="369">
<text-property name="text">country</text-property>
</label>
</cell>
</row>
</header>
<detail>
<row id="370">
<cell id="371">
<data id="372">
<property name="resultSetColumn">year</property>
</data>
</cell>
<cell id="373">
<data id="374">
<property name="resultSetColumn">info</property>
</data>
</cell>
<cell id="375">
<data id="376">
<property name="resultSetColumn">category</property>
</data>
</cell>
<cell id="377">
<data id="378">
<property name="resultSetColumn">title</property>
</data>
</cell>
<cell id="379">
<data id="380">
<property name="resultSetColumn">author</property>
</data>
</cell>
<cell id="381">
<data id="382">
<property name="resultSetColumn">country</property>
</data>
</cell>
</row>
</detail>
<footer>
<row id="383">
<cell id="384"/>
<cell id="385"/>
<cell id="386"/>
<cell id="387"/>
<cell id="388"/>
<cell id="389"/>
</row>
</footer>
</table>
</body>
</report>

This is an XML/DOM related issue: nodeName does not have # or // as prefix (that is XPath stuff) but just the element or attribute name, so not #category but just category, remove them in
if(nodeName == "#category")
and
else if(nodeName == "//author")
{
row["author"] = nodeText;
}
else if(nodeName == "//country")
{
row["country"] = nodeText;
}
The table in DOM Node javadoc shows what is returned by Node.getNodeName()
Additionally the nodes where you query category/author/country are wrong, see:
var book = nodes.item(x);
var nodeList = book.getChildNodes();
var nodeListLength = nodeList.getLength();
row["category"] = book.getAttribute("category");
for(int i=0; i<nodeListLength; i++)
{
var child = nodeList.item(i);
var nodeName = child.getNodeName();
var nodeText = child.getTextContent();
if(nodeName == "year")
{
row["year"] = nodeText);
}
else if(nodeName == "info")
{
row["info"] = nodeText);
}
else if(nodeName == "title")
{
row["title"] = nodeText);
}
else if(nodeName == "author")
{
row["author"] = child.getAttribute("name");
row["country"] = child.getAttribute("country");
}
Note: author is actually a list

jaxb null values while unmarshalling

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE xml>
<Settings version="3" xmlns="urn:Adapter-v3">
<Connections>
<Connection name ="A" description="DEV">
<SaveVersion version="M" siteURL="https://example.com" />
<Save>
<Id>id123</Id>
<Client>hello</Client>
</Save>
</Connection>
<Connection name ="B" description="DEV1">
<SaveVersion version="M" siteURL="https://example.com" />
<Auth>
<UserId>id123</UserId>
<Password>pass</Password>
</Auth>
</Connection>
</Connections>
<Mappings>
<Mapping cont="AA" auction="A1">
<Description>Desc</Description>
<Content
attr1="IO"
attr2="d"
attr3="Information"
attr4="false"
<Element enabled="false" count="200" prefix="DocLib_" itemPrefix="0" />
<Sub enabled="false" count="100" prefix="Folder_" itemPrefix="0" />
<FilenameA auction="N" delay="3" />
</Content>
</Mapping>
<Mapping cont="AB" auction="SharePointOLDev1">
<Description>Desc</Description>
<Content
attr1="IO"
attr2="d"
attr3="Information"
attr4="false"
<Element enabled="false" count="200" prefix="DocLib_" itemPrefix="0" />
<Sub enabled="false" count="100" prefix="1" itemPrefix="0" />
</Content>
</Mapping>
</Mappings>
<TypeMappings>
<TypeMapping Type="rfid" ext="msg" />
</TypeMappings>
</Settings>
Trying to unmarshal the above xml file to java objects.
Generated Settings.java and ObjectFactory.java file using xjc tool.These files are deployed in the src folder(along with other java files.)
WHen i unmarshal i get null valuues for Connections,Mappings and TypeMappings.
Is there any steps that i am missing?
public class JAXBImpl{
public static void main(String args[]){
JAXBContext jaxbContext;
Settings que= null;
jaxbContext = JAXBContext.newInstance(Settings.class);
Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
que= (Settings) jaxbUnmarshaller.unmarshal(file);
System.out.println(que.getConnections().getConnection());
}
}

How to change output coding of a XML of reading and writing at the same time in with Jdom?

I have this code, I want to read and want to write the "prueba3.xml" at the same time, the file is UTF8 but when I write the file, the encoding changes and displays strange characters, although I have added format.setEncoding("UTF-8"), it is not doing it correctly. Is it possible to change the output encoding to UTF8 with jdom SAXBuilder?
Input XML:
<?xml version="1.0" encoding="UTF-8"?>
<prueba>
<reg id="576340">
<dato cant="856" id="6" val="-1" num="" desc="ñápás" />
<dato cant="680" id="1" val="-1" num="" desc="résd" />
<dato cant="684" id="5" val="-1" num="" desc="..да и вообем" />
<dato cant="1621" id="1" val="-1" num="" desc="hi" />
<dato cant="1625" id="5" val="-1" num="" desc="Hola" />
</reg>
</prueba>
This is the code:
public static void main(String[] args) throws FileNotFoundException, JDOMException, IOException
{
//Se crea un SAXBuilder para poder parsear el archivo
File xml = new File("c:\\prueba3.xml");
Document doc = (Document) new SAXBuilder().build(xml);
Element raiz = doc.getRootElement();
//Recorremos los hijos de la etiqueta raíz
List articleRow = raiz.getChildren("reg");
for (int i = 0; i < articleRow.size(); i++) {
Element row = (Element) articleRow.get(i);
List images = row.getChildren("dato");
for (int j = 0; j < images.size(); j++) {
Element row2 = (Element) images.get(j);
String texto = row2.getAttributeValue("desc") ;
String id = row2.getAttributeValue("id");
if ((texto != null) && (texto !="") && (id.equals("1"))){
row2.getAttribute("desc").setValue("Raúl").toString();
}
}
Format format = Format.getRawFormat();
format.setEncoding("UTF-8");
XMLOutputter xmlOutput = new XMLOutputter(format);
xmlOutput = new XMLOutputter(format);
xmlOutput.output(doc, new FileWriter("c:\\prueba3.xml"));
}
System.out.println("fin");
}
Output XML:
<?xml version="1.0" encoding="UTF-8"?>
<prueba>
<reg id="576340">
<dato cant="856" id="6" val="-1" num="" desc="񡰡s" />
<dato cant="680" id="1" val="-1" num="" desc="Ra򬢠/>
<dato cant="684" id="5" val="-1" num="" desc="..?? ? ??????" />
<dato cant="1621" id="1" val="-1" num="" desc="Ra򬢠/>
<dato cant="1625" id="5" val="-1" num="" desc="Hola" />
</reg>
</prueba>
Greetings and thanks for your time.

This is a relatively common problem to encounter when using JDOM - especially in countries/regions with non-latin alphabets. In some senses I regret maintaining the use of Writer outputs at all in JDOM.
See the JavaDoc on XMLOutputter too: http://www.jdom.org/docs/apidocs/org/jdom2/output/XMLOutputter.html
The issue is that FileWriter uses the default encoding of the system to convert from the Writer to the underlying byte data. JDOM cannot control that conversion.
If you change the line of code:
xmlOutput.output(doc, new FileWriter("c:\\prueba3.xml"));
to use an OutputStream instead of a Writer:
try (OutputStream fos = new FileOutputStream("c:\\prueba3.xml")) {
xmlOutput.output(doc, fos);
}
... it will use the output as a byte-stream, and the systems' default encoding won't interfere with the output.
(P.S. There's no reason to assign the xmlOutput instance twice.)

I want to read attribute value of attribute "name" of Tag <Attribute> in which attribute "class" is present.. how do i do that?

<?xml version="1.0" encoding="UTF-8"?>
<Root xmlns:tns="http://www.example.org/book" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.example.org/book test.xsd ">
<Class name="AirwayBill">
<Attribute name="billNo" primary="true" />
<Attribute name="date" primary="false" />
<Attribute name="shipper" primary="false" class="Person" />
</Class>
<Class name="Person">
<Attribute name="perId" primary="true" />
<Attribute name="fname" primary="false" />
<Attribute name="lname" primary="false" />
</Class>
</Root>
I want to read attribute value of attribute "name" of Tag in which attribute "class" is present.. how do i do that? i am using javax.xml.parsers.DocumentBuilder and javax.xml.parsers.DocumentBuilderFactory classes to parse and read the xml file.

Probably there is a better way to do it, but this works:
public static void parseXml(){
File fXmlFile = new File("c://test.xml");
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder;
try {
dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(fXmlFile);
doc.getDocumentElement().normalize();
Element root = doc.getDocumentElement();
NodeList childList = root.getElementsByTagName("Class");
for (int i = 0; i<childList.getLength(); i++){
System.out.println("Class: " + childList.item(i).getAttributes().getNamedItem("name").getNodeValue());
NodeList attList = ((Element)childList.item(i)).getElementsByTagName("Attribute");
for (int j = 0; j<attList.getLength(); j++){
System.out.print(" Att: " + attList.item(j).getAttributes().getNamedItem("name").getNodeValue());
System.out.println(" primary " + attList.item(j).getAttributes().getNamedItem("primary").getNodeValue());
}
}
} catch (Exception e) {
e.printStackTrace();
}
}

how to get how many node with certain attributes with DOM

I have a task to parse an XML file with JDom in Eclipse. When I started to create the code, my code is only sufficient to get the length(how many) node with Tag name model. The task is more specific, I need to get the length of the node with tag name model which has attribute type. How should I modify my code to get the result?
This is part of my XML file
<container>
<models>
<model id="FM1" type="BoQ">
<meta
<phase phaseDesc="PRCR>SLCT>"/>
<domain domainCode="SPM.BOQ.RFP "/>
<levelOfDetail levelOfDetailCode="[4]"/>
</meta>
</model>
<model id="FM2" type="Object">
<meta>
<phase phaseDesc="PRCR>SLCT> "/>
<domain domainCode="BIM"/>
<levelOfDetail levelOfDetailCode="[4] "/>
</meta>
</model>
<model id="FM3">
<meta>
<phase phaseDesc="PRCR>SLCT>"/>
<domain domainCode="SPM.QTO"/>
<levelOfDetail levelOfDetailCode="[5]"/>
</meta>
</model>
<model id="FM4" type="BoQ">
<meta>
<phase phaseDesc="PRCR>SLCT>"/>
<domain domainCode="TSM.TSC"/>
<levelOfDetail levelOfDetailCode="[3]"/>
</meta>
</model>
</models>
<linkModels>
<linkModel id="LM1" type="QuantitySplit">
<meta>
<info>
<i k="ModelName" v="Linkmodell"/>
</info>
<domain domainCode="LKM.QSP" domainDesc="Link Model"/>
</meta>
<models>
<model id="FM1"/>
<model id="FM2"/>
<model id="FM3"/>
<model id="FM4"/>
</models>
</container>
public class xmldom {
public static void main(String[] args) {
Document xmlDoc = getDocument("./src/MMT_Angebot_Anfrage.xml");
System.out.println("Root: "+
xmlDoc.getDocumentElement().getNodeName());
NodeList model = xmlDoc.getElementsByTagName("model");
System.out.println("Number of Models " +
model.getLength());
String elementPhase = "phase";
String elementDomain = "domain";
String elementLOD = "levelOfDetail";
String attrPhase = "phaseCode";
String attrDomain = "domainCode";
String attrLOD = "levelOfDetailCode";
getElementAndAttrib(model, elementPhase, elementDomain, elementLOD, attrPhase, attrDomain, attrLOD);
}
private static Document getDocument(String docString) {
try {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setIgnoringComments(true);
factory.setIgnoringElementContentWhitespace(true);
factory.setValidating(true);
DocumentBuilder builder = factory.newDocumentBuilder();
return builder.parse(new InputSource(docString));
}
catch(Exception ex) {
System.out.println(ex.getMessage());
}
return null;
}
private static void getElementAndAttrib(NodeList model,
String elementPhase, String elementDomain, String elementLOD,
String attrPhase, String attrDomain, String attrLOD) {
try {
for(int i=0; i < model.getLength(); i++){
Node modelNode = model.item(i);
Element modelElement = (Element)modelNode;
NodeList phaseList = modelElement.getElementsByTagName(elementPhase);
NodeList domainList = modelElement.getElementsByTagName(elementDomain);
NodeList lodList = modelElement.getElementsByTagName(elementLOD);
Element phaseElement = (Element)phaseList.item(0);
Element domainElement = (Element)domainList.item(0);
Element lodElement = (Element)lodList.item(0);
NodeList elementList = phaseElement.getChildNodes();
System.out.println("=================================================");
System.out.println(attrPhase + " : "+ phaseElement.getAttribute(attrPhase));
System.out.println(attrDomain + " : "+ domainElement.getAttribute(attrDomain));
System.out.println(attrLOD + " : "+ lodElement.getAttribute(attrLOD));
}
}
catch(Exception ex) {
System.out.println(ex.getMessage());
}
}
}

You want to count the number of <model> elements that contains the type attribute? If, so XPath is the most suitable tool.
XPath xp = XPathFactory.newInstance().newXPath();
Double num = xp.evaluate("count(//model[#type])", doc, XPathConstants.NUMBER);
Btw, you are using DOM, not JDOM.