We Keep Coding

Trying to iterate over very similar Elements in an XML file. NOTE XML file is attribute less

Hullo, I have a question about xml and java. I have a weird XML file with no attributes and only Elements, im trying to zero in on a specific Element Stack, and then iterate over all of the similar element stacks.
<InstrumentData>
<Action>Entire Plot</Action>
<AppStamp>Vectorworks</AppStamp>
<VWVersion>2502</VWVersion>
<VWBuild>523565</VWBuild>
<AutoRot2D>false</AutoRot2D>
<UID_1505_1_1_0_0> ---- This is the part I care about, there are about 1000+ of these and they all vary slightly after the "UID_"---
<Action>Update</Action>
<TimeStamp>20200427192323</TimeStamp>
<AppStamp>Vectorworks</AppStamp>
<UID>1505.1.1.0.0</UID>
</UID_1505_1_1_0_0>
I am using dom4j as the xml parser and I dont have any issues spitting out all of the data I just want to zero in on the XML path.
This is the code so far:
public class Unmarshal {
public Unmarshal() {
File file = new File("/Users/michaelaboah/Desktop/LIHN 1.11.18 v2020.xml");
SAXReader reader = new SAXReader();
try {
Document doc = reader.read(file);
Element ele = doc.getRootElement();
Iterator<Element> it = ele.elementIterator();
Iterator<Node> nodeIt = ele.nodeIterator();
while(it.hasNext()) {
Element test2 = (Element) it.next();
List<Element> eleList = ele.elements();
for(Element elementsIt : eleList) {
System.out.println(elementsIt.selectSingleNode("/SLData/InstrumentData").getStringValue());
//This spits out everything under the Instrument Data branch
//All of that data is very large
System.out.println(elementsIt.selectSingleNode("/SLData/InstrumentData/UID_1505_1_1_0_0").getStringValue());
//This spits out everything under the UID branch
}
}
} catch (DocumentException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Also, I know there are some unused data types and variables there was a lot of testing

I think your answer is:
elementsIt.selectSingleNode("/SLData/InstrumentData/*[starts-with(local-name(), 'UID_')]").getStringValue()
I used this post to find this XPath and it works with the few xml lines you gave.

JAVA - How can i read an XML file and save each node to a string array

I new to java and XML.
My goal is to get info from my XML file and save it to a string array for later use in my code for testing my website.
The XML contains elements of each page divided to 3 categories: Name, Attribute, Text.
My first step was just to isolate the data i want and print it, and already got stuck.
Here is an example for my XML file (the original has a lot more nodes using the same structure):
<?xml version="1.0" encoding=""ISO-8859-1""?>
<config>
<HomeScreenName>
<Logo>Logo</Logo>
<Mainimage>Main image</Mainimage>
<Maintext>Main text</Maintext>
<Backupbutton>Backup button</Backupbutton>
<ViewBackupbutton>View Backup button</ViewBackupbutton>
<Version>Version</Version>
<Cancelaccountbutton>Cancel account button</Cancelaccountbutton>
</HomeScreenName>
<HomeScreenAttributes>
<Logo>/html/body/div[1]/div[1]</Logo>
<Mainimage>//*[#id="img-content"]</Mainimage>
<Maintext>/html/body/div[1]/div[3]/h3</Maintext>
<Backupbutton>/html/body/div[1]/div[3]/div[1]/a/span</Backupbutton>
<ViewBackupbutton>/html/body/div[1]/div[3]/div[2]/a/span</ViewBackupbutton>
<Version>//*[#id="version"]</Version>
<Cancelaccountbutton>//*[#id="unregister"]/p</Cancelaccountbutton>
</HomeScreenAttributes>
<HomeScreenText>
<Logo />
<Mainimage />
<Maintext>Secure backup</Maintext>
<Backupbutton>Back Up</Backupbutton>
<ViewBackupbutton>View Your Backups</ViewBackupbutton>
<Version>Version 1.0.3</Version>
<Cancelaccountbutton />
</HomeScreenText>
</config>
From this XML i would like to create 4 arrays:
First with just the name of each node:
array1 = [HomeScreenName, HomeScreenAttributes, HomeScreenText]
Then i want an array for each node attributes:
something like this: array2(from HomeScreenName) =
[Logo, Main image, Main text, Backup button, View Backup button, Version, Cancel account button]
I have 2 main problems:
How to get just the data i want and not everything from the XML.
How to save the data (I wanted arrays, but i'm open to suggstions).
Here is the code I have for printing every node in the XML file:
import java.io.File;
import java.io.IOException;
import java.util.List;
import org.jdom.Document;
import org.jdom.Element;
import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
public class readConfigXML{
public static void main(String[] args)
{
SAXBuilder builder = new SAXBuilder();
String folderPath = "C:\\Users\\udi\\Documents\\external\\XML\\";
String fileName = "configTest.xml";
String filePath = folderPath + fileName;
File xmlFile = new File(filePath);
try {
Document document = (Document) builder.build(xmlFile);
Element rootNode = document.getRootElement();
List configList = rootNode.getChildren();
for (int i = 0; i < configList.size(); i++)
{
Element node = (Element) configList.get(i);
List dataNodes = node.getChildren();
for (int j = 0; j < dataNodes.size(); ++j)
{
Element dataNode = (Element) dataNodes.get(j);
System.out.println(dataNode.getName());
}
}
}
catch (IOException io)
{
System.out.println(io.getMessage());
}
catch (JDOMException jdomex)
{
System.out.println(jdomex.getMessage());
}
}
}
Any help will be greatly appreciated!

Here is a simple code that can achieve what do you wanted. I used Lists in this code. To get just data you wanted, i do not have any other method than checking the nodes than have been read from the xml file.
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.jdom.Document;
import org.jdom.Element;
import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
public class readConfigXML {
// the nodes that we want to be read
public static List<String> wantedNodes = new ArrayList<String>() ;
// the attributes that we want to be read
public static List<String> wantedAttributes= new ArrayList<String>() ;
// init the nodes and attributes wanted here
public static void init(){
wantedNodes.add("HomeScreenName") ;
wantedNodes.add("HomeScreenAttributes") ;
wantedAttributes.add("Logo") ; // ...
}
public static void main(String[] args)
{
// here init your wanted nodes and attributes
init() ;
SAXBuilder builder = new SAXBuilder();
String folderPath = "C:\\Users\\udi\\Documents\\external\\XML\\";
String fileName = "configTest.xml";
String filePath = folderPath + fileName;
File xmlFile = new File(filePath);
List<String> nodes = new ArrayList<String>() ;
List<String> attributeNodes = new ArrayList<String>() ;
try {
Document document = (Document) builder.build(xmlFile);
Element rootNode = document.getRootElement();
List configList = rootNode.getChildren();
for (int i = 0; i < configList.size(); i++)
{
Element node = (Element) configList.get(i);
// check if this node is wanted
if(wantedNodes.contains(node.getName())){
nodes.add(node.getName()) ;
List dataNodes = node.getChildren();
for (int j = 0; j < dataNodes.size(); ++j)
{
Element dataNode = (Element) dataNodes.get(j);
// check if this attribute is wanted
if(wantedAttributes.contains(dataNode.getName())){
attributeNodes.add(dataNode.getValue()) ;
}
}
}
}
}
catch (IOException io)
{
System.out.println(io.getMessage());
}
catch (JDOMException jdomex)
{
System.out.println(jdomex.getMessage());
}
}
}

for the standard JDK package org.w3c.dom(I did not check but perhaps your org.jdom must have the same or similar:
How to get just the data i want and not everything from the XML.
each node has a node type. Then it may have a child of TEXT_NODE type. (Node Type is short and Node interface has constants for types. e.g.
if (dataNode.getNodeType() == Node.TEXT_NODE)
So when you traverse through node children - check if it is a TEXT_NODE then it is a data you need.
e.g. Element(node) <Logo>Logo</Logo> has a child text node with value "Logo" and you can get it buy calling: node.getTextContent().
BTW: Be careful with getTextContent() if it is mixed node (with text and other child elements) it will return texts from all children. I do not see it in your example, but first check the Node Type if it is Text Node you are good to use it.
How to save the data (I wanted arrays, but i'm open to suggestions).
It is up to you, but as long as Java does not have dynamic arrays it is better to use List as example LinkedList or ArrayList to capture data from unknown numbers of elements.
If you still want to have array, after all done get array from your resulting List by calling its toArray() method.
Also, for general purpose (if XML tree has unknown or large number of levels you need to use recursion to process child nodes). If it is not a case and you know exactly how deep your XML is, you can be fine to use nested for loops, but still recursion is preferred.

So the best way i could find was to edit my XML so that all the main child nodes will have the same name with different attributes like this:
<Page id="Home">
<Page id="Sign in">
And the code to get all the main Nodes names is this:
public static void main(String[] args)
{
String folderPath = "C:\\XML\\";
String fileName = "2.xml";
String xmlFile = folderPath + fileName;
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder;
try
/// gets the main nodes in the config and saves them to a list
{
dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(xmlFile);
doc.getDocumentElement().normalize();
//System.out.println("Root element is:" + doc.getDocumentElement().getNodeName());
NodeList mainList = doc.getElementsByTagName("Page");
List<String> list = new ArrayList<String>();
for (int temp = 0; temp < mainList.getLength(); temp++)
{
Node mainNode = mainList.item(temp);
Element eElement = (Element) mainNode;
list.add(eElement.getAttribute("id"));
System.out.println( list.get(temp));
}
System.out.println(list);
System.out.println("----------------------------------------");
}
catch(SAXException | ParserConfigurationException | IOException e1)
{
e1.printStackTrace();
}
}
And the output will be:
Home
Sign in
Register
Login
and as a list: [Home, Sign in, Register, Login]

HTML body returns empty (most of it) when calling from Jsoup [duplicate]

I have a problem using jsoup what I am trying to do is fetch a document from the url which will redirect to another url based on meta refresh url which is not working, to explain clearly if I am entering a website url named http://www.amerisourcebergendrug.com which will automatically redirect to http://www.amerisourcebergendrug.com/abcdrug/ depending upon the meta refresh url but my jsoup is still sticking with http://www.amerisourcebergendrug.com and not redirecting and fetching from http://www.amerisourcebergendrug.com/abcdrug/
Document doc = Jsoup.connect("http://www.amerisourcebergendrug.com").get();
I have also tried using,
Document doc = Jsoup.connect("http://www.amerisourcebergendrug.com").followRedirects(true).get();
but both are not working
Any workaround for this?
Update:
The Page may use meta refresh redirect methods

Update (case insensitive and pretty fault tolerant)
The content parsed (almost) according to spec
The first successfully parsed content meta data should be used
public static void main(String[] args) throws Exception {
URI uri = URI.create("http://www.amerisourcebergendrug.com");
Document d = Jsoup.connect(uri.toString()).get();
for (Element refresh : d.select("html head meta[http-equiv=refresh]")) {
Matcher m = Pattern.compile("(?si)\\d+;\\s*url=(.+)|\\d+")
.matcher(refresh.attr("content"));
// find the first one that is valid
if (m.matches()) {
if (m.group(1) != null)
d = Jsoup.connect(uri.resolve(m.group(1)).toString()).get();
break;
}
}
}
Outputs correctly:
http://www.amerisourcebergendrug.com/abcdrug/
Old answer:
Are you sure that it isn't working. For me:
System.out.println(Jsoup.connect("http://www.ibm.com").get().baseUri());
.. outputs http://www.ibm.com/us/en/ correctly..

to have a better error handling and case sensitivity problem
try
{
Document doc = Jsoup.connect("http://www.ibm.com").get();
Elements meta = doc.select("html head meta");
if (meta != null)
{
String lvHttpEquiv = meta.attr("http-equiv");
if (lvHttpEquiv != null && lvHttpEquiv.toLowerCase().contains("refresh"))
{
String lvContent = meta.attr("content");
if (lvContent != null)
{
String[] lvContentArray = lvContent.split("=");
if (lvContentArray.length > 1)
doc = Jsoup.connect(lvContentArray[1]).get();
}
}
}
// get page title
return doc.title();
}
catch (IOException e)
{
e.printStackTrace();
}

Trouble loading from XML file

I have a major problem. We can call this application "boat club system" and its a assignment in a UML course. But the problem that is really bothering me is that i can't load the "Member" Objects back to the Arraylist that is used when Exporting and Importing the objects.
I got some help with this by a friend, but the Importing won't work.
This is the Error i get when trying to import.
Exception in thread "main" java.lang.NullPointerException
at boat.Controller.SystemController.readFromSystem(SystemController.java:288)
at boat.Controller.SystemController.<init>(SystemController.java:26)
at boat.View.Console.<init>(Console.java:17)
at BoatMain.main(BoatMain.java:22)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
The problem is pointed on this specific line:
member.setMemberId(Integer.parseInt(element.get(i).getAttribute("memberId").getValue()));
The link to the whole application.
https://github.com/mjuu/boat/tree/master/src
The System controller class looks like this.
BTW, i am thinking of splitting up this class to more like System controller and Member controller for a more High Cohesion/Low coupling design. But only when i get the Import working correctly..
I don't know why it doesn't work. The member id is a INTEGER at first, but when exported its converted to a String. When imported it converted to a Int again. So I'm wondering if its something wrong there..
Would really appreciate help so i could continue.
System Controller
public void readFromSystem () {
File file = null;
Builder builder = null;
Document doc = null;
try {
file = new File(filePath);
builder = new Builder();
doc = builder.build(file);
Element root = doc.getRootElement();
Elements members = root.getChildElements();
for (int i = 0; i < members.size(); i++) {
Elements element = members.get(i).getChildElements();
Member member = new Member();
member.setMemberId(Integer.parseInt(element.get(i).getAttribute("memberId").getValue()));
member.setPersonId(element.get(0).getValue());
member.setName(element.get(1).getValue());
memberList.add(member);
if (members.get(i).getChildElements().size() == 3) {
Elements boats = element.get(2).getChildElements();
for (int j = 0; j < boats.size(); j++) {
Boat b = new Boat();
b.setBoatId(Integer.parseInt(boats.get(j).getAttribute("boatId").getValue()));
b.setBoatType(Integer.parseInt(boats.get(j).getChildElements().get(1).getValue()));
b.setBoatLength(boats.get(j).getChildElements().get(2).getValue());
}
}
}
} catch (IOException e) {
System.out.print("Could not read from the system");
} catch (nu.xom.ParsingException e) {
System.out.print("Parsing was unsuccessful!");
}
}

DOM Parser receiving NullPointerException on pure HTML RSS post

I'm going to try make this as clear as possible, although I'm not sure I'll succeed.
I've implemented a DOM parser in Android to parse a typical RSS feed based off some of the code found here. It works fine for almost all of the feeds I've tried however I just ran into a NullPointerException on the line theString = nchild.item(j).getFirstChild().getNodeValue(); (my code is lower down) on a certain post on a certain feed from a Blogger site. I know it's only this post because I rewrote the loop to ignore this single post and the error didn't appear and parsing continued just fine. Upon looking at this post within the actual RSS feed, it seems this post is entirely written in HTML (as opposed to just standard text) whereas the other posts which succeeded aren't.
Would this be the cause of the issue, or should I keep looking? And if this is indeed the issue, how would I go about solving it? Is there a way to ignore posts which are written in this way? I've tried looking for alternative examples to compare and try, but it seems that everyone has used the same base code for their tutorials.
The post I'm referring to is just a link, and a couple of lines of coloured text within <div> tags with some different fonts. I'd post it here, but I'm not sure the owner of the feed would want me to (I'll ask and update if able).
My parser:
try {
// Create required instances
DocumentBuilderFactory dbf;
dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
// Parse the xml
Document doc = db.parse(new InputSource(url.openStream()));
doc.getDocumentElement().normalize();
// Get all <item> tags.
NodeList nl = doc.getElementsByTagName("item");
int length = nl.getLength();
for (int i = 0; i < length; i++) {
Node currentNode = nl.item(i);
RSSItem _item = new RSSItem();
NodeList nchild = currentNode.getChildNodes();
int clength = nchild.getLength();
for (int j = 1; j < clength; j = j + 2) {
Node thisNode = nchild.item(j);
String theString = null;
String nodeName = thisNode.getNodeName();
theString = nchild.item(j).getFirstChild().getNodeValue();
if (theString != null) {
if ("title".equals(nodeName)) {
_item.setTitle(theString);
} else if ("description".equals(nodeName)) {
_item.setDescription(theString);
} else if ("pubDate".equals(nodeName)) {
String formatedDate = theString.replace(" +0000", "");
_item.setDate(formatedDate);
} else if ("author".equals(nodeName)) {
_item.setAuthor(theString);
}
}
}
_feed.addItem(_item);
}
} catch (Exception e) {
e.printStackTrace();
}
return _feed;
}
As I mentioned, I changed the text to ignore the (third) post causing the issue:
if(i != 3){
if (theString != null) {
if ("title".equals(nodeName)) {
_item.setTitle(theString);
} else if ("description".equals(nodeName)) {
_item.setDescription(theString);
} else if ("pubDate".equals(nodeName)) {
String formatedDate = theString.replace(" +0000", "");
_item.setDate(formatedDate);
} else if ("author".equals(nodeName)) {
_item.setAuthor(theString);
}
}
}
Which resulted in everything working as desired, just skipping the third post. Any help with this is appreciated, I've been searching for a while with no luck. I'd post my logcat but it's not very useful after the line I pasted at the start of this Q due to it going back through an AsyncTask.
Oh, and one of the ways I was thinking about solving it was just parse the description first instead of the title (rewriting the loop of course), and detecting if that was equal to NULL before continuing the parse. It'd be quite messy though, so I'm searching for an alternative.

Take a look at the HTML code you are trying to parse. I'm almost sure that the third post has no child. This is, it's empty. For example, this node would throw you an exception:
<Element></Element>
So, you must avoid calling getNodeValue before checking if the node has any childs:
theString = nchild.item(j).getFirstChild().getNodeValue();
To avoid this, you could make something like:
if (nchild.item(j).getFirstChild() != null)
//and your code
//...