We Keep Coding

Java XML Getting nodes from node list crashes program

Hello there.
As the title suggests, I currently have an issue in
my program. In the animation loader, I have a method that should
load an animation from a collada file. It gets an Element as an input.
The first thing I do is to collect the animation data. I do this by getting a node list with
NodeList sources = element.getElementsByTagName("source");
And then I iterate through that node list:
for(int i = 0; i < sources.getLength(); i++)
{
// Problem occurs here:
Element sourceElement = (Element) (sources.item(i));
String id = sourceElement.getAttribute("id");
if(id.equals(inputId))
inputSource = FloatArraySource.loadFromElement(sourceElement);
else if(id.equals(outputId))
outputSource = Matrix4fSource.loadFromElement(sourceElement);
else if(id.equals(interpolationId))
interpolationSource = StringArraySource.loadFromElement(sourceElement);
}
The problem occurs on the commented line, and it crashes (only sometimes) with this following exception
Cannot invoke "com.sun.org.apache.xerces.internal.dom.CoreDocumentImpl.changes()" because the return value of "com.sun.org.apache.xerces.internal.dom.NodeImpl.ownerDocument()" is null
I can start the application three times in a row, and it crashes roughly one of four times.
The strangest thing is the fact that it runs perfectly fine in debug mode.
So, I'd be very happy if you could help me out with this issue.
-Budschie
Edit: Some people wanted that I post the full stack trace, so here it is:
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "com.sun.org.apache.xerces.internal.dom.CoreDocumentImpl.changes()" because the return value of "com.sun.org.apache.xerces.internal.dom.NodeImpl.ownerDocument()" is null
at java.xml/com.sun.org.apache.xerces.internal.dom.NodeImpl.changes(NodeImpl.java:1887)
at java.xml/com.sun.org.apache.xerces.internal.dom.DeepNodeListImpl.item(DeepNodeListImpl.java:125)
at java.xml/com.sun.org.apache.xerces.internal.dom.DeepNodeListImpl.getLength(DeepNodeListImpl.java:116)
at de.budschie.engine.assets_management.newcollada.AnimationLoader.loadTransformAnimation(AnimationLoader.java:77)
at de.budschie.engine.assets_management.newcollada.AnimationLoader.loadAnimation(AnimationLoader.java:31)
at de.budschie.engine.assets_management.newcollada.ColladaLoader.loadCollada(ColladaLoader.java:60)
at de.budschie.engine.assets_management.DefaultResourceLoader.loadAll(DefaultResourceLoader.java:75)
at de.budschie.engine.main.MainWindow.gameLoop(MainWindow.java:192)
at de.budschie.engine.main.MainWindow.main(MainWindow.java:81)
Another edit:
Here's the way I load my collada files:
Element colladaTag = null;
try
{
colladaTag = getColladaTag(colladaFile);
} catch (Exception e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
Element libraryAnimations = (Element) colladaTag.getElementsByTagName("library_animations").item(0);
Element libraryControllers = (Element) colladaTag.getElementsByTagName("library_controllers").item(0);
Element libraryGeometries = (Element) colladaTag.getElementsByTagName("library_geometries").item(0);
NodeList meshesList = null, controllersList = null;
if(libraryGeometries != null)
{
meshesList = libraryGeometries.getElementsByTagName("geometry");
}
if(libraryControllers != null)
{
controllersList = libraryControllers.getElementsByTagName("controller");
}
if(libraryAnimations != null)
{
AnimationLoader.loadAnimation(colladaResult, libraryAnimations);
}
And here's what "getColladaTag()" looks like:
private static Element getColladaTag(String path) throws Exception
{
File file = new File(path);
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
try
{
DocumentBuilder docBuilder = factory.newDocumentBuilder();
Document doc = docBuilder.parse(file);
return doc.getDocumentElement();
}
catch(IOException | SAXException ex)
{
System.out.println("There is a problem with the file that couldn't be fixed.");
ex.printStackTrace();
}
return null;
}
Another small thing I noticed is that sometimes, the JVM itself crashesbecause of an access violation in the string builder...
Very important edit:Whilest debugging I found out that I can't import com.sun.org.apache.xerces.internal.dom.NodeImpl.
My program doesn't throw a ClassNotFoundException though...
So, could that be a reason why the GC is so confused?

You could change
for(int i = 0; i < sources.getLength(); i++)
{
// Problem occurs here:
Element sourceElement = (Element) (sources.item(i));
to
for ( Element sourceElement : sources )
{
which would remove sources.item(i). You could put System.out.println("Index: " + i): just above this line, which would give an indication of how far you get.
It looks like something is modifying the sources container while you are processing it.

private static <T extends YourColladaDataFormat> T loadColladaFile(String pathToXml) throws Exception {
// loads the XML Document, walks through it and returns your workable data model.
}
And then work with T.

HTML body returns empty (most of it) when calling from Jsoup [duplicate]

I have a problem using jsoup what I am trying to do is fetch a document from the url which will redirect to another url based on meta refresh url which is not working, to explain clearly if I am entering a website url named http://www.amerisourcebergendrug.com which will automatically redirect to http://www.amerisourcebergendrug.com/abcdrug/ depending upon the meta refresh url but my jsoup is still sticking with http://www.amerisourcebergendrug.com and not redirecting and fetching from http://www.amerisourcebergendrug.com/abcdrug/
Document doc = Jsoup.connect("http://www.amerisourcebergendrug.com").get();
I have also tried using,
Document doc = Jsoup.connect("http://www.amerisourcebergendrug.com").followRedirects(true).get();
but both are not working
Any workaround for this?
Update:
The Page may use meta refresh redirect methods

Update (case insensitive and pretty fault tolerant)
The content parsed (almost) according to spec
The first successfully parsed content meta data should be used
public static void main(String[] args) throws Exception {
URI uri = URI.create("http://www.amerisourcebergendrug.com");
Document d = Jsoup.connect(uri.toString()).get();
for (Element refresh : d.select("html head meta[http-equiv=refresh]")) {
Matcher m = Pattern.compile("(?si)\\d+;\\s*url=(.+)|\\d+")
.matcher(refresh.attr("content"));
// find the first one that is valid
if (m.matches()) {
if (m.group(1) != null)
d = Jsoup.connect(uri.resolve(m.group(1)).toString()).get();
break;
}
}
}
Outputs correctly:
http://www.amerisourcebergendrug.com/abcdrug/
Old answer:
Are you sure that it isn't working. For me:
System.out.println(Jsoup.connect("http://www.ibm.com").get().baseUri());
.. outputs http://www.ibm.com/us/en/ correctly..

to have a better error handling and case sensitivity problem
try
{
Document doc = Jsoup.connect("http://www.ibm.com").get();
Elements meta = doc.select("html head meta");
if (meta != null)
{
String lvHttpEquiv = meta.attr("http-equiv");
if (lvHttpEquiv != null && lvHttpEquiv.toLowerCase().contains("refresh"))
{
String lvContent = meta.attr("content");
if (lvContent != null)
{
String[] lvContentArray = lvContent.split("=");
if (lvContentArray.length > 1)
doc = Jsoup.connect(lvContentArray[1]).get();
}
}
}
// get page title
return doc.title();
}
catch (IOException e)
{
e.printStackTrace();
}

How to get data from an URL xml-file?

I'm making an android app - where I need to have weather-information. I've found this from yahoo weather. It's an XML and I want information such as: "day", "low" and "high".
Refer: http://weather.yahooapis.com/forecastrss?w=12718298&u=c
<yweather:forecast day="Sun" date="19 Feb 2012" low="-2" high="3" text="Clear" code="31"/>
(Line can be found in the bottom of the link)
I have no idea how to do this - please help. Source codes, examples and clues will be appreciated.

Here's the solution for future users:
InputStream inputXml = null;
try
{
inputXml = new URL("http://weather.yahooapis.com/forecastrss?w=12718298&u=c").openConnection().getInputStream();
DocumentBuilderFactory factory = DocumentBuilderFactory.
newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse(inputXml);
NodeList nodi = doc.getElementsByTagName("yweather:forecast");
if (nodi.getLength() > 0)
{
Element nodo = (Element)nodi.item(0);
String strLow = nodo.getAttribute("low");
Element nodo1 = (Element)nodi.item(0);
String strHigh = nodo1.getAttribute("high");
System.out.println("Temperature low: " + strLow);
System.out.println("Temperature high: " + strHigh);
}
}
catch (Exception ex)
{
System.out.println(ex.getMessage());
}
finally
{
try
{
if (inputXml != null)
inputXml.close();
}
catch (IOException ex)
{
System.out.println(ex.getMessage());
}
}
}

It's been a couple of years since I used XML in Android, but this was quite helpful to me when I started out: anddev.org

The link seems to be a feed. (which is XML, obviously). There are many feed-reader APIs in Java. So, here you go
Read feed documentation, http://developer.yahoo.com/weather/
Read how to parse/read the feed, Rome Library to read feeds Java
Pull out your desired fields.
I guess this is already done. (easily found on Google) http://www.javahouse.altervista.org/gambino/Articolo_lettura_feed_da_java_en.html

Android parse KML file for time

I've been trying to work out how to obtain the travel time between two locations (walking, driving etc...).
As I understand it, the only way to do this accurately is by retrieving a KML file from google, then parsing it.
Research has shown that it then needs to be parsed with SAX. The problem is, I can't seem to work out how to extract the correct variables (the time). Does anybody know if / how this can be done?
Many thanks for your help,
Pete.

Parsing XML (what KML basically is), using a SAX-Parser: http://www.dreamincode.net/forums/blog/324/entry-2683-parsing-xml-in-java-part-1-sax/
<kml>
<Document>
<Placemark>
<name>Route</name>
<description>Distance: 1.4 mi (about 30 mins)<br/>Map data ©2011 Tele Atlas </description>
</Placemark>
</Document>
</kml>
In the example you can see, that the guessed time is stored in the "description"-Tag. It's saved in the last "Placemark"-Tag in the KML-File and it has a "<name>Route</name>"-Tag.
Getting this Tag with the SAX-Parser and extracting the time using regex should be easy done.

Here's my JSOUP implementation for getting tracks
public ArrayList<ArrayList<LatLng>> getAllTracks() {
ArrayList<ArrayList<LatLng>> allTracks = new ArrayList<ArrayList<LatLng>>();
try {
StringBuilder buf = new StringBuilder();
InputStream json = MyApplication.getInstance().getAssets().open("track.kml");
BufferedReader in = new BufferedReader(new InputStreamReader(json));
String str;
while ((str = in.readLine()) != null) {
buf.append(str);
}
in.close();
String html = buf.toString();
Document doc = Jsoup.parse(html, "", Parser.xmlParser());
ArrayList<String> tracksString = new ArrayList<String>();
for (Element e : doc.select("coordinates")) {
tracksString.add(e.toString().replace("<coordinates>", "").replace("</coordinates>", ""));
}
for (int i = 0; i < tracksString.size(); i++) {
ArrayList<LatLng> oneTrack = new ArrayList<LatLng>();
ArrayList<String> oneTrackString = new ArrayList<String>(Arrays.asList(tracksString.get(i).split("\\s+")));
for (int k = 1; k < oneTrackString.size(); k++) {
LatLng latLng = new LatLng(Double.parseDouble(oneTrackString.get(k).split(",")[0]), Double.parseDouble(oneTrackString.get(k)
.split(",")[1]));
oneTrack.add(latLng);
}
allTracks.add(oneTrack);
}
} catch (Exception e) {
e.printStackTrace();
}
return allTracks;
}

stax - get xml node as string

xml looks like so:
<statements>
<statement account="123">
...stuff...
</statement>
<statement account="456">
...stuff...
</statement>
</statements>
I'm using stax to process one "<statement>" at a time and I got that working. I need to get that entire statement node as a string so I can create "123.xml" and "456.xml" or maybe even load it into a database table indexed by account.
using this approach: http://www.devx.com/Java/Article/30298/1954
I'm looking to do something like this:
String statementXml = staxXmlReader.getNodeByName("statement");
//load statementXml into database

I had a similar task and although the original question is older than a year, I couldn't find a satisfying answer. The most interesting answer up to now was Blaise Doughan's answer, but I couldn't get it running on the XML I am expecting (maybe some parameters for the underlying parser could change that?). Here the XML, very simplyfied:
<many-many-tags>
<description>
...
<p>Lorem ipsum...</p>
Devils inside...
...
</description>
</many-many-tags>
My solution:
public static String readElementBody(XMLEventReader eventReader)
throws XMLStreamException {
StringWriter buf = new StringWriter(1024);
int depth = 0;
while (eventReader.hasNext()) {
// peek event
XMLEvent xmlEvent = eventReader.peek();
if (xmlEvent.isStartElement()) {
++depth;
}
else if (xmlEvent.isEndElement()) {
--depth;
// reached END_ELEMENT tag?
// break loop, leave event in stream
if (depth < 0)
break;
}
// consume event
xmlEvent = eventReader.nextEvent();
// print out event
xmlEvent.writeAsEncodedUnicode(buf);
}
return buf.getBuffer().toString();
}
Usage example:
XMLEventReader eventReader = ...;
while (eventReader.hasNext()) {
XMLEvent xmlEvent = eventReader.nextEvent();
if (xmlEvent.isStartElement()) {
StartElement elem = xmlEvent.asStartElement();
String name = elem.getName().getLocalPart();
if ("DESCRIPTION".equals(name)) {
String xmlFragment = readElementBody(eventReader);
// do something with it...
System.out.println("'" + fragment + "'");
}
}
else if (xmlEvent.isEndElement()) {
// ...
}
}
Note that the extracted XML fragment will contain the complete extracted body content, including white space and comments. Filtering those on demand, or making the buffer size parametrizable have been left out for code brevity:
'
<description>
...
<p>Lorem ipsum...</p>
Devils inside...
...
</description>
'

You can use StAX for this. You just need to advance the XMLStreamReader to the start element for statement. Check the account attribute to get the file name. Then use the javax.xml.transform APIs to transform the StAXSource to a StreamResult wrapping a File. This will advance the XMLStreamReader and then just repeat this process.
import java.io.File;
import java.io.FileReader;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamConstants;
import javax.xml.stream.XMLStreamReader;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stax.StAXSource;
import javax.xml.transform.stream.StreamResult;
public class Demo {
public static void main(String[] args) throws Exception {
XMLInputFactory xif = XMLInputFactory.newInstance();
XMLStreamReader xsr = xif.createXMLStreamReader(new FileReader("input.xml"));
xsr.nextTag(); // Advance to statements element
while(xsr.nextTag() == XMLStreamConstants.START_ELEMENT) {
TransformerFactory tf = TransformerFactory.newInstance();
Transformer t = tf.newTransformer();
File file = new File("out" + xsr.getAttributeValue(null, "account") + ".xml");
t.transform(new StAXSource(xsr), new StreamResult(file));
}
}
}

Stax is a low-level access API, and it does not have either lookups or methods that access content recursively. But what you actually trying to do? And why are you considering Stax?
Beyond using a tree model (DOM, XOM, JDOM, Dom4j), which would work well with XPath, best choice when dealing with data is usually data binding library like JAXB. With it you can pass Stax or SAX reader and ask it to bind xml data into Java beans and instead of messing with xml process Java objects. This is often more convenient, and it is usually quite performance.
Only trick with larger files is that you do not want to bind the whole thing at once, but rather bind each sub-tree (in your case, one 'statement' at a time).
This is easiest done by iterating Stax XmlStreamReader, then using JAXB to bind.

I've been googling and this seems painfully difficult.
given my xml I think it might just be simpler to:
StringBuilder buffer = new StringBuilder();
for each line in file {
buffer.append(line)
if(line.equals(STMT_END_TAG)){
parse(buffer.toString())
buffer.delete(0,buffer.length)
}
}
private void parse(String statement){
//saxParser.parse( new InputSource( new StringReader( xmlText ) );
// do stuff
// save string
}

Why not just use xpath for this?
You could have a fairly simple xpath to get all 'statement' nodes.
Like so:
//statement
EDIT #1: If possible, take a look at dom4j. You could read the String and get all 'statement' nodes fairly simply.
EDIT #2: Using dom4j, this is how you would do it:
(from their cookbook)
String text = "your xml here";
Document document = DocumentHelper.parseText(text);
public void bar(Document document) {
List list = document.selectNodes( "//statement" );
// loop through node data
}

I had the similar problem and found the solution.
I used the solution proposed by #t0r0X but it does not work well in the current implementation in Java 11, the method xmlEvent.writeAsEncodedUnicode creates the invalid string representation of the start element (in the StartElementEvent class) in the result XML fragment, so I had to modify it, but then it seems to work well, what I could immediatelly verify by the parsing of the fragment by DOM and JaxBMarshaller to specific data containers.
In my case I had the huge structure
<Orders>
<ns2:SyncOrder xmlns:ns2="..." xmlns:ns3="....." ....>
.....
</ns2:SyncOrder>
<ns2:SyncOrder xmlns:ns2="..." xmlns:ns3="....." ....>
.....
</ns2:SyncOrder>
...
</Orders>
in the file of multiple hundred megabytes (a lot of repeating "SyncOrder" structures), so the usage of DOM would lead to a large memory consumption and slow evaluation. Therefore I used the StAX to split the huge XML to smaller XML pieces, which I have analyzed with DOM and used the JaxbElements generated from the xsd definition of the element SyncOrder (This infrastructure I had from the webservice, which uses the same structure, but it is not important).
In this code there can be seen Where the XML fragment has een created and could be used, I used it directly in other processing...
private static <T> List<T> unmarshallMultipleSyncOrderXmlData(
InputStream aOrdersXmlContainingSyncOrderItems,
Function<SyncOrderType, T> aConversionFunction) throws XMLStreamException, ParserConfigurationException, IOException, SAXException {
DocumentBuilderFactory locDocumentBuilderFactory = DocumentBuilderFactory.newInstance();
locDocumentBuilderFactory.setNamespaceAware(true);
DocumentBuilder locDocBuilder = locDocumentBuilderFactory.newDocumentBuilder();
List<T> locResult = new ArrayList<>();
XMLInputFactory locFactory = XMLInputFactory.newFactory();
XMLEventReader locReader = locFactory.createXMLEventReader(aOrdersXmlContainingSyncOrderItems);
boolean locIsInSyncOrder = false;
QName locSyncOrderElementQName = null;
StringWriter locXmlTextBuffer = new StringWriter();
int locDepth = 0;
while (locReader.hasNext()) {
XMLEvent locEvent = locReader.nextEvent();
if (locEvent.isStartElement()) {
if (locDepth == 0 && Objects.equals(locEvent.asStartElement().getName().getLocalPart(), "Orders")) {
locDepth++;
} else {
if (locDepth <= 0)
throw new IllegalStateException("There has been passed invalid XML stream intot he function. "
+ "Expecting the element 'Orders' as the root alament of the document, but found was '"
+ locEvent.asStartElement().getName().getLocalPart() + "'.");
locDepth++;
if (locSyncOrderElementQName == null) {
/* First element after the "Orders" has passed, so we retrieve
* the name of the element with the namespace prefix: */
locSyncOrderElementQName = locEvent.asStartElement().getName();
}
if(Objects.equals(locEvent.asStartElement().getName(), locSyncOrderElementQName)) {
locIsInSyncOrder = true;
}
}
} else if (locEvent.isEndElement()) {
locDepth--;
if(locDepth == 1 && Objects.equals(locEvent.asEndElement().getName(), locSyncOrderElementQName)) {
locEvent.writeAsEncodedUnicode(locXmlTextBuffer);
/* at this moment the call of locXmlTextBuffer.toString() gets the complete fragment
* of XML containing the valid SyncOrder element, but I have continued to other processing,
* which immediatelly validates the produced XML fragment is valid and passes the values
* to communication object: */
Document locDocument = locDocBuilder.parse(new ByteArrayInputStream(locXmlTextBuffer.toString().getBytes()));
SyncOrderType locItem = unmarshallSyncOrderDomNodeToCo(locDocument);
locResult.add(aConversionFunction.apply(locItem));
locXmlTextBuffer = new StringWriter();
locIsInSyncOrder = false;
}
}
if (locIsInSyncOrder) {
if (locEvent.isStartElement()) {
/* here replaced the standard implementation of startElement's method writeAsEncodedUnicode: */
locXmlTextBuffer.write(startElementToStrng(locEvent.asStartElement()));
} else {
locEvent.writeAsEncodedUnicode(locXmlTextBuffer);
}
}
}
return locResult;
}
private static String startElementToString(StartElement aStartElement) {
StringBuilder locStartElementBuffer = new StringBuilder();
// open element
locStartElementBuffer.append("<");
String locNameAsString = null;
if ("".equals(aStartElement.getName().getNamespaceURI())) {
locNameAsString = aStartElement.getName().getLocalPart();
} else if (aStartElement.getName().getPrefix() != null
&& !"".equals(aStartElement.getName().getPrefix())) {
locNameAsString = aStartElement.getName().getPrefix()
+ ":" + aStartElement.getName().getLocalPart();
} else {
locNameAsString = aStartElement.getName().getLocalPart();
}
locStartElementBuffer.append(locNameAsString);
// add any attributes
Iterator<Attribute> locAttributeIterator = aStartElement.getAttributes();
Attribute attr;
while (locAttributeIterator.hasNext()) {
attr = locAttributeIterator.next();
locStartElementBuffer.append(" ");
locStartElementBuffer.append(attributeToString(attr));
}
// add any namespaces
Iterator<Namespace> locNamespaceIterator = aStartElement.getNamespaces();
Namespace locNamespace;
while (locNamespaceIterator.hasNext()) {
locNamespace = locNamespaceIterator.next();
locStartElementBuffer.append(" ");
locStartElementBuffer.append(attributeToString(locNamespace));
}
// close start tag
locStartElementBuffer.append(">");
// return StartElement as a String
return locStartElementBuffer.toString();
}
private static String attributeToString(Attribute aAttr) {
if( aAttr.getName().getPrefix() != null && aAttr.getName().getPrefix().length() > 0 )
return aAttr.getName().getPrefix() + ":" + aAttr.getName().getLocalPart() + "='" + aAttr.getValue() + "'";
else
return aAttr.getName().getLocalPart() + "='" + aAttr.getValue() + "'";
}
public static SyncOrderType unmarshallSyncOrderDomNodeToCo(
Node aSyncOrderItemNode) {
Source locSource = new DOMSource(aSyncOrderItemNode);
Object locUnmarshalledObject = getMarshallerAndUnmarshaller().unmarshal(locSource);
SyncOrderType locCo = ((JAXBElement<SyncOrderType>) locUnmarshalledObject).getValue();
return locCo;
}