We Keep Coding

How to get data from a URL in new lines using JSOUP?

I'm scrapping IMDB chart of 250 movies. I want to store each movie name in an array, but I don't know why it puts all the movie names into the first index, i.e Array[0].
Below is my code.
Can anyone please help me out. I've to complete another project and this is the main thing that is needed.
If you can direct me any website or tutorial I'll be very thankful to you.
try {
Document doc = Jsoup.connect("http://www.imdb.com/chart/top").userAgent("Mozilla").get();
int counterVariable = 0;
for (Element el : doc.select(".lister-list")) {
mString[counterVariable] = el.select(".titleColumn").text();
totalNumberOfLines++;
counterVariable++;
}
} catch (Exception e) {
System.out.println("Sorry website couldn't be opened");
System.out.println(e);
}
System.out.println(mString[0]);// It's putting all the names into this index

The problem is that you have only one element matching selector .lister-list, so iterating over it does not make much sense. When you call el.select(".titleColumn").text(); Jsoup concatenates text from all matching elements. This is why you get all results in one element. Instead you can try to select all td tags with class tittleColumn that are children of tr element that are child of .lister-list
for (Element el : doc.select(".lister-list > tr > td.titleColumn")) {
mString[counterVariable] = el.text();
totalNumberOfLines++;
counterVariable++;
}
More about jsoup css selectors you can learn here.

Adding sub-element to element java (DOM)

Essentially, i'm creating an XML document from a file (a database), and then i'm comparing another parsed XML file (with updated information) to the original database, then writing the new information into the database.
I'm using java's org.w3c.dom.
After lots of struggling, i decided to just create a new Document object and will write from there from the oldDocument and newDocument ones i'm comparing the elements in.
The XML doc is in the following format:
<Log>
<File name="something.c">
<Warning file="something.c" line="101" column="23"/>
<Warning file="something.c" line="505" column="71" />
</File>
</Log>
as an example.
How would i go about adding in a new "warning" Element to the "File" without getting the pesky "org.w3c.dom.DOMException: WRONG_DOCUMENT_ERR: A node is used in a different document than the one that created it." exception?
Cutting it down, I have something similar to:
public static Document update(Element databaseRoot, Element newRoot){
Document doc = db.newDocument(); // DocumentBuilder defined previously
Element baseRoot = doc.createElement("Log");
//for each file i have:
Element newFileRoot = doc.createElement("File");
//some for loop that parses through each 'file' and looks at the warnings
//when i come to a new warning to add to the Document:
NodeList newWarnings = newFileToCompare.getChildNodes(); //newFileToCompare comes from the newRoot element
for (int m = 0; m < newWarnings.getLength(); m++){
if(newWarnings.item(m).getNodeType() == Node.ELEMENT_NODE){
Element newWarning = (Element)newWarnings.item(m);
Element newWarningRoot = (Element)newWarning.cloneNode(false);
newFileRoot.appendChild(doc.importNode(newWarningRoot,true)); // this is what crashes
}
}
// for new files i have this which works:
newFileRoot = (Element)newFiles.item(i).cloneNode(true);
baseRoot.appendChild(doc.importNode(newFileRoot,true));
doc.appendChild(baseRoot);
return doc;
}
Any ideas? I'm beating my head against the wall. First time doing this.

Going through with a debugger I verified that the document owners were correct. Using node.getOwnerDocument(), I realized that the newFileRoot was connected to the wrong document earlier when I created it, so I changed
Element newFileRoot = (Element)pastFileToFind.cloneNode(false);
to
Element newFileRoot = (Element)doc.importNode(pastFileToFind.cloneNode(false),true);
since later on when i was trying to add the newWarningRoot to newFileRoot, they had different Documents (newWarningRoot was correct but newFileRoot was connected to the wrong document)

Read the contents of child node from xml to Java

I have been trying to read this xml and fetch the values of id,active and MAIN_ID and store to string values in java as i need to process each values further. But i could read the Node document ,but iam not sure how to get the values of id ,active and MAIN_ID when we loop though the document. Can someone give me ideas to parse this xml and best way to do it.
<add>
<document>
<field name='id'>Summer id</field>
<field name='active' update='add'>yes</field>
<field name='MAIN_ID' update='add'>34242</field>
</doc>
<document>
<field name='id'>winter id</field>
<field name='active' update='add'>yes</field>
<field name='MAIN_ID' update='add'>5354</field>
</document>
<doc>
Right now this is my code .But iam not sure to retrive the child nodes based on fields.
DocumentBuilder builder = factory.newDocumentBuilder();
// create a new document from input stream and an empty systemId
Document doc = builder.parse(url);
// get the first element
Element element = doc.getDocumentElement();
System.out.println("element" + element);
// get all child nodes
NodeList nodes = element.getChildNodes();
// print the text content of each child
for (int i = 0; i < nodes.getLength(); i++) {
Node node = nodes.item(i);
if (node instanceof Element) {
//need to know how to parse the child elements for above code
}
}
} catch (Exception ex) {
ex.printStackTrace();
}
}

First consider not using attributes in XML to specify node meaning, so instead have something like this
...
<document>
<id>winter id</id>
<active update='add'>yes</active>
<MAIN_ID update='add'>5354</MAIN_ID>
</document>
...
If your document is large, consider learning about and using a SAX parser. However if you want to use DOM, look at the org.w3c.dom.Element interface. There are lots of step by step posts on SO on how to parse HTML using DOM, like this one. But you want getElementsByTagName for elements named "document", then pull the individual fields from that Element using the same methods to get id, active, etc.

Netbeans: How to populate JTree Dynamically?

I am developing a small desktop application in Java using NetBeans. At some point i need to read data from XML file and after reading that i need to store that in object of my custom class. I sucessfully did the above mentioned task (i.e I read XML data and store that data in object of my custom class). Now i want to populate a JTree with that object. Suppose my XML looks like this:
<Employees>
<Classification type="permanent">
<Emp>
<name>Emp1_Name<name/>
</Emp>
<Emp>
<name>Emp2_Name<name/>
</Emp> </Classification>
<Classification type="part time">
<Emp>
<name>Emp1_Name<name/>
</Emp>
<Emp>
<name>Emp2_Name<name/>
</Emp> </Classification>
</Classification>
</Employees>
Now i want my tree to look like this
Employees
Permanent Employees
Emp1_Name
Emp2_Name
Part Time Employees
Emp1_Name
Emp2_Name

This might be useful for you :
http://www.arsitech.com/xml/jdom_xml_jtree.php
http://www.wsoftware.de/SpeedJG/XMLTreeView.html
Code Taken From :
Java: How to display an XML file in a JTree
public JTree build(String pathToXml) throws Exception {
SAXReader reader = new SAXReader();
Document doc = reader.read(pathToXml);
return new JTree(build(doc.getRootElement()));
}
public DefaultMutableTreeNode build(Element e) {
DefaultMutableTreeNode result = new DefaultMutableTreeNode(e.getText());
for(Object o : e.elements()) {
Element child = (Element) o;
result.add(build(child));
}
return result;
}

You need to write the function to fetch the information of your application object, You can use SAX or DOM parser for that
DefaultMutableTreeNode root = new DefaultMutableTreeNode()
Object object = //function to fetch the information of your Object
// if you are storing all objects in a vector then read element of object
root.add((DefaultMutableTreeNode)object)

Adding a new node before the first child in dom tree

This is probably one of those easiest things to deal with but for some reason not working for me. I'm trying to add a new node after the root in dom tree.
Here's the original string:
<div class="discussionThread dt"><div class="dt_subject">2011 IS HERE!</div></div>
I'm trying to add a new node which is in the form of a string before . The final version should look like:
<div class="discussionThread dt"><div class="test">Test Val</div><div class="dt_subject">2011 IS HERE!</div></div>
As you can see, the new Test Val is being added immediately after the root div class. I've used few methods to place the node at the right place but its getting appended at the end.
Here's a sample which I referred from one of the earlier posts:
String newNode = "<div class="test">test</div>";
SAXReader reader = new SAXReader();
Document newNodeDocument = reader.read(new StringReader(newNode));
Document originalDoc = new SAXReader().read(new StringReader(content));
Element root = originalDoc.getRootElement();
Element givenNode = originalDoc.getRootElement();
givenNode.add(newNodeDocument.getRootElement());
This is resulting the node getting added at the end. I tried using insertBefore(), but didn't work out.
Any pointers will be highly appreciated.
Thanks

Why create a new Document or a new root Element? I think the shortest way is using Branch#content:
Returns the content nodes of this
branch as a backed List so that
the content of this branch may be
modified directly using the interface.
The List is backed by the Branch so
that changes to the list are reflected
in the branch and vice versa.
You just have to create the new Element and to add it to the root element through the List provided by content method (passing it the position index), this is my main:
public static void main(String[] args) throws DocumentException {
SAXReader reader = new SAXReader();
String xml = "<div class=\"discussionThread dt\"><div class=\"dt_subject\">2011 IS HERE!</div></div>";
Document document = reader.read(new StringReader(xml));
DefaultElement newElement = new DefaultElement("div");
newElement.addAttribute("class", "test");
newElement.add(new DefaultText("Test Val"));
List content = document.getRootElement().content();
if (content != null ) {
content.add(0, newElement);
}
System.out.println(document.asXML());
}
which prints out the following xml:
<div class="discussionThread dt"><div class="test">Test Val</div><div class="dt_subject">2011 IS HERE!</div></div>
In addition, you should also consider the use of xslt when you have to transform xml.

You're calling Element#add(Entity). From the Javadocs:
Adds the given Entity to this element. If the given node already has a parent defined then an IllegalAddException will be thrown.
So the new node you're adding will be added as a child of the node you're adding it to. You cannot add another node after the root node you have, because a document can have only one root node.
What you can do is create a new root node, then add the old root node and the new node as children of this new root node. Then set the root node of the document to the new root node.

Why not just create a new Document with the value you want, then append the other nodes to it?